spectre

exploiting check_passphrase (1)

exploiting check_passphrase (2)

big integer multiplication

• say we have two 64-bit integers \(x\), \(y\)

  • and want to 128-bit product, but our multiply instruction only does 64-bit products

• one way to multiply:

  ---

• divide \(x\), \(y\) into 32-bit parts: \(x=x_1\cdot2^{32}+x_0\) and \(y=y_1\cdot2^{32}+y_0\)

• then \(xy = x_1y_12^{64}+x_1y_0\cdot2^{32}+x_0y_1\cdot2^{32}+x_0y_0\)

  ---

• can extend this idea to arbitrarily large numbers

• number of smaller multiplies depends on size of numbers!

big integer timing attacks in practice (1)

• early versions of OpenSSL (TLS implementation) had timing attack

  • Brumley and Boneh, ‘‘Remote Timing Attacks are Practical’’ (Usenix Security ’03)

• attacker could figure out bits of private key from timing

  ---

• why? variable-time multiplication and modulus operations

  • got faster/slower depending on how input was related to private key
  • e.g. detect when attacker-value MOD private key part is close to 0

some webpage leakage (1)

• convenient feature 1: browser marks visited links

<script>
var the_color = window.getComputedStyle(
    document.querySelector('a[href=~"foo.com"]')
).color
if (the_color == ...) { ... }
</script>

• convenient feature 2: scripts can query current color of something

some webpage leakage (1)

• convenient feature 1: browser marks visited links

<script>
var the_color = window.getComputedStyle(
    document.querySelector('a[href=~"foo.com"]')
).color
if (the_color == ...) { ... }
</script>

• convenient feature 2: scripts can query current color of something
  • fix 1: getComputedStyle lies about the color
  • fix 2: limited styling options for visited links

some webpage leakage (2)

• one idea: script in webpage times loop that writes big array
• variation in timing depends on other things running on machine

side channels

• observing machine operation, indirectly

• common types of indirect observations

  • timing
  • power draw
  • electromagnetic radiation

• side channels can be security and privacy issues

  • often used to reveal internal/secret state

our focus

• we’re going to focus on timing-related leaks…
• and interaction with attacker input/processor design

  ---

• discovery: lots of cases where features that make code/processors fast
  let attackers learn much more than they should through timing

inferring cache accesses (1)

• suppose I time accesses to array of chars:

  • reading array[0]: 3 cycles
  • reading array[64]: 4 cycles
  • reading array[128]: 4 cycles
  • reading array[192]: 20 cycles
  • reading array[256]: 4 cycles
  • reading array[288]: 4 cycles
  • …

• what could cause this difference?

  • array[192] not in some cache, but others were

inferring cache accesses (2)

some psuedocode:

char array[CACHE_SIZE];
AccessAllOf(array);
*other_address += 1;
TimeAccessingArray();

• suppose during these accesses I discover that array[128] is slower to access
• probably because *other_address loaded into cache + evicted it
• what do we know about other_address? (select all that apply)

some complications

• caches often use physical, not virtual addresses

  • (and need to know about physical address to compare index bits)
  • (but can infer physical addresses with measurements/asking OS)
  • (often OS allocates contiguous physical addresses esp. w/‘large pages’)

• storing/processing timings evicts things in the cache

  • (but can compare timing with/without access of interest to check this)

• processor ‘‘pre-fetching’’ may load things into cache before access is timed

  • (but can arrange accesses to avoid triggering prefetcher
    and make sure to measure with memory barriers)

• some L3 caches use a simple hash function to select index instead of index bits

exercise: inferring cache accesses (1)

char *array;
array = AllocateAlignedPhysicalMemory(CACHE_SIZE);
LoadIntoCache(array, CACHE_SIZE);
if (mystery) {
    *pointer += 1;
}
if (TimeAccessTo(&array[index]) > THRESHOLD) {
    /* pointer accessed */
}

• suppose pointer is 0x1000188
• and cache (of interest) is direct-mapped, 32768 (\(2^{15}\)) byte, 64-byte blocks
• what array index should we check?

solution

array = AllocateAlignedPhysicalMemory(CACHE_SIZE);
LoadIntoCache(array, CACHE_SIZE);
if (mystery) { *pointer = 1; }
if (TimeAccessTo(&array[index]) > THRESHOLD) { /* pointer accessed */ }

• \(2^{15}\) byte direct mapped cache, \(64=2^{6}\) byte blocks
• 9 index bits, 6 offset bits
• 0x1000188: … 0000 0001 1000 1000

  ---

• array[0] starts at multiple of cache size — index 0, offset 0
• to get index 6, offset 0 array[0b1 1000 0000] = array[0x180]

aside

array = AllocateAlignedPhysicalMemory(CACHE_SIZE);
LoadIntoCache(array, CACHE_SIZE);
if (mystery) { *pointer += 1; }
if (TimeAccessTo(&array[index]) > THRESHOLD) {
    /* pointer accessed */
}

• will this detect when pointer accessed? yes
• will this detect if mystery is true? not quite
• … because branch prediction could started cache access

exercise: inferring cache accesses (2)

char *other_array = ...;
char *array;
array = AllocateAlignedPhysicalMemory(CACHE_SIZE);
LoadIntoCache(array, CACHE_SIZE);
other_array[mystery] += 1;
for (int i = 0; i < CACHE_SIZE; i += BLOCK_SIZE) {
    if (TimeAccessTo(&array[i]) > THRESHOLD) {
        /* found something interesting */
    }
}

• other_array at 0x200400, and interesting index is i=0x800, then what was mystery?

solution

array = AllocateAlignedPhysicalMemory(CACHE_SIZE);
LoadIntoCache(array, CACHE_SIZE);
other_array[mystery] += 1;
for (int i = 0; i < CACHE_SIZE; i += BLOCK_SIZE) {
    if (TimeAccessTo(&array[i]) > THRESHOLD) { ... }
}

• at i=0x800: … 0000 1000 0000 0000 (cache index = 0x20)
• other_array at 0x200400
• Q: 0x200400 + X has cache index 0x20?

exercise: inferring cache accesses (2)

char *array;
posix_memalign(&array, CACHE_SIZE, CACHE_SIZE);
LoadIntoCache(array, CACHE_SIZE);
if (mystery) {
    *pointer = 1;
}
if (TimeAccessTo(&array[index1]) > THRESHOLD ||
    TimeAccessTo(&array[index2]) > THRESHOLD) {
    /* pointer accessed */
}

• pointer is 0x1000188
• cache is 2-way, 32768 (\(2^{15}\)) byte, 64-byte blocks, ???? replacement
• what array indexes should we check?

reading a value — general pattern

char *array;
posix_memalign(&array, CACHE_SIZE, CACHE_SIZE);
AccessAllOf(array);
other_array[mystery] += 1;
for (int i = 0; i < CACHE_SIZE; i += BLOCK_SIZE) {
    if (CheckIfSlowToAccess(&array[i])) {
        ...
    }
}

• previous idea: learn bits of mystery that correspond to index bits

  • compute index bits of other_array + mystery as function of mystery
  • see how it matches index bits of array + i

reading a value — simpler case

char *array;
posix_memalign(&array, CACHE_SIZE, CACHE_SIZE);
AccessAllOf(array);
other_array[mystery * BLOCK_SIZE] += 1;
for (int i = 0; i < CACHE_SIZE; i += BLOCK_SIZE) {
    if (CheckIfSlowToAccess(&array[i])) {
        ...
    }
}

• if other_array is char* and starts at multiple of cache size
• and array[0x6*BLOCK_SIZE] is slow to access
• mystery == 0x6 + K * SET_COUNT

PRIME+PROBE

• name in literature: PRIME + PROBE
• PRIME: fill cache (or part of it) with values
• do thing that uses cache
• PROBE: access those values again and see if it’s slow

  ---

• (one of several ways to measure how cache is used)

  ---

• coined in attacks on AES encryption

example: AES (1)

• from Osvik, Shamir, and Tromer, ‘‘Cache Attacks and Countermeasures: the Case of AES’’ (2004)

• early AES implementation used lookup tables

• goal: detect index into lookup table

  • index depended on key + data being encrypted

• tricks they did to make this work

  • vary data being encrypted
  • subtract average time to look for what changes
  • lots of measurements

Meltdown fix

• HW: permissions check done with/before physical address lookup

  • was already done by AMD, ARM apparently?
  • now done by Intel

• SW: separate page tables for kernel and user space

  • don’t have sensitive kernel memory pointed to by page table
    when user-mode code running
  • unfortunate performance problem
  • exceptions start with code that switches page tables

mistraining branch predictor?

if (something) {
    CodeToRunSpeculatively()
}

• useful for attacks:
  have ‘something’ be false, but predicted as true

  ---

• one way: run lots of times with something true then do actually run with something false
• another way: learn how branch prediction caches work, run code that fills in caches in known way

reading a value — general pattern

char *array;
posix_memalign(&array, CACHE_SIZE, CACHE_SIZE);
AccessAllOf(array);
other_array[mystery] += 1;
for (int i = 0; i < CACHE_SIZE; i += BLOCK_SIZE) {
    if (CheckIfSlowToAccess(&array[i])) {
        ...
    }
}

• previous idea: learn bits of mystery that correspond to index bits

  • compute index bits of other_array + mystery as function of mystery
  • see how it matches index bits of array + i

reading a value — simpler case

char *array;
posix_memalign(&array, CACHE_SIZE, CACHE_SIZE);
AccessAllOf(array);
other_array[mystery * BLOCK_SIZE] += 1;
for (int i = 0; i < CACHE_SIZE; i += BLOCK_SIZE) {
    if (CheckIfSlowToAccess(&array[i])) {
        ...
    }
}

• if other_array is char* and starts at multiple of cache size
• and array[0x6*BLOCK_SIZE] is slow to access
• mystery == 0x6 + K * SET_COUNT

reading a value — * X * BLOCK_SIZE

char *array;
posix_memalign(&array, CACHE_SIZE, CACHE_SIZE);
AccessAllOf(array);
other_array[mystery * 8 * BLOCK_SIZE] += 1;
for (int i = 0; i < CACHE_SIZE; i += BLOCK_SIZE) {
    if (CheckIfSlowToAccess(&array[i])) {
        ...
    }
}

• if other_array is char* and starts at multiple of cache size
• and array[0x100*BLOCK_SIZE] is slow to access
• mystery * 8 = 0x100 + K * SET_COUNT
• mystery = 32 + K * SET_COUNT / 8

JavaScript

• JavaScript: scripts in webpages
• not supposed to be able to read arbitrary memory, but…
• can access arrays to examine caches
• and could take advantage of some browser function being vulnerable

  ---

• or — doesn’t even need browser to supply vulnerable code itself!

just-in-time compilation?

• for performance, compiled to machine code, run in browser
• not supposed to be access arbitrary browser memory
• example JavaScript code from paper:

if (index < simpleByteArray.length) {
    index = simpleByteArray[index | 0];
    index = (((index * 4096)|0) & (32*1024*1024-1))|0;
    localJunk ˆ= probeTable[index|0]|0;
}

• web page runs a lot to train branch predictor
• then does run with out-of-bounds index
• examines what’s evicted by probeTable access

supplying own attack code?

• JavaScript: could supply own attack code

  ---

• turns out also possible with kernel mode scenario
• trick: don’t need to actually run code for real
• … just need branch predictor to fetch it
  so it gets partially executed speculatively

other misprediction

• so far: talking about mispredicting direction of branch
• what about mispredicting target of branch in, e.g.:

// possibly from C code like:
//   (*function_pointer)();
jmp *%rax           

// possibly from C code like:
//      switch(rcx) { ... }
jmp *(%rax,%rcx,8)  

an idea for predicting indirect jumps

• Intel Haswell CPU did something similar to this

  • uses Hash(bits of last several jumps), not just last jmp

using mispredicted jump

• 1: find some kernel function with jmp *%rax

• 2: mistrain branch target predictor for it to jump to chosen code

  • use code at address that conflicts in ‘‘recent jumps cache’’
  • since only bottom bits are used, can set this up in user memory

• 3: have chosen code be attack code (e.g. array access)

  • either write special code OR
  • find suitable instructions (e.g. array access) in existing kernel code

• 4: run the kernel function

Spectre variants

• showed Spectre variant 1 (array bounds), 2 (indirect jump)

  • from original paper

• other possible variations:

  • could cause other things to be mispredicted

    • prediction of where functions return to?
    • values instead of which code is executed?

  • could use side-channel other than data cache changes

    • instruction cache
    • cache of pending stores not yet committed
    • contention for resources on multi-threaded CPU core
    • branch prediction changes
    • …

not just BLOCK_SIZE

char *array, *other_array;
// PRIME
posix_memalign(&array, CACHE_SIZE, CACHE_SIZE);
AccessAllOf(array);
// (some code we don't control)
other_array[mystery * N] += 1;  // previously: * BLOCK_SIZE
// PROBE
for (int i = 0; i < CACHE_SIZE; i += BLOCK_SIZE) {
    if (CheckIfSlowToAccess(&array[i])) {
    ...
    }
}

• 64KB (\(2^{16}\)B) direct-mapped cache with 64B blocks
• array[0x800] slow to access?
• other_array at 0x4000000 (index 0, offset 0)
• value of mystery if N = 1? N = 32 * 64?

solution (N=1)

\[\begin{align*} \left\lfloor\text{mystery} * N / \text{BLOCK_SIZE}\right\rfloor~\text{mod}~1024 & = & 32 \\ \left\lfloor\text{mystery} * N / \text{BLOCK_SIZE}\right\rfloor & = & 32 + 1024K \\ \end{align*}\]
let offset be some number in [0,BLOCK_SIZE):
\[\begin{align*} \text{mystery} * N & = & \text{BLOCK_SIZE}\times(32+1024Z) + \text{offset}\\ \text{mystery} & = & \text{BLOCK_SIZE}\times(32+1024Z) + N\times\text{offset} \\ \text{mystery} & = & 64\times(32+1024Z)+N\times\text{offset} \\ \end{align*}\]
N=1: mystery = \(2048\), \(2049\), \(2050\), …, \(2048+63\), \(64\cdot1024+2048\), \(64\cdot1024+2048+1\), …

exercise (N=32*64)

• what if N = 32*64
• recall: other_array[0] is set 0, offset 0
• other_array[mystery * N] is set 32
• possible values of mystery?

\[\begin{align*} \text{mystery}\cdot 32\cdot 64 & = & 64(32+1024Z) + \text{offset} \\ & = & 64\cdot32 + 65536Z + \text{offset}\\ \text{mystery} & = & 1 + \frac{65536}{64\cdot32}Z + \frac{\text{offset}}{64\cdot32} = 1+32Z \\ \end{align*}\]

alternate view

• learn index bits of mystery * N
• this example: bits 6–15

  ---

• N = 1, bits 6–15 of mystery
• N = 64, bits 0–9 of mystery
• N = 32*64 (\(2^{11}\)), bits 0–4 of mystery

exercise

char *array;
// PRIME
posix_memalign(&array, CACHE_SIZE, CACHE_SIZE);
AccessAllOf(array);
// (some code we don't control)
other_array[mystery * BLOCK_SIZE] += 1;
// PROBE
for (int i = 0; i < CACHE_SIZE; i += BLOCK_SIZE) {
    if (CheckIfSlowToAccess(&array[i])) {
    ...
    }
}

• 64KB (\(2^{16}\)B) direct-mapped cache with 64B blocks
• array[0x800] slow to access;
• other_array at 0x4000000
• value of mystery?

exercise solution (1)

• NUM_SETS = 64KB/64B = 1K (1024) sets

• array[0x800] has cache index 0x800/BLOCK_SIZE mod NUM_SETS

  • = cache index 32

• know other_array[mystery * BLOCK_SIZE] had same index

  ---

• other_array[0] at cache index 0

  • (0x4000000 / BLOCK_SIZE) mod NUM_SETS = 0

exercise solution (2)

• recall have found:

  • other_array[0] at index 0;
  • other_array[mystery*BLOCK_SIZE] has index 32 (same as array[0x800])

• other_array[X] at cache index (0 + X/BLOCK_SIZE mod NUM_SETS)

  • advanced by X/BLOCK_SIZE blocks
  • wrapping around after NUM_SETS blocks

• X = mystery * BLOCK_SIZE
• 32 = 0 + mystery mod NUM_SETS
• mystery = 32 or 32 \(\pm\) 1024 or 32 \(\pm\) 1024 \(\times\) 2 or etc.

variation: different starting location

• other_array starts at 0x4001440

• then other_array[0] at cache index

  • 0x4001440 / BLOCK_SIZE mod NUM_SETS = 0x51

• (0x51 + mystery * BLOCK_SIZE / BLOCK_SIZE) mod NUM_SETS = 32

• mystery = -49 or 975 or 1099 or …

variation: associative cache

char *array;
// PRIME
posix_memalign(&array, CACHE_SIZE, CACHE_SIZE);
AccessAllOf(array);
// (some code we don't control)
other_array[mystery * BLOCK_SIZE] += 1;
// PROBE
for (int i = 0; i < CACHE_SIZE; i += BLOCK_SIZE) {
    if (CheckIfSlowToAccess(&array[i])) { ...  }
}

• suppose 2-way 64KB cache instead of direct-mapped
• NUM_SETS = 64KB/2/64B = 512 sets
• array[0x800] still has cache index 32 (still)
• but now mystery can be \(32\) or \(32+512\) or \(32+512\cdot2\) or …

variation: associative cache (2)

char *array;
// PRIME
posix_memalign(&array, CACHE_SIZE, CACHE_SIZE);
AccessAllOf(array);
// (some code we don't control)
other_array[mystery * BLOCK_SIZE] += 1;
// PROBE
for (int i = 0; i < CACHE_SIZE; i += BLOCK_SIZE) {
    if (CheckIfSlowToAccess(&array[i])) { ...  }
}

• suppose 2-way 64KB cache w/ 64B and array[0x8800] is slow
• 0x8800/BLOCK_SIZE = 544 = 512 + 32
• since 512 sets total, still set index 32
• mystery still \(32\) or \(32+512\) or \(32+512\cdot2\) or …

exercise

• if 4-way 64KB cache w/64B blocks and something from cache set 32 evicted,
  then where could slow access be?

  • recall: 2-way cache: i=0x800, i=0x8800

• A. i=0x400, i=0x800, i=0x8400, i=0x8800

• B. i=0x800, i=0x8800, i=0x10800, i=0x18800

• C. i=0x800, i=0x4800, i=0x8800, i=0xc800

• D. i=0x800, i=0x4800, i=0x8800, i=0x10800

• E. something else

EVICT+RELOAD

• PRIME+PROBE: fill cache, detect eviction
• alternate idea EVICT+RELOAD:

unsigned char *probe_array;
posix_memalign(&probe_array, CACHE_SIZE, CACHE_SIZE);
access OTHER things to evict all of probe_array
if (something false) {
    read probe_array\[mystery * BLOCK_SIZE];
}
check which value from probe_array is faster

• requires code to access something you can access
• but often easier to setup/more reliable than PRIME+PROBE

into exploit: Meltdown

uint8_t* probe_array = new uint8_t[256 * 4096];
// ... Make sure probe_array is not cached
uint8_t kernel_memory_val = *(uint8_t*)(kernel_address);
uint64_t final_kernel_memory = kernel_memory_val * 4096;
uint8_t dummy = probe_array[final_kernel_memory];
// ... catch page fault
// ... in signal handler, determine which of 256 slots in probe_array is cached