exercise: throughput/latency (1)

• suppose cycle time is 500 ps
• exercise: latency of one instruction?

  A. 100 ps	B. 500 ps	C. 2000 ps	D. 2500 ps	E. something else

• exercise: throughput overall?

  A. 1 instr/100 ps	B. 1 instr/500 ps	C. 1 instr/2000ps	D. 1 instr/2500 ps
  E. something else

exercise: throughput/latency (2)

• double number of pipeline stages (to 10) + decrease cycle time from 500 ps to 250 ps — throughput?

  A. 1 instr/100 ps	B. 1 instr/250 ps	C. 1 instr/1000ps	D. 1 instr/5000 ps
  E. something else

diminishing returns: uneven split

• Can we split up some logic (e.g. adder) arbitrarily?
• Probably not…

a data hazard

addq %r8, %r9   // R8 + R9 -> R9
addq %r9, %r8   // R9 + R8 -> R9
addq ...
addq ...

data hazard compiler solution

addq %r8, %r9
nop
nop
addq %r9, %r8

• one solution: change the ISA

  • all addqs take effect three instructions later
    (assuming can read register value while it is being written back)

• make it compiler’s job

• problem: recompile everytime processor changes?

data hazard hardware solution

addq %r8, %r9
// hardware inserts: nop
// hardware inserts: nop
addq %r9, %r8

• how about hardware add nops?

• called stalling

• extra logic:

  • sometimes don’t change PC
  • sometimes put do-nothing values in pipeline registers

making guesses

        cmpq %r8, %r9
        jne LABEL
        xorq %r10, %r11
        movq %r11, 0(%r12)
        ...

LABEL:  addq %r8, %r9
        imul %r13, %r14
        ...

• speculate (guess): jne won’t go to LABEL
• right: 2 cycles faster!; wrong: undo guess before too late

opportunity

// initially %r8 = 800,
//           %r9 = 900, etc.
0x0: addq %r8, %r9 
0x2: addq %r9, %r8
...

opportunity 2

// initially %r8 = 800,
//           %r9 = 900, etc.
0x0: addq %r8, %r9 
0x2: nop
0x3: addq %r9, %r8
...

exercise: forwarding paths

• in subq, %r8 is _____ addq.

• in xorq, %r9 is _____ addq.

• in andq, %r9 is _____ addq.

• in andq, %r9 is _____ xorq.

  • assume regfile cannot read value while being written
  • A: not forwarded from
  • B-D: forwarded to decode from {execute,memory,writeback} stage of

unsolved problem

• value from memory stage too late for forwarding alone

• combine stalling and forwarding to resolve hazard

• assumption in diagram: hazard detected in subq’s decode stage

  • (since easier than detecting it in fetch stage)

why can’t we…

exercise

• suppose 5-stage pipeline
  • using forwarding, stalling like we discussed
• 1 ns cycle time
• 5% are branches
  • 60% correct predictions
• 1% are uses of value just after loading it from memory
• assume negligible cache misses
• estimated throughput?

exercise solution

• 97% 1 cycle (until next useful instruction fetched)
• 40% of 5% = 2% 3 cycles
  • 2 “wasted” fetches for misprediction
• 1% 2 cycles
  • 1 “wasted” fetch for data hazard stall
• average cycles/instruction = 1$\(97% + 3\)\(2% + 1\)$1% = 1.05
• 1 billion instructions / second \(\div\) 1.05 cycles/instruction = 0.952 billion instr/sec

hazards versus dependencies

• dependency — X needs result of instruction Y?

  • has potential for being messed up by pipeline
  • (since part of X may run before Y finishes)

• hazard — will it not work in some pipeline?

  • before extra work is done to ‘‘resolve’’ hazards
  • multiple kinds: so far, data hazards, control hazards

pipeline with different hazards

• example: 4-stage pipeline: fetch/decode/execute+memory/writeback

                   // 4 stage  // 5 stage
  addq %rax, %r8   //          // W
  subq %rax, %r9   // W        // M
  xorq %rax, %r10  // EM       // E
  andq %r8,  %r11  // D        // D
  

• (assuming register file does not read while writing)
• addq/andq is hazard with 5-stage pipeline
• addq/andq is not a hazard with 4-stage pipeline
• more hazards with more pipeline stages

exercise: different pipeline

• split execute into two stages: F/D/E1/E2/M/W
• result only available near end of second execute stage
• where does forwarding, stalls occur?

exercise: different pipeline

• split execute into two stages: F/D/E1/E2/M/W

exercise: forwarding paths (2)

• in subq, %r8 is _____ addq.

• in subq, %r9 is _____ addq.

• in andq, %r9 is _____ subq.

• in andq, %r9 is _____ addq.

  • A: not forwarded from
  • B-D: forwarded to decode from {execute,memory,writeback} stage of

exercise: predict+forward (1)

• if jle is correctly predicted:

  • in andq, %r9 is ______ addq.

  • in andq, %r8 is ______ subq.

  • A: not forwarded from [assume read while writing requires forwarding]

  • B-D: forwarded to decode from {execute,memory,writeback} stage of

exercise: predict+forward (2)

• if jle is mispredicted + resolved after jle’s execute:

  • in andq, %r9 is _____ addq.

  • in andq, %r9 is ___ subq.

  • A: not forwarded from [assume read while writing requires forwarding]

  • B-D: forwarded to decode from {execute,memory,writeback} stage of