This schedule is tentative more than two weeks in advance. Assignments are due at 11:59PM on the day indicated on the schedule unless otherwise noted. Note that there may be some changes to assignments that are not yet released; we still link to their writeups in the hopes that they will be useful.

DateTopicAssignment
Week 1
Mon 16 Jan
(no class)
Wed 18 Jan

Overview / Malware Terminology

[ ]

(Audio is missing for the first ~15 minutes.)

Topics:

  • Malware definitions
  • Some Malware history
  • Course logistics
  • Ethics pledge
ethics pledge released
VM released
Week 2
Mon 23 Jan

Virtual Machines

[ ]

Topics:

  • Process/System VMs
  • Utility for malware analysis
  • VM escape/detection
Wed 25 Jan

Topics:

  • VM (review)
  • definitive ISA reference: via Intel, via AMD
  • calling convention review
  • floating point/SSE2
  • addressing modes
  • segmentation (incomplete)
Fri 27 Jan
VM due
RE released
Week 3
Mon 30 Jan

Topics:

ethics pledge due
Wed 01 Feb

x86 asm 3 / Viruses 1

[ ]

Topics:

  • x86 instruction encoding (finish)
  • virus example: Vienna (start)
Fri 03 Feb
RE due
ASM released
Week 4
Mon 06 Feb

Topics:

  • virus example: Vienna (finish)
  • viruses: hiding code in programs
  • viruses: invoking hidden code (start)

(Szor reference: chapter 4.)

Wed 08 Feb

Viruses 3 / Signature-based detection

[ ]

Topics:

  • viruses: invoking hidden code (finish)
  • regular expressions and lex
  • on anti-virus effectiveness

(Szor reference: chapter 11.)

Further reference: Hanno Böck’s talk “In Search of Evidence-Based IT-Security

Fri 10 Feb
ASM due
TRICKY released
Week 5
Mon 13 Feb

Beyond signatures / Anti-anti-virus (1)

[ ]

Topics:

  • heuristic detection
  • behavior-based detection
  • polymorphic malware
  • packers and packer detection (start)

(Szor reference: chapter 11, chapter 7.)

Wed 15 Feb

Anti-anti-virus (2)

[ ]
  • packers and packer detection (finish)
  • metamorphic case studies
  • anti-reverse engineering
    • anti-disassembly
    • anti-emulation
    • anti-goat
    • anti-debugging (start)

(Szor reference: chapter 7, chapter 6)

Further reference (1): Lakhotia et al, “Are Metamorphic Viruses Really Invincible?” (part 1 in VB Dec’04 and part 2 in VB Jan’05 and appendix).

Fri 17 Feb
TRICKY due
LEX released
Week 6
Mon 20 Feb

Anti-anti-virus

[ ]

Topics:

  • anti-debugging (finish)
  • retroviruses / tunneling / stealth
  • Nasi article

(Szor reference: chapter 7)

Further reference: Nasi, “Bypass Antivirus Dynamic Analysis: Limitations of the AV model and how to exploit them

Wed 22 Feb

Stack Smashing

[ ]

Topics:

  • definitions: vulnerabilities, exploits
  • buffer overflows
  • basic stack smashing

(Szor reference: chapter 10)

Further reference: Aleph1, “Stack smashing for fun and profit”, Phrack issue 48

Fri 24 Feb
LEX due
Week 7
Mon 27 Feb

notes; drawing 1 2 3; RE example: C file; objdump

Wed 01 Mar

Midterm 1

Fri 03 Mar
OVER released
Week 8
Mon 06 Mar
(no class)
Wed 08 Mar
(no class)
Week 9
Mon 13 Mar

More Buffer Overflows, Part 1

[ ]

Topics:

  • stack smashing review
  • not-so-great GDB demo
  • stack canaries
  • pointer subterfuge (start)

Further reference:

(Szor reference: chapter 10)

Wed 15 Mar

More Buffer Overflows, Part 2

[ ]

Topics:

  • pointer subterfuge
  • non-return address targets
  • frame pointer overwrite
  • heap structure exploits (start)
Fri 17 Mar
Week 10
Mon 20 Mar

More Buffer Overflows, Part 3

[ ]

Topics:

  • heap structure exploits
  • double-free
  • use-after-free
  • format string exploits (start)

(Szor reference: chapter 13)

Wed 22 Mar

Buffer Overflow Mitigations

[ ]

Topics:

  • format string exploits (finish)
  • mitigations continued
  • stack canary review
  • shadow stacks
  • ASLR (start)

Format string example archive, directory.

Fri 24 Mar
OVER due
FORMAT released
Week 11
Mon 27 Mar

Mitigations / Return-Oriented Programming

[ ]

Topics:

  • ASLR (finish)
  • NX
  • return-oriented programming

Further reference: Payer, “Too much PIE is bad for performance

Further reference: Roemer et al, “Return-Oriented Programming: Systems, Languages, and Applications

Wed 29 Mar

Topics:

  • ROP (finish)
  • C library functions and bounds checking
  • adding bounds checking in C
Fri 31 Mar
Sat 01 Apr
FORMAT due (extended)
Week 12
Mon 03 Apr

notes

Wed 05 Apr

Midterm 2

Fri 07 Apr
ROP released
Week 13
Mon 10 Apr

Topics:

  • Memory error detectors
  • Fuzzing, black and white box
Wed 12 Apr

Safe systems languages

[ ]

Topics:

  • Static analysis
  • Rust as a case study
Fri 14 Apr
ROP due
FUZZ released
Week 14
Mon 17 Apr

Command injection

[ ]

Topics:

  • SQL injection
  • Command injection
  • Taint tracking

Further Reference: OWASP page on Command Injection

Wed 19 Apr

Web Security: Overview / XSS

[ ]

Topics:

  • Shellshock command injection example
  • Cookies and Referrers
  • Cross-Site Scripting (XSS)

Further Reference: OWASP page on Cross-Site Scripting

Fri 21 Apr
FUZZ due
Week 15
Mon 24 Apr

Web Security: SOP / CSRF

[ ]

Topics:

  • Same Origin Policy
  • Cross-Site Request Forgery
  • Clickjacking

Further Rererence: Cross-Site Request Forgeries: Exploitation and Prevention

Wed 26 Apr

Sandboxing / Privilege Separation

[ ]

Topics:

  • User Tracking
  • Sandboxing and Privilege Separation

Further Reference: Cookieless Monster: Exploring the Ecosystem of Web-Based Device Fingerprinting

Further Reference: The Security Architecture of the Chromium Browser

Fri 28 Apr
Week 16
Mon 01 May

notes

CHALLENGE released
Wed 03 May
(no class)
Week 17
Thu 11 May

Final exam

2PM–3:30PM, lecture room

CHALLENGE due