• malware definitions, types
• course logistics
• ethics considerations

References:

• taxonomy of malware: Szor chapter 2 (taxonomy of malware); NIST SP 800-83R1 section 2
• Team Cymru, “The Underground Economy: Priceless” (2006)
• Caballero et al. “Measuring Pay-per-Install: The Commoditization of Malware Distribution” (2011)
• Wegberg et al. “Plug and Prey? Measuring the Commoditization of Cybercrime via Online Anonymous Markets” (2018)
• Thomas et al. “Framing Dependencies Introduced by Underground Commoditization” (2015)
• Chatterjee et al. “The Spyware used in Intimate Partner Violence” (2018)

• definitive sources
• calling convention review
• floating point/SSE2
• addressing modes
• segmentation

References:

• definitive x86-64 ISA reference: via Intel, via AMD
• Linux x86-64 calling convention references:
  • x86-64 ABI that Linux uses (based on processor-generic System V ABI); section 3.2.3 is the calling convention

• x86 instruction encoding
• virus example: Vienna

References:

• Szor chapter 4

• virus example: Vienna (finish)
• executable formats
• viruses: hiding code in programs

References:

• Introduction to ELF
• ELF specification
  • x86-64 extensions (as used by Linux)
  • Linux extensions

• viruses: hiding code in programs (finish)
• viruses: invoking hidden code
  • and boot-loaders and secure boot
  • and dynamic linking

• (if needed) finish dynamic linking
• aside: on the cat-and-mouse game of malware detection
• integrity protection: tripwire, application whitelisting
• regular expressions and flex

(Szor reference: chapter 11.)

• regular expressions and flex (finish)
• behavior-based detection
• packers (start)

(Szor reference: chapter 11, chapter 7.)

• packers (finish)
• polymorphic/metamorphic malware
• anti-reverse engineering
  • emulation detection
• anti-goat

(Szor reference: chapter 7, chapter 6)

Further reference (1): Lakhotia et al, “Are Metamorphic Viruses Really Invincible?” (part 1 in VB Dec’04 and part 2 in VB Jan’05 and appendix).

• messing up debuggers
• (possibly) messing up disassembly
• “virtual machine”/emulator based obfuscation

• (briefly) root-kit style techniques

• buffer overflows (start)

(Szor reference: chapter 7)

Further reference: Nasi, “Bypass Antivirus Dynamic Analysis: Limitations of the AV model and how to exploit them”

• buffer overflows
• basic stack smashing
• finding stack addresses
• shellcode

Further reference:

• GDB cheat sheet

(Szor reference: chapter 10)

Further reference: Aleph1, “Stack smashing for fun and profit”, Phrack issue 48

• integer overflows
• simple mitigation: stack canaries
• information disclosure
• non-contiguous overflows
• format string exploits
• non-return address targets (start)
  • virtual tables

(Szor reference: chapter 10)

• non-return address targets (finish)
• pointer subterfuge
• guard pages

Further reference:

• Röttger, “Finding and exploiting ntpd vulnerabilities”
• Pincus and Baker, “Beyond Stack Smashing: Recent Advances in Exploiting Buffer Overflows”

• write XOR execute
• return-oriented programming

(Szor reference: chapter 13) Further reference: Roemer et al, “Return-Oriented Programming: Systems, Languages, and Applications” Further reference: Further reference: Payer, “Too much PIE is bad for performance”

The slides PDF contains some additional explanation slides for a few of the exercises (added after lecture) that aren’t in the recordings.

• OBFUSCATE retrospective
• ROP (finish)
  • jump-oriented programming
  • bootstrapping ROP chains
• ASLR, revisited (start)

• ROP review
• ASLR contineud
  • defeating ASLR example: CVE-2021-3156
• heap structure exploits
• use-after-free (start)

• use-after-free (finish)
• double-free
• C library functions and bounds checking

• C library functions and bounds checking (finish)
• adding bounds checking in C (with minimal code changes, hopefully?)

• bounds checking (finish)
• simple fuzzing
• symbolic execution

• symbolic execution (finish)
• greybox fuzzing
• static analysis (start)

• static analysis (finish)
• Taint tracking

• Rust as example

• other kinds of smart references/pointers
• other ideas for language-enforcement
• least privilege
• privilege separation as an application design

• unmodified application isolation
  • problem of enumerating system calls
• chroot
• SELinux — labels and mandatory access control rules
• Linux namespaces
• containers

• sandboxing without OS support
• OS X sandbox

• Qubes

• Android permissions and humans
• Android permission omissions

• notes

released 9PM 12 May; due 9PM 13 May