Creating a C++/Assembly Project

Step 1- Create Project

Add a cpp source file with a main() function. For this tutorial, we will create a main() function that will call an assembler function named clear(). clear() is of type void and requires no parameters and will be located in a separate file called clear.asm. Since it is in a separate file, clear() will need to be declared at the beginning of the file. Our main() function will look like this:

// Declaration of assembly function
// extern "C" instructs compiler to use C calling conventions
extern "C" void clear();

int main() {
clear();
return 1;
}

The phrase "extern "C"" in the declaration tells the compiler to use the C calling convention. This effects things like how parameters are passed to the routine and whether the calling function or the called function has responsibility for saving and restoring the registers. Most importantly for us, it prevents name-mangling. When C++ code is compiled, the names of functions are changed drastically to include extra information. This is commonly called name-mangling. The language C also changes the name, but it merely prepends an underscore to the front of the name. The "extern "C"" tells the compiler to do this simpler, C-style name change instead of the more complex, standard C++ name-mangling.
Leaving out the "extern "C"" will result in an error like:

asm_tut.obj : error LNK2001: unresolved external symbol "void __cdecl clear(void)" (?clear@@YAXXZ)

Step 2- Add Assembly Code

Add the file that contains your assembly source code to the project. If this hasn't been created yet, you can do this by selecting FileView in the Project Window, right-clicking on the project's name and selecting "Add files to project..." When the dialog box appears, type in the name you want the assembly code file to be saved as (in our case, clear.asm). VC++ will warn you that the file does not exist and ask if you want to create a reference to it in the project anyway. Select Yes. Expand the tree listing in the project window until you see the name of the assembly file (clear.asm). Double-click the file name. VC++ will ask if you want to create a new file with that name. Select Yes. A new file will be created and opened in the editor.

Enter you assembly code. For this tutorial, we will clear the EAX and EBX registers. To do this, we'll use this code:

.586              ;Target processor.  Use instructions for Pentium class machines
.MODEL FLAT, C    ;Use the flat memory model. Use C calling conventions
.STACK            ;Define a stack segment of 1KB (Not required for this example)
.DATA             ;Create a near data segment.  Local variables are declared after
                  ;this directive (Not required for this example)
.CODE             ;Indicates the start of a code segment.


clear PROC

xor eax, eax
xor ebx, ebx

ret

clear ENDP
END

Step 3- Set Custom Build Commands

We now provide the commands that VC++ will use to compile the assembly code. Visual C++ does not compile assembly source code. It must call an external program named ml.exe to perform the compilation. The command line must be added to Custom Build options of the Project Settings. To do this, right-click on the assembly filename (In the picture below, this is factorial.asm) in the Project window. Select "Properties..." from the pop-up menu.

Select the "Custom Build Step" folder. And update the a) Command Line and b) Outputs fields as shown in the screen shot below.

a) The Command Line field. This is the actual command which will be executed to build the file factorial.asm. It requires running ml.exe (the
assembler) on your input file. You need to be sure that the first part of the command actually _IS_ where you have downloaded the ml.exe file. (you don't need to add the .exe extension). The following will work if you had the file ml.exe in the directory C:\cs216\x86. If you have spaces in any of the directories in the path name then you may need quotes around "$(inputpath)" .

C:\cs216\x86\ml /c /Cx /coff $(inputpath)

b) The Outputs field. This tells what the name of the file created by the build step should be. In this case it will be the name of the input file
with an .obj file extension.

After setting these two fields, hit o.k. and you should be ready to build.
 

Select "OK" to close the dialog box.

Step 4- Compile and Link

Debugging a C++/Assembly Project

Step 1- Stop the Program

Step 2- Open the Disassembly Window

The Disassembly Window shows the underlying assembly instructions for C++ instructions. The actual C++ code we wrote is listed in black. The disassembled code is listed in grey after the C++ instruction. These are the actual assembly instructions which will be executed as the program is run. The yellow arrow, indicating the next instruction to be executed, is present in this window. The arrow, however, is not pointing to the C++ instruction clear(), but rather to the assembly instruction

00401038 call @ILT+0(_clear) (00401005)

The Disassembly Window allows stepping through code line-by-line just as the normal debugger window does; however, it traces the execution of assembly instructions instead of the higher C++ instructions.

Step 3- Display Registers and Memory

You can see the register values for EAX, EBX, ECX, EDX, ESI, EDI, ESP, and EBP, as well as some other registers and status flags present in the processor (these registers are from a Pentium Pro; registers in other processors may be different although the 8 listed will be present). You can right click in the register window to select which registers (FP, MMX, etc.) will be displayed.

To examine memory, select Debug->Windows->Registers from the Debug menu (these menu options probably won't appear until you have run the program).. It appears like this:

It provides a memory dump with the memory address on the left, the hexadecimal values of the memory contents on the right, and the ASCII representation of the hex values on the right. A particular memory location can be displayed by typing the address in the text box at the top of the window. The window shown above displays the beginning of our program. The first instruction is in memory location 0x00401020 and is 0x55. This is the hexadecimal encoding of 'push ebp'. The next six numbers on the line show subsequent memory locations. In this example, a total of seven memory locations is shown on each line in this window. Note that the window can be re-sized to change the number of bytes displayed per line. The final column is the ASCII characters for the memory locations.  This will usually be garbage unless you are viewing a memory region that has text stored in it. You can right click in the memory window and adjust how the contents of memory are grouped (by 1, 2, 4, or 8 bytes) and displayed (as signed or unsigned integers, floating point numbers, hex, etc.).

Step 4- Step through the Code

Pressing F11 once executes the call instruction. We're now at a jmp instruction that will jump us to beginning of the clear() function.

Pressing F11 again takes us to the first instruction of the actual clear() function. The Disassembly Window now looks like this:

Notice that the yellow arrow is pointing to the first of our two xor calls. The Registers window at this point is unchanged. Pressing the F11 key again executes the first xor statement, clearing the EAX register. The Registers window is now:

The EAX is now 0x0. Pressing F11 again clears the EBX register. Pressing F11 again returns from the clear() function and places us below our C++ command "return 1;". Since we've finished debugging the crucial part of our code, we can press F5 to Go and quickly finish the program.