Assembly

Behind the wizard's curtain

A talk by Ciro Durán (@chiguire)
<ciro.duran@gmail.com>
December 2015

Press 's' for the speaker notes

Hello!

My name is Ciro Durán, and during the next hour we will be talking about assembly.

In this talk we will do an introduction to the assembly language. Assembly language is one of the fundamental building blocks of software, and yet it is often overlooked as compilers and interpreters usually "just work", and there's no need to look deeper than stepping on the debugger of the managed language you're using.

Yet, when there is nowhere else to go, or you have lost your symbol files, the disassembly is not a horrible place, nor is assembly the language of the devil. It's rather a tool with very simple, but very fast parts, which build upon themselves to offer the modern civil life you're enjoying.

After this talk, you'll no longer see assembly as the place where Matrix symbols go to mingle, but rather you'll see assembly more like a service tunnel you'll probably have to go down from time to time. What was probably black magic and "it's all turtles all the way down", will become simply a series of simple instructions that are quickly executed.

Let's honour the fallen from the Ringworld and prevent another disaster like this

Order of the day

Processors' generalities
6502 processor assembly
x86 processor assembly
The gap between assembly and high-level languages

What's a processor?

What's so scary?

Maybe this frontier is what usually scares us. It is a reminder that all our perfect software world is held up by sometimes imperfect electronics and pesky physics. Therefore, we have high-level languages and their safe and proven lands. High-level languages are abstractions, and abstractions are VERY useful for real-world production. However, ALL ABSTRACTIONS ARE LEAKY (Spolsky dixit), and high-level languages are no exception. There might be some ambiguities in your language design, most of all in a language so complex and that has taken so long to standardise such as C++. This results in some subtle differences between compilers, or the platforms you're running programs on. If you run into these moments, being aware of what's happening underneath will prove valuable in your debugging endeavours.

Where do I start to learn?

For the moment, the important part for you is that a processor offers a number of operations, followed by some parameters. This is all part of the processor design, and once it's released it's quite difficult to change, so this has the advantage that once you learn all of this your knowledge won't become obsolete too soon!

The way to learn assembly, rather than watching it, is to start writing things with it. Most of the tools for this are fortunately free and widely available in the internet. For this talk, we will aim to learn from a simple abstracted platform, with some memory available, some memory-mapped video, and a couple of useful interfaces. We will learn how to program the 6502, and then we'll apply some similarities to the x86 architecture.

Today's agenda

Processors' generalities
6502 processor assembly
x86 processor assembly
The gap between assembly and high-level languages

The 6502 processor

Your weapons

3 registers: X, Y, and A (accumulator)
Stack Pointer (SP)
Program Counter (PC)
Processor Flags (Zero, Carry, Decimal mode, among others)
Did I mention that, except for the PC, this is all 8-bit?

When you're writing assembly, you need to know what you can count with. You have 3 registers, X, Y and A (often called accumulator). Each register counts with different operations, and you'll see very soon how they differ.

You also have a Stack Pointer, this one is initialised at $ff and gets decremented each time you push a value. There are a couple of instructions that make use of the stack and are quite helpful.

The Program Counter is the only that's 16-bit. It holds an address in memory, and it gets increased as the processor reads instructions. The actual value depends on the instruction that it reads. Some instructions are one byte long, others two and three bytes.

The Processor Flags are contained in a byte, so each flag is a bit that is turned on or off. You have 8 of these flags. Two useful flags are the Zero flag, which is turned on when the last result is zero, and the Carry flag, which affects all additions and substractions.

Please notice that, except for the PC, all values here are of one byte, this means a value can be from $00 to $ff. Think about that, all of you that have benefited for years of having 32-bit values.

Let's now see the processor instructions, but we're not listing them. We can actually start learning how they work by using them directly.

First Program in 6502

A game for the 6502

Load, Store, Memory and Numbers

Addressing Memory

We can address memory in several ways. The first one is directly specifying it, as in $0200.

We can also specify memory with one byte, e.g. $c6. This is the zero page mode, and it corresponds to the address $00c6. It's quite useful as the CPU takes less cycles to read it, so use it as much as you can.

There's also indirect memory, in which we specify a memory address. The CPU will read the first memory address and the next one and that will become a memory address. In the 6502, the first memory address tells the least significant bits of the address, while the next one tells the most significant ones. For us humans this means that we specify the last two digits in the first byte, and the first two digits in the second byte ($0267 would be stored in two bytes as $67 in the first one, and $02 in the second one).

Only the jumping instruction supports simple indirection. For storing the X, Y and A registers we've got another mode, the indexed indirection. In this case we will specify a memory address from a value in memory, but we will use either the X or Y register to add to that value.

Adding and substracting

Branching

Now you're thinking with p̶o̶r̶t̶a̶l̶s̶ assembly

Today's Agenda

Processors' generalities
6502 processor assembly
x86 processor assembly
The gap between assembly and high-level languages

x86 processor assembly

Your weapons

Eight 32-bit registers, 6 for general purpose (yay!), and 2 with special purpose
- EAX, EBX, ECX, EDX, ESI, EDI
- ESP (Stack pointer)
- EBP (Base pointer)
EFLAGS register (32 bits for processor state and result operations)
EIP - Instruction Pointer
64-bit adds 8 more registers, and the E is exchanged by an R.
The CPU has 2 modes of running:
- Real mode, Protected mode.

EAX, EBX, ECX and EDX have an inner 16-bit subsection called correspondingly AX, BX, CX and DX. Each one has a subsection corresponding to the most significant 8-bits (AH), and the least significant 8-bits (AL).

The ESP and EBP register still retain their special meaning. The ESP is a Stack Pointer, in a similar way to the one in the 6502, and the EBP is the Base Pointer, an idea we'll explore in just a bit.

The EFLAGS register has bits with similar functionality as the 6502, e.g. the Carry bit! We should make sure it behaves the same way, anyway, by writing a small program that confirms our suspicions.

When the computer is booted it starts as a 16-bit CPU and the Boot loader/OS changes the mode to switch to 32-bit.

Addressing Memory

We'll limit ourselves to flat addressing.

Let's not forget about segments, though (CS, DS, ES, SS)

Static data declarations
- Precede with .DATA
- DB, DW, DD (1, 2 or 4 byte size)
- [name] [size] [initial value] (e.g. foo DD 32567)
- We can declare some primitive arrays with DUP

Addressing Memory

x86 CPUs addresses are 32-bits wide (e.g. 0xdeadbeef)
mov eax, [ebx] → gets value from address specified in ebx
mov [var], ebx → stores register value in address specified in constant
mov eax, [esi-4] → gets value from address specified in ebx ± 4 bytes
mov [esi+eax], foo → stores value in memory to address at esi + eax
mov edx, [esi+4*ebx] → gets value from address specified in in esi + 4 * ebx

Assembly instructions

Data movement instructions

mov, push, pop, lea

Arithmetic/logic

add, sub, inc, dec
imul, idiv
and, or, xor, not, neg
shl, shr

Control flow

jmp
je, jne, jz, jg, jge, jl, jle, jo
cmp
call, ret

These are some common instructions. Some of these were available in the 6502. You should study their behaviour anyway, and notice any similarities and differences.

mov - same as 6502, direct memory-to-memory moves are not allowed.
push/pop - push and pop values from the stack
lea - load an address from memory, does not load contents of memory address.
add/sub/inc/dec - take a guess
imul/idiv - integer multiplication and division
and/or/xor/not/neg - bitwise operations.
shl/shr - shift bits left and right. 6502 has both.
jmp - jump to a certain address
je/jne/jz/jg/jge/jl/jle/jo - a much more varied way to branch conditionally
cmp - compare, very similar to the one in the 6502
call/ret - call and return, we'll see these in a minute

Calling Conventions

cdecl, stdcall.

You've seen call/ret instructions. It'd be wonderful you could call subroutines and pass them parameters, this way you could reuse code! (heh)

Well, there are conventions in x86 assembly that allow you to interact with code written by someone else (such as DLLs). Two of the most common ones are the C calling convention (cdecl), or stdcall (Microsoft specific, and used to interact with the Windows API). The convention states how the calling code must pass the parameters, and how the called code must read the parameters.

In most cases, conventions use the x86 stack, basing themselves in push, pop, call and ret. The subroutine parameters are passed on the stack, but the code must also save the current subroutine state, as the registers and local variables.

The code helps itself with the ESP and EBP to point towards the current function's variables and parameters. The Stack Pointer points towards the last value in the stack, and the Base Pointer points towards the start of the local variables of the current function. Note that you could ignore using the Base Pointer to do this job, and use it as an extra register for general purposes; this optimization is so common that is has its own name: the Frame Pointer Optimization.

What about interrupts?

Interrupts are the way to signal the CPU about something. The interrupt stops the CPU from what is doing and starts doing something else.

You have hardware interrupts (or Interrupt ReQuests, IRQ), triggered by hardware devices, software interrupts, that transfer control to the operating system kernel, and exceptions. If you try to do an illegal operation on the processor (such as dividing by zero, or a General Protection Fault), the process will detect this problem, and transfer control to a handler to manage the exception.

Since we are talking here about assembly that interacts with an operative system, you will probably not have too much interaction with them, as operative systems try to isolate you from interrupts.

It's worth to know they're there and that you should probably study them anyway.

Let's see x86 assembly in action

We won't have time to fully explore x86 assembly. Remember, this talk is to convince you that it's a good idea to learn it! But in order to learn it, there's no outher route than start writing programs in it. In this case, I'm going to show you a simple and free tool to write assembly for Microsoft Windows.

You might think there's a lot of boilerplate and hand-helding going on here; you'd probably like to write bare metal hardware-interacting assembly because you're rad. But in this modern world you'll learn more easily if you ignore certain things for the moment, and interact with several OS functionalities. Unless you want to write your own boot loader or driver, which is something very cool too! But we'll stick to application programs for the moment.

Here's Microsoft Assembler, or MASM32. Apart from all the basic instructions I mentioned earlier, it also has "Macros". These are words that are substituted by assembly code. This is one level below actual functions, and most of the code in the MASM library is written this way, so you can easily do things while you learn the ropes and go deeper in your assembly understanding.

In this case, we'll create a boilerplate project, compile it, link it and run it. Study the code, and start writing something!

Here's Microsoft Assembler, MASM32

We can create a project automatically

It generates some boilerplate code

With a convenient batch file that assembles and links the code

Here's the build output

And here's the code in action

Today's Agenda

Processors' generalities
6502 processor assembly
x86 processor assembly
The gap between assembly and high-level languages

The gap between assembly and high-level languages

The disassembly

You will ask yourselves how does your debugger (i.e. Visual Studio) know how your code and the assembly instructions relate? Well, that's what symbols are for. The symbols are produced each time you compile your code, and they are tightly related to the state of your code by the time you compile.

The symbols are stored in a format called PDB, rather undocumented. All the debuggers will look for these, and you can set where to look for them. Microsoft has software that allows you to upload and store the symbols so your programmers can use. You can even set it so you can have private symbols for your programmers, and public symbols for your clients, if you make software intended to be used by others programmers.

As your project grows, and the amount of programmers increase, you should work out on having an easy way to store and distribute the symbols of your builds. You will also need good debugging tools to see where your program is standing at. Visual Studio has a good debugging environment, but there is also windbg, which is free.

I did not mention assembly for *NIX-like systems, but if you're interested in that, there are also quite mature tools for it that you can use, such as the venerable gdb.

The combination of the symbols and disassembly might assist you when debugging.

Let's compile a simple Hello World console application

Let's set a breakpoint, run it, and go to disassembly

Woah, that's a lot of instructions, what's 945858h?

Turns out we can watch it and see it's the pointer to the string

Closing Remarks

“Although, some people can program in assembly language and understand the intricacy of the spacecraft [Voyager programme], most younger people can't or really don't want to.” - Suzanne Dodd (NASA JPL)

You can also practice here http://challenges.re/

We're reaching the end of this talk. I cannot emphasize enough the importance of learning assembly. Not just for fun, as shown in this talk, but also to dispel any magic that we may consider processors have. Processors are machines, upon which we have built a complex layer of software, which in turn might or might not be easy to understand. Assembly is part of the key to understand it.

Writing assembly is an essential step for understanding it. But it is just as essential to be able to read assembly written by others. Reverse engineering, like writing assembly, is some kind of lost art due to not being something that is easily found or widely promoted. Here are some exercises that might help you to develop your assembly reading and understanding skills.

Thanks for being here!

Your questions are welcome!

http://ciroduran.com
@chiguire
ciro.duran@gmail.com

Browse this talk (press 's' for notes) at
http://ciroduran.com/talks/assembly