Programming Forums - View Single Post - Self modifying code, pipelining and interrupts

mitakeet · Jun 16th, 2005, 6:11 PM

This is a follow up question to an old thread I happened to come across on DevShed: http://forums.devshed.com/showpost.p...9&postcount=18. In it, Scorpions4ever uses some tricks to detect if a program is running in a debugger:

Quote:

(pseudo-asm code)

(Toggle Plain Text)

MOV AX, 4Ch
MOV [someaddr], AX
MOV AH, 30h
INT 21
CMP AL, 5

Now looking at this code, the last three instructions appear to be getting the DOS version and checking if the major version # is 5 (Int 21h with AH set as 30h returns DOS version). The code's not so innocent after all though. The first two instructions move the value 4c into some specific memory location. In fact, the first two instructions change the value of the third instruction so it reads:
MOV AH, 4Ch
Now what does calling INT 21h with AH=4Ch do? This is the DOS function to terminate a program and return to the command interpreter.

Ok, so now we're getting cute. If the program merely terminates itself, what's the purpose of all this code, you ask? Here's where the true voodoo takes place. A Pentium chip pipelines its instructions to improve performance. So the first two instructions end up overwriting some area of memory, but the instruction that is occupying that area of memory is already loaded into the pipeline and hence it executes as:
MOV AH, 30h
whereas the memory where the instruction resided now reads:
MOV AH, 4Ch

So, the program gets the DOS version and merrily goes along its way, exactly as it appears to do.

Now comes the real tricky part -- if your cracker is running the program through a debugger, the debugger puts the CPU in single-step mode (i.e.) it does NOT pipeline the instructions. Hence, when the CPU gets to the third instruction and sees:
MOV AH, 4Ch
and then runs INT 21 and terminates the program!

I am reading a hardware book on pipelining and am in a section that deals with the logic of interrupts and it seems to me that if there is an interrupt during the execution of these instructions that the program will restart after pulling the changed instructions from memory and promptly terminate (just like what would be expected to happen if a debugger were running). I suppose this would be a low probability event, but certainly not a no probability event. Have I analyzed this correctly or am I still too new to hardware pipelining?