Understanding "Magic" Pointers and Offsets

Статья
08/07/2007

With this blog post I try to explain how "magic" pointers and offsets work.

I just copied the term "magic" to refer to these kinds of pointers or offsets:

dd poi(0x129514 + 0x18) + 0x8 L2

du poi(0x0007de95)

du poi(poi(poi(0x129514 + 0x9c)) + 0x4)

dd poi(0x129514 + 0x34)

To use an analogy: it is similar to the "magic number" term we use for programs that access a value using a number instead of using a constant, like:

b = 3.14; // Magic number.

Instead of:

const float PI = 3.14;

b = PI;

I decided to blog about it because sometimes I run into people that look at these and wonder how these expressions can return valid data and how to create them.

Actually it's very simple!

Let me explain: The commands above display values or Unicode strings from specific locations.

These locations are selected based on pointers and offsets. These offsets usually come from structs (struct is the short term for structure. It’s a C/C++ reserved word) or classes. The structs or classes can be seen when using private symbols or source code access. To create them it's necessary to have private symbols or source code access because you need to see the class or struct to be able to create a command that displays information for a specific field without using any symbol.

Ok, at this point you may wonder why not use the variable names instead of using offsets.

The answer is because sometimes you just don't have private symbols.

Thus, when you use specific offsets from a specific address, you work around the lack of symbols.

This is great; therefore, I use this approach in some of my scripts. That's why it works for you using just public symbols or no symbols at all, like this one here. The downside is that your command or script depends on the specific offsets and pointers. In other words, if the internal class or struct changes, you might need to change your command, or you won't get the right information anymore.

This is funny: very often I see that inexperienced debuggers get impressed when they see a command using several layers of offsets.

Probably they don’t know the person who created the command memorized it because he uses it over and over. The person created it because he/she had access to the source code!

Yes, this is how we can create these kinds of commands using these magic pointers! :)

A lot of engineers at Microsoft, including myself, have a cheat sheet file where we save many "magic" pointers and offsets.

Some of the commands I have:

- Extracts PerfMon counters from mscorsvr/mscorwks.

- Extracts ASP page number being executed from an ASP call.

- Extracts ASP template from an ASP call.

- Extracts ASP source code from an ASP call.

- Extracts SQL command from OLEDB call.

- Extracts Connection string from OLEDB call.

- Extracts HRESULT from call stack.

- Extracts COM Object from OLE32 call.

- Extracts Remote Server IP Address from Winsock call.

We do that because we might forget the specific offsets and have a situation where we don't have private symbols access, for example, when working from home without being able to access the office computer or when working from a customer's site.

When we have this situation, we just open our cheat sheet file and use the command to get the information we want without needing to read the source code.

Whenever you see this kind of "magic" pointer or offset you should consider it might not work with newer product versions. Keep it in mind, and you won't be frustrated.

That said; let me exemplify how this “magic” pointer works and what might happen when the struct or class being used changes.

Source code for the application MagicPointer:

#include "stdafx.h"

#include <conio.h>

struct stData

{

TCHAR szName[21];

int nVersion;

TCHAR cLetter;

int nCode;

// int nCode2;

TCHAR szSomeSentence[51];

};

void DoSomething(stData* pst);

int _tmain(int argc, _TCHAR* argv[])

{

stData* pstStruct = new stData();

DoSomething(pstStruct);

_getch();

delete pstStruct;

return 0;

}

void DoSomething(stData* pst)

{

_tcscpy_s(pst->szName, L"Test");

pst->nVersion = 5;

pst->cLetter = L'D';

pst->nCode = 3;

_tcscpy_s(pst->szSomeSentence, L"Some Sentence");

}

I used Visual C++ 2005 and compiled the application above using a Debug build.

When debugging this application using symbols, I have the structure below after executing the last line from DoSomething():

Local var @ 0x17fe44 Type stData*

0x005f5600

+0x000 szName : [21] "Test"

+0x02c nVersion : 5

+0x030 cLetter : 0x44 'D'

+0x034 nCode : 3

+0x038 szSomeSentence : [51] "Some Sentence"

Since I have symbols and source code access, I can see the offsets above for each field.

Let’s suppose I want to dump the szSomeSentence field.

I can use:

0:000> du @@c++(pst->szSomeSentence)

005f5638 "Some Sentence"

Or:

0:000> ?? (wchar_t*) pst->szSomeSentence

wchar_t * 0x005f5638

"Some Sentence"

Cool, huh? I’m using the C++ sintax from Windbg.

I also know the address of the DoSomething() method:

004114f0 MagicPointer!DoSomething (struct stData *)

Now, let’s try the same process but without using symbols.

I reloaded the application and broke into the debugger when the _getch() was called.

As you can see below, the approaches above didn’t work because I cannot use the variable names anymore.

How can I overcome it? Once I know the offsets, since I can see the source code and private symbols, I can create a magic pointer.

First, I put a breakpoint when the function is about to return. At this point the data will be already assigned to the struct.

Then I dump the Unicode string, but this time using pointers and offsets, like:

bp 00411560 ß Break point when the method is about to end.

0:000> kvn

# ChildEBP RetAddr Args to Child

00 0017fe3c 00411478 008d5600 00000000 00000000 MagicPointer!DoSomething+0x70 (FPO: [Non-Fpo]) (CONV: cdecl) [c:\development\my tools\personal blog\article #17\magicpointer\magicpointer\magicpointer.cpp @ 40]

01 0017ff48 00411bd6 00000001 008d1188 008d1278 MagicPointer!wmain+0x88 (FPO: [Non-Fpo]) (CONV: cdecl) [c:\development\my tools\personal blog\article #17\magicpointer\magicpointer\magicpointer.cpp @ 24]

02 0017ff98 00411a1d 0017ffac 776119f1 7efde000 MagicPointer!__tmainCRTStartup+0x1a6

03 0017ffa0 776119f1 7efde000 0017ffec 77c2d109 MagicPointer!wmainCRTStartup+0xd

04 0017ffac 77c2d109 7efde000 00178231 00000000 kernel32!BaseThreadInitThunk+0xe

05 0017ffec 00000000 0041108c 7efde000 00000000 ntdll!_RtlUserThreadStart+0x23

008d5600 is our struct.

Then I use:

0:000> du 008d5600 + 0x38

008d5638 "Some Sentence"

Here it is!

Even without using symbols for the MagicPointer application I can get the information because I know the offsets.

Now, what happens when we change the struct? It might break our magic pointer.

Let’s see…

Remove the comment from the commented field nCode2 and rebuild the application.

Reload Windbg and, without using symbols for the MagicPointer application, use the breakpoint above and the same instructions.

bp 00411560

kvn

Then use:

0:000> kvn

# ChildEBP RetAddr Args to Child

WARNING: Stack unwind information not available. Following frames may be wrong.

00 0017fe3c 00411478 00365600 00000000 00000000 MagicPointer+0x11560

01 0017ff48 00411bd6 00000001 00361188 00361278 MagicPointer+0x11478

02 0017ff98 00411a1d 0017ffac 776119f1 7efde000 MagicPointer+0x11bd6

03 0017ffa0 776119f1 7efde000 0017ffec 77c2d109 MagicPointer+0x11a1d

04 0017ffac 77c2d109 7efde000 00177837 00000000 kernel32!BaseThreadInitThunk+0xe

05 0017ffec 00000000 0041108c 7efde000 00000000 ntdll!_RtlUserThreadStart+0x23

du 00365600 + 0x38

What happened?

Now we got garbage because the field szSomeSentence is not at the offset 0x38 anymore.

However, if you use symbols and the instructions I used before referring to variable names, it works without problems!

So we need to change our offset. Of course, as we have access to the source code and symbols, we can do that very easily and create an instruction that again dumps the szSomeSentence value when not using symbols. Actually we could get the same information by reverse engineering the application if you don’t have access to private symbols or source code. I’m not considering this approach here. We don’t need to do that if we have or had access to the source code.

Ok, so now we know there’s a new field. It’s an integer, so it uses a double word, 4 bytes on Win32.

What do we need to do? Shift our offset.

The new one is:

du <address> + 0x3c

Let’s try it…

It worked!

Of course, this is a very simple example. Usually we have classes that have properties that are pointers to structs and so on. The expressions become more complex although the principle is the same.

Actually, it sounds complex but it’s simple.

Comments

Anonymous
August 11, 2007
Men, you mean some people are trying to use WinDbg without any clue about structures and offsets ???
Anonymous
August 12, 2007
Yes, it happens because not all Windbg users have programming knowledge. It easy to copy and paste a magic pointer from a blog and use it, however, it requires some basic programming knowledge to understand what's going on under the hood, then everything makes sense.
Anonymous
September 11, 2007
The comment has been removed
Anonymous
September 11, 2007
Hi, I think I know how to help you here. If you want to get the address from static variables you may use: !DumpHeap -mt 0x02eb49d0 <-- MethodTable from m_bInited above If you use !DumpObject with the addresses you get you'll see they correspond to the right field. I'm not sure if you are familiar with this link, but it's helpful. It has all SOS commands: http://msdn2.microsoft.com/en-us/library/bb190764(vs.80).aspx Thanks
Anonymous
September 11, 2007
Hello, Roberto Thanks for answer... Unfortunately, I have tried many methods that getting addresses of static variables (methods specified in article http://msdn.microsoft.com/msdnmag/issues/05/05/JITCompiler/default.aspx). I use "dumpheap" when i want getting objects from the heaps, but static variable (ex. m_bInited) shouldn't be stored in the heaps. Below i placed the output of command "dumpheap": 0:000> !dumpheap -mt 0x02eb49d0

Heap 0 Address MT Size Gen total 0 objects

Heap 1 Address MT Size Gen total 0 objects

Heap 2 Address MT Size Gen total 0 objects

Heap 3 Address MT Size Gen total 0 objects

total 0 objects :-( but somehow "dumpclass" command takes static variables values in fact

Anonymous
September 12, 2007
Hello .rip, I have an idea. Let me show you a real situation. I'm debugging one of my tools: Netwiz. 2.0.50727.832 retail Workstation mode Stack: OS Thread Id: 0x14a4 (0) ESP EIP 0012f314 7c90eb94 [InlinedCallFrame: 0012f314] System.Windows.Forms.UnsafeNativeMethods.WaitMessage() 0012f310 7b087438 System.Windows.Forms.Application+ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(Int32, Int32, Int32) 0012f3b0 7b086e95 System.Windows.Forms.Application+ThreadContext.RunMessageLoopInner(Int32, System.Windows.Forms.ApplicationContext) 0012f41c 7b086cd3 System.Windows.Forms.Application+ThreadContext.RunMessageLoop(Int32, System.Windows.Forms.ApplicationContext) 0012f44c 7b0665fe System.Windows.Forms.Application.Run(System.Windows.Forms.Form) 0012f45c 024601f9 NetWiz.Program.Main() 0012f69c 79e7be1b [GCFrame: 0012f69c] I chose a method call. 0:004> !name2ee *!System.Windows.Forms.Application.Run Module: 790c2000 (mscorlib.dll)

Module: 01f72380 (sortkey.nlp)

Module: 01f72010 (sorttbls.nlp)

Module: 01f52c24 (NetWiz.exe)

Module: 7a71e000 (System.dll)

Module: 7b444000 (System.Windows.Forms.dll) Token: 0x060012ca MethodDesc: 7b5ab910 Name: System.Windows.Forms.Application.Run() JITTED Code Address: 7b0eed5c

Token: 0x060012cb MethodDesc: 7b4b36e0 Name: System.Windows.Forms.Application.Run(System.Windows.Forms.Form) JITTED Code Address: 7b0665d0

Token: 0x060012cc MethodDesc: 7b5ab918 Name: System.Windows.Forms.Application.Run(System.Windows.Forms.ApplicationContext) JITTED Code Address: 7b0eed94

Module: 7ae72000 (System.Drawing.dll)

Module: 648ec000 (System.Configuration.dll)

Module: 639ea000 (System.Xml.dll) I chose one of the MethodDesc above: 0:000> !dumpmd 7b4b36e0 Method Name: System.Windows.Forms.Application.Run(System.Windows.Forms.Form) Class: 7b46f078 MethodTable: 7b46f0dc mdToken: 060012cb Module: 7b444000 IsJitted: yes m_CodeOrIL: 7b0665d0 0:004> !DumpClass 7b46f078 Class Name: System.Windows.Forms.Application mdToken: 020001db (C:WINDOWSassemblyGAC_MSILSystem.Windows.Forms2.0.0.0__b77a5c561934e089System.Windows.Forms.dll) Parent Class: 790f8a18 Module: 7b444000 Method Table: 7b46f0dc Vtable Slots: 4 Total Method Slots: 5f Class Attributes: 100101 NumInstanceFields: 0 NumStaticFields: 12 <-- Great 0n12 Static Fields. MT Field Offset Type VT Attr Value Name 7a74da98 4001014 144 ....EventHandlerList 0 static 00000000 eventHandlers 790f9244 4001015 148 System.String 0 static 00000000 startupPath 790f9244 4001016 14c System.String 0 static 00000000 executablePath 790f8a7c 4001017 150 System.Object 0 static 00000000 appFileVersion 790ffe7c 4001018 154 System.Type 0 static 00000000 mainType 790f9244 4001019 158 System.String 0 static 00000000 companyName 790f9244 400101a 15c System.String 0 static 00000000 productName 790f9244 400101b 160 System.String 0 static 00000000 productVersion 790f9244 400101c 164 System.String 0 static 00000000 safeTopLevelCaptionSuffix 79103c00 400101d 964 System.Boolean 1 static 1 useVisualStyles 7b477e54 400101e 168 ...ms.FormCollection 0 static 0290a4c4 forms 790f8a7c 400101f 16c System.Object 0 static 028c2af4 internalSyncObject 79103c00 4001020 968 System.Boolean 1 static 0 useWaitCursor 79103c00 4001021 96c System.Boolean 1 static 0 useEverettThreadAffinity 79103c00 4001022 970 System.Boolean 1 static 0 checkedThreadAffinity 79103c00 4001023 974 System.Boolean 1 static 0 exiting 790f8a7c 4001024 170 System.Object 0 static 028c2b00 EVENT_APPLICATIONEXIT 790f8a7c 4001025 174 System.Object 0 static 028c2b0c EVENT_THREADEXIT You can see the class has 12 static fields and some of them are variables. Now I'm going to select the variable "exiting" that is System.Boolean: 79103c00 4001023 974 System.Boolean 1 static 0 exiting 0:004> !DumpHeap -mt 79103c00 Loading the heap objects into our cache. Address MT Size 028d264c 79103c00 12 2 System.Boolean 028d3478 79103c00 12 2 System.Boolean 028d34f8 79103c00 12 2 System.Boolean I think when you tried to setup the breakpoints the objects were never used yet. Let me prove my theory. I restarted NetWiz and instead of clicking Next a few times I just broke into the debugger after loading the application: 0:003> !DumpClass 7b46f078 Class Name: System.Windows.Forms.Application mdToken: 020001db (C:WINDOWSassemblyGAC_MSILSystem.Windows.Forms2.0.0.0__b77a5c561934e089System.Windows.Forms.dll) Parent Class: 790f8a18 Module: 7b444000 Method Table: 7b46f0dc Vtable Slots: 4 Total Method Slots: 5f Class Attributes: 100101 NumInstanceFields: 0 NumStaticFields: 12 MT Field Offset Type VT Attr Value Name 7a74da98 4001014 144 ....EventHandlerList 0 static 00000000 eventHandlers 790f9244 4001015 148 System.String 0 static 00000000 startupPath 790f9244 4001016 14c System.String 0 static 00000000 executablePath 790f8a7c 4001017 150 System.Object 0 static 00000000 appFileVersion 790ffe7c 4001018 154 System.Type 0 static 00000000 mainType 790f9244 4001019 158 System.String 0 static 00000000 companyName 790f9244 400101a 15c System.String 0 static 00000000 productName 790f9244 400101b 160 System.String 0 static 00000000 productVersion 790f9244 400101c 164 System.String 0 static 00000000 safeTopLevelCaptionSuffix 79103c00 400101d 964 System.Boolean 1 static 0 useVisualStyles 7b477e54 400101e 168 ...ms.FormCollection 0 static 00000000 forms 790f8a7c 400101f 16c System.Object 0 static 028ff260 internalSyncObject 79103c00 4001020 968 System.Boolean 1 static 0 useWaitCursor 79103c00 4001021 96c System.Boolean 1 static 0 useEverettThreadAffinity 79103c00 4001022 970 System.Boolean 1 static 0 checkedThreadAffinity 79103c00 4001023 974 System.Boolean 1 static 0 exiting 790f8a7c 4001024 170 System.Object 0 static 028ff26c EVENT_APPLICATIONEXIT 790f8a7c 4001025 174 System.Object 0 static 028ff278 EVENT_THREADEXIT And... 0:003> !DumpHeap -mt 79103c00 Loading the heap objects into our cache. Address MT Size Nothing on the managed heap! Every static variable is stored on the managed heap, regardless of whether it is declared within a reference type or a value type. They are like strings, always stored on the heap. But now I use the following breakpoint: ba r4 79103c00 <-- Stops on read or write. Then I press 'g' and pass control to the application. Just after that I get: ModLoad: 605d0000 605d9000 C:WINDOWSsystem32mslbui.dll ModLoad: 77120000 771ab000 C:WINDOWSsystem32OLEAUT32.DLL ModLoad: 64890000 6498a000 C:WINDOWSassemblyNativeImages_v2.0.50727_32System.Configurationc86724b99cb7eb7830f5c29bf2e5133eSystem.Configuration.ni.dll ModLoad: 637a0000 63d02000 C:WINDOWSassemblyNativeImages_v2.0.50727_32System.Xmld79992ddeba22e1133e9d956346ac52aSystem.Xml.ni.dll Breakpoint 0 hit eax=c377e1ff ebx=00000000 ecx=79103c00 edx=002b3418 esi=0012d598 edi=002b3418 eip=79e8575a esp=0012d50c ebp=0012d5b0 iopl=0 nv up ei pl zr na pe nc cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246 mscorwks!TypeHandle::GetManagedClassObject+0x15: 79e8575a 0f855e122800 jne mscorwks!TypeHandle::GetManagedClassObject+0x47 (7a1069be) [br=0] Conclusion: I think the only reason you don't see the static variables on the managed heap is because the method using it was not Jitted yet so there's no instance of the object. It's easy to prove that. You need to use a breakpoint after the moment the object was created then verify the heap and try your 'ba' breakpoint. Please, let me know if it helped. Thanks

Anonymous
March 03, 2008
I really like using C/C++ expressions from WinDbg. It’s a natural way to extract information from C and
Anonymous
March 03, 2008
I really like using C/C++ expressions from WinDbg. It’s a natural way to extract information from C and

Поделиться через