Asynchronous COM for Windows Vista and Win7 – memory overwrite bug

COM technology (Component Object Model) has been developing for almost twenty years and work of the vast majority of system components is still based on it. I think that this will continue in the nearest future. Asynchronous COM is an option of using this technology. Some applications of the early 2000s used it to build heavy-loaded servers.

Things are not so rosy for Asynchronous COM. Asynchronous COM was presented by Microsoft along with Windows 2000 release, and was intended to be a solution to one of the problems when using the classic COM – client freezing while waiting for a response from a server. The latter may either be processing the request for too long or be not operational at all, which can result in the customer’ hanging.

Everything was just great until the moment of Windows Vista release. Then the problems started ..

Some kind of strange behavior of applications using asynchronous COM became visually observable. Crashes were appearing here and there in a totally unpredictable way. Sometimes an application just would not start; sometimes it could cause an illegal operation, etc. Only long and painful debugger work resulted in detecting that OLE32.dll was the problem. It was overwriting the memory.

A possible solution

Here is a function code segment:

CAsyncUnknownMgr::IBegin_QueryMultipleInterfaces

      xor   edi,edi
      ...
      lea   ecx,[ebp-8]
      push  ecx
      push  edi
      push  0Ch
      xor   eax,eax
      push  eax
      push  ebx
      call  __allmul
      push  edx
      push  eax
      call  ULongLongToUInt
      cmp   eax,edi
      push  eax
      jge   @f
      ...
@@:   push  edi
      push  dword ptr [g_hHeap]
      call  dword ptr [pfnHeapAlloc]

If you write this is C, the code segment will look like this:

      int cb;
      void *p;
      UINT nBlockSize;
      ULONGLONG ullVar;
      ...
      cb = ULongLongToUInt(12 * ullVar, &nBlockSize);
      p = pfnHeapAlloc(hHeap, 0, cb);

Let me remind you of HeapAlloc и ULongLongToUInt functions prototypes:

      HRESULT ULongLongToUInt(ULONGLONG ullOperand, UINT *puiResult);
      LPVOID  HeapAlloc(HANDLE hHeap, DWORD dwFlags, DWORD dwBytes);

Thus, CAsyncUnknownMgr::IBegin_QueryMultipleInterfaces transmits a value, which is returned from ULongLongToUInt, as a block size into HeapAlloc. And since ULongLongToUInt returns HRESULT and the status that is being returned is always S_OK, a zero size block allocation is always requested (S_OK == 0). Then the CAsyncUnknownMgr::IBegin_QueryMultipleInterfaces code copies nBlockSize data bytes into this block, which leads to memory overwriting and to application collapse in the nearest future as a result.

So what does the fact that this problem had not only been missed during the testing procedures of Windows Vista, but also wasn’t fixed in Windows 7, tell us? Probably that Microsoft itself almost does not use the asynchronous COM. Or it does not use it in a simplified mode (please see the next paragraph) without asynchronous QueryInterface. My guess is  that the last assumption might be true.

Conclusion: asynchronous COM is dying

As I have already mentioned, applications use the asynchronous COM in order to get rid of freezing during a server call. It’s not only a COM method call that might be hanged, but also an attempt to request an interface from the server might result in this. In other words, we are talking about using AsyncIUnknown::QueryInterface. This is the source of the problem. It’s interesting that a bug with memory overwriting appeared in Windows Vista as a result of calling AsyncIUnknown::QueryInterface, while a AsyncIUnknown::QueryInterface call was just blocked in Windows 7 (some relevant modifications were made in NdrpCloneInOnlyCorrArgs, if someone is interested).

It is possible that it was blocked due to some horrible IBegin_QueryMultipleInterfaces manifestations, which remain a mystery to Microsoft. A code for modification of OLE32.DLL code was written at the moment of the launch of our components , since our program product had been used for Windows 2000 and XP and still needs a full use of asynchronous COM. I’ve described the technology that was used for these purposes in my article “Self-modifying programs”.