Intermittent Crashes with .NET visual controls embedded in PowerBuilder

[Former Member]

After adding a few .NET (4.0) controls embedded in our PowerBuilder (12.1 Build 7217) windows as OLE objects, we are now getting random hard crashes in our application. It is a hard crash so exception handling in our application does not catch the error, the app simply crashes.

We have been able to reproduce it through automated UI tests, and it usually occurs after opening and closing the same set of three or four windows less than 100 times.

The most common error codes we get are the three below:

0xC0000005  EXCEPTION_ACCESS_VIOLATION  The thread tried to read from or write to a virtual address for which it does not have the appropriate access.
0x80000003  EXCEPTION_BREAKPOINT A breakpoint was encountered.
0xc0000409  Unknown software exception (Application Error)

Here are some sample errors in the Windows Event Viewer:

Faulting application name: [OUR APP NAME], version: 1.0.0.1, time stamp: 0x512de438
Faulting module name: unknown, version: 0.0.0.0, time stamp: 0x00000000
Exception code: 0x80000003
Fault offset: 0x048495b4
Faulting process id: 0x2fc
Faulting application start time: 0x01cfa28c02e07932
Faulting application path: C:\Program Files (x86)\..........[our app exe]
Faulting module path: unknown
Report Id: 9c101a2e-0e7f-11e4-815e-0026b9806e38

Faulting application name: [OUR APP NAME], version: 1.0.0.1, time stamp: 0x512de438
Faulting module name: clr.dll, version: 4.0.30319.18444, time stamp: 0x52717e84
Exception code: 0xc0000005
Fault offset: 0x000fe810
Faulting process id: 0x2fc
Faulting application start time: 0x01cfa28c02e07932
Faulting application path: C:\Program Files (x86)\..........[our app exe]
Faulting module path: C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll
Report Id: a5b51191-0e7f-11e4-815e-0026b9806e38

Faulting application name: [OUR APP NAME], version: 1.0.0.1, time stamp: 0x512de438
Faulting module name: ntdll.dll, version: 6.1.7601.18229, time stamp: 0x51fb1072
Exception code: 0xc0000029
Fault offset: 0x00090892
Faulting process id: 0x272c
Faulting application start time: 0x01cfa29978c86751
Faulting application path: C:\Program Files (x86)\..........[our app exe]
Faulting module path: C:\WINDOWS\SysWOW64\ntdll.dll
Report Id: 983c1451-0e8d-11e4-a4b9-b8ca3a74f207

The application is not even crashing on our code, but usually somewhere in the CLR code or the PBVM code. The callstack from the crash dump does not even include any of our functions, it seems to be purely .NET framework/CLR code. The callstack below is what we usually see, but even this is not consistent from one crash to the next:

clr.dll!string "d:\\iso_whid\\x86fre\\base\\ntos\\rtl"...()  + 0x4f3b9b bytes 
ntdll.dll!ExecuteHandler@20()  + 0x24 bytes
ntdll.dll!_KiUserExceptionDispatcher@8()  + 0xe bytes 
0b1cb4b4() 
clr.dll!GetFastContextCookie()  + 0x1d bytes 
clr.dll!CtxEntry::EnterContextCallback()  + 0x41 bytes 
ole32.dll!CRemoteUnknown::DoCallback()  + 0x3b bytes 
rpcrt4.dll!_Invoke@12()  + 0x30 bytes 
rpcrt4.dll!_NdrStubCall2@16()  + 0x217 bytes 
rpcrt4.dll!_CStdStubBuffer_Invoke@12()  + 0x82 bytes 
ole32.dll!SyncStubInvoke()  + 0x30 bytes 
ole32.dll!StubInvoke()  + 0x73 bytes 
ole32.dll!CCtxComChnl::ContextInvoke()  + 0xdb bytes 
ole32.dll!MTAInvoke()  + 0x1a bytes
ole32.dll!STAInvoke()  + 0x4c bytes
ole32.dll!AppInvoke()  + 0x8d bytes
ole32.dll!ComInvokeWithLockAndIPID()  + 0x21b bytes
ole32.dll!ComInvoke()  + 0x71 bytes
ole32.dll!ThreadDispatch()  + 0x1a bytes 
ole32.dll!ThreadWndProc()  + 0x93 bytes
user32.dll!_InternalCallWinProc@20()  + 0x28 bytes 
user32.dll!_UserCallWinProcCheckWow@32()  + 0xa2 bytes 
user32.dll!_DispatchMessageWorker@8()  + 0xc8 bytes
user32.dll!_DispatchMessageW@4()  + 0xf bytes 
ole32.dll!CCliModalLoop::PeekRPCAndDDEMessage()  - 0x7982 bytes
ole32.dll!CCliModalLoop::BlockFn()  + 0x1143 bytes 
ole32.dll!_CoWaitForMultipleHandles@20()  + 0xb7 bytes 
clr.dll!MsgWaitHelper()  + 0x6f bytes 
clr.dll!Thread::DoAppropriateAptStateWait()  + 0x12524 bytes 
clr.dll!Thread::DoAppropriateWaitWorker()  + 0xe9 bytes
clr.dll!Thread::DoAppropriateWait()  + 0x60 bytes 
clr.dll!Thread::JoinEx()  + 0x83 bytes 
clr.dll!Thread::Join()  + 0x16 bytes 
clr.dll!RCW::Initialize()  + 0x1cb81f bytes
clr.dll!RCW::CreateRCW()  + 0x62 bytes 
clr.dll!COMInterfaceMarshaler::CreateObjectRef()  + 0x4e bytes 
clr.dll!COMInterfaceMarshaler::FindOrCreateObjectRef()  + 0x8c bytes 
clr.dll!GetObjectRefFromComIP()  + 0x125 bytes 
clr.dll!UnmarshalObjectFromInterface()  + 0x1b bytes 
clr.dll!StubHelpers::InterfaceMarshaler__ConvertToManaged()  + 0xc6 bytes 
System.Windows.Forms.ni.dll!7ba745e0() 
[Frames below may be incorrect and/or missing, no symbols loaded for System.Windows.Forms.ni.dll]  
clr.dll!_COMToCLRDispatchHelper@28()  + 0x28 bytes 
clr.dll!BaseWrapper<Stub *,FunctionBase<Stub *,&DoNothing<Stub *>,&StubRelease<Stub>,2>,0,&CompareDefault<Stub *>,2>::~BaseWrapper<Stub *,FunctionBase<Stub *,&DoNothing<Stub *>,&StubRelease<Stub>,2>,0,&CompareDefault<Stub *>,2>()  + 0x175b8b bytes
clr.dll!COMToCLRWorkerBody()  + 0x80 bytes 
clr.dll!COMToCLRWorkerDebuggerWrapper()  + 0x34 bytes 
clr.dll!_COMToCLRWorker@8()  + 0x12b bytes 
04e7a2c6() 
PBVM120.DLL!10b1a856()

Even when we strip down the embedded .NET control to a simple/empty container with no child controls on it, and no other significant code being called, we still get the crash. Note that we are only making simple function calls into the .NET control with basic data type parameters.

We have been able to drastically improve stability by manually calling Dispose on all the .NET controls and forcing garbage collection, such that now we don't seem to get the error at all when showing certain embedded .NET controls, but we still get the error for others.

This does not appear to be a memory leak issue as such, since the app is only using around 100MB of RAM when it crashes. Although it does appear to be some kind of memory corruption issue, given that it is a hard crash in the CLR code, usually with an access violation error, and given that calling Dispose on our controls seems to reduce the problem occurrence.

Since the app seems to crash randomly in the CLR code we have been unable to pinpoint the exact cause of the issue, all we can say is that it only seems to occur when opening PowerBuilder windows in our app that have embedded .NET controls.

We updated the .NET version installed on our machines and applied the hotfix that looked like it would resolve this issue (below), but this has not helped.

http://support.microsoft.com/kb/2640103

Any help would be greatly appreciated, as we have been unable to find any evidence on the web that others are having this issue.

[Former Member]

Hi Graham:

We've seen this issue in the past related to the "The Interop Forms Toolkit". The issue we saw was caused by the Event Sink being called after the control is destroyed, PB has already terminated the connection to the Event Sink and the control before the control is destroyed. The release call is incorrect and hence the problem is within "The Interop Forms Toolkit". You might check with Microsoft to see if they certify it outside the Microsoft suite of products.

Cheers...Bob