PowerBuilder 12.5.2 app crashing often

[Former Member]

Hi,

We recently updated our software from PB 8 to PB 12.5.2 Build 5550. Our client's are now beginning to have issues with our software crashing to desktop (a few times a day per user) and the Windows error reporting message displays. The issue seems random (it happens throughout the program at no discernible point or task).

I've tried to read around on the web and it sounds like 12.5.2 isn't very stable...

Does anyone else have any experience with this type of problem? At this point, I'm not sure what we can do about it?

Thanks,

Dan

[Former Member]

Hi,

Brief update; I narrowed the crashing down to when a window with a COM object on it closes, I'll get a few (or more) access violations:

(a24.b84): Access violation - code c0000005 (first chance)

First chance exceptions are reported before any exception handling.

This exception may be expected and handled.

eax=00000000 ebx=00000000 ecx=0018d7dc edx=00620174 esi=0f9049a8 edi=00000001

eip=6eca4e14 esp=0018d7c8 ebp=0018d800 iopl=0         nv up ei pl nz na po nc

cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202

clr!SafeReleaseHelper+0x75:

6eca4e14 ff7008          push    dword ptr [eax+8]    ds:002b:00000008=????????

0:000> kp

ChildEBP RetAddr 

0018d800 6eca4e8a clr!SafeReleaseHelper+0x75

0018d838 6edf8e3a clr!SafeRelease+0x2f

0018d870 6edf8d8d clr!IUnkEntry::Free+0x62

0018d884 6edf8e96 clr!RCW::ReleaseAllInterfaces+0x1a

0018d8b4 6edf8ec2 clr!RCW::ReleaseAllInterfacesCallBack+0xc4

0018d8e8 6edf9105 clr!RCW::Cleanup+0x22

0018d8f4 6edf85d3 clr!RCWCleanupList::ReleaseRCWListRaw+0x18

0018d924 6edf90e3 clr!RCWCleanupList::ReleaseRCWListInCorrectCtx+0x10a

...

Sometimes the program recovers and continues and sometimes it crashes (sometimes with a STATUS_STACK_BUFFER_OVERRUN).

So I read a few articles on COM and RCW:

http://blogs.msdn.com/b/vcblog/archive/2006/09/20/762884.aspx

http://www.voyce.com/index.php/2010/01/21/beware-of-using-stack-based-com-objects-from-net/

The COM object is an in house custom COM object that uses .NET third party controls, so I do have some .NET access but since I'm using the COM object in PowerBuilder classic, there's not much I can do about the .NET garbage collection of the COM object.

I've added a function to my COM object to do clean up, dispose of everything and do .net garbage collection. I'm trying out calling this function in the closequery event of any window that has the COM object on it. Ya, so pretty much just trying any clean up/garbage collecting that I can think of and hoping that it works.

So far that's where I'm at...

Dan

Message was edited by: Dan Black

[Former Member]

Hi Dan,

I'am facing the  'exe stopped working' problem (PB app created via PB 12.5.2 build 5006). The crash is only faced few times but in random flows. In the event log I faced this PBVM125.DLL fault module in the event logger. So all quiet similar like you reported.

So today I am quit curious on the outcome of your research.
Were you able to solve this ?

Regards Nathalie

[Former Member]

HI Nathalie,

Are you using .NET controls in COM wrappers within your PB application?

That's where we were having issues. There's a few recent articles with examples of how to get that working, although I've never tried them.

We scrapped the .NET control as we could not get it stable.

Dan

[Former Member]

Hi Dan,

No, no .NET controls are used in our PB CLASSIC application.

Our solution was initially created in PB4, this has been migrated to PB8, PB12.5 and today we have PB 12.5.2 build 5006.
This fatal crash is faced on our older modules, where we did not face any problem before.

We do note that it is always the same DLL, which is set in the EVENT LOG.  (PBVM125.dll)

During research we noted that until today we could not reproduce the problem if we run the exe as administrator (which is not an option in the field)

We do start to believe that maybe Microsoft updates/Security start to conflicts with the PB deployement dll's. Do you have any experience on this ?

We've experimented already with 12.6 but same problem is faced, at random a crash is faced and it only happens from time to time

Regards Nathalie

[Former Member]

Hi Nathalie,

Sorry, I don't really have experience with Microsoft updates/security and PB stability.

I can suggest that you can contact PB support, they should be able to guide you in generating crash logs / diagnostic reports that they can use to diagnose the problem. But I don't think they'll do that for free.

Dan

[Former Member]

Hi,

We've added all kinds of exception handling, and that seems to have caught a few issues, but we still have clients running our software and the software is crashing 2 to 5 times a day per workstation.

I've been seeing more and more crashes around the PBVM125.dll in particular.

I know that's a powerbuilder dll but I'm not sure what I can do about tracking down what the issue would be?

Has anyone ever purchased a technical support contract from Sybase? What's their response time like?I'm wondering if this is the kind of thing they'd be able to track down?

Thanks,

Dan

[Former Member]

Hi Dan:

Are the crashes happening with all clients? If not, can you confirm the DLLs loaded by the failing clients using Process Explorer (http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx)? If all clients are failing, you can use ADPlus (http://support.microsoft.com/kb/286350) to capture a crash dump which can then be forwarded to development once you open a support contract.

Let me know if you need more detail.

Cheers...Bob

[Former Member]

Hi Bob,

All of our clients are experiencing crashing to varying degrees. The most popular DLLs that the program crashes on are MSVCR80.dll, clr.dll and PBMV125.dll.

What about the DLLs should I be confirming? The versions? They are all the same version (12.5.2.5550).

Thanks,

Dan

[Former Member]

Hi Dan:

Can you send me the Process Explorer DLL report which shows the file version and path of all DLLs your PB application would have loaded including MSVCR80.DLL, CLR.DLL so I can confirm that the application is not loading an incorrect version?

The Process Explorer display consists of two sub-windows. The top window always shows a list of the currently active processes, including the names of their owning accounts, whereas the information displayed in the bottom window depends on the mode that Process Explorer is in: if it is in handle mode you'll see the handles that the process selected in the top window has opened; if Process Explorer is in DLL mode you'll see the DLLs and memory-mapped files that the process has loaded. Process Explorer also has a powerful search capability that will quickly show you which processes have particular handles opened or DLLs loaded. You can save the DLL report by using File > SaveAs. Please send me the resulting file.

Let me know if you need anything else.

Cheers...Bob

[Former Member]

Hi Bob,

Here you go: https://www.dropbox.com/s/vqdi67rniq2rkdc/healthq.exe.txt

Thanks for taking a look at this,

Dan

[Former Member]

HI Dan:

Nothing obvious from my review of the DLLs loaded by your PB 12.5.2 application other than 3rd party references. Have you tried using ADplus (http://support.microsoft.com/kb/286350) to catch the reamining failures? This may give us more clues to the failure(s). Any in-depth analysis by our developers would require a support contract.

Cheers...Bob

[Former Member]

Hi Bob,

I'm installing the Windows Debugging Tools now, looks like there's lots of option to wade my way through but it's definitely worth a shot. I'll post a reply if I do manage to narrow down the failures.

Thanks,

Dan

[Former Member]

UPDATE:

Now our app is being built and running on the same PB dlls but we still have crashing issues:

Quite often there is a .Net Runtime Error followed by the application crash.

Here is the .NET error from the Event Viewer:

Log Name:      Application

Source:        .NET Runtime

Date:          19/08/2013 5:25:37 PM

Event ID:      1026

Task Category: None

Level:         Error

Keywords:      Classic

User:          N/A

Computer:      CORTHTS.LBOBH.cloud.local

Description:

Application: healthq.exe

Framework Version: v4.0.30319

Description: The process was terminated due to an unhandled exception.

Exception Info: exception code c0000005, exception address 10BF50D8

Stack:

Event Xml:

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

  <System>

    <Provider Name=".NET Runtime" />

    <EventID Qualifiers="0">1026</EventID>

    <Level>2</Level>

    <Task>0</Task>

    <Keywords>0x80000000000000</Keywords>

    <TimeCreated SystemTime="2013-08-19T23:25:37.000000000Z" />

    <EventRecordID>1733717</EventRecordID>

    <Channel>Application</Channel>

    <Computer>CORTHTS.LBOBH.cloud.local</Computer>

    <Security />

  </System>

  <EventData>

    <Data>Application: healthq.exe

Framework Version: v4.0.30319

Description: The process was terminated due to an unhandled exception.

Exception Info: exception code c0000005, exception address 10BF50D8

Stack:

</Data>

  </EventData>

</Event>

...followed by the application error from the Event Viewer:

Log Name:      Application

Source:        Application Error

Date:          19/08/2013 5:25:39 PM

Event ID:      1000

Task Category: (100)

Level:         Error

Keywords:      Classic

User:          N/A

Computer:      CORTHTS.LBOBH.cloud.local

Description:

Faulting application name: healthq.exe, version: 4.0.0.1, time stamp: 0x51835e47

Faulting module name: PBVM125.dll, version: 12.5.2.5550, time stamp: 0x51835eb3

Exception code: 0xc0000005

Fault offset: 0x000f50d8

Faulting process id: 0x2c74

Faulting application start time: 0x01ce9d31eebac5c7

Faulting application path: C:\HQ4\healthq.exe

Faulting module path: C:\HQ4\PBVM125.dll

Report Id: a849f453-0926-11e3-8c96-0050568c78dc

Event Xml:

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

  <System>

    <Provider Name="Application Error" />

    <EventID Qualifiers="0">1000</EventID>

    <Level>2</Level>

    <Task>100</Task>

    <Keywords>0x80000000000000</Keywords>

    <TimeCreated SystemTime="2013-08-19T23:25:39.000000000Z" />

    <EventRecordID>1733718</EventRecordID>

    <Channel>Application</Channel>

    <Computer>CORTHTS.LBOBH.cloud.local</Computer>

    <Security />

  </System>

  <EventData>

    <Data>healthq.exe</Data>

    <Data>4.0.0.1</Data>

    <Data>51835e47</Data>

    <Data>PBVM125.dll</Data>

    <Data>12.5.2.5550</Data>

    <Data>51835eb3</Data>

    <Data>c0000005</Data>

    <Data>000f50d8</Data>

    <Data>2c74</Data>

    <Data>01ce9d31eebac5c7</Data>

    <Data>C:\HQ4\healthq.exe</Data>

    <Data>C:\HQ4\PBVM125.dll</Data>

    <Data>a849f453-0926-11e3-8c96-0050568c78dc</Data>

  </EventData>

</Event>

I don't have much experience reading these so I'm not sure where to take this now?

Thanks,

Dan

[Former Member]

OH ... I thought you were compiling a Win32 native EXE from PB Classic.

Are you using PB.Net or compiling into a Winform EXE from PB Classic?

[Former Member]

Hi Chris,

We are compiling a Win32 native exe from PB classic, I suppose you're saying that if that's the case we shouldn't be getting a .NET error.

We do have a VB.NET COM Interop object in our app, "unhandled exception" should have lead me to that.

There are other crashes that aren't unhandled exceptions, they look like this:

Reporting queued error: faulting application healthq.exe, version 4.0.0.1, faulting module clr.dll, version 4.0.30319.1008, fault address 0x0052d3e9.

Thanks,

Dan

[Former Member]

Hi Dan;

  Exactly ... if its a Win32 EXE then you should not be getting .Net run-time errors!   

  So, it seems to me that the problem could revolve around a .net component and not the PB application itself. More like how the native EXE steps up or uses the .Net component or how the .net component interacts with the Win32 world. My guess is that your issue lies in the boundary between the two worlds.

HTH

Regards ... Chris

[Former Member]

Make sure the methods in your .Net component have proper exception handling (if possible).

[Former Member]

Hi,

Thanks for all of your input, here's an update:

The crashing is happening at various parts of the program, nothing that we've been able to narrow down to a specific business case or powerbuilder object.

Yes the PB 8 to 12.5 conversion was fun. Pretty sure we got all the Ansi to Unicode things fixed, mostly the issues were with converting to and from blobs and strings in and out of the database and file system. A lot of the external function definitions were also modified to account for Ansi vs Unicode. But I think if it were either of these issues it would lead us to a specific business case or pb object.

Now Chris mentioned duplicate PBVMs causing problems...

Our program has the following dlls for PB12.5 in the application folder (all seem to be from the proper build):

libjcc.dll

libjtml.dll

libj.util.dll

nlwnsck.dll

PBACC125.DLL

PBDWE125.DLL

PBODB125.DLL

pbodb125.ini

PBRTC125.DLL

PBSHR125.DLL

PBVM125.DLL

tp15.dll

tp15_tls.dll

tp15_wnd.dll

tp4ole15.ocx

We also have so old, companion, admin apps for backups etc. in PB 8 also in the application folder so the PB 8 dlls are in there as well:

pbdwe80.dll

pbodb80.dll

pbrtc80.dll

pbvm80.dll

The PB 8 apps aren't run by the users, they would only be run by our support staff, and we figured that since they're named differently than the PB 12.5 dlls it wouldn't be an issue. We were wrong about that?

.... and in the middle of writing this I decided to check the build machine to see what version of PB our exe/pbds are being built with... 12.5.1.4595

Oh... well let's hope that's the issue, we're going to do a quick patch and get it out. I'll hopefully repost next week with an update..

Thanks,

Dan

[tobias]

Hello Dan,

sounds as if you found the problem. 😉

Just one sentence about different runtime.dlls in one folder: If they are from different "major"-releases, as in your case, it is not a problem in general. That is why the names contain the PB-versionnumber. A problem could be around libjcc.dll. Do you use the actual version or the one from PB8?

Having two versions from e.g. 12.5.1 and 12.5.2 in one folder is not possible, but having the applications devided in two folders makes it possible.

And last but not least: Migration is always fun!

Ciao,

Tobias

[Former Member]

If I had a nickel for every time the build machine had incorrect dlls I'd have ....

.

.

.

.

. at least 25 or 30 cents.

[arnd_schmidt]

Dan,

If you built your app with PB 12.5.1,  I do not think that there is a guarantee that this app is running well using PB 12.5.2 Runtimes.

[tobias]

Hello,

same here. PB 12.5.2 5500 is not that stable. My first upgrade-installation although was not able to start the classic-IDE. Invalid licence, but no chance to re-enter a valid licence. Really strange.

Today we stay at 12.5.1 4953 and are quite happy.

But, as Chris mentioned - it could be a migration issue or something around dll/ocx/etc.

Could you isolate the problem(s) to a business case or better a pb-object?

regards,

Tobias

[Former Member]

Hi,

We are in 12.5.1.4015 for production.

We used to take only the Maint/Update EBFs, but no Maint/Update EBF has been released since February 2012, so I was thinking that the 12.5.2 was a kind of maintenance revision (I don't understand what means SP02 PL01).

So I installed 12.5.2.5550 recently and i didn't see any issue for the moment, and it comes with some fixes.

Now you're worrying me a bit, did you know some specific issues, what means "not stable" ?

Regards

Guillaume

[Former Member]

Hi Dan;

   FWIW: Unfortunately, that was my experience with v12.5.2 as well. I have my clients holding on v12.1  which have been very stable for them. I have a few applications running on 12.5.1 as well that seem to be very well behaved as well.

   Now - upgrading from PB 8 though could be another issue ... especially crossing the PB 10.0 (Unicode) release as well as PB changing its event sequencing as well. Your application stability could be related to some old ANSI PB 8 code that just does not like W7/64bit.

   The other obvious stability issue can be related to duplicate PBVM run-time DLL's of different versions / builds. Mixing these can certainly cause your application's instability or running run-time DLL's that don't match what your IDE uses to build the EXE with can also be your demise.

Regards ... Chris

[Former Member]

Also external API methods may have changed too.

- Matt