PB Component not receiving data from Web Service

[Former Member]

Hello all;

I am have a PB 11.2 component running on EAServer 5.2 server, which is used to fetch and forward PDF reports to a .jsp webpage on our website.  The PB component gets its data from a  .NET Web Service, which arrives in the form of two byte arrays.  The report generally works; however, from time to time, esp. when the jagsrv.exe is using a lot of memory (but not always) the data never arrives; that is, the NVO which should be populated with the data, does not receive the two byte arrays. In fact, it receives nothing at all.  

The Web Service  is definitely working correctly; I sniffed the network with WireShark, and discovered that the entire PDF, from the beginning XML tag to the end, is arriving at our EAServer correctly.  The question is, why does the PB Component fail to  receive the data?

------------ dst_rpt_org_contact_list_dotnet - function of_get_org_contact_list_report -- Code snippet starts ---------------

try 

  // Create the soap connection object

  inv_conn = Create soapconnection

  ll_rc = inv_conn.createinstance( inv_reportservice, is_webservice_name, as_url)

  if (ll_rc <> 0) then

       ls_errors = "Failed to create web service " + is_webservice_name + "."

       return -1

  end if

  inv_cst_return_results =  inv_reportservice.getcontactlistreport(al_stakeholder_id, true  )

  If IsValid (inv_cst_return_results) Then

       ll_rc = UpperBound(inv_cst_return_results.ib_pdf1)

       ls_error_msg = inv_cst_return_results.is_error_msg

  else

       ls_errors = "The Web Service's return object inv_cst_return_results was not instantiated. Stakeholder Id was "  + String(al_stakeholder_id)

       return -1

  end if

  if ll_rc < 1 then

       ls_errors += " The web service " + is_webservice_name + " did not return any array elements. Error msg: "   + ls_error_msg

       return -1

  end if

catch ( SoapException e )

  ls_errors = "SoapException - Cannot invoke the web service " + is_webservice_name + "." + e.Text

  return -1

catch (PBXRuntimeError e2)

  ls_errors = "PBXRuntimeError error when invoking the web service " + is_webservice_name + "." + e2.Text

  return -1

catch (RuntimeError e3)

  ls_errors = "Unknown runtime error when invoking the web service " + is_webservice_name + "." + e3.Text

  return -1

------------ Code snippet ends --------------------------------------------------------------------

inv_cst_return_results is a  non-visual object with properties of two byte arrays, ib_pdf1[] and ib_pdf2[], and a string  is_error_msg

inv_conn is an instance variable of type soapconnection

The failures are reported by an email which basically says the call to the function failed.  None of the error messages inside the function are returned.

I would appreciate any advice that you might be able to offer.

thanks

Marvin Haagsma

Systems Analyst, Alberta Gaming and Liquor Commission

St. Albert, AB

mhaagsma [AT] aglc [dot] ca

[Former Member]

Hi Marvin;

  Anything suspicious in the EAS logs?

FWIW: Sounds like an EAS resource (ie: memory)  issue to me (just a guess).

Regards ... Chris

[Former Member]

Thanks for the reply, Chris;

I alluded to memory issues in my post, but didn't explain in detail, so here goes:  - I monitor the memory usage of jagsrv.exe closely, using the Task Manager.  The problem usually occurs whenever memory usage gets above 1 Gb, but not always. 

I have noticed that jagsrv.exe memory usage always increases whenever the server is busy (esp. weekdays), possibly due to memory leaks in the server itself.  I have never been able to figure out why. 

In fact, if I don't do a manual restart, somewhere around 1.8 Gb memory usage the server experiences a PB Heap Exception and crashes. However, this data problem also occurs, at times, when memory use is very low; even at .25 Gb memory usage we have seen this problem.  So I am flummoxed as to what the cause is. 

We have taken to restarting the server automatically twice daily (thereby kicking out all logged on users -( I'm sure they have some choice comments to make about our website))-  just to clear the cobwebs. Sometimes we manually restart it during the day when the memory warnings appear in the server log.

We are switching our corporate software to .NET and IIS, but plenty of PB code still remains in use;  our secure website is on EAServer 5.5 (using CynergySystems EAF4.0) and our site is so popular that the old clunker can't keep up. 

Marvin

[Former Member]

Hi Marvin;

1) I'm confused ... are you on EAS 5.2 or 5.5?

2) What O/S version are you running EAS on?

3) Have you tried the 3G OS/ windows switch?

4) What kind of hardware are you using?

5) Have you thought about running an EAS cluster?

6) Do you know of my EAS Monitor application?

STD Foundation Classes - Browse /Example Applications/EAServer at SourceForge.net

Regards ... Chris

[Former Member]

Hello Chris;

Thanks for your reply;
First off, here are answers to your questions:
---------------------------------------
1) I'm confused ... are you on EAS 5.2 or 5.5?
We are running EAServer 5.5 (Jaguar CTS - Component Transaction Server/Version 5.5.0)

2) What O/S version are you running EAS on? 
Windows 2003 Server SP2

3) Have you tried the 3G OS/ windows switch?  
Yes.

4) What kind of hardware are you using? 
Virtual Machine -  Dual 2.3 GHz Intel Pentium Pro

5) Have you thought about running an EAS cluster? 
No.

6) Do you know of my EAS Monitor application?
Yes, I still have a version of it that I created years ago.    

---------------------------------

Next, here is some new information we've learned since my last post.

We tested using Apache JMeter which allows us to issue concurrent calls to our EAServer .JSP pages.  Apache JMeter allowed to reliably duplicate the problem.

When testing concurrent calls, we found that the first call to the service works everytime; however, all of the other concurrent calls fail.

Using Wireshark, I could monitor network traffic and see that the Web Service is being called successfully every time, and, in fact, the entire payload (the PDF file) is arriving at EAServer; however, for some unknown reason, the client component on EAServer is not able to access it.

If we turn concurrency 'OFF' all calls are successful, although much slower as everyone is queueing for the component. We even tried concurrent calls against different components, which exhibit the same behaviour. Our conclusion is that the web service client calls within EAServer might be sharing memory.

We also tried the 'Bind Thread' setting, which made no difference either way.

We have not yet been able to get the component to successfully receive and process the payload for 2nd, 3rd, nth current calls.

HTH
Marvin

[Former Member]

Hi Marvin;

   Any chance that you can try a newer version of PB?

   I think that you may have exhausted the EAS route as the version 6.x series has other adverse side effects.

Regards ...Chris

[Former Member]

Hi   Marvin

pdf   < 4m   ok?

pdf  < 8m   ok ?

......

pdf < 40m ok?

.........

we sure easerver 5.2   retrive <38m     data  [zlib ]   ok!

get_fullstate()   return  blob   use  zlib

Longlong ll_srclen

string ls_storage

string ls_log

ls_storage = this.object.datawindow.storage

ll_srclen = Long(ls_storage)

if ll_srclen > 38000000 then       

  ls_log = is_classname + "~t" + String(today(),'yyyy-mm-dd') + ' ' + String(Now(),'hh:mm:ss.fff')+ ' ll_srclen='+string(ll_srclen)

        of_log_test('get_fullstate',ls_log)    

        setNull(arg_dest)  

        return arg_dest

end if

if this.getfullstate(lblob_src) < 0 then    

     setNull(arg_dest)    

     return arg_dest

end if

// allocate buffer space

ll_srclen  = Len(lblob_src) ll_destlen = (ll_srclen * 1.01) + 12

// >38m not proc ，

if ll_destlen > 38000000 then       

   ls_log = is_classname + "~t" + String(today(),'yyyy-mm-dd') + ' ' + String(Now(),'hh:mm:ss.fff')+ ' ll_srclen='+string(ll_srclen)   

    of_log_test('get_fullstate',ls_log)   

    setNull(arg_dest)   

    return arg_dest

end  if

//Space(limt)  over

lblob_dest = Blob(Space(ll_destlen))

// compress the blob

ll_rc = compress(lblob_dest, ll_destlen, lblob_src, ll_srclen)

CHOOSE CASE ll_rc  

  CASE 0           arg_dest = BlobMid(lblob_dest, 1, ll_destlen)

END CHOOSE

Return arg_dest

[Former Member]

Our code was created a decade ago; current version uses PB11.2 and the CynergySystems EAF4  which only works with EAServer 5.5 and below.  Years ago CynergySystems abandoned their EAF, so in order to move ahead to EAS6 we would have to re-write the EAF Framework ourselves.  ( Instead of doing that, we chose to switch to ASP.NET )    However, our secure website still uses our original PB code which was working well; the problem I've described started occurring when the secure website became much more popular, with many more hits than we had originally experienced.  We still plan to re-write the site with ASP.NET, but, for the moment,  this recurring PDF generation failure is a genuine PITA as the server needs to be restarted to clear the problem, which annoys our customers big time.

AS far as going to  PB12.6, we used it to create .NET web services (hosted on IIIS) since we had the datawindows all ready, and could re-use them.  We discovered some interesting (and annoying)  bugs, but we managed to create .NET versions of our old PowerBuilder code in the end.  I suppose we could try to generate the EAServer client code with PB12.6 (classic mode); I'll have to discuss that option with Herb.

I am currently trying a different option, that is, to create the PDF on the IIS7.5 Application Server, and to just go fetch it using the file system (instead of having it returned in a byte array).

[Former Member]

The size of the PDFs are quite small; generally under 500KB.

[Former Member]

Hi Marvin;

  I think that rewriting your PB application in another tool and trying the IIs based web service alternative as a short term fix is the better path (IMHO) as EAS is set for EOL this year anyway.

Regards ... Chris