Unhandled exceptions from ESP

[Former Member]

Hello,

From time to time I am getting unhandled exceptions from ESP (5.1 v4 using .NET interface). This is happening only for the high rates of updates (300 records/s). ESP is dying internally, I am not even able to catch exception.

That is the exception from .NET Runtime:

Application: AvidPublisher.exe Framework Version: v4.0.30319 Description: The process was terminated due to an unhandled exception. Exception Info: System.AccessViolationException Stack:  at .SYBASE.Esp.SDK.EspPublisher.publish(SYBASE.Esp.SDK.EspPublisher*, SYBASE.Esp.SDK.EspPublishable*, SYBASE.Esp.SDK.EspError*)  at SYBASE.Esp.SDK.NetEspPublisher.publish(SYBASE.Esp.SDK.NetEspMessageWriter, SYBASE.Esp.SDK.NetEspError)  at FXDerivatives.Common.StreamingPlatform.EspHelper.Publish(System.String, System.Object[][], Int32)  at FXDerivatives.Common.StreamingPlatform.EspHelper.PublishToStream(System.String, System.Collections.Generic.IEnumerable`1, Int32, PublishMode)  at FXDerivatives.Common.Aleri.AleriLogWriter.PublishToAleri(System.String, System.String, System.String, FXDerivatives.Common.Aleri.AleriLogLevel, System.String, System.String, System.Collections.Generic.Dictionary`2)  at FXDerivatives.Common.Aleri.AleriLogWriter.Write(FXDerivatives.Common.Aleri.AleriLogLevel, System.String, System.String, System.Collections.Generic.Dictionary`2)  at FXDerivatives.Common.Aleri.AleriLogWriter.Write(FXDerivatives.Common.Aleri.AleriLogLevel, System.String, System.String, FXDerivatives.Common.MarketData.CurrencyPair, System.String)  at AvidPublisher.Publishers.Publisher.LogToAleri(FXDerivatives.Common.Aleri.AleriLogLevel, System.String, System.String, System.String, System.String)  at AvidPublisher.Publishers.Publisher.PublishAll(System.String, System.String, System.Collections.Generic.Dictionary`2)  at AvidPublisher.Publishers.Publisher+<>c__DisplayClass2.b__0(System.Object, System.Collections.Generic.Dictionary`2)  at FXDerivatives.Common.Utility.Bundler`2+DistributeEventHandler[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].Invoke(System.Object, System.Collections.Generic.Dictionary`2)  at FXDerivatives.Common.Utility.Bundler`2[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].DistributeEvents(System.Object, System.Timers.ElapsedEventArgs)  at System.Timers.Timer.MyTimerCallback(System.Object)  at System.Threading.TimerQueueTimer.CallCallbackInContext(System.Object)  at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)  at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)  at System.Threading.TimerQueueTimer.CallCallback()  at System.Threading.TimerQueueTimer.Fire()  at System.Threading.TimerQueue.FireNextTimers()  at System.Threading.TimerQueue.AppDomainTimerCallback()

That is the Applciation error:

Faulting application name: AvidPublisher.exe, version: 0.0.0.0, time stamp: 0x54083da6 Faulting module name: MSVCR100.dll, version: 10.0.40219.1, time stamp: 0x4d5f0c22 Exception code: 0xc0000005 Fault offset: 0x00010a5d Faulting process id: 0xd78 Faulting application start time: 0x01cfc82a6a680ae9 Faulting application path: D:\FXDeriv\Apps\AvidPublisher\AvidPublisher.exe Faulting module path: C:\Windows\system32\MSVCR100.dll Report Id: 48d4d149-341f-11e4-ae8a-0050569a4c0f

Can you please advise? Are you experiencing similar problems?

[former_member217348]

Hi Jacek,

Can you share the esp_server.log file and the cluster log file? What does the project CCL look like? Where is the ESP project running - what environment? Would you be able to provide us with enough so that we can reproduce the problem?

Thanks,

Alice

[Former Member]

Hello,

Please see attached file (exception occurred around 11:36 ).

Version of ESP: 5.1 sp4.

Environment:  Linux esplni01 2.6.32-279.5.1.el6.x86_64 #1 SMP Tue Jul 24 13:57:35 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux

It is happening only for the high rates of updates. When reproducing on my PC I do not have specific scenario, it is happening randomly.

cluster (log).txt.zip

MaDFX (ccl).txt.zip

esp_server-2 (log).txt.zip

[former_member217348]

Hi Jacek,

Also, can you tell me more about your application - is it publishing 300 records/s from a single publisher? Or does your application have multiple threads and each is publishing data that adds up to 300 records/s?

Thanks,

Alice

[Former Member]

Pardon, if I wasn't clear. there are many types of applications that are actually failing, so there is no specific scenario. I have around 20 separate applications which are listening for changes and 1 aplication which copy all data from Aleri to ESP. This app never fails, as it is basically publishing around 15 different datatables sequentially. But those 20 separate applications are failing randomly when trying to consume data, and then save data back to ESP if required. there is 1 more place when application is failing, when different threads are subscribing to stream, and ESP crashes when we call: _subscriber.connect(espError);

[former_member217348]

Hi Jacek,

There are no known issues for what you are describing. Please log a technical support case and provide a reproduction.

Thank you,

Alice