JDBC socketRead freezes the webserver

[Former Member]

Hi,

today a webserver freezes, with more than 42 blocked threads, to inspect the problem i get a thread dump, from running instance, and the problematic thread is:

"http-80-19" daemon prio=10 tid=0x0000002af8bbe000 nid=0x72fc runnable [0x000000004f45f000..0x000000004f460b30]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:129)
        at com.sap.dbtech.rte.comm.BasicSocketComm.receiveConnect(BasicSocketComm.java:707)
        at com.sap.dbtech.rte.comm.BasicSocketComm.dbConnectExchange(BasicSocketComm.java:789)
        at com.sap.dbtech.rte.comm.BasicSocketComm.connectDB(BasicSocketComm.java:233)
        at com.sap.dbtech.rte.comm.SocketComm$1.open(SocketComm.java:38)
        at com.sap.dbtech.jdbc.DriverSapDB.openConnection(DriverSapDB.java:966)
        at com.sap.dbtech.jdbc.DriverSapDB.openByURL(DriverSapDB.java:891)
        at com.sap.dbtech.jdbc.DriverSapDB.connect(DriverSapDB.java:208)
        - locked <0x0000002aa4978bc0> (a com.sap.dbtech.jdbc.DriverSapDB)
        at org.apache.tomcat.dbcp.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
        at org.apache.tomcat.dbcp.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:294)
        - locked <0x0000002aa515d318> (a org.apache.tomcat.dbcp.dbcp.PoolableConnectionFactory)
        at org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:974)
        at org.apache.tomcat.dbcp.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:96)
        at org.apache.tomcat.dbcp.dbcp.BasicDataSource.getConnection(BasicDataSource.java:880)
        ...

analysis of the thread dump, show us that is a problem when open new connections, and freezes on java.net.SocketInputStream.socketRead0, i checked the system and have space for more sessions, then its not a case of TASK LIMIT, follow is a sample of one thread waiting the above.

"http-80-18" daemon prio=10 tid=0x0000002af7f74000 nid=0x72fb waiting for monitor entry [0x000000004ef5e000..0x000000004ef5fdb0]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at org.apache.tomcat.dbcp.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:294)
        - waiting to lock <0x0000002aa515d318> (a org.apache.tomcat.dbcp.dbcp.PoolableConnectionFactory)
        at org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:974)
        at org.apache.tomcat.dbcp.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:96)
        at org.apache.tomcat.dbcp.dbcp.BasicDataSource.getConnection(BasicDataSource.java:880)
        ...

after 2 minutes, the problematic thread reach the end, and the webserver unfreeze, and come back processing requests.

The MaxDB and JDBC version is 7.6.03.07

note this problem dont occurs every time, or every day, just occurs in a random times, once per week, maybe have a reason on database side, that i dont know, any idea, how to check and inspect this type of problem?

Best regards

Clóvis

[Former Member]

no answer until now

[Former Member]

Again, but in another place, when pool try to check if connection is Closed, this one, blocked 35 other threads, and freezes the webserver for 2 minutes.

"Timer-1" daemon prio=10 tid=0x0000002af6ddf000 nid=0x3118 runnable [0x0000000047c48000..0x0000000047c48d30]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:129)
        at com.sap.dbtech.rte.comm.BasicSocketComm.receiveData(BasicSocketComm.java:577)
        at com.sap.dbtech.rte.comm.BasicSocketComm.receive(BasicSocketComm.java:666)
        at com.sap.dbtech.rte.comm.JdbcCommunication.execute(JdbcCommunication.java:41)
        at com.sap.dbtech.jdbc.ConnectionSapDB.execute(ConnectionSapDB.java:536)
        - locked <0x0000002aad5d66b0> (a com.sap.dbtech.jdbc.ConnectionSapDB)
        at com.sap.dbtech.jdbc.ConnectionSapDB.execute(ConnectionSapDB.java:461)
        at com.sap.dbtech.jdbc.ConnectionSapDB.isClosed(ConnectionSapDB.java:746)
        at com.sap.dbtech.jdbc.trace.Connection.isClosed(Connection.java:400)
        at org.apache.tomcat.dbcp.dbcp.DelegatingConnection.isClosed(DelegatingConnection.java:346)
        at org.apache.tomcat.dbcp.dbcp.PoolableConnectionFactory.validateConnection(PoolableConnectionFactory.java:324)
        at org.apache.tomcat.dbcp.dbcp.PoolableConnectionFactory.validateObject(PoolableConnectionFactory.java:312)
        at org.apache.tomcat.dbcp.pool.impl.GenericObjectPool.evict(GenericObjectPool.java:1217)
        - locked <0x0000002aa4a60cb0> (a org.apache.tomcat.dbcp.pool.impl.GenericObjectPool)
        at org.apache.tomcat.dbcp.pool.impl.GenericObjectPool$Evictor.run(GenericObjectPool.java:1341)
        at java.util.TimerThread.mainLoop(Timer.java:512)
        at java.util.TimerThread.run(Timer.java:462)

   Locked ownable synchronizers:
        - locked <0x0000002aa4a60cb0> (a org.apache.tomcat.dbcp.pool.impl.GenericObjectPool)
        at org.apache.tomcat.dbcp.pool.impl.GenericObjectPool$Evictor.run(GenericObjectPool.java:1341)
        at java.util.TimerThread.mainLoop(Timer.java:512)
        at java.util.TimerThread.run(Timer.java:462)

   Locked ownable synchronizers:
        - None