SAP File System is getting full

[Former Member]

Hi,

The SAP File system is getting full frequently.

We are also reducing by deleting the file. But then also after some hours it is getting Full again  (/usr/sap).

Can you please tel the possible root causes for the File system ( /usr/sap) is getting full frequently.

Regards,

Gayathri

[Former Member]

can anybody reply please

[manumohandas82]

Hi ,

Please check this SAP note

2107491 - JCo and JRFC traces grow up and start filling up the file system

Thanks ,

Manu

[former_member182657]

Hi,

Are you using parameters stat/max_files,stat/compression & stat/file ? If yes , then could you share the values for these.

Try to set value to lower for parameter stat/max_files and recheck the system again.

Regards,

Gaurav

[Former Member]

Hi,

In application server, the value of the parameters maintained is

stat/max_files = 48

stat/compression = ON

stat/file  =   /usr/sap/SID/DVEBMGS01/data/stat.

The trace files  ( dev_jrfc*)  are getting generated in the below path every one minute

zerp<sid>app1% date

Thu Jan 22 09:54:48 EST 2015

zerp<sid>app1% pwd

/usr/sap/SID/D11/j2ee/cluster/server0

**********************************************

-rw-r--r--   1 <SID>ADM   sapsys   8388961 Jan 22 09:49 dev_jrfc.20150122_094923.trc

-rw-r--r--   1 <SID>ADM   sapsys 8388961 Jan 22 09:50 dev_jrfc.20150122_095005.trc

-rw-r--r--   1 <SID>ADM   sapsys 8388961 Jan 22 09:50 dev_jrfc.20150122_095048.trc

-rw-r--r--   1 <SID>ADM   sapsys 8388961 Jan 22 09:51 dev_jrfc.20150122_095133.trc

-rw-r--r--   1 <SID>ADM   sapsys 8388961 Jan 22 09:52 dev_jrfc.20150122_095216.trc

-rw-r--r--   1 <SID>ADM   sapsys 8388961 Jan 22 09:52 dev_jrfc.20150122_095259.trc

-rw-r--r--   1 <SID>ADM   sapsys 8388961 Jan 22 09:53 dev_jrfc.20150122_095343.trc

-rw-r--r--   1 <SID>ADM   sapsys 8388961 Jan 22 09:54 dev_jrfc.20150122_095427.trc

Logs in the trace file dev_jrfc.trc

********************************************************

Error> occured  >Thu Jan 22 09:58:14,771<

>RfcIoException:

    message: CPIC-CALL: SAP_CMACCPTP3 on convId: 00000000

LOCATION    SAP-Gateway on host zerp<SID>ci / sapgw01

ERROR       registration of tp ZERP<SID>CI_PORTAL_<SID> from host

            sblz<SID>ap1.SSS.SSSSSS.com not allowed

TIME        Thu Jan 22 09:58:14 201

RELEASE     701

COMPONENT   SAP-Gateway

VERSION     2

RC          720

MODULE      gwxxrd.c

LINE        3736

COUNTER     275682012

    Return code: RFCIO_ERROR_SYSERROR(5)

*******************************************************************************************************

Resetting the trace files will not help but it will delete the old files only.

But, Please tel what could be the root cause for the increase in the above trace file (dev_jrfc*) in the path (/usr/sap/SID/D11/j2ee/cluster/server0).

kindly advice what could be the root cause.

Regards,

gayathri

[manumohandas82]

Hi Gayathri ,

Open  your Visual Administrator and  traverse

Cluster -> Server-> Services->JCo RFC Provider and check  whether the RFC mentioned in the trace logs are  working fine . On the special settings tab check whether the tracing is enabled . If enabled disable the same

Thanks ,

Manu

[manumohandas82]

Hi Gayathri ,

Are you on a Unix system ? If yes please execute the following commands and provide the screen shot

df -g

1 )

     Check in /usr/sap/<SID>/DV<>/work  directory for any files like core*

2 ) Stats files in //usr/sap/<SID>/DV<>/Data  can also grow 

Thanks ,

Manu

[Former Member]

Hi,

We have  tried removing the old  files of 2014 from location /usr/sap/SID/DVEBMGS01/j2ee/cluster/server0.  which starts with dev_jrfc.

now the percentage is    86%    /usr/sap

There is one core File present in /usr/sap/<SID>/DV<>/work in System

-rw----r-x   1 <SID>adm   sapsys   357066145 Nov  4  2011 core

2)Stat Files are also growing rapidly

Stats files in //usr/sap/<SID>/DV<>/Data  can also grow.

-rw-r--r--   1 <SID>ADM   sapsys    325002 Jan 20 08:00 stat38

-rw-r--r--   1 <SID>ADM   sapsys 324222 Jan 20 09:00 stat39

-rw-r--r--   1 <SID>ADM   sapsys 491178 Jan 20 10:00 stat40

-rw-r--r--   1 <SID>ADM   sapsys 18524 Jan 20 10:26 astat26

-rw-r--r--   1 <SID>ADM   sapsys 5086 Jan 20 10:28 astat27

-rw-r--r--   1 <SID>ADM   sapsys 373149 Jan 20 11:00 stat41

-rw-r--r--   1 <SID>ADM   sapsys 339532 Jan 20 12:00 stat42

-rw-r--r--   1 <SID>ADM   sapsys 454431 Jan 20 13:00 stat43

-rw-r--r--   1 <SID>ADM   sapsys 345663 Jan 20 14:00 stat44

-rw-r--r--   1 <SID>ADM   sapsys 306283 Jan 20 15:00 stat45

-rw-r--r--   1 <SID>ADM   sapsys 358601 Jan 20 16:00 stat46

-rw-r--r--   1 <SID>ADM   sapsys 1923 Jan 20 16:31 astat28

-rw-r--r--   1 <SID>ADM   sapsys 369032 Jan 20 17:00 stat47

-rw-r--r--   1 <SID>ADM   sapsys        10 Jan 20 18:34 rslgcpid

-rw-r--r--   1 <SID>ADM   sapsys        10 Jan 20 18:34 rslgspid

-rw-r--r--   1 <SID>ADM   sapsys 285790 Jan 20 19:00 stat48

-rw-r--r--   1 <SID>ADM   sapsys 396109 Jan 20 20:00 stat49

-rw-r--r--   1 <SID>ADM   sapsys 390275 Jan 20 21:00 stat50

-rw-r--r--   1 <SID>ADM   sapsys 3528 Jan 20 21:28 astat29

-rw-r--r--   1 <SID>ADM   sapsys 343940 Jan 20 22:00 stat51

-rw-r--r--   1 <SID>ADM   sapsys 334410 Jan 20 23:00 stat52

-rw-r--r--   1 <SID>ADM   sapsys 343846 Jan 21 00:00 stat53

-rw-r--r--   1 <SID>ADM   sapsys 332255 Jan 21 01:00 stat54

-rw-r--r--   1 <SID>ADM   sapsys 333649 Jan 21 02:00 stat55

-rw-r--r--   1 <SID>ADM   sapsys 328296 Jan 21 03:00 stat56

-rw-r--r--   1 <SID>ADM   sapsys 348266 Jan 21 04:00 stat57

-rw-r--r--   1 <SID>ADM   sapsys 348250 Jan 21 05:00 stat58

-rw-r--r--   1 <SID>ADM   sapsys 348989 Jan 21 06:00 stat59

-rw-r--r--   1 <SID>ADM   sapsys 346060 Jan 21 07:00 stat60

-rw-r--r--   1 <SID>ADM   sapsys 344400 Jan 21 08:00 stat61

-rw-r--r--   1 <SID>ADM   sapsys 344525 Jan 21 09:00 stat62

-rw-r--r--   1 <SID>ADM   sapsys 466174 Jan 21 10:00 stat63

-rw-r--r--   1 <SID>ADM   sapsys 346478 Jan 21 11:00 stat64

-rw-r--r--   1 <SID>ADM   sapsys 346320 Jan 21 12:00 stat65

-rw-r--r--   1 <SID>ADM   sapsys 3420 Jan 21 12:28 astat30

-rw-r--r--   1 <SID>ADM   sapsys 605756 Jan 21 13:00 stat66

-rw-r--r--   1 <SID>ADM   sapsys 346642 Jan 21 14:00 stat67

-rw-r--r--   1 <SID>ADM   sapsys 346962 Jan 21 15:00 stat68

-rw-r--r--   1 <SID>ADM   sapsys 344197 Jan 21 16:00 stat69

-rw-r--r--   1 <SID>ADM   sapsys 343961 Jan 21 17:00 stat70

-rw-r--r--   1 <SID>ADM   sapsys 346538 Jan 21 18:00 stat71

-rw-r--r--   1 <SID>ADM   sapsys 346651 Jan 21 19:00 stat72

-rw-r--r--   1 <SID>ADM   sapsys 345731 Jan 21 20:00 stat73

-rw-r--r--   1 <SID>ADM   sapsys 362718 Jan 21 21:00 stat74

-rw-r--r--   1 <SID>ADM   sapsys 353230 Jan 21 22:00 stat75

-rw-r--r--   1 <SID>ADM   sapsys 347095 Jan 21 23:00 stat76

-rw-r--r--   1 <SID>ADM   sapsys 356790 Jan 22 00:00 stat77

-rw-r--r--   1 <SID>ADM   sapsys 348890 Jan 22 01:00 stat78

-rw-r--r--   1 <SID>ADM   sapsys 346444 Jan 22 02:00 stat79

-rw-r--r--   1 <SID>ADM   sapsys 347345 Jan 22 03:00 stat80

-rw-r--r--   1 <SID>ADM   sapsys 343864 Jan 22 04:00 stat81

-rw-r--r--   1 <SID>ADM   sapsys 356942 Jan 22 05:00 stat82

-rw-r--r--   1 <SID>ADM   sapsys 1953 Jan 22 05:28 astat

-rw-r--r--   1 <SID>ADM   sapsys 345594 Jan 22 06:00 stat83

-rw-r--r--   1 <SID>ADM   sapsys 362133 Jan 22 07:00 stat84

drwxr-xr-x   4 <SID>ADM   sapsys        91 Jan 22 07:00 .

-rw-r--r--   1 <SID>ADM   sapsys       184 Jan 22 07:00 stat

This issue is occurring quite frequently now, there is need to dig into the cause creating thousand of tracefiles daily. (8-10GB appx).

Can you please tel the reason may be the core or increase in Stat files

Regards,

Gayathri

[manumohandas82]

Hi   Gayathri ,

You may remove the core files from the work directory  .

You can do  clearing of trace files from SM50 ( RESET trace files twice )

As you have mentioned it is an ABAP + Java system . Is the Java restarting  with Memory issues ?

If it does it will dump core files into the /usr/sap/SID/DVEBMGS01/j2ee/cluster/server0/log directory .  [ These files will be quite large and can be safely removed ] . Also check the  archive directory in this directory

Check whether trace is enabled in your system  [ switch off in ST01  and also any tracing for work process in SM50 ]

Thanks ,

Manu

[Former Member]

Hi,

This file system increase is in Application server  (app1).

in st01, this trace is switch off only already

THERE IS NO CORE FILE present in app1 server but core file present in CI server.

the default trace file are getting increased in the path  /usr/sap/<SID>/D11/j2ee/cluster/server0

in the default trace file we could see below error

*************************************************

Error> occured  >Thu Jan 22 08:31:47,833<

>RfcIoException:

    message: CPIC-CALL: SAP_CMACCPTP3 on convId: 00000000

LOCATION    SAP-Gateway on host zerpfbeci / sapgw01

ERROR       registration of tp ZERPFBECI_PORTAL_FBE from host

            sblzfbeap1.SSS.SSSSSS.com not allowed

TIME        Thu Jan 22 08:31:47 201

RELEASE     701

COMPONENT   SAP-Gateway

VERSION     2

RC          720

MODULE      gwxxrd.c

LINE        3736

COUNTER     268194876

Return code: RFCIO_ERROR_SYSERROR(5)<

*********************************************************

can you please tel where is the secinfo file. maintained and where to check

regards,

gayathri.K

[Former Member]

Hello Gayathri,

You can delete the core files under work directory. These are created when a WP is terminated. Check the following notes for more info:

39261 - Core file is too large

16513 - File system full - what can you do?

Go through the below note, regarding stat files

6833 - Deleting statistics file, size of statistics file

regards,

pavan

[manumohandas82]

Hi Gayathri ,

Have you enabled logging for Java . Check the below link . You can  reduce or disable the tracing from visual admin

Logging and Tracing - Identity Management - SAP Library

What is the size of these default traces . ? 

You may remove the old default traces if you need to .

Thanks ,

Manu

[former_member182657]

Hi Gayathri,

Follow SCN doc at

and SCN thread

Also check for enabled developer traces,try to reset traces from SM50 or disable and enable only if required.

Regards,

[Former Member]

Hello,

it is a good question which needs more investigation. By the way, which files are you deleting, when you are trying to reduce the size? And what is the size of your /usr/sap/<SID> location?

From my point of view there can be two subfolders, which can increase its size continuously:

     1) /usr/sap/<SID>/<instance>/data

               Here are stat* files stored and in default they are stored for 48 hours and then deleted. If your filesystem is not large enough to store 48 files of these, then you should increase size of filesystem.

     2) /usr/sap/<SID>/<instance>/log

               Here are audit_<date> files stored continuusly. You can SAPCAR the older files or delete them, if you think you don't need them anymore.

Kind regards