cancel
Showing results for 
Search instead for 
Did you mean: 

Ruby and Unicode

Former Member
0 Kudos

Hi Piers, Hi Everybody,

because of the better performance we continue the discussion startet in the comments to the Weblog: <a href="/people/piers.harding/blog/2006/05/18/ruby-on-rails-with-ajax">Ruby on Rails with AJAX</a> here.

To reduce possible influences I now use the test.rb script coming with the Ruby SAPRFC Lib avaliable at http://www.piersharding.com/download/ruby/. I've used the latest version saprfc-0.19.tar.gz. I added the Lines

                   :codepage => "4103",
                   :unicode => 1,

to the call of SAP::Rfc.new. The result when I call the script with "ruby test.rb" is:

GROUP    104     KEY     C       MESSAGE         F
RFC Call/Exception: Connection Failed   Error group: 104        Key: C  Message: F

the file dev_rfc.trc contains:

**** ERROR file opened at 20060605 212258 CEST, SAP-REL 620,0,1909 RFC-VER 3 748231 MT-SL
T:1076356192 ======> F
T:1076356192 <* RfcReceive [1] : returns 3:RFC_SYS_EXCEPTION

in rfc31072_1076356192.trc i see:


**** ERROR file opened at 20060605 212258 CEST, SAP-REL 620,0,1909 RFC-VER 3 748231 MT-SL
T:1076356192 ======> F
T:1076356192 <* RfcReceive [1] : returns 3:RFC_SYS_EXCEPTION
sarge-www:/usr/local/src/saprfc-0.19/test# more rfc31072_1076356192.trc

**** Trace file opened at 20060605 212258 CEST, SAP-REL 620,0,1909 RFC-VER 3 748231 MT-SL
resize I/O buffer to 16000 bytes
>>>> [1] <unknown>    : EXT  <ac: 1> sapserver.test.com >>> OPEN
Instrument: ab_drvstate create uuid {44848493-8492-7960-E200-81FB7B3B1509}
>>>> [1] <unknown>    : EXT  <ac: 2> sapserver.test.com >>> 39277992 (39277992)
  -{44848493-8492-7960-E200-81FB7B3B1509}


>>> RfcOpenEx ...
 Got following connect_param string:
   ASHOST=sapserver.test.com SYSNR=00 LANG=EN CLIENT=002 
	USER=developer PASSWD=******* TRACE=1 CODEPAGE=4103
Send RFCHEADER: 01/LIT/IEEE/SPACE/4103
Send UNICODE-RFCHEADER: cp:4103/ce:IGNORE/te:REJECT/cs:1/rc:0x00000023
Instrument: send_rfcuuid {44848493-8492-7960-E200-81FB7B3B1509}

 >>> Logon check: calling RFCPING

>>> RfcCall [1] ...
*> RfcCall
  FUNCTION RFCPING
        handle = 1
        parameter   = <NULL>
        tables      = <NULL>
Instrument:  RfcCallNew send the uuid to the partner 
{44848493-8492-7960-E200-81FB7B3B1509}
>>>> [1] <unknown>    : EXT  <ac: 3> sapserver.test.com >>> WRITE (39277992)
  -{44848493-8492-7960-E200-81FB7B3B1509}
000000 | D9C6C3F0 F0F0F0F0 F0F0F0E3 01010008 |................|
000010 | 01010101 04010003 01010103 00040000 |................|
000020 | 020B0103 0106000B 04010003 01000100 |................|
000030 | 00002301 06000700 0F313237 2E302E30 |..#......127.0.0|
000040 | 2E312020 20202020 00070011 00014500 |.1      ......E.|
000050 | 11001200 04363230 20001200 13000436 |.....620 ......6|
000060 | 32302000 13000800 20736172 67652D77 |20 ..... sarge-w|
000070 | 77772020 20202020 20202020 20202020 |ww              |
000080 | 20202020 20202020 20000800 0600803C |         ......<|
000090 | 756E6B6E 6F776E3E 00000000 00000000 |unknown>........|
0000a0 | 00000000 00000000 00000000 00000000 |................|
0000b0 | 00000000 00000000 00000000 00000000 |................|
0000c0 | 00000000 00000000 00000000 00000000 |................|
0000d0 | 00000000 00000000 00000000 00000000 |................|
0000e0 | 00000000 00000000 00000000 00000000 |................|
0000f0 | 00000000 00000000 00000000 00000000 |................|
000100 | 00000000 00000000 00000000 00000000 |................|
000110 | 06051400 10938484 44928460 79E20081 |........D..`y...|
000120 | FB7B3B15 09051401 30000472 75627901 |.{;.....0..ruby.|
000130 | 30011100 09444556 454C4F50 45520111 |0....DEVELOPER..|
000140 | 0117000C A1554E98 574715B2 D5179A0C |.....UN.WG......|
000150 | 01170114 00033030 32011401 15000145 |......002......E|
000160 | 01150501 00010105 01050200 00050200 |................|
000170 | 0B000336 3230000B 01020007 52464350 |...620......RFCP|
000180 | 494E4701 02051400 10938484 44928460 |ING.........D..`|
000190 | 79E20081 FB7B3B15 090514FF FF0000FF |y....{;.........|
0001a0 | FF000000 00000000 00000000 00000000 |................|
>>>> [1] <unknown>    : EXT  <ac: 4> sapserver.test.com >>> FLUSH(WRITE) (39277992)
  -{44848493-8492-7960-E200-81FB7B3B1509}
<* RfcCall [1] : returns 0:RFC_OK
>>>> [1] <unknown>    : EXT  <ac: 5> sapserver.test.com >>> FLUSH(WRITE) (39277992)
  -{44848493-8492-7960-E200-81FB7B3B1509}

>>>> Listen/WaitForRequest (counter = 5)
>>>> [1] <unknown>    : EXT  <ac: 6> sapserver.test.com >>> LISTEN (39277992)
  -{44848493-8492-7960-E200-81FB7B3B1509}
000000 | 01010008 01010101 04010003 01010103 |................|
000010 | 00040000 020B0103 0106000B 04010003 |................|
000020 | 01000200 00002301 06001600 08310031 |......#......1.1|
000030 | 00300030 00001600 07001E31 0030002E |.0.0.......1.0..|
000040 | 00310032 0038002E 00330036 002E0031 |.1.2.8...3.6...1|
000050 | 00350020 00200020 00000700 11000233 |.5. . . .......3|
000060 | 00001100 12000836 00320030 00200000 |.......6.2.0. ..|
000070 | 12001300 08360032 00300020 00001300 |.....6.2.0. ....|
000080 | 08004064 00650063 00720030 0034005F |..@d.e.c.r.0.4._|
000090 | 00430052 0034005F 00300030 00200020 |.C.R.4._.0.0. . |
0000a0 | 00200020 00200020 00200020 00200020 |. . . . . . . . |
0000b0 | 00200020 00200020 00200020 00200020 |. . . . . . . . |
0000c0 | 00200000 08000601 003C756E 6B6E6F77 |. .......<unknow|
0000d0 | 6E3E0000 00000000 00000000 00000000 |n>..............|
0000e0 | 00000000 00000000 00000000 00000000 |................|
0000f0 | 00000000 00000000 00000000 00000000 |................|
000100 | 00000000 00000000 00000000 00000000 |................|
000110 | 00000000 00000000 00000000 00000000 |................|
000120 | 00000000 00000000 00000000 00000000 |................|
000130 | 00000000 00000000 00000000 00000000 |................|
000140 | 00000000 00000000 00000000 00000000 |................|
000150 | 00000000 00000000 00000000 00000000 |................|
000160 | 00000000 00000000 00000000 00000000 |................|
000170 | 00000000 00000000 00000000 00000000 |................|
000180 | 00000000 00000000 00000000 00000000 |................|
000190 | 00000000 00000000 00000000 00000000 |................|
0001a0 | 00000000 00000000 00000000 00000000 |................|
0001b0 | 00000000 00000000 00000000 00000000 |................|
0001c0 | 00000000 00000000 00000605 14001093 |................|
0001d0 | 84844492 846079E2 0081FB7B 3B150905 |..D..`y....{;...|
0001e0 | 14050000 00050004 03002E43 0041004C |...........C.A.L|
0001f0 | 004C005F 00460055 004E0043 00540049 |.L._.F.U.N.C.T.I|
000200 | 004F004E 005F004E 004F0054 005F0046 |.O.N._.N.O.T._.F|
000210 | 004F0055 004E0044 00040304 02004046 |.O.U.N.D......@F|
000220 | 0075006E 00630074 0069006F 006E0020 |.u.n.c.t.i.o.n. |
000230 | 006D006F 00640075 006C0065 00200022 |.m.o.d.u.l.e. ."|
000240 | 00524643 50494E22 0020006E 006F0074 |.RFCPIN". .n.o.t|
000250 | 00200066 006F0075 006E0064 002E0004 |. .f.o.u.n.d....|
000260 | 02FFFF00 00FFFF00 00000000 00000000 |................|
<* RfcListen [1] : returns 0:RFC_OK

>>> RfcReceive [1] ...
Received RFCHEADER: 01/LIT/IEEE/SPACE/4103
Received UNICODE-RFCHEADER: cp:4103/ce:IGNORE/te:REJECT/cs:2/rc:0x00000023
Instrument: ab_rfccheck_uuid compare uuid's {44848493-8492-7960-E200-81FB7B3B1509}
======> F
>>>> [1] <unknown>    : EXT  <ac: 7> sapserver.test.com >>> 
	CLOSE abrfcrcv_mt.c 400 (39277992)
  -{44848493-8492-7960-E200-81FB7B3B1509}
*> RfcReceive
        handle = 1
        parameter   = <NULL>
        tables      = <NULL>
         Data conversion On

<* RfcReceive [1] : returns 3:RFC_SYS_EXCEPTION
>>>> [1] <unknown>    : EXT  <ac: 8> sapserver.test.com >>> 
	FREE abrfcio_mt.c 3245 (39277992)
  -{44848493-8492-7960-E200-81FB7B3B1509}

**** Trace file opened at 20060605 212258 CEST, SAP-REL 620,0,1909 
	RFC-VER 3 748231 MT-SL
<<< RfcOpenEx failed

It looks exactly like the problem I've described in the Topic: Forum.

I have real doubts that it will work because there is a special Unicode RFC SDK avaliable. Why should it work to connect with the Non Unicode Version.

Regards

Gregor

Accepted Solutions (1)

Accepted Solutions (1)

Former Member
0 Kudos

Hi Gregor,

Looking at the trace it seems to strangely truncate the name of the function that is being called => RFCPING. This is a dummy function that you can see in SE37, used purely to test connectivity.

I've had a look at the RFCSDK documentation again, and there is specific talk about compiling for a UNICODE environment => eg. you must use librfccm.so, and -DSAPwithUNICODE is supposed to be added to CFLAGS. Also, there is considerable evidence that there are a lot of C helper functions that are supposed to be used to ensure that UNICODE data is preserved when being moved arround - this probably needs to be done to any RFC adapter.

The problem I have is that I do not have access to a UNICODE system so I cant do any build testing to sort this out.

So - if anyone out there has a system that they can give me access to - then I'll do the work.

Cheers.

gregorw
Active Contributor
0 Kudos

Hi Piers,

I've compiled now with Option -DSAPwithUNICODE when I run the test.rb I get this error:

ruby: relocation error: 
/usr/local/lib/site_ruby/1.8/i386-linux/saprfc.so: undefined
symbol: RfcEnvironmentU

Hope it helps. Unfortenately I'm not a C programmer.

Regards

Gregor

Former Member
0 Kudos

OK - thats a bit of progress because it must be doing something different. I think that this error is because the librfc library you have is not the UNICODE one. If you look at note 413708 - it describes where to get the RFC library from. The library you need according to this is librfcum.so (not librfccm.so as I said previously).

Cheers,

Piers Harding.

gregorw
Active Contributor
0 Kudos

Hello Piers,

I've now found the time to try out the ruby saprfc-0.20 with Unicode support. Unfortunately during compillation I got this error when I run "ruby setup.rb setup":

sarge-www:/usr/local/src/saprfc-0.20# ruby setup.rb setup

---> lib

---> lib/SAP

<--- lib/SAP

<--- lib

---> ext

---> ext/saprfc

make

gcc -fPIC -Wall -g -O2 -fPIC -I. -I/usr/lib/ruby/1.8/i386-linux -I/usr/lib/ruby/1.8/i386-linux -I/usr/local/src/saprfc-0.20/ext/saprfc -DHAVE_SAPRFC_H -DHAVE_SAPITAB_H -I/usr/local/src/rfcsdk-640-unicode/include -c saprfc.c

saprfc.c: In function `SAPRfc_getStructure':

saprfc.c:961: warning: unused variable `a_index'

saprfc.c:962: warning: unused variable `tab_index'

saprfc.c:964: warning: unused variable `i'

saprfc.c:965: warning: unused variable `j'

saprfc.c: In function `SAPRfc_installStructure':

saprfc.c:1106: warning: unused variable `irow'

saprfc.c:1103: warning: `exception' might be used uninitialized in this function

saprfc.c: In function `SAPRfc_Register':

saprfc.c:1695: warning: unused variable `funcname'

saprfc.c:1696: warning: unused variable `iface'

saprfc.c: In function `SAPRfc_Process':

saprfc.c:1736: warning: unused variable `env'

gcc -shared -L/usr/sap/rfcsdk/lib -lrfc -lrfc -L"/usr/local/src/rfcsdk-640-unicode/lib" -L"/usr/lib" -o saprfc.so saprfc.o -lruby1.8 -lstdc++ -lpthread -ldl -lm -lpthread -ldl -lcrypt -lm -lc

/usr/bin/ld: cannot find -lrfc

collect2: ld returned 1 exit status

make: *** [saprfc.so] Error 1

setup.rb:658:in `command': system("make") failed (RuntimeError)

from setup.rb:667:in `make'

from setup.rb:1299:in `setup_dir_ext'

from setup.rb:1573:in `__send__'

from setup.rb:1573:in `traverse'

from setup.rb:1571:in `dive_into'

from setup.rb:1571:in `traverse'

from setup.rb:1575:in `traverse'

from setup.rb:1574:in `each'

... 8 levels...

from setup.rb:829:in `__send__'

from setup.rb:829:in `invoke'

from setup.rb:776:in `invoke'

from setup.rb:1619

I've tried to run in ext/saprfc:

gcc -shared -L/usr/sap/rfcsdk/lib -lrfc -lrfc -L"/usr/local/src/rfcsdk-640-unicode/lib" -L"/usr/lib" -o saprfc.so saprfc.o -lruby1.8 -lstdc++ -lpthread -ldl -lm -lpthread -ldl -lcrypt -lm -lc

but it returned "collect2: ld returned 1 exit status". So I've added a -v to the gcc and got a message "/usr/bin/ld: cannot find -lrfc". I've deleted the two "-lrfc -lrfc" and it compiled finaly.

When I try now to run the test script test.rb I get this error:

sarge-www:/usr/local/src/saprfc-0.20# ruby test/test.rb

ruby: relocation error: /usr/local/lib/site_ruby/1.8/i386-linux/saprfc.so: undefined symbol: RfcInstallStructure2

Regards

Gregor

Former Member
0 Kudos

Hi Gregor - I've modified the README file (not published yet) - you need to modify the default build process as per this:

== Unicode

In general Unicode is an evil that should be eliminated from the universe for the greater good of man kind....

but seeing that we are stuck with it, I have attempted to marry saprfc, librcu* and ruby together, as best I can.

Ruby at best has support for UTF-8, which at every stage needs to be translated into UTF-16 for SAP's unicode implementation.

You need to consider this carefully for what ever you try and do - you MUST work in UTF-8!!!

TO get this working you must compile saprfc against librcu* and libsapu16*. - get these from the usual instructions found at http://service.sap.com/connectors.

Additionally - you will most likely be on Linux, and therefore require a program from SAP called u16lit.pl. This must be retrieved from OSS note 763741, and placed in the root directory of the unpacked source distribution of saprfc eg. if you get saprfc-0.20.tar.gz => unpack and place in the directory saprfc-0.20 - this is where the compilation process will look for this program (from extconf.rb).

build with:

ruby setup.rb clean

ruby setup.rb config --with-rfcsdk-dir=/path/to/rfcsdk --with-unicode=yes

ruby setup.rb setup

Let me know how you get on - I suspect that there is so more debugging to be done.

Cheers.

gregorw
Active Contributor
0 Kudos

Hi Piers,

with that tip the compilation worked well. When I've starded the test.rb it worked well the first time till it aborted with an error. In ST22 I see:

An exception occurred. This exception is dealt with in more detail below

. The exception, which is assigned to the class 'CX_SY_DYNAMIC_OSQL_SEMANTICS',

was neither

caught nor passed along using a RAISING clause, in the procedure

"RFC_READ_TABLE" "(FUNCTION)"

.

Since the caller of the procedure could not have expected this exception

to occur, the running program was terminated.

The reason for the exception is:

The field name "&#16718;&#17741;&#19488;&#19273;&#8261;&#21287;&#20545;&#9548;&#18002;&#9539;&#8231;††††††††††††††††††††††††††††††††††††††††††††††††††††

†††††††††" contains more than 30 characters which is too

long. If the field name was not specified until runtime in an internal

table, there is an ABAP/4 programming error.

If the field name was specified statically in the program, there is

an internal error.

The error suggests that the length defined for the field name

in the ABAP/4 Dictionary is different from that in the database

interface.

When I now tried to run the testscript again this error occurs:

connect id: 1

is_connected: true

sapinfo: {"RFCIPADDR"=>"10.128.36.15 ", "RFCMACH"=>" 560", "RFCDBHOST"=>"DECR04 ", "RFCDATABS"=>"CR4 ", "RFCDBSYS"=>"ORACLE ", "RFCINTTYP"=>"LIT", "RFCSI_RESV"=>" ", "RFCHOST2"=>"decr04 ", "RFCHOST"=>"decr04 ", "RFCTZONE"=>" 3600", "RFCSAPRL"=>"620 ", "RFCFLOTYP"=>"IE3", "RFCDAYST"=>"X", "RFCOPSYS"=>"Windows NT", "RFCSYSID"=>"CR4 ", "RFCPROTO"=>"011", "RFCKERNRL"=>"620 ", "RFCDEST"=>"decr04_CR4_00 ", "RFCCHARTYP"=>"4103"}

RFC Call/Exception: RFCTYPE_UNKNOWN_HANDLE

Group Error group 104

Key RFC_ERROR_SYSTEM_FAILURE

Message See RFC trace file or SAP system log for more details

RFC Call/Exception: RFCTYPE_UNKNOWN_HANDLE Error group: 104 Key: RFC_ERROR_SYSTEM_FAILURE Message: See RFC trace file or SAP system log for more details

I will send you the trace files via personal mail.

Regards

Gregor

Former Member
0 Kudos

Hi Gregor,

It looks like everything compiles correctly, and that the first few calls are negotiated correctly. It seems that after that for the funciton module RFC_READ_REPORT it is not installing the structure correctly, as it doesn't like the UTF16 description of it.

The change in behaviour between the first and second time that you run test.rb is because of this line:

SAP::Rfc.useCache = true

If you remove this, and remove the directory that has been created called .rfc_cache then the program will revert to looking up the interface description everytime.

Really - I need to get proper access to a UNICODE system so that I can debug this properly.

Cheers.

gregorw
Active Contributor
0 Kudos

Hi Piers,

we are one step further. I can run the program now like at the first start. Unfortenately the Line with Unicode Characters is displayed as:

& Test Unicodezeichen: Deutsch: 303266303244303274303226303204303234303237 Slowenisch: 304215304207

305276305241304221304214

the original was:

*& Test Unicodezeichen: Deutsch: öäüÖÄÜß Slowenisch: &#269;&#263;žš&#273;&#268;&#262;ŽŠ&#272;

but the error:

./lib/SAP/Rfc.rb:923:in `call': CALL Error: EXCEPT SYSTEM_FAILURE GROUP 104 KEY RFC_ERROR_SYSTEM_FAILURE MESSAGE Field name in SQL statement is too long. (RuntimeError)

from test/test-siteco.rb:60

still occurs.

I've sent you the trace files. I talk to my Basis Colleagues if I can get a VPN Connection for you. I hope you can use the Cisco VPN Client on your Linux machine?

Regards

Gregor

gregorw
Active Contributor
0 Kudos

Hello,

it seems that the SDN Forums are not rearly UNICODE aware :-). Let's try something in Japan: &#24180;&#24230;&#31532;2&#22235;&#21322;&#26399;&#26989;&#32318;&#27010;&#35201;&#12398;&#12362;&#30693;&#12425;&#12379;.

Regards

Gregor

gregorw
Active Contributor
0 Kudos

Hi Piers,

with the new test.rb Version it's working until:

./lib/SAP/Rfc.rb:1896: [BUG] Segmentation fault

ruby 1.8.2 (2005-04-11) [i386-linux]

Aborted

Regards

Gregor

Former Member
0 Kudos

After a long while, we (Gregor and I ) have done a lot of work on reengineering the RFC connectors for Ruby and Perl.

The new Ruby conector that can be compiled for unicode support is available at: http://raa.ruby-lang.org/project/saprfc/

and the new version for Perl is at: http://search.cpan.org/~piers/SAP-Rfc-1.45/

Enjoy!

former_member583013
Active Contributor
0 Kudos

Congratulations to you both! Piers and Gregor, you just show how the community members can work to reach a common goal -:)

Little bit out of topic

Does anyone know the creator of the SAPRFC connector for PHP??? I'm mean...who really knows him...Because I write him a long time ago, but he never answer me, and I think that the connector had some bugs that must be fixed...The important one is being able to fill a TABLES parameters when calling a function...Not by using WebServices -;)

Little bit out of topic

Greetings,

Blag.

gregorw
Active Contributor
0 Kudos

Hello Blag,

please open a new toppic in the PHP Forum because this thread is already solved. Have you tried already this functions:

saprfc_table_init -- Init a internal table

saprfc_table_insert -- Insert a line to an internal table

saprfc_table_modify -- Modify a line of an internal table

? I think Craig also tried to contact Eduard Koucky but also never got a response.

Regards

Gregor

former_member583013
Active Contributor
0 Kudos

Thanx for pointing me that Gregor, the new post is done -;)

BTW, I have used that functions, but they doesn't work for me...

Greetings,

Blag.

Former Member
0 Kudos

Hi Gregor,

I have the same error as you - and very strange behaviour if I change my own ruby test script. Sometimes it appears, sometimes not, sometimes a "*** glibc detected *** corrupted double-linked list:" error appears.

After long and intensive debugging saprfc-0.31 on Linux, valgrind gave me a hint: It seems that the "free(uptr)" in line 1797 is too much - the pointer is already freed in line 1776. After commenting out and recompiling, the error was gone.

I'm interested if this is a solution for your problem, too.

Piers, can you commit that?

Best regards,

Thomas

Answers (0)