on 07-15-2013 10:26 PM
Hi Experts,
I have been facing encoding issue while using SOAP Adapter. The scenario is a synchronous one from Proxy <--> SAP PI <--> SOAP.
1. I have not used any Java mapping and it is normal one to one mapping, so the encoding should be UTF-8
2. I have used ISO-8859-1 encoding in the adapter engine which has worked, but still on the target webserver, the data seems to be corrupted and could see '?' symbols.
3. I have gone through various SAP Forums links, but couldnt find suitable exact answer to drill down the issue.
Does it has something to do with the encoding setting in SAP ECC System?
Looking forward to hear from you.
Regards,
N. Jayanth Kumar.
Please check with data fields any weird characters coming from the sender system. UTF 8 is right encoding type. You normally dont need to change. Are you getting any data like foreign characters or image or so? If so, then correct it in the sender system or need the appropriate encoding type to send. Send the same data via soapUI and see how it works
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Bhaskar,
When tested from SOAP UI Client, the target webserver when viewed, it has the characters rightly appearing. Below are characters we have been using.
Radox „2In1” Dušas Želeja Un Šampūns Vīriešiem Εε ΖζΜμΠπΣσΤτЛлščфûâΞξ,ΧχΣσΘθגÔô • Ʒʒ • ǮǯŊŋ• ÑñDždžǼǽ • ǢǣĤĥ Ľľ • Ȑȑ •й йд дÖö - ק20130811
ΣτοματικοΔSæ åΠερПастаъбиý žÉψç㚎eŻőá šľΘρεψηςłosóŠ æä،о®SHY§¥‰ ÇƉ¶€0712A - 20130811
Even when we pushed from PI RWB manually, we are getting response that the file has been accpeted, but still there are '?' mark symbols.
Is it something where the encoding has to be set in ECC System as well? Request you to please help.
Radox „2In1” Dušas Želeja Un Šampūns Vīriešiem Εε ΖζΜμΠπΣσΤτЛлščфûâΞξ,ΧχΣσΘθגÔô • Ʒʒ • ǮǯŊŋ• ÑñDždžǼǽ • ǢǣĤĥ Ľľ • Ȑȑ •й йд дÖö - ק20130811
ΣτοματικοΔSæ åΠερПастаъбиý žÉψç㚎eŻőá šľΘρεψηςłosóŠ æä،о®SHY§¥‰ ÇƉ¶€0712A - 20130811
What do you expect, when you use characters, which are not part of ISO-8859-1?
Please check wikipedia first!
Hi Stefan,
Yes I have used the UTF-8 initially and I didnt use any other character set coding. When I pushed the message from SOAP UI Client, the characters appear correctly on target system with successful response coming back.
When we pushed the same from ECC, we are getting successful response back, but characters on target system Portal doesnt seem to appear correctly. There are '?' symbols.
We are connecting to target system via SOAP Receiver which has a proxy configured with a specific host and port. Does it have any difference?
More over, the characters from the Integration engine also seem to appear correct when extracted. Not sure where the exact issue lies in. Can you please help us here Stefan.
Regards,
Jayanth Kumar
Hi Jayanth,
If you want to change the encoding from UTF-8 to ISO-8859-1, here is the excerpt from
http://www.stechno.net/sap-notes.html?view=sapnote&id=856597
A: The SOAP receiver adapter can use any character encoding supported by the local JDK. The request message from the SOAP receiver is normally encoded in UTF-8. If you want to change this encoding, for instance to iso-8859-1, you can set parameter XMBWS.XMLEncoding to iso-8859-1 in the module configuration for the SOAP adapter module. This setting is for the outgoing SOAP message and has no effect on the incoming SOAP message. For the incoming SOAP message, any code page supported by the local JDK is accepted.
Worth a try....
Hope it helps!
Ambrish
In the beginning of the thread you said:
" I have used ISO-8859-1 encoding in the adapter engine which has worked, but still on the target webserver, the data seems to be corrupted and could see '?' symbols."
Now you say:
"Yes I have used the UTF-8 initially and I didnt use any other character set coding."
Maybe you can describe the whole scenario more precisely.
Can you figure out which characters are represented as "?" in the target? The greek, cyrillic or any others?
Hi Stefan,
Sure, I will explain the scenario more precisely. The flow is a synchronous flow as below.
SAP ECC (Proxy) <--> PI <--> Target system (SOAP).
We are connecting to target system via SOAP Receiver which has a proxy configured with a specific host and port. Does it really have any difference in handling special characters?
Should the WSDL File also have the "UTF-8" encoding when we upload into PI system?
This is a simple one to one mapping 'without' any specific encodings that have been set. So usually it should be UTF-8
When the special characters are being triggered from ECC which goes via PI, the target server gives us error response because of the special characters in the file which it is unable to recognize.
I have put all remaining characters aside and tested with below special characters that were triggered to see if it works: I think there are cyrillic characters as well in the input file.
ΣτοματικοΔSæ åΠερПастаъбиý žÉψç㚎eŻőá šľΘρεψηςłosóŠ
When the response comes back to ECC from target system, it has below errors that the request has been rejected due to the below.
ΣÄομαÄικοΔSæ åΠεÃÂßðÑÂтðъñøý žÉÈç㚎eŻőá šľΘÃÂεÈηÂłosóÅÂ
When I open the error XML in taken from PI the above characters appear.
I have added then ISO-8859-1 seeburger Bic module in SOAP Receiver channel and it didnt throw error response, but the characters on the target portal were appearing as '?' symbols.
When we push the same characters from SOAP UI Client using UTF-8, it works correctly and the characters appear rightly on portal with successful response back.
I am not sure where is the exact issue. I have also tried to do below things.
1. Used Java mapping to return the data in UTF-8 encoding as below but not success.
private final String utf ="UTF-8";
byte[] buffer = new byte[in.available()];
int bblen = in.read(buffer);
String str = new String(buffer, utf
System.out.println(str);
out.write(str.getBytes(utf));
out.close();
2. I have also used various modules anonymizer bean, XMBWS.XMLEncoding to UTF-8, checked the check box to keep encoded headers, but still no response.
3. The ABAP Connection has the unicode settings already.
4. There is another inbound interface from same webserver SOAP --> PI --> Proxy which successfully handles special characters in PI and they appear correctly in ECC.
Is it something needs to be done in ECC while sending the data to PI, must be encoded in UTF-8 in ABAP Code? When it works for inbound without anything, not sure why is it not working for outbound. Request you to please help me here.
Regards,
N. Jayanth Kumar.
Hi Bhaskar,
When pushed from SOAP UI Client the characters appear correctly on target system portal. When pushed from ECC, they dont appear correctly. The source is an ECC system via proxy connectivity. Does the ABAP code needs to be amended to send everything to PI in UTF-8 format.
We are connecting to target system via SOAP Receiver which has a proxy configured with a specific host and port. Does it really have any difference in handling special characters?
Regards,
N. Jayanth Kumar.
Hi Stefan,
I could see the characters correctly appearing in sxmb_moni of PI system. The response message simply says if it is accepted or rejected. We are directing the message to Target system using a SOAP URL as well as by configuring a proxy additionally. Will that cause a difference Stefan?
when I dont use ISO-8859-1 the messages completely gets rejected
The ECC system is a unicode system as per below screenshot.
Hi Stefan,
The scope which we are implementing is for entire Europe which should cover for all special characters is what had been informed to us. Target webservice provider has informed that they would need UTF-8. The strange thing here is, when we push a message from SOAP UI Client with all the characters it works well. But when pushed from PI, it throws error on target system
Is it something that needs to be done in ECC side Stefan.
I will find out what are the exact characters that are required here.
Regards,
N. Jayanth Kumar.
ΣÄομαÄικοΔSæ åΠεÃÂßðÑÂтðъñøý žÉÈç㚎eŻőá šľΘÃÂεÈηÂłosóÅÂ
When I open the error XML in taken from PI the above characters appear.
How do you open this? In fact that are UTF-8 characters displayed as ISO-8859-1.
What just came into my mind: How do you generate test data in ECC? Do you type the data yourself or do they come from the database?
Hi Stefan,
I have included the mapping and included the below module in SOAP Receiver channel.
localejbs/AF_Modules/MessageTransformBean Local Enterprise Bean transform
transform Transform.ContentType text/xml;charset=utf-8
It has worked for below characters
Hi Stefan,
I had removed the java mapping and module and tested which has thrown errors already.
private final String utf ="UTF-8";
byte[] buffer = new byte[in.available()];
int bblen = in.read(buffer);
String str = new String(buffer, utf
System.out.println(str);
out.write(str.getBytes(utf));
out.close();
The Java mapping with latest Messagetransform Bean module is giving success for the special characters. I will make tests below and will let you know.
1. Remove Java mapping and modules and test again
2. Remove Java mapping and include MessageTransformBean and test.
On the other side, I had made some tests from SOAP UI Client with UTF-8 encoding set, where I had pushed the characters where no PI or ECC would be involved.
ΖζΜμΠπΣσΤ τЛлščфûâ Ξξ,ΧχΣσΘθגÔô
The target system has thrown errors from SOAP UI client as well, which says there is an issue on Target application?
Regards,
Jayanth Kumar.
Hi Stefan,
There are unwanted characters that are not in scope like vietnamese also there were really some spaces that were added in ECC and when they removed the spaces and sent the messages it worked. I have removed the Java mapping and included the Message transform bean which worked. I will keep you updated if we still face any errors. Thanks a lot for your support Stefan. Your inputs have been really very useful.
Regards,
Jayanth Kumar.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Jayanth,
As i understand, the words coming as part of your payload are latvian and these characters are not supported by ISO-8859-1. You might need to refer to specific code page for baltic language (Cp1257).
Please try using Cp1257 encoding in your SOAP sender channel and lets see if it works.
Regards,
Anurag
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Jayanth - Not sure you got solution to issue.
Below Module configuration In the SOAP receiver adapter worked for me. I was in the same situation.
1 | sap.com/com.sap.aii.af.soapadapter/XISOAPAdapterBean | Local Enterprise Bean | soap |
2 | localejbs/AF_Modules/MessageTransformBean | Local Enterprise Bean | transform |
3 | AF_Modules/TextCodepageConversionBean | Local Enterprise Bean | convert |
convert | Conversion.charset | UTF-8 |
transform | Transform.ContentType | text/xml |
Thanks.
Hi All,
I made changes in the Java code as below
private final String utf ="UTF-8";
byte[] buffer = new byte[in.available()];
int bblen = in.read(buffer);
String str = new String(buffer, utf);
//str = str.replaceAll(utf, latin);
System.out.println(str);
out.write(str.getBytes(utf));
out.close();
and pushed the data to target system without any encoding in Adapter level in any of module configuration, but I still get to see errors in response that the target system is unable to recognize the special characters. Can you please help
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Inaki,
For confirmation purposes, I had pushed the data from PI RWB directly and still the response threw errors. In parallel can you pelase help m ein how do we see the encoding that has been set in ECC whilst the data is being triggered.
Regards,
Jayanth
Message was edited by: Jayanth Kumar
Hi,
If you are using an ABAP proxy to send the message, go to sxmb_moni in the ECC system. With abap proxy the connection is direct between integration engines (dual PI system), with RWB also uses http protocol, im not sure, but the test is a bit different for the real one. I mean, even if you get a right answer with a java mapping or xsl mapping, you should check if the data is coming rigtht to PI from ECC.
Regards,
Hi Inaki,
When I saw the special characters sxmb_moni from inbound message in PI, the characters look o, strangely they are throwing errors yet
The special characters appear as below in PI.
ΣτοματικοΔSæ åΠερПастаъбиý žÉψç㚎eŻőá šľΘρεψηςłosóŠ
and when it seem to reach the target system via SOAP Receiver channel (I havent not used any module as it is UTF-8, I have used only to convert the content in java mapping in UTF-8)
they appear to be as below when we got the response back with errors as below.
ΣÄομαÄικοΔSæ åΠεÃÂÂÂßðÑÂÂтðъñøý žÉÈç㚎eŻőá šľΘÃÂÂÂεÈηÂłosóÅÂÂ
Hi,
If i understand you right when the message come to PI looks right. Have you tested to use trace.addInfo instruction before and and after of the treatment in the java mapping?, it's to check the rightness in that step. May be the problem is in the SOAP adapter, try to put Dont use SOAP Envelope and you make with an XSL or the same java mapping the SOAP envelope.
Regards,
Hi Jayanth,
The input to this java mapping
private final String utf ="UTF-8";
byte[] buffer = new byte[in.available()];
int bblen = in.read(buffer);
String str = new String(buffer, utf);
//str = str.replaceAll(utf, latin);
System.out.println(str);
out.write(str.getBytes(utf));
out.close();
might not be in correct encoding. You have mentioned encoding scheme as "UTF-8" (line 01) and reading the input in the same encoding (line 04).
I would suggest to change the value of the variable "utf" to some other encoding(if this does not work use other encoding too) as shown below
private final String utf ="ISO-8859-1";
byte[] buffer = new byte[in.available()];
int bblen = in.read(buffer);
String str = new String(buffer, utf);
str = str.replaceAll(utf, "UTF-8");
System.out.println(str);
out.write(str.getBytes("UTF-8"));
out.close();
Regards
Anupam
Hi Jayanth,
If you have read this document http://scn.sap.com/docs/DOC-16138 and you still get the problem you can try with this note Note 945570 - PI Sheet Input characters in Unicode system changed to #
Regards.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Inaki,
I have gone through the Note 94550. It has a comment as below.
The Internet Explorer encodes POST data in accordance with HTML page
encoding. The following article from Microsoft explains this:
http://support.microsoft.com/default.aspx?scid=kb;en-us;303612
In the current case the PI Sheet encoding is UTF-16. But as described in
the article the POST data is sent in UTF-8.
Is it something which we can force to use "UTF-8" by using a java mapping while returning the output file as below?
finalOutput.getBytes("UTF-8")
where the finalOutput is a string which has the data.
Hi Jayanth,
I think a java mapping wouldn't resolve your problem because the data could be reaching to this point corrupted. Have you checked in the ECC that the data is outgoing correctly?, check the sxmb_monitor in the ECC. Check how the XML is coming to PI in the first pipeline step, if the data is corrupted, you can't resolve anything with a java mapping, if the data is correct, you can try with a java mapping or xsl transformation.
<xsl:output method="xml" encoding="UTF-8"/>
Regard.
Hi Jayanth,
You can convert entire payload to UTF-8 using java mapping. Please check this wonderful "how to" guide http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/502991a2-45d9-2910-d99f-8aba5d79f... by Stefan Grube . Additionally you can check this thread https://scn.sap.com/thread/2068827
Regards
Anupam
Hi Anupam,
I am using the below code to return the output in UTF-8
public void execute(InputStream inputStream, OutputStream outputStream)
throws StreamTransformationException {
// TODO Auto-generated method stub
try {
// Initializing Trace
docInput = new BufferedReader(new InputStreamReader(inputStream));
} catch (Exception e) {
throw new StreamTransformationException("Unable to parse input document - ".concat(e.getMessage()));
}
try {
while ((inputLine = docInput.readLine()) != null) {
inputData.append(inputLine);
}
finalOutput = inputData.toString();
outputStream.write(finalOutput.getBytes("UTF-8"));
Hi Anupam,
I have used the below code in simple.
private final String utf ="UTF-8";
byte[] buffer = new byte[in.available()];
int bblen = in.read(buffer);
String str = new String(buffer, utf);
//str = str.replaceAll(utf, latin);
System.out.println(str);
out.write(str.getBytes(utf));
out.close();
User | Count |
---|---|
87 | |
10 | |
10 | |
10 | |
7 | |
6 | |
6 | |
5 | |
5 | |
4 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.