Extended ascii values

[Former Member]

Hi all,

I have to process ansi  .txt files which can contain extended high ascii values , a few examples

ä	132
à	133
å	134
ç	135
ê	136
ë	137
è	138
ï	139
î	140
ì	141
Ä	142

When i save some parts of this txt file , i end up having "unreadable" characters in my datawindows.

At the moment i am using FileReadEx to read the contents of my file.

Backend : SQL Anywhere 11.0.1.2713

-- Database CHAR collation: 1252LATIN1, NCHAR collation: UCA

-- Connection Character Set: windows-1252

1. I am looking ultimately to display the characters in my datawindow they way i receive them in my txt files.

2. If that can't be done, i will settle for a replace function that will search for these high values and replaces them by others. Question here is how to search for these high ascii values in a string/blob ?

TIA

John

[Former Member]

Something to note is that Unicode text files have two bytes at the beginning that indicate what format the file uses. Ansi text files do not have these two encoding bytes.

You might have to read the file into a Blob and then convert it to a string:

ls_data = String(lblob_data, EncodingAnsi!)

Then you can use ImportString to put the data in the DataWindow.

[Former Member]

Hi John,

Can you please attach a simple file with the problematic characters.
Are you using PB 12.6 ?

I am not sure but I suspect some bug when converting ansi characters to unicode.

Regards.
Abdallah.

[Former Member]

Hi John;

  Newer versions of PowerBuilder are UTF-16LE (double byte) Unicode based. That means that handing the extended ANSI character set is no problem.

  Your challenge here will be to make sure that you read the ANSI characters properly to preserve their ANSI meaning (for example, reading your external file into a Blob variable) - then, using the String() method to convert the ANSI character to an Unicode equivalent.

  The other way around this would be to save the .TXT file in Unicode format with the above extended characters already in a Unicode encoded format. Then a simple FileImport() might suffice to load your DataWindows.

HTH

Regards ... Chris

[Former Member]

There are a few things here that may lead to issues:

* Does the text file have to correct encoding in the first place. Open it in Notepad and do File | Save As, if I remember correctly the encoding is displayed here

* Are you reading it in with the correct encoding, i.e. you need to specify in FileOpen the correct encoding

* Are you displaying it in the datawindow in a font that supports display of the characters in question. I believe Arial is the standard font that displays most characters correctly

* Don't use the PowerBuilder debugger to check the data as I believe it doesn't display the data as it would be displayed in the datawindow, view in in a running application in a datawindow

I think if you get all of those correct then it should work properly.

This article may have some pointers: How to read/convert EncodingUTF8 to EncodingANSI in PowerBuilder? - Stack Overflow

PB help file on reading a file and checking it's encoding: SyBooks Online (Archive)

[Former Member]

Do you open the file with EncodingANSI! ?

Do you read data into string or blob? If you read to string Powerbuilder should convert it automatically to Unicode.

[Former Member]

Hi John

If I have understood your problem correctly.

You can use the CharA function to set a string equal to the ascii code.

ls_char = CharA(133)

Then use the PosA command to find it in a string

ll_pos = PosA(mylongstring, ls_char)

Then use the MidA function to replace the characters.

Hope that helps

David