Re: Encoding conversion problem

Hello Andrea,


Andrea wrote:
> Hi Silvio,
> the settings are taken from the DB2 instance of a customer (and I
> can't change them). The very same code works, of course, without
> problems with a DB2 instance configured with ISO-8859-1.
> 
I already expected you could not change this but it was worth
suggesting...

I do not really understand why a Euro sign would work with 8859-1 since 
that does not contain that character as far as I am aware of.

> The problem arises also when a C program stores to DB a string with
> non-IBM850 valid characters: another C program can read the string
> without problems while Java can't; so the string is not corrupted when
> saved to DB but someone (JDBC driver? Java I/O?) looses something when
> I read the field with Resultset.getString(int index) and I can't
> convert it correctly (or I haven't found the right way to do it yet,
> if it exists...).
> 

It will probably depend on how you access the DB from C (which I have 
never done with DB2) but it does not sound surprising that the C binding 
would just pass through byte sequences. As long as you stay inside 8-bit 
character encoding and always interpret them in the same codepage then 
this usually works (which is why many programmers are totally unaware of 
character encoding issues).

Since Java programs handle strings as sequences of characters from the 
Unicode character set all interfaces with external character storage 
needs to be done encoding aware. This means that when doing input Java 
String objects are created from byte sequences ALWAYS assuming an 
encoding, if not explicitly specified defaulting to the platform 
default. The other way around String objects are converted to byte 
sequences when doing output as well ALWAYS using an encoding, again 
defaulting to the platform default.

> BTW: a test has been made on DB2 with a table with a field declared
> CHAR(n) FOR BIT DATA and Java code works without problems reading and
> writing non-IBM850 characters.
> 

That makes sense. What I said before means that if you do not specify 
different encodings the same default will be used for input and output. 
If the storage medium leaves the intermediate bytes alone (which any 
binary database column type would do, just like a binary file would) 
then output will again match input, again allowing encoding unaware 
programmers to get away with it.

> Having read your feedback (THANK YOU EVERYONE!) I would say that
> there's no way to read back those characters in Java in my
> application.
> 

The problem is probably that the default encoding in Java turns your 
Unicode character into something the database is unwilling to store as 
is in this encoding and therefore mutilates trying to fix something. 
When queried the database then returns a byte sequence different from 
what was put in.
After that modification the JDBC driver has no chance of restoring the 
original value.

> Thanks again,
> Andrea


Best regards,

Silvio