7
Programming with Unicode

• Put a prefix N before a string literal that is enclosed with single quote marks. This explicitly indicates that the following string literal is an NCHAR string literal. For example, N'12-SEP-1975' is an NCHAR string lite ral.
• Mark a string literal with single quote marks. Because Oracle supports im plicit conversions to SQL NCHAR datatypes, a string literal is converted to a SQL NCHAR datatype wherever n ecessary.

  ---

  Note:

  When a string literal is included in a query and the query is submitted through a client-side tool suc h as SQL*Plus, all the queries are encoded in the client's character set and then converted to the server's database character set be fore processing. Therefore, data loss can occur if the string literal cannot be converted to the server database character set.

  < hr>
• Use the NCHR(n< code>) SQL function, which returns a unit of character code in the national character set, which is AL16UTF16 or UTF8. The res ult of concatenating several NCHR(n) functions is NVARCHAR2 data. In thi s way, you can bypass the client and server character set conversions and create an NVARCHAR2 string directly. For examp le, NCHR(32) represents a blank character.

  Because NCHR(< code>n) is associated with the national character set, portability of the resulting value is limited to appl ications that run in the national character set. If this is a concern, then use the UNISTR function to remove portabilit y limitations.

• Use the UNISTR('string') SQL function. UNISTR('string') converts a string to the national character set. To ensure portabi lity and to preserve data, include only ASCII characters and Unicode encoding in the following form: \xxxx, where xxxx is the hexadecimal value of a character code value in UTF-16 encoding format. For example, UNISTR('G\0061ry') represents 'Gary'. The ASCII characters are converted to the database character set and then to the national character set. The Unicode encoding is converted directly to the national character set.

The l ast two methods can be used to encode any Unicode string literals.

Using the UTL_FILE Package with NCHAR Data

The UTL_FILE package was enhanced in Oracle9i to handle Unicode national character set data. The following functions and procedures were added:

• FOPEN_NCHAR

  This function opens a file in Unicode for i nput or output, with the maximum line size specified. With this function, you can read or write a text file in Unicode instead of in the database character set.

• GET_LINE_NCHAR

  This procedure reads text from the open file identified by the file handle and places the text in the out put buffer parameter. With this procedure, you can read a text file in Unicode instead of in the database character set.

• PUT_NCHAR

  This procedure wr ites the text string stored in the buffer parameter to the open file identified by the file handle. With this procedure, you can writ e a text file in Unicode instead of in the database character set.

• P UT_LINE_NCHAR

  This procedure writes the text string stored in the buffer paramet er to the open file identified by the file handle. With this procedure, you can write a text file in Unicode instead of in the databa se character set.

• PUTF_NCHAR
  < p class="BP1">This procedure is a formatted PUT_NCHAR procedure. With this procedure, you can write a text file in Unico de instead of in the database character set.

  See Also: PL/SQL Packages and Types Reference for more information about the UTL_FILE package

OCI Programming with Unicode

OCI is the lowest -level API for accessing a database, so it offers the best possible performance. When using Unicode with OCI, consider these topics:< /p>

• OCIEnvNlsCreate() Function for Unicode Programming
• OCI Unicode Code Conver sion
• When the NLS_LANG Character Set is UTF 8 or AL32UTF8 in OCI
• Binding and Defining S QL CHAR Datatypes in OCI
• Binding and Defini ng SQL NCHAR Datatypes in OCI
• Binding and D efining CLOB and NCLOB Unicode Data in OCI

  See Also: Chapter 10, "OCI Pro gramming in a Global Environment"

OCIEnvNlsCreate() Function for Unicode Programming

The OCIEnvNlsCreate() function is used to specify a SQL CHAR character set and a SQL NCHAR character set when the OCI environment is created. It is an enhanced version of the OCIEnvCreate() function and has extended arguments for two character set IDs. The OCI_UTF16ID UTF-16 character set ID replaces the Unicode mode introduced in Oracle9i release 1 (9.0.1). For example:

OCIEnv *envhp;
status = OCIEnvNlsCreate((OCIEnv **)&envhp,
(ub4)0,
(void *)0,
(void *(*) ()) 0,
(void *(*) ()) 0,
(void(*) (
)) 0,
(size_t) 0,
(void **)0,
(ub2)OCI_UTF16ID, /* Metadata and SQL
 CHAR character set */
(ub2)OCI_UTF16ID /* SQL NCHAR character set */);

The Unicode mode, in which the OCI_UTF16 flag is used with the OCIEnvCreate() function, is deprecated.

When OCI_UTF16ID is specified for both SQL CHAR and SQL NCHA R character sets, all metadata and bound and defined data are encoded in UTF-16. Metadata includes SQL statements, user names, error messages, and column names. Thus, all inherited operations are independent of the NLS_LANG setting, and all metat ext data parameters (text*) are assumed to be Unicode text datatypes (utext*) in UTF-16 encoding.

To prepare the SQL statement when the OCIEnv() function is initialized with the OCI_UTF 16ID character set ID, call the OCIStmtPrepare() function with a (utext*) string. The following example run s on the Windows platform only. You may need to change wchar_t datatypes for other platforms.

const wchar_t sqlstr[] = L"SELECT * FROM ENAME=:ename";
...
OCIStmt* stmt
hp;
sts = OCIHandleAlloc(envh, (void **)&stmthp, OCI_HTYPE_STMT, 0,
NULL);
status = OCIStmtPrepare(stmthp, errhp,(const text*)sqlstr,
wcslen(sqlstr),

                      OCI_NTV_SYNTAX, OCI_DEFAULT);

To bind and d efine data, you do not have to set the OCI_ATTR_CHARSET_ID attribute because the OCIEnv() function has alre ady been initialized with UTF-16 character set IDs. The bind variable names must be also UTF-16 strings.

/* Inserting Unicode data */
OCIBindByName(stmthp1, &bnd1p, errhp, (const text*)L":ename",
(sb4)wcslen(L":ename"),
              (void *) ename, sizeof(ename), SQLT_STR, (void
*)&insname_ind,
              (ub2 *) 0, (ub2 *) 0, (ub4) 0, (ub4 *)0,
OCI_DEFAULT);
OCIAttrSet((void *) bnd1p, (ub4) OCI_HTYPE_BIND, (void *)
&ename_col_
len,
           (ub4) 0, (ub4)OCI_ATTR_MAXDATA_SIZE, errhp);
...
/*
 Retrieving Unicode data */
OCIDefineByPos (stmthp2, &dfn1p, errhp, (ub4)1, (void *)ename,
               (sb4)sizeof(ename), SQLT_STR, (void *)0, (ub2 *)0,
(ub2*)0,

    (ub4)OCI_DEFAULT);

The OCIExecute() function per forms the operation.

OCI Unicode Code Conversion

Unicod e character set conversions take place between an OCI client and the database server if the client and server character sets are diff erent. The conversion occurs on either the client or the server depending on the circumstances, but usually on the client side.

Data Integrity

You can lose data during conversion if you call an OCI API inapprop riately. If the server and client character sets are different, then you can lose data when the destination character set is a smalle r set than the source character set. You can avoid this potential problem if both character sets are Unicode character sets (for exam ple, UTF8 and AL16UTF16).

When you bind or define SQL NCHAR datatypes, you sho uld set the OCI_ATTR_CHARSET_FORM attribute to SQLCS_NCHAR. Otherwise, you can lose data because the data i s converted to the database character set before converting to or from the national character set. This occurs only if the database c haracter set is not Unicode.

OCI Performance Implications When Using Unicode

Redundant data conversions can cause performance degradation in your OCI applications. These conversions occur in two cases:

• When you bind or define SQL CHAR datatypes and set the OCI_ATTR_CHARSET_FORM attribute to SQLCS_NCHAR, data conversions take place from client character set to the national database character set, and from the national character set to the database character set. No data loss is expected, but two conversions happen, even though it requires only one.
• When you bind or define SQL NCHAR datatypes and do not set OCI_ATTR_CHARSET_FORM, data conversions take place from client characte r set to the database character set, and from the database character set to the national database character set. In the worst case, d ata loss can occur if the database character set is smaller than the client's.

To avo id performance problems, you should always set OCI_ATTR_CHARSET_FORM correctly, based on the datatype of the target colu mns. If you do not know the target datatype, then you should set the OCI_ATTR_CHARSET_FORM attribute to SQLCS_NCHA R when binding and defining.

Table 7-3 conta ins information about OCI character set conversions.

Table 7-3 OCI Character Set Conversions  

Datatypes for OCI Client Buffer	OCI_ATTR_CHARSET_FORM	Datatypes of the Target Column in the Database	< a name="1006495"> Conversion Between	Comments
utext	SQLCS_IMPLICIT	CHAR, VARCHAR2, CLOB	UTF-16 and database character set in OCI	No un expected data loss
utext	SQLCS_NCHAR	NCHAR, NVARCHAR2, NCLOB	UTF-16 and national character set in OCI	No unexpected data loss
utext	SQLCS_NCHAR	CHAR, VARCHAR2, CLOB	UTF-16 and national character set in OCI National character set and database character set in database server	No unexpected data loss, but may degrade performance because the conversion goes through the national character set
utext	SQLCS_IMPLICIT	NCHAR, NVARCHAR2, NCLOB	UTF-16 and database character set in OCI Database character set and national character set in database server	Data loss may occur if the database character set is not Unicode
text	SQLCS_IMPLICIT	CHAR, VARCHAR2, CLOB	NLS_LANG character set and database character set in OCI	No unexpect ed data loss
text	SQLCS_NCHAR	NCHAR, NVARCHAR2, NCLOB	NLS_LANG character set and national character set in OCI	No unexpected data loss
text	SQLCS_NCHAR < /td>	CHAR, VARCHAR2, CLOB	< a name="1006567"> NLS_LANG character set and national character set in OCI National character set and database character set in database server	No unexpected data loss, but may degrade performance because the conversion goes through the national character set< /p>
text	SQLCS_IMPLICIT	NCHAR, NVARCHAR2, NCLOB	< code>NLS_LANG character set and database character set in OCI Database character set and national character set in database server	Data loss may occur b ecause the conversion goes through the database character set

OCI Unicode Data Expansion

Data conversion can result in data expansion, which can cause a buffer to overflow. For binding operations, you need to set the OCI_ATTR_MAXDATA_SIZE attribute to a large enough size to hold the expanded data on the server. If this is difficult to do, then you need to consider changing the table schema. For defining operations, client application s need to allocate enough buffer space for the expanded data. The size of the buffer should be the maximum length of the expanded dat a. You can estimate the maximum buffer length with the following calculation:

1. Get the column data byte size.
2. Multiply it by the maximum number of bytes for each character in the client character set.

Th is method is the simplest and quickest way, but it may not be accurate and can waste memory. It is applicable to any character set co mbination. For example, for UTF-16 data binding and defining, the following example calculates the client buffer:

ub2 csid = OCI_UTF16ID;
oratext *selstmt = "SELECT ename FROM emp";
counter = 1;
... 
OCIStmtPrepare(stmthp, errhp, selstmt, (ub4)strlen((char*)selstmt),
<
a name="1006596">               OCI_NTV_SYNTAX, OCI_DEFAULT);
OCIStmtExecute ( svchp, stmthp, errhp, (ub4)0
, (ub4)0,
                 (CONST OCISnapshot*)0, (OCISnapshot*)0,
                 OCI_D
ESCRIBE_ONLY);
OCIParamGet(stmthp, OCI_HTYPE_STMT, errhp, &myparam, (ub4)counter);
OC
IAttrGet((void*)myparam, (ub4)OCI_DTYPE_PARAM, (void*)&col_width,
           (ub4*)0, (ub4)OCI_ATTR_DATA_SI
ZE, errhp);
... 
maxenamelen = (col_width + 1) * sizeof(utext);
cbu
f = (utext*)malloc(maxenamelen);
...
OCIDefineByPos(stmthp, &dfnp, errhp, (ub4)1, (vo
id *)cbuf,
                (sb4)maxenamelen, SQLT_STR, (void *)0, (ub2 *)0,

   (ub2*)0, (ub4)OCI_DEFAULT);
OCIAttrSet((void *) dfnp, (ub4) OCI_HTYPE_DEFINE, (void *) &csid,
           (ub4) 0, (ub4)OCI_ATTR_CHARSET_ID, errhp);
OCIStmtFetch(stmthp, errhp, 1, OCI_FETCH_NEXT
, OCI_DEFAULT);
...

When the NLS_LANG Character Set is UTF8 or AL32UTF8 in OCI

You can use UTF8 and AL32UTF8 by setting NLS_LANG for OCI client applications. If you do not need supplementary characters, then it does not matter whether you choose UTF8 or AL32UTF8. However, if your OCI applications might h andle supplementary characters, then you need to make a decision. Because UTF8 only supports characters of up to three bytes, no supp lementary character can be represented in UTF8. In AL32UTF8, one supplementary character is represented in one code point, totalling four bytes.

Do not set NLS_LANG to AL16UTF16, because AL16UTF16 is the nationa l character set for the server. If you need to use UTF-16, then you should specify the client character set to OCI_UTF16ID, using the OCIAttrSet() function when binding or defining data.

Oracle Corporation recommends that you access SQL NCHAR datatypes using UTF-16 binding or defining when using OCI. Beginning with Oracle9i, SQL NCHAR datatypes are Unicode datatypes with an encoding of eit her UTF8 or AL16UTF16. To access data in SQL NCHAR datatypes, set the OCI_ATTR_CHARSET_FORM attribute to SQLCS_NCHAR between binding or defining and execution so that it performs an appropriate data conversion without data loss . The length of data in SQL NCHAR datatypes is always in the number of Unicode code units.

...
ub2 csid = OCI_UTF16ID;
ub1 cform = SQLCS_NCHA
R;
utext ename[100]; /* enough buffer for ENAME */
... 
/* Insertin
g Unicode data */ 
OCIBindByName(stmthp1, &bnd1p, errhp, (oratext*)":ENAME",

      (sb4)strlen((char *)":ENAME"), (void *) ename,
              sizeof(ename), SQLT_STR, (void *)&insnam
e_ind, (ub2 *) 0,
              (ub2 *) 0, (ub4) 0, (ub4 *)0, OCI_DEFAULT); 
OCIAttrSet((
void *) bnd1p, (ub4) OCI_HTYPE_BIND, (void *) &cform, (ub4) 0,
           (ub4)OCI_ATTR_CHARSET_FORM, errhp
); 
OCIAttrSet((void *) bnd1p, (ub4) OCI_HTYPE_BIND, (void *) &csid, (ub4) 0,

    (ub4)OCI_ATTR_CHARSET_ID, errhp);
OCIAttrSet((void *) bnd1p, (ub4) OCI_HTYPE_BIND, (void *) &ename_col_
len,
           (ub4) 0, (ub4)OCI_ATTR_MAXDATA_SIZE, errhp); 
... 

/* Retrieving Unicode data */
OCIDefineByPos (stmthp2, &dfn1p, errhp, (ub4)1, (void *)ename,
                (sb4)sizeof(ename), SQLT_STR, (void *)0, (ub2 *)0, (ub2*)0,
                (ub4)OCI_DE
FAULT); 
OCIAttrSet((void *) dfn1p, (ub4) OCI_HTYPE_DEFINE, (void *) &csid, (ub4) 0,

           (ub4)OCI_ATTR_CHARSET_ID, errhp); 
OCIAttrSet((void *) dfn1p, (ub4) OCI_HTYPE_DEFINE, (void *) &
cform, (ub4) 0,
           (ub4)OCI_ATTR_CHARSET_FORM, errhp); 
...

Binding and Defining CLOB a nd NCLOB Unicode Data in OCI

In order to write (bind) and read (defi ne) UTF-16 data for CLOB or NCLOB columns, the UTF-16 character set ID must be specified as OCILobWri te() and OCILobRead(). When you write UTF-16 data into a CLOB column, call OCILobWrite() as follows:

...
ub2 csid = OCI_UTF16ID;
err =
 OCILobWrite (ctx->svchp, ctx->errhp, lobp, &amtp, offset, (void *) buf,
                   (ub4) BUF
SIZE, OCI_ONE_PIECE, (void *)0,
                   (sb4 (*)()) 0, (ub2) csid, (ub1) SQLCS_IMPLICIT); 

The amtp parameter is the data length in number of Unicode co de units. The offset parameter indicates the offset of data from the beginning of the data column. The csid parameter must be set for UTF-16 data.

To read UTF-16 data from CLOB columns, call OCILobRead() as follows:

...
ub2 csid = OCI_UTF16I
D;
err = OCILobRead(ctx->svchp, ctx->errhp, lobp, &amtp, offset, (void *) buf,

                 (ub4)BUFSIZE , (void *) 0, (sb4 (*)()) 0, (ub2)csid,
                 (ub1) SQLCS_IMPLICIT);
<
a name="1006716">

The data length is always represented in the number of Unicode cod e units. Note one Unicode supplementary character is counted as two code units, because the encoding is UTF-16. After binding or defi ning a LOB column, you can measure the data length stored in the LOB column using OCILobGetLength(). The returning value is the data length in the number of code units if you bind or define as UTF-16.

err = OCILobGetLength(ctx->svchp, ctx->errhp, lobp, &lenp);

• Using the VARCHAR Pro*C/C++ datatype or the native C/C++ text datatype, a program can access Unicode data stored in SQL CHAR datatypes of a UTF8 or AL32UTF8 database. Alternatively, a program could use the C/C++ native text type.
• Using the UVARCHAR Pro*C/C++ datatype or th e native C/C++ utext datatype, a program can access Unicode data stored in NCHAR datatypes of a database.
• Using the NVARCHAR Pro*C/C++ datatype, a program can access Unicod e data stored in NCHAR datatypes. The difference between UVARCHAR and NVARCHAR in a Pro*C/C++ program is that the data for the UVARCHAR datatype is stored in a utext buffer while the data for the NVARCHAR datatype is stored in a text datatype.

Pro*C/C++ does n ot use the Unicode OCI API for SQL text. As a result, embedded SQL text must be encoded in the character set specified in the N LS_LANG environment variable.

This section contains the following topics:

• Pro*C/C++ Data Conversion in Unicode< /li>
• Using the VARCHAR Datatype in Pro*C/C++
• Using the NVARCHAR Datatype in Pro*C/C++
• Using the UVARCHAR Datatype in Pro*C/C++

Pro*C/C++ Data Conversion in Unicode

Data conversion occurs in the OCI layer, but it is the Pro*C/C++ preprocessor that instructs OCI which conversion path should be taken based on the datatypes used in a Pro*C/ C++ program. Table 7-4 illustrates the conversion paths:

Table 7-4 Pro*C/C++ Bind a nd Define Data Conversion  

< td class="Formal">

NVARCHAR

< td class="Formal">

UTF-16 to and from the national character set happens in OCI

Pro*C/C++ Datatype	SQL Datatype	Conversion Path
VARCHAR or text	CHAR	NL S_LANG character set to and from the database character set happens in OCI
VARCHAR or text	NCHAR	N LS_LANG character set to and from database character set happens in OCI Database char acter set to and from national character set happens in database server
NCHAR	NLS_LANG character set t o and from national character set happens in OCI
NVARCHAR	CHAR	NLS_LANG character set to and from national char acter set happens in OCI National character set to and from database character set in databa se server
UVARCHAR or utext	NCHAR
UVARCHAR or u text	CHAR	UTF-16 to and from national character set happens in OCI Nation al character set to database character set happens in database server

Using the VARCHAR Datatype in Pro*C/C++

The Pro*C/C++ VARCHAR datatype is preprocessed to a struct wi th a length field and text buffer field. The following example uses the C/C++ text native data type and the VARCHAR Pro*C/C++ datatypes to bind and define table columns.

#in
clude <sqlca.h> 
main() 
{ 
   ... 

/* Change to STRING datatype:    */ 
   EXEC ORACLE OPTION (CHAR_MAP=STRING) ; 
   text e
name[20] ;                  /* unsigned short type */ 
   varchar address[50] ;             /* Pro*C/C++ varcha
r type */ 

   EXEC SQL SELECT ename, address INTO :ename, :address FROM emp; 
   /* ename is NULL-terminated */ 
   printf(L"ENAME = %s, ADDRESS = %.*s\n", ename, address.len, a
ddress.arr); 
   ... 
} 

When you use the VARCHAR datatype or native text datatype in a Pro*C/C++ program, the preprocessor assu mes that the program intends to access columns of SQL CHAR datatypes instead of SQL NCHAR datatypes in the database. The preprocessor generates C/C++ code to reflect this fact by doing a bind or define using the SQLCS_IMPLICIT value for the OCI_ATTR_CHARSET_FORM attribute. As a result, if a bind or define variable is bound to a column of SQL NCHAR datatypes in the database, then implicit conversion occurs in the database server to convert the data from the databa se character set to the national database character set and vice versa. During the conversion, data loss occurs when the database cha racter set is a smaller set than the national character set.

Using the NVARCHAR Datatype in Pro*C/C++

The Pro*C/C++ NVARCHAR datatype is similar to the Pro*C/C++ VARCHAR datatype. It s hould be used to access SQL NCHAR datatypes in the database. It tells Pro*C/C++ preprocessor to bind or define a text bu ffer to the column of SQL NCHAR datatypes. The preprocessor specifies the SQLCS_NCHAR value for the O CI_ATTR_CHARSET_FORM attribute of the bind or define variable. As a result, no implicit conversion occurs in the database.

If the NVARCHAR buffer is bound against columns of SQL CHAR datatype s, then the data in the NVARCHAR buffer (encoded in the NLS_LANG character set) is converted to or from the national character set in OCI, and the data is then converted to the database character set in the database server. Data can be lost when the NLS_LANG character set is a larger set than the database character set.

Using the UVARCHAR Datatype in Pro*C/C++

The UVARCHAR datatype is preprocessed to a struct with a length field and utext buffer field. The following example code contains two host variables, ename and address. The ename host variable is declared as a utext buffer containing 20 Unicode char acters. The address host variable is declared as a uvarchar buffer containing 50 Unicode characters. The len and arr fields are accessible as fields of a struct.

#include &
lt;sqlca.h> 
#include <sqlucs2.h> 

main() 
{ 
   ... 
   /* Change to STRING datatype:    */ 
   EXEC
ORACLE OPTION (CHAR_MAP=STRING) ; 
   utext ename[20] ;                  /* unsigned short type */ 
 uvarchar address[50] ;               /* Pro*C/C++ uvarchar type */ 

   EXEC
SQL SELECT ename, address INTO :ename, :address FROM emp; 
   /* ename is NULL-terminated */ 
wprintf(L"ENAME = %s, ADDRESS = %.*s\n", ename, address.len, 
address.arr); 
... 
} 

When you use the UVARCHAR dat atype or native utext datatype in Pro*C/C++ programs, the preprocessor assumes that the program intends to access SQL NCHAR datatypes. The preprocessor generates C/C++ code by binding or defining using the SQLCS_NCHAR value for OCI_ATTR_CHARSET_FORM attribute. As a result, if a bind or define variable is bound to a column of a SQL NCHAR datatype, then an implicit conversion of the data from the national character set occurs in the database server. However, there is no data lost in this scenario because the national character set is always a larger set than the database character set.

JDBC Programming w ith Unicode

Oracle provides the following JDBC drivers for Java prog rams to access character data in an Oracle database:

• The JDBC OCI driver
• The JDBC thin driver
• The JDBC server-side internal driver
• The JDBC server-side thin driver

Java programs can insert or retrieve character data to and from columns of SQL CHAR and NCHAR datatypes. Specifically, JDBC enables Java programs to bind or define Java strings to SQL C HAR and NCHAR datatypes. Because Java's string datatype is UTF-16 encoded, data retrieved from or in serted into the database must be converted from UTF-16 to the database character set or the national character set and vice versa. JD BC also enables you to specify the PL/SQL and SQL statements in Java strings so that any non-ASCII schema object names and string lit erals can be used.

At database connection time, JDBC sets the server NLS_LANGUAGE and NLS_TERRITORY parameters to correspond to the locale of the Java VM that runs the JDBC driver. This operation en sures that the server and the Java client communicate in the same language. As a result, Oracle error messages returned from the serv er are in the same language as the client locale.

This section contains the following topic s:

• Binding and Defining Java St rings to SQL CHAR Datatypes
• Binding and Def ining Java Strings to SQL NCHAR Datatypes
• U sing the SQL NCHAR Datatypes Without Changing the Code
• Data Conversion in JDBC
• Using o racle.sql.CHAR in Oracle Object Types
• Restr ictions on Accessing SQL CHAR Data with JDBC

Binding and Defining Java Strings to SQL CHAR Datatypes

Oracle JDBC drivers allow you to access SQL CHAR datatypes in the database using Java string bind or define variables. The following code illustrates how to bind a Java string to a CHAR column.

int employee_id = 12345;
String last_name = "Joe";
Prepared
Statement pstmt = conn.prepareStatement("INSERT INTO" +
    "employees (last_name, employee_id) VALUES (?, ?)")
;
pstmt.setString(1, last_name);
pstmt.setInt(2, employee_id);
pstm
t.execute();                 /* execute to insert into first row */
employee_id += 1;                      /* n
ext employee number */
last_name = "\uFF2A\uFF4F\uFF45";    /* Unicode characters in name */

pstmt.setString(1, last_name);
pstmt.setInt(2, employee_id);
pstmt.execute();
         /* execute to insert into second row */

You can define t he target SQL columns by specifying their datatypes and lengths. When you define a SQL CHAR column with the datatype and the length, JDBC uses this information to optimize the performance of fetching SQL CHAR data from the column. The follo wing is an example of defining a SQL CHAR column.

OraclePreparedStatement pstm
t = (OraclePreparedStatement)
     conn.prepareStatement("SELECT ename, empno from emp");
pstmt.defineColumnType(1,Types.VARCHAR, 3);
pstmt.defineColumnType(2,Types.INTEGER);
Res
ultSet rest = pstmt.executeQuery();
String name = rset.getString(1);
int id = reset.getIn
t(2);

You need to cast PreparedStatement to Or aclePreparedStatement to call defineColumnType(). The second parameter of defineColumnType() is the datatype of the target SQL column. The third parameter is the length in number of characters.

Binding and Defining Java Strings to SQL NCHAR D atatypes

For binding or defining Java string variables to SQL NCHAR datatypes, Oracle provides an extended PreparedStatement which has the setFormOfUse() method t hrough which you can explicitly specify the target column of a bind variable to be a SQL NCHAR datatype. The following c ode illustrates how to bind a Java string to an NCHAR column.

int employee_id
= 12345;
String last_name = "Joe"
oracle.jdbc.OraclePreparedStatement pstmt =
    (oracle.jdbc.OraclePreparedStatement)
    conn.prepareStatement("INSERT INTO employees (last_nam
e, employee_id) 
    VALUES    (?, ?)");
pstmt.setFormOfUse(1, oracle.jdbc.OraclePrepared
Statement.FORM_NCHAR);
pstmt.setString(1, last_name);
pstmt.setInt(2, employee_id);
pstmt.execute();                 /* execute to insert into first row */
employee_id += 1;
                 /* next employee number */
last_name = "\uFF2A\uFF4F\uFF45";    /* Unicode characters in name
*/
pstmt.setString(1, last_name);
pstmt.setInt(2, employee_id);
pst
mt.execute();                 /* execute to insert into second row */

You can define the target SQL NCHAR columns by specifying their datatypes, forms of use, and lengths. JDBC uses thi s information to optimize the performance of fetching SQL NCHAR data from these columns. The following is an example of defining a SQL NCHAR column.

OraclePreparedStatement pstmt = (OraclePreparedSt
atement)
     conn.prepareStatement("SELECT ename, empno from emp");
   pstmt.defineColum
nType(1,Types.VARCHAR, 3, 
OraclePreparedStatement.FORM_NCHAR);
   pstmt.defineColumnType
(2,Types.INTEGER);
   ResultSet rest = pstmt.executeQuery();
   String name = rset.getStr
ing(1);
   int id = reset.getInt(2);

To def ine a SQL NCHAR column, you need to specify the datatype that is equivalent to a SQL CHAR column in the fir st argument, the length in number of characters in the second argument, and the form of use in the fourth argument of defineCol umnType().

You can bind or define a Java string against an NCHAR column without explicitly specifying the form of use argument. This implies the following:

• If you do not specify the argument in the setString() method, then JDBC assumes that the bind or define variable is for the SQL CHAR column. As a result, it tries to convert them to the database character set. When th e data gets to the database, the database implicitly converts the data in the database character set to the national character set. D uring this conversion, data can be lost when the database character set is a subset of the national character set. Because the nation al character set is either UTF8 or AL16UTF16, data loss would happen if the database character set is not UTF8 or AL32UTF8.
• Because implicit conversion from SQL CHAR to SQL NCHAR dataty pes happens in the database, database performance is degraded.

In addition, if you bi nd or define a Java string for a column of SQL CHAR datatypes but specify the form of use argument, then performance of the database is degraded. However, data should not be lost because the national character set is always a larger set than the databas e character set.

Using the SQL NCHAR Datatypes Without Changing the Code

A gl obal flag has been introduced in the Oracle JDBC drivers for customers to tell whether the form of use argument should be specified b y default in a Java application. This flag has the following purposes:

• Existing applications accessing the SQL CHAR datatypes can be migrated to support the SQL NCHAR da tatypes for worldwide deployment without changing a line of code.
• Applications do not need to call the setFormOfUse() method when binding and defining a SQL NCHAR column. The applicatio n code can be made neutral and independent of the datatypes being used in the backend database. With this flag, applications can be e asily switched from using SQL CHAR or SQL NCHAR.

The global flag is specified in the command line that invokes the Java application. The syntax of specifying this flag is as follows:

java -Doracle.jdbc.defaultNChar=true <application class>

With this flag specified, the Oracle JDBC drivers assume the presence of the form of use argument for all bind and define operations in the application.

If you have a database schema that consi sts of both the SQL CHAR and SQL NCHAR columns, then using this flag may have some performance impact when accessing the SQL CHAR columns because of implicit conversion done in the database server.

< div align="center">

See Also: < a href="ch7progr.htm#1006926">"Data Conversion in JDBC" for more information about the perform ance impact of implicit conversion

Data Conversion in JDBC

Because Java strings are always encoded in UTF-16, JDBC drivers transparently convert data from the database character set to U TF-16 or the national character set. The conversion paths taken are different for the JDBC drivers:

• Data Conversion for the OCI Driver
• Data Conversion for Thin Drivers
• Data Conversion for the Server-Side Internal Driver

Data Conversion for the OCI D river

For the OCI driver, the SQL statements are always converted to the database character set by the driver before it is sent to the database for processing. When the database character set is neithe r US7ASCII nor WE8ISO8859P1, the driver converts the SQL statements to UTF-8 first in Java and then to the database character set in C. Otherwise, it converts the SQL statements directly to the database character set. For Java string bind or define variables, Table 7-5 summarizes the conversion paths taken for different scenarios.

Table 7-5 OCI Dr iver Conversion Path  

Data Conve rsion for Thin Drivers

SQL statements are always converted to either the database character set or to UTF-8 by the driver before they are sent to the database for processing. When the database characte r set is either US7ASCII or WE8ISO8859P1, the driver converts the SQL statement to the database character set. Otherwise, the driver converts the SQL statement to UTF-8 and notifies the database that a SQL statement requires further conversion before being processed . The database, in turn, converts the SQL statements from UTF-8 to the database character set. The database, in turn, converts the SQ L statement to the database character set. For Java string bind and define variables, the conversion paths shown in Table 7-6 are taken for the thin driver.

Table 7-6 Thin Driver Conversion Path  

Form of Use	SQL Data type	Conversion Path
Const.CHAR (Default)	CHAR	Java string to and from data base character set happens in the JDBC driver
Const.CHAR (Default)	NCHAR	Java string to and from database character set happ ens in the JDBC driver. Data in the database character set to and from national character se t happens in the database server
Const.NCHAR	NCHAR	Java string to and from national character set happens in the JDBC driver
Const.NCH AR	CHAR	Java string to and from national character set happens in the JDBC driver Data in national character set to and from database character set happens in the database server

< td class="Formal">

Const.CHAR (Default)

Form of Use	SQL Datatype	Database Charact er Set	Conversion Path
CHAR	US7ASCII or WE8ISO8859P1	Java string to and from the database character set happens in the th in driver
Const.CHAR (Default)	NCHAR	US7ASCII or WE8ISO8859P1	Java string to and from the database character set happens in the thin driver. Data in t he database character set to and from the national character set happens in the database server
Const.CHAR (Default)	CHAR	no n-ASCII and non-WE8ISO8859P1	Java string to and from UTF-8 happens in the thin driver. Data in UTF-8 to and from the database character set happens in the data base server
Const.CHAR (Default)	NCHAR	non-ASCII and non-WE8ISO8859P1	Java string to and from UTF-8 happens in the thin driver. Data in UTF-8 to and f rom national character set happens in the database server
Const.NCHAR	CHAR		Java string to and from the national character set happens in the thin driver. Data in t he national character set to and from the database character set happens in the database server
Const.NCHAR	NCHAR		Java string to and from the national character set happens in the thin driver

Data Conversion for the Server-Side Internal Driver

All data co nversion occurs in the database server because the server-side internal driver works inside the database.

Using oracle.sql.CHAR in Oracle Obje ct Types

JDBC drivers support Oracle object types. Oracle objects ar e always sent from database to client as an object represented in the database character set or national character set. That means th e data conversion path in "Data Conversion in JDBC" does not ap ply to Oracle object access. Instead, the oracle.sql.CHAR class is used for passing SQL CHAR and SQL NCHAR data of an object type from the database to the client.

This section includes the following topics:

• oracle.sql.CHAR
• Accessing SQL CHAR and NCHAR Attributes with orac le.sql.CHAR

oracle.sql.CHAR

The oracle.sql.CHAR class has a special functionality for conversion of character data. The Oracle character set is a key attribute of the oracle.sql.CHA R class. The Oracle character set is always passed in when an oracle.sql.CHAR object is constructed. Without a kn own character set, the bytes of data in the oracle.sql.CHAR object are meaningless.

The oracle.sql.CHAR class provides the following methods for converting character data to strings:

• getString()

  Conver ts the sequence of characters represented by the oracle.sql.CHAR object to a string, returning a Java string object. If the character set is not recognized, then getString() returns a SQLException.

• toString()

  Identical to getString(), except that if the character set is not recognized, then toString() returns a hexadecimal representation of the oracle.sql.CHAR data and does not returns a SQLException.

• getStringWithReplacement()

  Identical to getString(), except that a default replacement character replaces characters that have no Unicode representation in the character set of this oracle.sql.CHAR object. This default character varies among character sets, but it is often a question mark.

< a name="1008664">

You may want to construct an oracle.sql.CHAR object yourself (to pass into a prepar ed statement, for example). When you construct an oracle.sql.CHAR object, you must provide character set information to the oracle.sql.CHAR object by using an instance of the oracle.sql.CharacterSet class. Each instance of the oracle.sql.CharacterSet class represents one of the character sets that Oracle supports.

Complete the following tasks to construct an oracle.sql.CHAR object:

1. Create a CharacterSet instance by calling the static CharacterSet.m ake() method. This method creates the character set class. It requires as input a valid Oracle character set (OracleId)< /code>. For example:

       int OracleId = CharacterSet.JA16SJIS_CHARSET; // this is character se
   t 832
       ...
       CharacterSet mycharset = CharacterSet.make(OracleId);
   
   

   Each character set that Oracle supports has a unique predefined OracleId. The OracleId can always be referenced as a character set specified as Oracle_character_set_name_CHARSET where Oracle_character_set_name is the Oracle character set.

2. Construct an oracle.sql.CHAR object. Pass to the constructor a string (or the b ytes that represent the string) and the CharacterSet object that indicates how to interpret the bytes based on the chara cter set. For example:

       String mystring = "teststring";
       ...
       oracle.sql.CHAR mychar = new oracle.sql.CHAR(teststring, mycharset);
   
   

   The oracle.sql.CHAR class has multiple constructors: they can take a string, a byte array, or an object as input along with the CharacterSet object. In the case of a string, the string is converted to the character set indicated by the CharacterSet object before being placed into the oracle.sql.CHAR object.

The server (database) and the client (or application running on the client) can use different character sets. When you use the methods of this class to transfer data between the server and the client, the JDBC drivers must con vert the data between the server character set and the client character set.

Accessing SQL CHAR and NCHAR Attributes with oracle.sql.CHAR

The following is an example of an object type created using SQL:

CREATE TYPE person_type AS OBJECT (name VARCHAR2(30), address NVARCHAR(256), age NUMBER); CREATE TABLE employees (id NUMBER, person PERSON_TYPE);

The Java class corresponding to this object type can be constructed as follows:

publi
c class person implement SqlData  
{
   oracle.sql.CHAR name;
   or
acle.sql.CHAR address;
   oracle.sql.NUMBER age;
   // SqlData interfaces
   getSqlType() {...}
   writeSql(SqlOutput stream) {...}
   readSql(SqlInput stre
am, String sqltype) {...}
}

The oracle.sql. CHAR class is used here to map to the NAME attributes of the Oracle object type, which is of VARCHAR2 datat ype. JDBC populates this class with the byte representation of the VARCHAR2 data in the database and the Character Set object corresponding to the database character set. The following code retrieves a person object from the employees table:

TypeMap map = ((OracleConnection)conn).getTypeMap();
map.put("PERSON
_TYPE", Class.forName("person"));
conn.setTypeMap(map);
    .       .        .
    .       .        .
ResultSet rs = stmt.executeQuery("SELECT PERSON FROM EMPLOYEES");
rs.next();
person p = (person) rs.getObject(1);
oracle.sql.CHAR sql_name = p.
name;
oracle.sql.CHAR sql_address=p.address;
String java_name = sql_name.getString();
String java_address = sql_address.getString();

The getString() method of the oracle.sql.CHAR class converts the byte array from the database character set or national character se t to UTF-16 by calling Oracle's Java data conversion classes and returning a Java string. For the rs.getObject(1) call to work, the S qlData interface has to be implemented in the class person, and the Typemap map has to be set up to indicate the mapping of the object type PERSON_TYPE to the Java class.

Restrictions on Accessing SQL CHAR Data with JDBC

< a name="1008746">

This section contains the following topics:

• SQL CHAR Data Size Restriction With the JDBC Thin Driver
• Character Integrity Issues in a Multibyte Database Environment

SQL C HAR Data Size Restriction With the JDBC Thin Driver

If the database character set is neither ASCII (US7ASCII) nor ISO Latin1 (WE8ISO8859P1), then the JDBC thin driver must impose size restrictions for SQL CHAR bind parameters that are more restrictive than normal database size limitations. This is necessary to allow for data expansion during conversion.

The JDBC thin driver checks SQL CHAR bind s izes when a setXXX() method (except for the setCharacterStream() method) is called. If the data size exceeds the size restriction, then the driver returns a SQL exception (SQLException: Data size big ger than max size for this type") from the setXXX() call. This limitation is n ecessary to avoid the chance of data corruption when conversion of character data occurs and increases the length of the data. This l imitation is enforced in the following situations:

• Using the J DBC thin driver
• Using binds (not defines)
• Using SQL CHAR datatypes
• Connecting to a datab ase whose character set is neither ASCII (US7ASCII) nor ISO Latin1 (WE8ISO8859P1)

Whe n the database character set is neither US7ASCII nor WE8ISO8859P1, the JDBC thin driver converts Java UTF-16 characters to UTF-8 enco ding bytes for SQL CHAR binds. The UTF-8 encoding bytes are then transferred to the database, and the database converts the UTF-8 encoding bytes to the database character set encoding.

This conversion to the cha racter set encoding can result in an increase in the number of bytes required to store the data. The expansion factor for a database character set indicates the maximum possible expansion in converting from UTF-8 to the character set. If the database character set i s either UTF8 or AL32UTF8, then the expansion factor (exp_factor) is 1. Otherwise, the expansion factor is equal to the maximum character size (measured in bytes) in the database character set.

Table 7-7 shows the database size limitations for SQL CHAR data and the JDBC thin dr iver size restriction formulas for SQL CHAR binds. Database limits are in bytes. Formulas determine the maximum allowed size of the UTF-8 encoding in bytes.

Table 7-7 Maximum SQL CHAR Bind Sizes  

Datatype	Maximum Bind Size Allowed by Database	Formula for Determini ng the Maximum Bind Size, Measured in UTF-8 Bytes
CHAR	< p class="TB">2000 bytes	4000/exp_factor
VA RCHAR2	4000 bytes	4000/exp_factor
LONG	231 - 1 bytes	(231 - 1)/exp_factor

The formulas guar antee that after the data is converted from UTF-8 to the database character set, the size of the data does not exceed the maximum siz e allowed in the database.

The number of UTF-16 characters that can be supported is determi ned by the number of bytes for each character in the data. All ASCII characters are one byte long in UTF-8 encoding. Other character types can be two or three bytes long.

Table 7-8 lis ts the expansion factors of some common server character sets. It also shows the JDBC thin driver maximum bind sizes for CHAR and VARCHAR2 data for each character set.

Table 7-8 Expansion Factor and Maximum Bind Size for Common Se rver Character Sets  

Server Character Set	Expansion Factor	JDBC Thin Driver Maximum Bind Size for SQL CHAR Data, Measured in UTF-8 Bytes
WE8DE C	1	4000 bytes
JA 16SJIS	2	2000 bytes
JA16EUC	3	1333 bytes
AL32UTF8	1	4000 bytes

Character Integrity Issues in a Multibyte Database Environment

ODBC and OLE DB Programming with Unicode

You should use the Oracle ODBC driver or Oracle Provider for OLE DB to access the Oracle server when using a Windows plat form. This section describes how these drivers support Unicode. It includes the following topics:

• Unicode-Enabled Drivers in ODBC and OLE DB
• OCI Dependency in Unicode
• ODBC and OLE DB Code Conversion in Unicode
• ODBC Unicode Datatypes
• < a href="ch7progr.htm#1007269">OLE DB Unicode Datatypes
• ADO Access

Unicode-Enabled Drivers in ODBC and OLE DB

O racle's ODBC driver and Oracle Provider for OLE DB can handle Unicode data properly without data loss. For example, you can run a Uni code ODBC application containing Japanese data on English Windows if you install Japanese fonts and an input method editor for enteri ng Japanese characters.

Oracle provides ODBC and OLE DB products for Windows platforms only . For Unix platforms, contact your vendor.

OCI Dependency in Unicode

OCI Unic ode binding and defining features are used by the ODBC and OLE DB drivers to handle Unicode data. OCI Unicode data binding and defini ng features are independent from NLS_LANG. This means Unicode data is handled properly, irrespective of the NLS_LA NG setting on the platform.

S ee Also: "OCI Programming with Unicode"

ODBC and OLE DB Code Conversion in Unicode

In general, no redundant data conversion occurs unless you specify a different client datatype from that of the server. If you bind Unicode buffer SQL_C_WCHAR with a Unicode data column like NCHAR, for example, then ODB C and OLE DB drivers bypass it between the application and OCI layer.

If you do not specify datatypes before fetching, but call SQLGetData with the client datatypes instead, then the conversions in Table 7-9 occur.

Table 7-9 ODBC Implicit Binding Code Conversions  

Datatypes of ODBC Client Buffer	Datatypes of the Target Column in the D atabase	Fetch Conversions	Comments
SQL_C_WCHAR	CHAR, VARCHAR2, CLOB	[If the database character set is a subset of the NLS_LANG character set, then the conversions occ ur in the following order: Database character set NLS_LANG UTF-16 in OCI UTF-16 in ODBC	No unexpected data loss May degrade performance if database character set is a sub set of the NLS_LANG character set
SQL_C_CHAR	CHAR,< br> VARCHAR2, CLOB	If database character set is a subset of NLS_LANG character set: Database character set to NLS_LANG in OCI If database character set is NOT a subset of NLS_LANG character set: Database character set, UTF-16, to NLS_LANG character set in OCI and ODBC	No unexpected data loss May degrade performance if database character set is not a subset of NLS_LANG character set< /p>

You must specify the datatype for inserting and updating operatio ns.

The datatype of the ODBC client buffer is given when you call SQLGetData b ut not immediately. Hence, SQLFetch does not have the information.

Because the ODBC driver guarantees data integrity, if you perform implicit bindings, then redundant conversion may result in performance degrada tion. Your choice is the trade-off between performance with explicit binding or usability with implicit binding.

OLE DB Code Conversions

Unlike ODBC, OLE DB only enables you to perform implicit bindings for ins erting, updating, and fetching data. The conversion algorithm for determining the intermediate character set is the same as the impli cit binding cases of ODBC.

Table 7-10 OLE DB Implicit Bindings

No unexpected data loss

May degrade p erformance if database character set is a subset of NLS_LANG character set

Datatypes of OLE_DB Client Buffer	Datatypes of the Target Column in the Database	In-Bin ding and Out-Binding Conversions	< /a> Comments
DBTYPE_WCHAR	CHAR, VARCHAR2, CLOB	If database character set is a subset of the NLS_LANG character set: D atabase character set to and from NLS_LANG character set in OCI. NLS_LANG character set to UTF-16 in OLE DB If database character set is NOT a subset of NLS_LANG character set: Database character set to and from UTF-16 in OCI
DBTYPE_CHAR	CHAR, VARCHAR2, CLOB	If database character set is a subset of the NLS_LANG character set: Datab ase character set to and from NLS_LANG in OCI If database character set is not a subset of NLS_LANG character set: Da tabase character set to and from UTF-16 in OCI. UTF-16 to NLS_LANG character set in OLE DB	< a name="1007225"> No unexpected data loss May degrade performance if data base character set is not a subset of NLS_LANG character set

ODBC Unicode Datatypes

In ODBC Unicode applications, use SQLWCHAR to store Unicode data. All stan dard Windows Unicode functions can be used for SQLWCHAR data manipulations. For example, wcslen counts the number of characters of SQLWCHAR data:

SQLWCHAR sqlStmt[] = L"select ename fro
m emp";
len = wcslen(sqlStmt);

Microsoft's ODBC 3.5 specification defines three Unicode datatype identifiers for the SQL_C_WCHAR, SQL_C_WVARCHAR, and SQL_WLONGVARCHAR clients; and three Unicode datatype identifiers for servers SQL_WCHAR, SQL_WVARCHAR< /code>, and SQL_WLONGVARCHAR.

For binding operations, specify datatypes for bo th client and server using SQLBindParameter. The following is an example of Unicode binding, where the client buffer Name indicates that Unicode data (SQL_C_WCHAR) is bound to the first bind variable associated with the Unicode column (SQL_WCHAR):

SQLBindParameter(StatementHandle, 1, SQL_PARAM_INPUT, SQL
_C_WCHAR,
SQL_WCHAR, NameLen, 0, (SQLPOINTER)Name, 0, &Name);

Table 7-11 represents the datatype mappings of the ODBC Unicode dataty pes for the server against SQL NCHAR datatypes.

Table 7-11 Server ODBC Unicode Datatype Mapping

< /tr>

ODBC Datatype	Oracle Datatype
SQL_WCHAR	NCHAR
SQL_WVARCHAR	NVARCHAR2
SQL_WLONGVARCHAR	NCLOB

According to ODBC specifications, SQL_WCHAR, SQL_WVARCH AR, and SQL_WLONGVARCHAR are treated as Unicode data, and are therefore measured in the number of characters inst ead of the number of bytes.

OLE DB Unicode Datatypes

OLE DB offers the wchar_t, BSTR, and OLESTR datatypes for a Unicode C client. In practice, wchar_t is the most common datatype and the others are for specific purposes. The following example assigns a static SQL statement:

wchar_t *sqlStmt = OLESTR("SELECT ename FROM emp");

The OLESTR macro works exactly like an "L" modifier to indicate the Unicode string. If you need to allocat e Unicode data buffer dynamically using OLESTR, then use the IMalloc allocator (for example, CoTaskMe mAlloc). However, using OLESTR is not the normal method for variable length data; use wchar_t* inste ad for generic string types. BSTR is similar. It is a string with a length prefix in the memory location preceding the s tring. Some functions and methods can accept only BSTR Unicode datatypes. Therefore, BSTR Unicode string mu st be manipulated with special functions like SysAllocString for allocation and SysFreeString for freeing m emory.

Unlike ODBC, OLE DB does not allow you to specify the server datatype explicitly. Wh en you set the client datatype, the OLE DB driver automatically performs data conversion if necessary.

Table 7-12 illustrates OLE DB datatype mapping.

Table 7-12 OLE DB Datatype Mapping

OLE DB Datatype	Oracle Datatype
DBTYPE_WCHAR	NCHAR or NVARCHAR2

If DBTYPE_BSTR is specified, then it is assumed to be DBTYPE_WCHAR be cause both are Unicode strings.

ADO Access

ADO is a high-level API to access database with the OLE DB and ODBC drivers. Most database application developers use the ADO interface on Windows because it is easily accessible from Visual Basic, the primary scripting language for Active Server Pages (ASP) for the Internet Information Server (IIS) . To OLE DB and ODBC drivers, ADO is simply an OLE DB consumer or ODBC application. ADO assumes that OLE DB and ODBC drivers are Unic ode-aware components; hence, it always attempts to manipulate Unicode data.

XML Programming with Unicode

XML support of Unicode is essential for software development for global markets so that text information can be exchanged in any language. Unicode uniformly supports almost every character and language, which makes it much easier to suppo rt multiple languages within XML. To enable Unicode for XML within an Oracle database, the character set of the database must be UTF- 8. By enabling Unicode text handling in your application, you acquire a basis for supporting any language. Every XML document is Unic ode text and potentially multilingual, unless it is guaranteed that only a known subset of Unicode characters will appear on your doc uments. Thus Oracle recommends that you enable Unicode for XML. Unicode support comes with Java and many other modern programming env ironments.

This section includes the following topics:

• Writing an XML File in Unicode with Java
• Reading an XML File in Unicode with Java
• Parsing an XML Stream in Unicode with Java

Writing an XML File in Unic ode with Java

A common mistake in reading and writing XML files is u sing the Reader and Writer classes for character input and output. Using Reader and Writ er for XML files should be avoided because it requires character set conversion based on the default character encoding of the runtime environment.

For example, using FileWriter class is not safe because it converts the document to the default character encoding. The output file can suffer from a parsing error or data loss if the docum ent contains characters that are not available in the default character encoding.

UTF-8 is popular for XML documents, but UTF-8 is not usually the default file encoding for Java. Thus using a Java class that assumes the defa ult file encoding can cause problems.

The following example shows how to avoid these proble ms:

import java.io.*;
import oracle.xml.parser.v2.*;

public class I18nSafeXMLFileWritingSample 
{
  public static
 void main(String[] args) throws Exception
  {
    // create a test document
    XMLDocument         doc  = new XMLDocument();
    doc.setVersion( "1.0" );

    doc.appendChild(doc.createComment( "This is a test empty document." ));
    doc.appendChild(doc.createEleme
nt( "root" ));
    
    // create a file
    File                fi
le = new File( "myfile.xml" );

    // create a binary output stream to write to the file
 just created
    FileOutputStream    fos  = new FileOutputStream( file );

    // create a Writer that converts Java character stream to UTF-8 stream
    OutputStreamWriter  osw
  = new OutputStreamWriter( fos, "UTF8" );

    // buffering for efficiency
    Writer              w    = new BufferedWriter( osw );

    // create a Print
Writer to adapt to the printing method
    PrintWriter         out  = new PrintWriter( w );
<
/a>
    // print the document to the file through the connected objects
    doc.print( ou
t ); 
  }
}

Reading an XML File in Unicode with Java

Do not read XML files as text input. When reading an XML document stored in a file system, use the parser to a utomatically detect the character encoding of the document. Avoid using a Reader class or specifying a character encodin g on the input stream. Given a binary input stream with no external encoding information, the parser automatically figures out the ch aracter encoding based on the byte order mark and encoding declaration of the XML document. Any well-formed document in any supported encoding can be successfully parsed using the following sample code:

import java.io.*;
import oracle.xml.parser.v2.*;

public class I18nSafeXMLFileReadingSam
ple 
{
  public static void main(String[] args) throws Exception

{
    // create an instance of the xml file
    File                 file = new File( "my
file.xml" );

    // create a binary input stream
    FileInputStre
am      fis  = new FileInputStream( file );

    // buffering for efficiency
    BufferedInputStream  in = new BufferedInputStream( fis );

    // get an in
stance of the parser
    DOMParser  parser = new DOMParser();


// parse the xml file
    parser.parse( in );
  }
}

Parsing an XML Stream in Unicode with Java

When the source of an XML document is not a fi le system, the encoding information is usually available before reading the document. For example, if the input document is provided in the form of a Java character stream or Reader, its encoding is evident and no detection should take place. The parser can begin pa rsing a Reader in Unicode without regard to the character encoding.

The following is an exa mple of parsing a document with external encoding information:

import java.io.*;
import java.net.*;
import org.xml.sax.*;
import oracle.xml.parser.v2.*;

public class I18nSafeXMLStreamReadingSample 
{

public static void main(String[] args) throws Exception
  {
    // create an instance of
the xml file
    URL  url = new URL( "http://myhost/mydocument.xml" );

    // create a connection to the xml document    
    URLConnection  conn = url.openConnection();

    // get an input stream
    InputStream  is = conn.getInputStream();
<
a name="1007697">
    // buffering for efficiency
    BufferedInputStream  bis = new
BufferedInputStream( is );

    /* figure out the character encoding here
              */
    /* a typical source of encoding information is the content-type header */
    /* we assume it is found to be utf-8 in this example                   */
    String  charset = "utf-
8";

    // create an InputSource for UTF-8 stream
    InputSource
 in = new InputSource( bis );
    in.setEncoding( charset );
    

   // get an instance of the parser
    DOMParser  parser = new DOMParser();

    // parse the xml stream
    parser.parse( in );  
  }
}

---

Pr evious Next

Copyright © 1996, 2003 Oracle Co rporation All Rights Reserved.

Home < a href="../../nav/portal_3.htm"> Book List Contents Index Master Index Feedback