SYSTEM AND METHOD FOR DETERMINING A CHARACTER ENCODING SCHEME

The present invention provides a system and method for determining an appropriate (e.g., best-fit) character encoding scheme including the steps of (i) generating a set of reference characters based on a reference character encoding scheme and a first set of bytes; (ii) generating a set of test characters based on a test character encoding scheme and said first set of bytes; (ii) generating a set of test bytes based on said test character encoding scheme and said set of test characters; (iv) generating a set of comparison characters based on said reference character encoding scheme and said set of test bytes; and (v) comparing said set of reference characters to said set of comparison characters. In one embodiment of the present invention, the aforementioned steps are implemented as a JAVA based software program with Unicode (e.g., USC2) as the reference character encoding scheme. The present invention can determine the degree of match between the reference character encoding scheme and the test character encoding scheme. If the degree of match is within a predefined threshold, the test character encoding scheme can be selected as the character encoding scheme for a particular set of bytes or can be saved for comparison with other test character encoding schemes.