ISO/IEC 2022
493590
224992577
2008-07-11T10:33:10Z
Tedickey
1942604
rv - projects (which add no information) belong on the talk page
'''ISO 2022''', more formally '''ISO/IEC 2022''' "Information Technology—Character code structure and extension techniques", is an [[International Organization for Standardization|ISO]] standard (equivalent to the [[Ecma International|ECMA]] standard '''ECMA-35''') specifying
* a technique for including multiple character sets in a single [[character encoding]], and
* a technique for representing character sets which cannot be represented in 7 bits.
Unlike [[ISO 8859]] character encodings which use 8 [[bit]]s for every character, the ISO 2022 encodings are variable size encodings typically using either 8 or 16 bits per character. Several character encodings use ISO 2022 mechanisms. For example, '''ISO-2022-JP''' is a widely used character encoding for the Japanese language.
== Introduction ==
Many languages or [[language family|language families]] not based on the [[Latin alphabet]] such as [[greek language|Greek]], [[russian language|Russian]], [[Arabic language|Arabic]], or [[Hebrew language|Hebrew]] have historically been represented on computers with 8-bit [[extended ASCII]] encodings including the [[ISO 8859]] family of character sets. Written [[East Asian]] languages, specifically [[Chinese language|Chinese]], [[Japanese language|Japanese]], and [[Korean language|Korean]], use far more characters than can be represented in an 8-[[bit]] computer [[byte]] and were first represented on computers with language-specific [[DBCS|double byte encodings]].
ISO 2022 was developed as a technique to attack both of these problems: to represent characters in multiple character sets within a single character encoding, and to represent large character sets.
Being based on [[ISO/IEC 646|ISO 646]], ISO 2022 exhibits many of ISO 646's properties. For example, the [[most significant bit]] of each byte does not carry any meaning; this allows ISO 2022 (like ISO 646) to be easily transmitted through 7-bit communication channels. (This 7-bit property also forms the basis of the [[Extended Unix Code|EUC]] code.)
To represent multiple character sets, the ISO 2022 character encodings include [[escape sequence]]s which indicate the character set for characters which follow. The escape sequences are registered with ISO and are often three characters long starting with the [[ASCII]] ESCAPE character (hexadecimal 1B, octal 33). These character encodings require data to be processed sequentially in a forward direction since the correct interpretation of the data depends on the most recently encountered escape sequence.
To represent large character sets, ISO 2022 builds on ISO 646's property that 1 byte can define 94 graphic (printable) characters (in addition to space and 33 control characters). Using two bytes, it is thus possible to represent up to 8836 (94×94) characters; and, using three bytes, up to 830584 (94×94×94) characters. For the two-byte character sets, the code point of each character is normally specified in so-called ''kuten'' form (sometimes called ''quwei'', especially when dealing with [[GB2312]] and related standards), which specifies a zone (''ku'' or ''qu''), and the point (''ten'') or position (''wei'') of that character within the zone.
The escape sequences therefore do not only declare which character set is being used, but also, by knowing the properties of these character sets, know whether a 94-, 8836-, or 830584-character (or some other sized) encoding is being dealt with.
In practice, the escape sequences declaring the national character sets may be absent if context or convention dictates that a certain national character set is to be used. For example, RFC 1922, which defines ISO-2022-CN, allows ASCII SHIFT characters to be used instead of escape sequences.
Although the ISO 2022 character sets are still in common use, particularly ISO-2022-JP, most modern [[e-mail]] applications are converting to use the simpler [[Unicode]] character encodings such as [[UTF-8]].
== Code structure ==
ISO/IEC 2022 coding specifies a two-layer mapping between character codes and displayed characters. [[Escape sequence]]s allow any of a large registry of graphic character sets to be "designated" into one of four working sets, named G0 through G3, and shorter control sequences specify the working set that is "invoked" to interpret bytes in the stream.
Character codes from the 7-bit ASCII graphic range (0x20–0x7F) are referred to as "GL" codes, being on the left side of a character code table, while codes from the "high ASCII" range (0xA0–0xFF), if available, are referred to as the "GR" codes.
By default, GL codes specify G0 characters, and GR codes specify G1 characters, but this may be modified with control codes:
{| class="wikitable"
! Code !! Abbr. !! Name !! Effect
|-
| <tt>0x0F</tt> || SI<br>LS0 || [[Shift In]]<br>Locking shift zero || GL encodes G0 from now on
|-
| <tt>0x0E</tt> || SO<br>LS1 || [[Shift Out]]<br>Locking shift one || GL encodes G1 from now on
|-
| <tt>ESC 0x6E</tt> (n) || LS2 || Locking shift two || GL encodes G2 from now on
|-
| <tt>ESC 0x6F</tt> (o) || LS3 || Locking shift three || GL encodes G3 from now on
|-
| <tt>0x8E<br>ESC 0x4E</tt> (N) || SS2 || Single shift two || GL encodes G2 for next character only
|-
| <tt>0x8F<br>ESC 0x4F</tt> (O) || SS3 || Single shift three || GL encodes G3 for next character only
|-
| <tt>ESC 0x7E</tt> (~) || LS1R || Locking shift one right || GR encodes G1 from now on
|-
| <tt>ESC 0x7D</tt> (}) || LS2R || Locking shift two right || GR encodes G2 from now on
|-
| <tt>ESC 0x7C</tt> (|) || LS3R || Locking shift three right || GR encodes G3 from now on
|}
Each of the four working sets may be a 94-character set or a 94<sup>n</sup>-character set. Additionally, G1 through G3 may be a 96- or 96<sup>n</sup>-character set. When one of the latter is invoked in the GL region, the space and delete characters (codes 0x20 and 0x7F) are not available.
There are additional (rarely used) features for switching control character sets, but this is a single-level lookup: the 0x00–0x1F range is the C0 control character set, the 0x80–0x9F range is the C1 control character set, and there are escape sequences which switch in various alternatives. It is required that any C0 character set include the ESC character at position 0x1B, so that further changes are possible.
As seen in the SS2 and SS3 examples above, single control characters from the C1 control character set may be invoked using only 7 bits using the sequences <code>ESC 0x40 (@)</code> through <code>ESC 0x5F (_)</code>. Additional control functions are assigned in the range <code>ESC 0x60 (`)</code> through <code>ESC 0x7E (~)</code>. While this article describes escape sequences using the corresponding ASCII characters, they are actually defined in terms of byte values, and the graphic assigned to that byte value may be altered without affecting the control sequence.
Escape sequences to designate character sets take the form <code>ESC ''I'' [''I''...] ''F''</code>, where there are one or more intermediate ''I'' bytes from the range 0x20–0x2F, and a final ''F'' byte from the range 0x40–0x7F. (The range 0x30–0x3F is reserved for private-use ''F'' bytes.) The ''I'' bytes identify the type of character set and the working set it is to be designated to, while the ''F'' byte identifies the character set itself.
{| class="wikitable"
! Code !! Hex !! Abbr. !! Name !! Effect
|-
| <tt>ESC ! ''F''</tt> || <tt>1B 21 ''F''</tt> || CZD || C0-designate || ''F'' selects a C0 control character set to be used.
|-
| <tt>ESC " ''F''</tt> || <tt>1B 22 ''F''</tt> || C1D || C1-designate || ''F'' selects a C1 control character set to be used.
|-
| <tt>ESC % ''F''</tt> || <tt>1B 25 ''F''</tt> || DOCS || Designate other coding system || ''F'' selects an 8-bit code; use <code>ESC % @</code> to return to ISO/IEC 2022.
|-
| <tt>ESC % / ''F''</tt> || <tt>1B 25 2F ''F''</tt> || DOCS || Designate other coding system || ''F'' selects an 8-bit code; there is no standard way to return.
|-
| <tt>ESC ( ''F''</tt> || <tt>1B 28 ''F''</tt> || GZD4 || G0-designate 94-set || ''F'' selects a 94-character set to be used for G0.
|-
| <tt>ESC ) ''F''</tt> || <tt>1B 29 ''F''</tt> || G1D4 || G1-designate 94-set || ''F'' selects a 94-character set to be used for G1.
|-
| <tt>ESC * ''F''</tt> || <tt>1B 2A ''F''</tt> || G2D4 || G2-designate 94-set || ''F'' selects a 94-character set to be used for G2.
|-
| <tt>ESC + ''F''</tt> || <tt>1B 2B ''F''</tt> || G3D4 || G3-designate 94-set || ''F'' selects a 94-character set to be used for G3.
|-
| <tt>ESC - ''F''</tt> || <tt>1B 2D ''F''</tt> || G1D6 || G1-designate 96-set || ''F'' selects a 96-character set to be used for G1.
|-
| <tt>ESC . ''F''</tt> || <tt>1B 2E ''F''</tt> || G2D6 || G2-designate 96-set || ''F'' selects a 96-character set to be used for G2.
|-
| <tt>ESC / ''F''</tt> || <tt>1B 2F ''F''</tt> || G3D6 || G3-designate 96-set || ''F'' selects a 96-character set to be used for G3.
|-
| <tt>ESC $ ( ''F''</tt> || <tt>1B 24 28 ''F''</tt> || GZDM4 || G0-designate multibyte 94-set || ''F'' selects a 94<sup>n</sup>-character set to be used for G0.
|-
| <tt>ESC $ ) ''F''</tt> || <tt>1B 24 29 ''F''</tt> || G1DM4 || G1-designate multibyte 94-set || ''F'' selects a 94<sup>n</sup>-character set to be used for G1.
|-
| <tt>ESC $ * ''F''</tt> || <tt>1B 24 2A ''F''</tt> || G2DM4 || G2-designate multibyte 94-set || ''F'' selects a 94<sup>n</sup>-character set to be used for G2.
|-
| <tt>ESC $ + ''F''</tt> || <tt>1B 24 2B ''F''</tt> || G3DM4 || G3-designate multibyte 94-set || ''F'' selects a 94<sup>n</sup>-character set to be used for G3.
|-
| <tt>ESC $ - ''F''</tt> || <tt>1B 24 2D ''F''</tt> || G1DM6 || G1-designate multibyte 96-set || ''F'' selects a 96<sup>n</sup>-character set to be used for G1.
|-
| <tt>ESC $ . ''F''</tt> || <tt>1B 24 2E ''F''</tt> || G2DM6 || G2-designate multibyte 96-set || ''F'' selects a 96<sup>n</sup>-character set to be used for G2.
|-
| <tt>ESC $ / ''F''</tt> || <tt>1B 24 2F ''F''</tt> || G3DM6 || G3-designate multibyte 96-set || ''F'' selects a 96<sup>n</sup>-character set to be used for G3.
|}
Note that the registry of ''F'' bytes is independent for the different types. The 94-character graphic set designated by <code>ESC ( A</code> through <code>ESC + A</code> is not related in any way to the 96-character set designated by <code>ESC - A</code> through <code>ESC / A</code>. And neither of those are related to the 94<sup>n</sup>-character set designated by <code>ESC $ ( A</code> through <code>ESC $ + A</code>, and so on; the final bytes must be interpreted in context. (Indeed, without any intermediate bytes, <code>ESC A</code> is a way of specifying the C1 control code 0x81.)
Also note that C0 and C1 control character sets are independent; the C0 control character set designated by <code>ESC ! A</code> (which happens to be the NATS control set for newspaper text transmission) is not the same as the C1 control character set designated by <code>ESC " A</code> (the [[CCITT]] attribute control set for [[Videotex]]).
Additional ''I'' bytes may be added before the ''F'' byte to extend the ''F'' byte range. This is currently only used with 94-character sets, where codes of the form <code>ESC ( ! ''F''</code> have been assigned. At the other extreme, no multibyte 96-sets have been registered, so the sequences above are strictly theoretical.
== ISO 2022 character sets ==
Character encodings using ISO 2022 mechanism include:
* '''ISO-2022-JP'''. A widely used encoding for Japanese. Starts in ASCII and includes the following escape sequences
** <tt>ESC ( B </tt> to switch to ASCII (1 byte per character)
** <tt>ESC ( J </tt> to switch to [[JIS X 0201|JIS X 0201-1976]] (ISO 646:JP) Roman set (1 byte per character)
** <tt>ESC $ @ </tt> to switch to [[JIS X 0208|JIS X 0208-1978]] (2 bytes per character)
** <tt>ESC $ B </tt> to switch to [[JIS X 0208|JIS X 0208-1983]] (2 bytes per character)
* '''ISO-2022-JP-1'''. The same as ISO-2022-JP with one additional escape sequence
** <tt>ESC $ ( D</tt> to switch to [[JIS X 0212|JIS X 0212-1990]] (2 bytes per character)
* '''ISO-2022-JP-2'''. A multilingual extension of ISO-2022-JP. The same as ISO-2022-JP-1 with the following additional escape sequences
** <tt>ESC $ A </tt> to switch to [[GB2312|GB 2312-1980]] (2 bytes per character)
** <tt>ESC $ ( C</tt> to switch to [[KSX1001|KS X 1001-1992]] (2 bytes per character)
** <tt>ESC - A </tt> to switch to [[ISO 8859-1]] high part, Extended Latin 1 set (1 byte per character)
** <tt>ESC - F </tt> to switch to [[ISO 8859-7]] high part, Basic Greek set (1 byte per character)
* '''ISO-2022-JP-3'''. The same as ISO-2022-JP with three additional escape sequences
** <tt>ESC ( I </tt> to switch to [[JIS X 0201|JIS X 0201-1976]] Kana set (1 byte per character)
** <tt>ESC $ ( O</tt> to switch to [[JIS X 0213|JIS X 0213-2000]] Plane 1 (2 bytes per character)
** <tt>ESC $ ( P</tt> to switch to [[JIS X 0213|JIS X 0213-2000]] Plane 2 (2 bytes per character)
* '''ISO-2022-JP-2004'''. The same as ISO-2022-JP-3 with one additional escape sequence
** <tt>ESC $ ( Q</tt> to switch to [[JIS X 0213|JIS X 0213-2004]] Plane 1 (2 bytes per character)
* '''ISO-2022-KR'''. An encoding for Korean.
** <tt>ESC $ ( C</tt> to switch to [[KSX1001|KS X 1001-1992]]<ref name="ksx">{{cite web |url=http://examples.oreilly.com/cjkvinfo/AppL/ksx1001.pdf |title=KS X 1001:1992}}</ref> <ref name="ksc">{{cite web |url=http://www.itscj.ipsj.or.jp/ISO-IR/149.pdf |title=KS C 5601:1987|date=1988-10-01}}</ref>, previously named KS C 5601-1987 (2 bytes per character)
* '''ISO-2022-CN'''. An encoding for Chinese.
** <tt>ESC $ ( A</tt> to switch to [[GB2312|GB 2312-1980]] (2 bytes per character)
** <tt>ESC $ ( G</tt> to switch to [[CNS11643|CNS 11643-1992]] Plane 1 (2 bytes per character)
** <tt>ESC $ ( H</tt> to switch to CNS 11643-1992 Plane 2 (2 bytes per character)
* '''ISO-2022-CN-EXT'''. The same as ISO-2022-CN with six additional escape sequences
** <tt>ESC $ ( E</tt> to switch to [[ISO-IR-165]] (2 bytes per character)
** <tt>ESC $ ( I</tt> to switch to CNS 11643-1992 Plane 3 (2 bytes per character)
** <tt>ESC $ ( J</tt> to switch to CNS 11643-1992 Plane 4 (2 bytes per character)
** <tt>ESC $ ( K</tt> to switch to CNS 11643-1992 Plane 5 (2 bytes per character)
** <tt>ESC $ ( L</tt> to switch to CNS 11643-1992 Plane 6 (2 bytes per character)
** <tt>ESC $ ( M</tt> to switch to CNS 11643-1992 Plane 7 (2 bytes per character)
The character after the <code>ESC</code> (for single-byte character sets) or <code>ESC $</code> (for multi-byte character sets) specifies the type of character set and working set that is designated to. In the above examples, the character <code>(</code> (0x28) designates a 94-character set to the G0 character set. This may be replaced by <code>)</code>, <code>*</code> or <code>+</code> (0x29–0x2B) to designate to the G1–G3 character sets.
Two of the codes above are 96-character codes, and in the above examples, the character <code>-</code> (0x2D) designates to the G1 character set. This may be replaced with <code>.</code> or <code>/</code> (0x2E or 0x2F) to designate to the G2 or G3 character sets. As mentioned earlier, a 96-character set may not be designated to the G0 set.
There are three special cases for multi-byte codes. The code sequences <code>ESC $ @</code>, <code>ESC $ A</code>, and <code>ESC $ B</code> were all registered before the ISO 2022 standard was finalized, so must be accepted as synonyms for the sequences <code>ESC $ ( @</code> through <code>ESC $ ( B</code> to designate to the G0 character set. The latter form may also be used, and may be adapated by changing the <code>(</code> character to designate to the G1 through G3 character sets.
The standard also defines a way to specify coding systems that do not follow its own structure. Of particular interest, the sequence <code>ESC % G</code> designates the [[UTF-8]] coding system, which does not reserve the range 0x80–0xAF for control characters.
== See also ==
*[[ISO/IEC 646]]
*[[C0 and C1 control codes]]
*[[CJK]]
*[[Mojibake]]
==References==
*Lunde, Ken. ''CJKV Information Processing''. Cambridge, Massachusetts: O'Reilly & Associates, 1998. ISBN 1-56592-224-7.
<references/>
== External links ==
* [http://www.iso.org/ International Organization for Standardization]
* [http://www.ecma-international.org/publications/standards/Ecma-035.htm ECMA-35], equivalent to ISO/IEC 2022 and freely downloadable.
* [http://www.itscj.ipsj.or.jp/ISO-IR/ International Register of Coded Character Sets to be Used with Escape Sequences], a full list of assigned character sets and their escape sequences
*[http://tronweb.super-nova.co.jp/characcodehist.html History of Character Codes in North America, Europe, and East Asia]
*[ftp://ftp.ora.com/pub/examples/nutshell/ujip/doc/cjk.inf CJK.INF: a document on encoding Chinese, Japanese, and Korean (CJK) languages, including a discussion of the various variants of ISO 2022]. Also [http://examples.oreilly.com/cjkvinfo/doc/cjk.inf available by HTTP].
;[[Request for Comments|RFCs]]
* RFC 1468: description of ISO-2022-JP
* RFC 2237: description of ISO-2022-JP-1
* RFC 1554: description of ISO-2022-JP-2
* RFC 1922: description of ISO-2022-CN and ISO-2022-CN-EXT
* RFC 1557: description of ISO-2022-KR
[[Category:ISO standards|#02022]]
[[Category:IEC standards]]
[[Category:Character sets]]
[[Category:Ecma standards]]
[[de:ISO 2022]]
[[fr:ISO 2022]]
[[ja:ISO/IEC 2022]]
[[zh:ISO/IEC 2022]]