KANJIDIC Project

From EDRDG Wiki
Jump to navigation Jump to search

The KANJIDIC Project

(Note that this page in the process of being rewritten, so be patient with any aspects that seems incomplete.)

Introduction

The KANJIDIC project, which began in 1991, has the goal of compiling and distributing comprehensive information on the kanji used in Japanese text processing. It covers the 13,108 kanji in three main Japanese standards:

Three data files are distributed by this project:

  • the KANJIDIC2 file, which is in XML format and Unicode/UTF-8 coding, and contains information about all 13,108 kanji. (download)
  • the KANJIDIC file, which in in EUC-JP coding and covers the 6,355 kanji in JIS X 0208. For this there is the

There is also a combined overview of the KANJIDIC/KANJD212 files.

Content & Format

The database and distributed data files contain an entry for each of the kanji, with each entry containing a number of fields of data about the kanji. The data is described in the following table. The format of the distributed files as as follows:

  • the KANJIDIC and KANJD212 files are text files with one line per kanji and the information fields separated by spaces. The format of each line is:
    • the kanji itself followed by the hexadecimal form of the JIS "ku-ten" coding, e.g. "亜 3021";
    • information fields beginning with one or two-letter codes as per the table below. For example "S10" indicates a stroke count of 10;
    • the Japanese readings of the kanji. ON readings (音読み) are generally in katakana and KUN readings (訓読み) in hiragana. An exception is the set of kokuji for measurements such as centimetres, where the reading is in katakana. Hyphens are used to indicate prefixes/suffixes, and '.' indicates the portion of the reading that is okurigana. There may be several classes of reading fields, with ordinary readings first, followed by members of the other classes, if any. The current other classes, and their tagging, are:
      • where the kanji has special nanori (i.e. name) readings, these are preceded the marker "T1";
      • where the kanji is a radical, and the radical name is not already a reading, the radical name is preceded the marker "T2".
    • the meanings (usually in English). Each field begins with an open brace '{' and ends at the next close brace '}'.
  • the KANJIDIC2 file is in XML and is structured according to its DTD (Document Type Definition). The DTD contains extensive annotations and is intended to be the primary documentation for the file. This sample illustrates the structure of a typical entry. Information fields are grouped by type within entities such as <dic_number> and <query_code>, with specific values indicated by an attribute code. For example the kanji 亜 has the number 43 in the original Nelson kanji dictionary and 81 in the New Nelson. This is recorded in the XML file as:
<dic_number>
<dic_ref dr_type="nelson_c">43</dic_ref>
<dic_ref dr_type="nelson_n">81</dic_ref>
....
</dic_number>
Kanjidic Information Fields
Field Kanjidic Code
(if any)
Group Entity Entity/Attribute Comment
Kanji none literal
JIS code-point codepoint cp_value cp_type="jis208" e.g. 亜 is "3021" in KANJIDIC and
"16-01" in KANJIDIC2
Unicode code-point U codepoint cp_value cp_type="ucs"
Radical (KangXi Zidian) B/C radical rad_value rad_type="classical" Where Nelson uses the KangXi radical this has a "B" code, otherwise it has a "C" code
Radical (Nelson) B radical rad_value rad_type="nelson_c"
Grade G misc grade The "grade" of the kanji.
- G1 to G6 indicates the grade level as specified by the Japanese Ministry of Education for kanji that are to be taught in elementary school (1006 Kanji). These are sometimes called the kyōiku (education) kanji and are part of the set of jōyō (daily use) kanji;
- G8 indicates the remaining jōyō kanji that are to be taught in secondary school (additional 1130 Kanji);
- G9 and G10 indicate jinmeiyō ("for use in names") kanji which in addition to the jōyō kanji are approved for use in family name registers and other official documents. G9 (649 kanji, of which 640 are in KANJIDIC) indicates the kanji is a "regular" name kanji, and G10 (212 kanji of which 130 are in KANJIDIC) indicates the kanji is a variant of a jōyō kanji.
Stroke count S misc stroke_count The stroke count of the kanji. If more than one, the first is considered the accepted count, while subsequent ones are common miscounts. (See the section later in this document on counting strokes for some of the rules applied especially to radicals.)
Frequency-of-use ranking F misc freq The 2,501 most-used characters have a ranking which expresses the relative frequency of occurrence of a character in modern Japanese. The data is based on an analysis of word frequencies in the Mainichi Shimbun over 4 years by Alexandre Girardi. Note: (a) these frequencies are biased towards words and kanji used in newspaper articles, and (b) the relative frequencies for the last few hundred kanji so graded is quite imprecise.

(This table will be extended as time goes on.)

Radical and Stroke Counting Rules

These rules apply to:

  1. the stroke-counts themselves;
  2. the stroke counts in the SKIP codes. Where this results in a SKIP which differs from that in the NJECD, or in the non-NJECD SKIPs provided by Jack Halpern, the Jack Halpern version is included prefixed with "ZR".

Radicals

The radicals listed below are ones where there are differing approaches to the counting of radicals in the various references. The stroke counting in this file does not strictly follow any reference, but tends to more aligned to Halpern.

  1. B54 ENNYOU - 廴. Traditionally counted as 3 strokes, but more recently often counted as 2. S&H count this as 2; Nelson, Halpern, Koujien, etc, count it is 3. I treat it as 3.
  2. B97 URI - 瓜. Traditionally counted as 5 strokes, as the middle portion looks like a katakana ム. Modern glyphs invariably make it look like 6 strokes. Nelson says it is 5 strokes. Halpern does too, but then counts the shape as 6 in other kanji. Koujien says 6, as do S&H. I treat it as 6.
  3. B113 SHIMESU e.g. 礼, is counted as 4 strokes in that form, and 5 strokes in its older form, 祀. 18 kanji are in the 4-stroke form and 20 are in the 5-stroke form. (Nelson and S&H count it as 4; Halpern counts it as 4 or 5. [See Note 1.])
  4. B131 SHIN/KERAI 臣. Counted as 7 (Nelson counts it as 6, Halpern as 7 (in the book), and S&H as both for different kanji.)
  5. B136 MAI ASHI 舛. Counted as 7 (traditionally counted as 6, in accordance with the older writing of `ヰ'. Nelson counts as 6, S&H as 7, and Halpern as 7 for 常用 and 人名用漢字 and 6 for the rest.) Note this is also applied to counting 絳 and for kanji with the 韋 pattern.
  6. B140 KUSA-KANMURI e.g. 苛 always counted as 3 strokes (Halpern counts this 4 strokes for the (mostly level 2) kanji where the older form is often printed.) Note that this has been carried through to kanji where this element is not the indexing radical, such as 朦.
  7. B162 SHIN-NYUU e.g. 遙 or 逢 counted as 3 or 4 strokes. (Nelson and S&H count it as 2 strokes, and Halpern as either 3 or 4.) [See Note 1 below.]
  8. B163 OOZATOZUKIRI & B170 KOZATO-HEN 邦 and 阡 always counted as 3 strokes (Nelson and S&H count it as 2, Halpern as 3.) This also applies where it appears mid-kanji, such as in 橢.
  9. B184 SHOKU HEN 食, 飢, etc.is counted as 8 strokes in the 飢 form, and as 9 strokes in the 飭 and 餐 forms. (Nelson and S&H count it as 8 strokes, and Halpern as 8 or 9.) [See Note 1. below.]
  10. B199 MUGI 麦 always counted as 7 strokes, except for 麥 & 麩 where it is counted as 11. (Nelson and Halpern do the same, and S&H avoid treating it as a radical, but count it as 12 in the remainder.)
  11. The ROO or OI radical (老) has a variant consisting of the top 4 strokes. For example, it is in 者. Traditionally, this variant had an extra dot, and was counted as 5 strokes. I'm counting it as 4 throughout.

Other Stroke Patterns

  1. While the pattern 臼 is a 6-stroke radical, the top half of 叟 is made up of three distinct parts totalling 8 strokes. Note that this also is the case with 嫂, 溲, 艘 and 痩 despite the simplification in the JIS glyphs.
  2. 牙 (KIBA HEN) is a problem. It is classically counted as 4 strokes, but these days has a flick that makes it effectively 5. Halpern, Nelson and S&H usually have it as 5 strokes, so I'm standardizing on that.
  3. Another little horror is 旡 (MU or NASHI), which is classically counted as 4 strokes. The most common variant has 5 strokes, but looks like 6. Halpern, S&H and the Classical Nelson count this as 4 strokes, and the New Nelson as 5. I'm making it 5 too.
  4. The JUU or ASHIATO radical is at the bottom of 禽 and 禺. It is traditionally counted as 5 strokes, although sometimes it looks like 4. I'm using 5 throughout.
  5. A related shape is ム, as in 瓜, 孤, 弧, etc. This is sometimes counted as two strokes (both Nelsons) and sometimes as three strokes (Halpern, S&H). Classically it is regarded as two strokes. I am using 6 strokes for 瓜.
  6. The pattern to the left of 敝, which appears in several kanji, e.g. 幣 and 瞥, has 8 strokes. (There are 3 strokes at the top as in 尚.)
  7. The "east" pattern (東) has 8 strokes. There is an older form in which there are two strokes in the box (柬). It is counted as 8 strokes here in the 東 form (e.g. 諌) and 9 in the 柬 form, as in 諫.
  8. The pattern at the bottom of 雋 is counted as 4 strokes in modern dictionaries, although traditionally it was 5.
  9. The pattern 巻, which appears in several kanji, is counted as 9 strokes. Several dictionaries count it as either 8 or 9.
  10. The pattern on the left of 収 is variously handled as 2 strokes or 3 strokes. As more recent dictionaries make it 4, I will do so too.
  11. The 攵 pattern has 3 and 4-stroke versions, and sometimes the glyphs can be confusing as to which is used. In the 緻 kanji, for example, it is traditionally counted as 3, but Spahn & Hadamitzky count it as 4 and the Nelsons include both.

Note: The JIS X 0208-1990 standard does not formally specify the precise glyphs used for kanji, however the glyphs it uses in the published version have become de facto standards for many font compilations. In the published standard, for several kanji, e.g. 辿/迚, 礼/祀, 飢/飭, the JIS level one kanji use the simpler form, and the Level 2 kanji use the older more complex form. Just to make matters worse, many fonts for JIS X 0208 kanji are based on the bit-maps specified in JIS X 9051-1984 standard, which defines the 16x16 patterns for JIS X 0208-1983 characters. According to Ken Lunde: "This standard was not very good, and JSA is no longer supporting it." Anyway, JIS X 9051-1984 had the simpler form for all these bushu in both Levels 1 and 2, as well as having simplifications of kanji like 濾. Thus, as the font foundries have freedom to choose whichever glyphs they like, what you see on your screen may well not agree with these rules. All the rules in this appendix relate to the glyphs as published in the JIS X 0208-1990 standard, and as appearing in font compilations based on them.

History

(some comments by Jim Breen)

KANJIDIC began around 1991 as two files: jis1detl.lst and jis2detl.lst, which were later merged into a single file.

The first file was compiled initially from the file "kinfo.dat" supplied by Stephen Chung, who in turn compiled his file from a file prepared by Mike Erickson. I originally added about 1900 "meanings" by James Heisig keyed in by Kevin Moore from the book "Remembering The Kanji". I later added the meanings from Rik Smoody's files, compiled when he was working for Sony in Japan. These appear to have been based on Nelson.

The second file was compiled from a complete JIS2 list with Bushu and stroke counts kindly supplied to me by Jon Crossley, to which I added Nelson numbers, yomikata and meanings extracted from Rik Smoody's file.

Theresa Martin was an early assister with this file, particularly with tracking down and correcting many mistranscribed yomikata (the old zu/dzu, oo/ou, ji/dji, etc. problems).

Jeffrey Friedl did a major overhaul in September-October 1992, in which he added the original frequency rankings, Halpern codes, SKIP patterns, updated the grading ("G" fields) to reflect the modern Jouyou lists, corrected radical numbers, corrected stroke counts and readings to fall in line with modern usage.

Magnus Halldorsson corrected some erroneous Halpern numbers, and provided them for a lot of the radicals. He provided the list of Heisig indices, which he originally compiled himself, then verified and expanded using lists from Richard Walters and Antti Karttunen. He also passed on to me the list of Gakken indices compiled by Antti Karttunen.

Lee Collins provided the Unicode mappings.

Iain Sinclair has provided the yomikata, meanings and S&H indices of many of the obscure JIS2 kanji.

Christian Wittern, a Sinologist working at Kyoto University, sent me a monster file prepared by Dr Urs App from Hanazono College. From this I have extracted the Four Corner and Morohashi information. Christian also provided the original Pinyin details, which were later replaced. I am very grateful for these significant contributions.

In March 1994 the Morohashi indices were proof-read and corrected by Christian.

Alfredo Pinochet supplied all the Henshall numbers.

Ingar Holst has provided considerable assistance in regularizing the Bnnn and Cnnn radical classifications to remove some errors that were in the original JIS2 file, and to make it all conform to Nelson's classification.

In mid-1993 I withdrew the SKIP codes from the distributed file as it appeared that their presence violated Jack Halpern's copyright on these codes. Jeffrey Friedl contacted Jack about this, and Jack obtained permission from his publisher for the codes to be included subject (initially) to copyright and usage restrictions. In March 1994 the Halpern indices and SKIP codes were checked against an extract from Jack's files, and the "Z" mis-classification codes added, again from his files. Jack has also made a lot of useful comments and suggestions about the content and format of the file. I am most grateful to Jack for his permission and assistance, and also to Jeffrey for making the contact.

In May 1995, a number of updates took place. Jeffrey Friedl established contact with James Heisig, and obtained a further set of his indices. I contacted Mark Spahn (via the "honyaku" mailing list) and he kindly provided most of the missing S&H descriptors, and Jack Halpern released to me the SKIP codes of the kanji not in the New Japanese-English Character Dictionary. For all this material I am most grateful.

In August 1995, I added the O'Neill index numbers. These were compiled by Jenny Nazak, David Rosenfeld and myself. Thanks to Jenny & David for their assistance.

In January and February 1996 the Morohashi numbers were checked thoroughly against two important sources: a file of Unicode-Morohashi data (Uni2Dict) which was prepared by Koichi Yasuoka from the allocation in the JIS X 0221 standard, and the review draft of the proposed revision of the JIS X 0208 standard, which was prepared by the INSTAC Committee, and made available in a text file, thus enabling comparisons. All the mismatches between the three files were examined against the Morohashi text, and extensive corrections made to all three files. I am grateful to Koichi Yasuoka and Masayuki Toyoshima for their considerable assistance in this task.

In March 1996 the Korean readings were added. They were provided by Dr Charles Muller, then of of Toyo Gakuen University, to whom I am most grateful. Chuck's compilation of Korean readings is extremely thorough and scholarly, and I am pleased to be able to incorporate them.

In April 1996 the readings of all the kanji were compared with those in the JIS X 0208 draft, and a number of corrections and additions made.

In May 1996 I carried out a "unification" of the readings of the KANJIDIC and KANJD212 files, wherein all the readings of the "itaiji" were brought into line. The identification of these itaiji was drawn from a file posted to the fj.kanji group by Taichi Kawabata (kawabata@is.s.u-tokyo.ac.jp), which was compiled at the ETL from the itaiji identification in the JIS X 0208 and JIS X 0212 standards. I corrected a few errors, and added some extra sets which were indicated in the JIS X 0208-1996 draft.

In July 1996 the Pinyin details were completely replaced by a new set. The original Pinyin were from an earlier compilation by Christian Wittern, and and contained many errors. Two more reliable sources had become available: the Uni2Pinyin file compiled by Koichi Yasuoka, which is based in part on the TONEPY.tit by Yongguang Zhang; and the PYCHAR set of readings of Big5 hanzi compiled by Christian Wittern. The Pinyin currently in the KANJIDIC file is a combination of the two, following the order in the Uni2Pinyin file.

In August 1996 I corrected a few more missing and erroneous Nelson numbers, using a massive Nelson list prepared by Wolfgang Cronrath. He also flagged the kokuji, so I added these to the readings fields as "{(kokuji)}".

Also in August 1996 I deleted the handful of former "XJxxxx" cross-references, and replaced them with a much more comprehensive set, so that they now represent all the recognized "itaiji". The file I used for this was the corrected itaiji file mentioned above.

In April 1997 I corrected a large number of bushu codes. Many of these had been identified as errors by Jean-Luc Leger who analyzed and examined all the Nelson bushu. I also identified and added a large number of missing Cnnn codes.

Also in April 1997 I added the S&H "Kanji & Kana" indices. These had been keyed by Olivier Galibert (Olivier.Galibert@mines.u-nancy.fr). (There must be an outbreak of kanji interest on Nancy.)

In February 1998, the long-awaited inclusion of the "New Nelson" numbers took place. I had been waiting for the editor of the New Nelson, John Haig, to supply a list (as he had agreed some years before), but in the meantime, Jean-Luc Leger keyed a list, so they are now available.

Also between December 1997 and February 1998 a large number of Level 2 kanji had their stroke counts corrected to bring them into line with the counting principles used in the Level 1 kanji. This usually aligned the counts with those used in the New Nelson and in S&H. Appendix E of this document was amended to reflect this. The leg-work in tracking this material down was done by Wolfgang Cronrath.

During December 1998 & Jan 1999 I updated the stroke counts of many of the Level 2 kanji, using an analysis of them carried out by Wolfgang Cronrath. I also added the De Roo codes, which had been keyed by Jasmin Blanchette, who also typed the explanatory material. I contacted Fr De Roo in Tokyo who readily agreed to the inclusion of the codes.

The extension of the S&H Kana & Kanji numbers to the 2nd edition was done by Enrique Sanchez Rosa.

The Hangul versions of the Korean readings (which only appear in the XML version) were provided by Francis Bond and Kyonghee Paik.

I did the Tuttle card numbers myself.

James Rose provided the numbers from Crowley's "The Kanji Way to Japanese Language Power", Sakade's "A Guide To Reading and Writing Japanese", and also for that book's 3rd Edition edited by Henshall, Seeley & De Groot.

The "Kodansha's compact Kanji guide" codes were provided by Richard Fremmerlid.

The "Kanji in Context" codes were provided by Randy Foreman.

The Spanish kanji meanings (which appear in the XML format, and may also appear in special versions of KANJIDIC) were compiled by Francisco Gutierrez and provided by Gabriel Sanroman.

Alain Thierion translated the meanings of the kanji into French, and also provided the Maniette numbers.

Andrew Slater provided updates to the JLPT numbers, and additional numbers for the Japanese Flashcards series.