2. TEI architecture support for the I18N and L10N process
2.1. Unicode
The first priority in internationalizing the TEI is to ensure
clean support for character sets throughout the system. With this in mind,
the P5 revision of the TEI made substantial changes in its dealings
with characters. As the W3C (http://www.w3.org/International/) recommend, in the TEI scheme:
- Unicode is the only supported character encoding schema. This
means that entities for characters are deprecated, and the
recommended daily use is for UTF-8 encoded text, as in
<note> [一]乘龍二句：《李太白集》卷五作「乘龍天飛， 目瞻兩角」，是。[二]神藥：同上作「仙藥」。</note>
- There is a clean mechanism to use non-Unicode characters
- all appropriate text content models are set to allow a mixture
of CDATA and
<g>(where<g>is a reference to a non-Unicode character) - all elements have an attribute xml:lang to record the language used
- there are no places where an attribute is used to hold pure text
<glyph>
element in the TEI header. In the following example, we define a new
character and assign it to a position in the Unicode Private Use Area
(PUA); we also prode a standardized form as a fallback:
<charDesc>
<glyph xml:id="z103">
<glyphName>LATIN LETTER Z WITH TWO STROKES</glyphName>
<mapping type="standardized">Z</mapping>
<mapping type="PUA">U+E304</mapping>
</glyph>
</charDesc>
This can now be referred to using the <gi> element, as in
<g ref="#z103"/>
At this point we will expect the processing application to work
out what to do (either show the PUA character, if it can, or the standardized
form). Other facilities in the <charDesc> element allow the
user to provide an image file which has a picture of the character.
It is also possible to override what appears in the text by using
markup like this
<g ref="#z103">z</g>
where the content of the <g> element can be used immediately
without any lookup.Where a character is simply a relatively unimportant
variant on a Unicode character, the
user does not need to define a point in PUA, but can simply use
<charDesc> to describe the variation.
2.2. TEI literate programming
The TEI is written in a high-level markup language for specifying
XML schemas and their documentation. This language is an XML
vocabulary known as ODD (One Document Does it all), and
is one of the TEI modules. This provides a literate programming
language for production and documentation of any XML schema, with
three important characteristics:
- The element and attribute sets making up the schema are formally specified using a special XML vocabulary
- The specification language also includes support for macros (like DTD entities, or schema patterns), a hierarchical class system for attributes and elements, and the creation of pre-defined groups of elements known as modules.
- Content models for elements and attributes are written using an embedded RELAXNG XML notation, but tools are available to generate schemas in any of RELAXNG, DTD language, or W3C schema.
- Documentation describing the supported elements, attributes, value lists etc is managed along with their specification, together with use cases, examples, and other supporting material.
The ODD language has allowance for translating element name,
attribute names, and descriptions, and for preserving information to
allow canonicalisation. The technical documentation elements
(<gloss> and <desc>) for TEI elements and attributes etc
can be specified multiple times, in different languages, distinguished
by the standard xml:lang attribute. There is also a container
(<equiv>) to specify the relationship of an element, attribute
or value to standardised schemes.
Each definition of a new primary object (element or attribute) has
associated description and examples. A complete example
of a definition is as follows:
<elementSpec
xmlns="http://www.tei-c.org/ns/1.0"
module="header"
ident="taxonomy">
<gloss>taxonomy</gloss>
<desc>defines a typology used to classify texts either implicitly, by
means of a bibliographic citation, or explicitly by a structured
taxonomy.</desc>
<content>
<rng:choice xmlns:rng="http://relaxng.org/ns/structure/1.0">
<rng:oneOrMore>
<rng:ref name="category"/>
</rng:oneOrMore>
<rng:group>
<rng:group>
<rng:ref name="model.biblLike"/>
</rng:group>
<rng:zeroOrMore>
<rng:ref name="category"/>
</rng:zeroOrMore>
</rng:group>
</rng:choice>
</content>
<exemplum>
<egXML xmlns="http://www.tei-c.org/ns/Examples">
<taxonomy xml:id="tax.b">
<bibl>Brown Corpus</bibl>
<category xml:id="tax.b.a">
<catDesc>Press Reportage</catDesc>
<category xml:id="tax.b.a1">
<catDesc>Daily</catDesc>
</category>
<category xml:id="tax.b.a2">
<catDesc>Sunday</catDesc>
</category>
<category xml:id="tax.b.a3">
<catDesc>National</catDesc>
</category>
<category xml:id="tax.b.a4">
<catDesc>Provincial</catDesc>
</category>
<category xml:id="tax.b.a5">
<catDesc>Political</catDesc>
</category>
<category xml:id="tax.b.a6">
<catDesc>Sports</catDesc>
</category>
</category>
<category xml:id="tax.b.d">
<catDesc>Religion</catDesc>
<category xml:id="tax.b.d1">
<catDesc>Books</catDesc>
</category>
<category xml:id="tax.b.d2">
<catDesc>Periodicals and tracts</catDesc>
</category>
</category>
</taxonomy>
</egXML>
</exemplum>
</elementSpec>
The important things to note here are that the content model for the
element is expressed in RELAXNG, which references other elements only
by the names of classes to which they belong; and that the
the worked example is well-formed XML embedded in its own namespace.
This specification may be processed to produce a DTD, a
RELAXNG schema, an XSD schema, or documentation in various forms. The objects identified by the ident attribute in the TEI
can be given an alternate name by use of the
<altIdent> element; so
the example above could be rewritten as
<elementSpec
xmlns="http://www.tei-c.org/ns/1.0"
module="header"
ident="taxonomy">
<altIdent xml:lang = "fr">taxinomie</altIdent>
....
Providing a French name for the element.
How does this work in the schema, where other elements
might refer to ‘taxonomy’? The normal schema, using RELAXNG
compact syntax, has the definition
taxonomy =
## (taxonomy) defines a typology used to classify texts either implicitly, by
## means of a bibliographic citation, or explicitly by a structured
## taxonomy.
element taxonomy { taxonomy.content, taxonomy.attributes }
taxonomy.content = category+ | (model.biblLike, category*)
taxonomy.attributes = att.global.attributes, empty
in which the element <taxonomy> is
defined by the containing pattern ‘taxonomy’; it is the
pattern name which other elements use, not the
element name. If the schema were translated into Chinese, it would
look like this:
taxonomy =
element 分類學法 { taxonomy.content, taxonomy.attributes }
...
where the pattern name remains the same. This type of schema
markup is generated by the TEI tools, picking up the information
from <altIdent>. The descriptions work in the same way. We can
expand the TEI source to add French translations alongside the English
originals, and the appropriate text can be passed to the generated schemas or
documentation:
<elementSpec
xmlns="http://www.tei-c.org/ns/1.0"
module="header"
ident="taxonomy">
<altIdent xml:lang = "fr">taxinomie</altIdent>
<gloss>taxonomy</gloss>
<gloss xml:lang="fr">Taxinomie</gloss>
<desc>defines a typology used to classify texts either implicitly, by
means of a bibliographic citation, or explicitly by a structured
taxonomy.</desc>
<desc xml:lang="fr">L'élément Taxinomie <gi>taxonomy</gi> définit
une typologie employée pour classer des textes soit implicitement au
moyen d’une citation bibliographique, soit explicitement au moyen
d’une taxinomie structurée.</desc>
....
[We thank Pierre Yves Duchemin for these translations.]What does a translated schema look like in practice?
If we take a Spanish play, and translate the element names to Spanish
(thank to Alejando Bia for this work), a text like this
will be much more familiar-looking to encoders in Spanish-speaking
countries:
<cuerpo>
<div1 tipo="part">
<div2 tipo="act">
<encabezado tipo="main">Jornada primera</encabezado>
<div3 tipo="scene">
<encabezado tipo="main">Cuadro único</encabezado>
<acotacion formato="centered">
<resaltado formato="bold">(Salen </resaltado>REBOLLEDO,
<resaltado formato="bold">la</resaltado> CHISPA<resaltado formato="bold">
soldados</resaltado>.<resaltado formato="bold">)</resaltado>
</acotacion>
<dialogo>
</dialogo>
</div3>
</div2>
</div1>
</cuerpo>
This file will not work with normal TEI publishing tools,
or be suitable for archiving, but it is straightforward to
write a transformation (eg in XSL) which reads the TEI source
with the element names and <altIdent> information, and
puts the text back to canonical form.2.3. TEI applications
TEI applications, as well as the texts themselves, need to have
developed internationalised interfaces. For example, an application
with turns TEI XML into HTML for web display, and provides a
heading such as ‘Contents’ when it meets
<divGen type="toc"/>, will have to provide appropriate
translations. The TEI XSL family maintained by Sebastian Rahtz, for
example (http://www.tei-c.org/Stylesheets/teic/), can
operate in many languages:
| ISO Language code | Text |
| en | Contents |
| de | Inhalt |
| ro | Cuprins |
| fr | Contenu |
| pt | Índice geral |
| es | Contenidos |
| slv | Vsebina |
| sv | Innehåll |
| ch-TW | 內容 |
| sr | Sadržaj |
| ja | 目次 |
| pl | Spis treści |
| hi | Mula Shabda |
| th | เนื้อหา |
| nl | Inhoud |
| ru | Оглавление |
| tr | İçerik |
| bg | Съдържание |
| el | Περιεχόμενα |
2.4. TEI schema-making tools
The ODD language files need to be processed to produce schemas in
the chosen language. This is done by a set of XSLT scripts, which can
either be run on a command-line, or as a web service
called Roma (http://www.tei-c.org.uk/Roma/). This currently has support
for varying the languages of its interface, but must also allow for
supporting the following output schemes:
- canonical: English names, descriptions in English
- local descriptions: English names, descriptions in chosen language
- local names: names designed to make sense to a speaker of the chosen language, descriptions in English
- fully localized: both names and descriptions in chosen language
2.5. The application of the W3C ITS guidelines to TEI work
An Internationalisation Tag Set working group (under
the chairmanship of Yves Savourel, Enlaso) is writing a Recommendation
(if it is accepted) for the World Wide Web Consortium about markup
which encodes information for translators and localisors. The current
state can be found at http://www.w3.org/International/its (this document is
itself written using the TEI ODD language). The ITS consists of a set
of elements and attributes for annotating a text with information for
further processing, covering Internationalization:
- Markup for bidirectional text
- Ruby annotation
- Language identification
- Translatability of content
- The localization process in general
- Terminology markup
The primary ITS notion is that
information about elements and attributes can be supplied
- in a document schema
- in an external rules file
- in a rule section in an instance file
- attached to instance elements
- translate
- should this object be translated?
- locInfo
- Is there some localisation hint?
- locInfoType
- What type of hint is it?
- term
- Does this object describe a technical term?
- termRef
- Where is the term defined?
- dir
- What is the text direction?
- rubyText
- Is there some Ruby annotation?
A complete example of a TEI text marked up with
a combination of ITS rules and ITS local markup
looks like this:
<TEI>
<teiHeader>
<its:rules>
<its:ns
its:prefix="t"
its:uri="http://www.tei-c.org/ns/1.0"/>
<its:translateRule translate="no" selector="//t:body/t:p"/>
</its:rules>
</teiHeader>
<text>
<body>
<p>Hello <hi>world</hi>
</p>
<p
its:translate="yes">translate me</p>
</body>
</text>
</TEI>
where the ITS rules say that <p> elements should not normally
be translated, but the second <p> has an explicit override.If we take a TEI ODD document, we can express the relationship
between the structural elements and the documentation elements with
the following ITS rules, which says that the default is to
not translate anything, but gives a set of elements which
are to be translated:
<its:rules>
<its:ns prefix="tei" uri="http://www.tei-c.org/ns/1.0"/>
<its:translateRule translate="no" selector="//tei:*"/>
<its:translateRule translate="yes" selector="//tei:desc"/>
<its:translateRule translate="yes" selector="//tei:gloss"/>
<its:translateRule translate="yes" selector="//tei:valDesc"/>
<its:translateRule translate="yes" selector="//tei:p[@rend='dataDesc']"/>
<its:translateRule translate="yes" selector="//tei:remarks"/>
</its:rules>
Using this information, we can show graphically in Figure 1, Example of ITS implementation, using an ITS tool, which elements need a
translated equivalent (those in green).
Figure 1. Example of ITS implementation
For the purposes of the formal translation procedure advocated by this paper, the ITS procedure provides a good framework.
