Re: [Koha-devel] Charsets: Moving to utf-8

[Mathew]

Hi,

I dont respond often on this list, but I thought the following information may 
be useful...

> OK ... during the bugsquash session the issue of charsets came up.
> (in response to 885:; Non-ISO8859-1 charset support broken in CSS
> theme in 2.2). Our goal for 2.4 would be to move everything--db
> storage, and templates -- to utf-8. The question is, how?

[snip]

> So we're looking for suggestions on how to make the switch. What do
> we need to consider:
> 
>  HTML::Template limitations (should we be looking at Template::Toolkit)
>  Database Issues
>  Translation Issues
>  Other stuff

At my work we provide a browser-based application which is translated into a 
number of languages. Some of the interesting points:

- we only support UTF8 as the charset, as any other charset in simply 
insufficient (Google does this too).

- all translations are stored within a database table, then have the template 
lookup any static text (text generated from within Perl, can be translated 
before being sent to the templates.

- since templates lookup their own translations, you dont need a 
template-per-language.

- since translations are stored within a database, you can update the 
translations at run-time, without needing to restart / reload your updated 
translations

- web access to the database translations allow multiple people to localise the 
translations, at the same time.

- right-to-left text is also easily supported

To do some of these, we modified HTML::Template to suport custom TMPL_xxx tags, 
so that we could implement a tag for the static text lookup.  ie: our templates 
look like:

...
  <TMPL_CATGETS "A phrase containing a template variable [_1], that gets 
inserted here",some_var>
...

where some_var is a Perl generated TMPL_VAR value.

We combine H::T with the Locale::MakePhrase module so that the output can 
handle singular / plural / etc based on the string/numeric value of some_var.

For some extra information, go here:

   http://members.optusnet.com.au/mathew

Hope this helps,
Mathew

PS. Some languages, particularily Asian languages, contain many pen-strokes 
within a glyph.  Often these languages need the text point-size to be slightly 
larger than European languages... this is just a tidbit that I have found 
through experience, particularily if a single template is to be used for every 
language.