Unweaving the Web

Since ZK 2.2.0 is possible to add i18n support by using i3-label*.properties files to store localized strings and use resource class Labels to retrieve them. This approach is the most extended in ZK for localizing ZUL files, and it was introduced by Minjie Zha in his smalltalk “I18N Implementation in ZK”.

However, this approach has its drawbacks. It requires developers to think first of a key for the text they are going to localize, store it in a i3-label*.properties file together with the localized text, an enable the mechanisms necessary to retrieve keys from a ZUL file. In addition, ZUL files are flooded with keys which sometimes have an obscure meaning.

GNU Gettext utilities follow a similar approach, but with the clear advantage of saving developers the burden of thinking of a key. Developers simply write down text in their source files as they would normally do, together with Gettext utilities, that very same text will serve as a key for all localized texts.

On my previous posts I gave a brief introduction to GNU gettext, how to settup Getttext Commons in a Java enviroment, and how to use Gettext from a Java project. Before going ahead with this post, I recommend you to take a look at those articles in case you haven’t done that yet, as from here on I will refer to some of the concepts and examples explained on them.

First of all, check you have sucessfully installed and configured Gettext Commons in your project.

Now what we are going to modify I18nHelper.java class explained on my last post. I am not going to get very much in detail here, what I am basically going to do is to modify getI18n() method so it knows through ZK engine which Locales end-user is asking for, loading it in case it exists, or using a default one (Click to download final version of I18nHelper.java).

After that we are going to create a taglib which is simply a facade for I18nHelper.

Name it i18n.tld and save it under your /WEB-INF/tld/ directory (Click to download i18n.tld).

After that, you are ready to include i18n.tld taglib from a ZUL file. Include it and use the i18n prefix accordingly. For instance,

<zk>
    < ?taglib uri="/WEB-INF/tld/i18n.tld" prefix="i18n" ?>
    <window>
        <button label="${i18n:_('Add')}" />
    </window>
</zk>

Load it on your favorite browser and see what happens. Basically what we are doing here is to rely on I18nHelper, which exposes a _ function for localizing strings passed as parameter. I18nHelper works against a specific resource bundle, a binary file compiled as the result of a gettext process. A resource bundle contains localized strings and knows how to convert msgids, or keys, to locale strings.

As you may have seen in Using Gettext Commons from Java files, localizing strings in Java files was an easy task. We used GNU Gettext utilities to parse down Java files, searching for strings marked with the _ function. The text within this function served as a key for generating a keys.pot file. Later that file was localized to a locale.po, es.po (Spanish localized file, for instance), and with the help of msgfmt command was possible to generate a resource bundle that could be used later from Java files for localizing texts dinamically. This same philllosophy applies to the process of localizing ZUL files.

The first obstacle that we need to surpass to generate a resource bundle successfully is parsing ZUL files searching for texts wrapped by ${i18n:_(‘%s’)}. GNU gettext utilities support a myriad of programming languages but unfortunately gettext does not support XML files. However, this is not a problem as parsing a XML file is much easier than parsing source code of a programming language. I wrote a small Perl script: gettext_zul.pl, which does exactly that. Run it like this:

gettext_zul.pl --dir path_to_zul_files --keys existing_keys.pot_file

You can ommit the –keys parameter, so a new keys.pot will be created in your current directory. This option can be useful in case you have an existing keys.pot file, generated prior as the result of processing a set of Java files for example.

Once your keys.pot file is up-to-date, create a localized version out of it (See I18n with GNU Gettext utilities). Lastly, run msgfmt to generate a locale resource bundle, Messages_XX.class, out of locale.po.

Supporting arguments

Generally text messages need extra parameters. For example, message “Confirm deleting element?” should be localized as ‘”Confirm deleting {0}?”, item.name’. To sort out this problem, we can extend I18nHelper and overload _ method with extra arguments.

public static String _(String str) {
return getI18n().tr(str);
}
 
public static String _(String text, Object o1) {
return getI18n().tr(text, o1);
}
 
public static String _(String text, Object o1, Object o2) {
return getI18n().tr(text, o1, o2);
}
 
public static String _(String text, Object o1, Object o2, Object o3) {
return getI18n().tr(text, o1, o2, o3);
}
 
public static String _(String text, Object o1, Object o2, Object o3,
Object o4) {
return getI18n().tr(text, o1, o2, o3, o4);
}
 
public static String _(String text, Object[] objects) {
return getI18n().tr(text, objects);
}

To make use of these new functions, we need to exposed them in i18n.tld tag-lib. Tag libs do not support function overloading, so we need to provide different names for each function. For instance, we may add a new function, __, that receives one extra parameter.

I18n as a Macrocomponent

Another way of supporting arguments from ZUL files is to encapsulate I18nHelper funcionality into a Macrocomponent.

First of all, create a HTMLMacroComponent, save it as i18n.zul at webapp/common/components/ (Click to download i18n.zul)

<zk>
    <label value="@{self.i18n}" />
</zk>

Create its corresponding Java file (Click to download I18n.java) and save it as I18n.java at webapp/common/components/

And finally add this new macrocomponent to your lang-addon.xml file.

Now you are ready to use it from a ZUL page.

<i18n value="Confirm deleting {0} ?" arg0="@{item.name}"/>

Macrocomponent Vs Taglib

Most part of the time we may just need to localize values inside attributes, in this case using a tag-lib is the right way to go, for example:

<button label="${i18n:_('Accept')}"/>

Tag-libs are OK for static texts. Static texts are evaluated only once when the ZUL page is rendered for the first time. When showing texts with data bindings, we must use i18n macrocomponent. In most cases, strings with arguments have their arguments bound to dynamic data. As rule of thumb, whenever there are arguments, i18n macrocomponent is the right choice.

<i18n value="Confirm deleting {0} ?" arg0="@{item.name}"/>

However, enabling i18n to support a different number of arguments can be convenient if we need to substitute an argument inside a literal value. Consider the following example:

<window title="Error ${requestScope['javax.servlet.error.status_code']}"/>

This is a very particular case but it still can happen in your code. Adding an extra __ function in your taglib (see above) can solve this problem.

<window title="${i18n:__('Error', requestScope['javax.servlet.error.status_code'])}">
</window>

NOTE: Do not interpolate requestScope variable as it is already inside another interpolation (${i18n:…})

In this last chapter I have reviewed a new approach to internationalize texts in ZUL files. This approach is based on GNU gettext, probably the most wide-spread way of localizing texts in the FOSS world. This method provides clear advantages to developers as they do not need to spend time thinking of meaningless keys and keeping track of them manually across different localized files, which in the long-run means saving time and automatizing the whole process of translating and localizing.

Download all files on this post: i18n.tar.gz.

On my previous posts I introduced GNU gettext utilities and showed how to setup Gettext Commons library for Java. In this new post, I’ll focus on how to use Gettext Commons library for i18n Java files.

The result of  tagging and post-processing source code using GNU gettext utilities is generally a binary file which can later be used from a target programming language. In Java, this binary file is called a resource bundle, with extension .class. There are as many resource bundle files as locale.po files exist in our application. For each locale, there’s a Messages_XX.class file. Basically, a resource bundle is a collection of msgids and its corresponding locale translations. Let’s see how it works:

class MyClass {
   I18n i18n = I18NFactory.getI18n(this.getClass(),
         new Locale("Es", "es"),
         org.xnap.commons.i18n.I18nFactory.FALLBACK);
 
   public void render() {
      Listheader listheader = new Listheader(i18n.tr("Header"));
   }
}

For i18n a text string in source code, we need to instantiate a I18n object before by calling I18NFactory.getI18(). This static method returns a specific resource bundle, depending on the calling paremeters. In the example above, I asked for a resource bundle for locale es_ES (Spanish language, Spanish dialect from Spain). The third parameter is a flag sign indicating that in case such resource bundle doesn’t exist a default resource bundle will be returned. A default resource bundle does actually nothing, it simply returns i18n marked strings as they are.

On the render() method, I wrapped “Resource” within a i18n.tr function. This function is used to mark texts to be internationalized. Later we should parametrized gettext command properly so it can recognize tr as a marker, and fetch msgids from it. On the other hand, typing i18n.tr every time we need to mark a text, plus instantiating a i18n object for every .java file,  can be tedious and repetitive. Fortunately, this can be improved:

public class I18nHelper {
I18n i18n = I18nFactory.getI18n(I18nHelper.class,
new Locale("Es", "es"),
 org.xnap.commons.i18n.I18nFactory.FALLBACK);
 
 public static String _(String str) {
 return i18n.tr(str);
}

So, later we just need to add a static import of _ from other Java file, and we’re ready to go (in case of using _ as marker, call xgettext with -k _)

import static org.navalplanner.web.I18nHelper._;
 
class MyClass {
   public void render() {
      Listheader listheader = new Listheader(_("Header"));
   }
}

In this post, I have reviewed how to use Gettext Commons java library for i18n strings spread across source code. On my next post, I will focus on the more specific case of i18n ZUL pages, UI XML files from ZK framework, using Gettext, of course

In my previous post: I18n with GNU Gettext utilities, I covered the basis of GNU gettext utilities. In this new post, I’ll explain how to use gettext to internationalize .java files.

Gettext Commons is Java library that makes use of GNU gettext utilities. It will enable us to mark texts to be localized in source code as well as taking care of retrieving localized strings. So, we can think of Gettext Commons as a bridge between our project and MO (Message Object) files generated as result of a gettext workflow.

Gettext Commons comes as .jar. There’s even a Maven repository for it. So, if you’re running a Maven2 project, add the following lines to your POM file:

Apart from gettext-commons.jar library, there’s also a Gettext Maven plugin. This plugin comes apart from the .jar library, and it’s not necessary for using gettext from Java.

Unfortunately Gettext Commons website documentarion is a bit outdated, lacking information about how to setup the plugin, latest version, and parameters for every goal. For installing the plugin add the following lines to your POM file:

Let’s check first how the plugin works. Gettext Maven plugin is basically a wrapper  of some GNU gettext commands: xgettext, msmerge and msgfmt. Run the following maven goals from command line to:

• mvn gettext:gettext, parses .java files and generates keys.pot file.
• mvn gettext:merge, executes previous command plus generates locale files (es.po, pt.po, jp.po, etc).
• mvn gettext:dist, executes previous command plus generates a MO (Message Object) file, Messages_XX.class, for every locale file.

Each goal accepts a different range of parameters. Set parameters under <configuration> tag. In the example above, I’ve set up three parameters:

• poDirectory, directory where to place locale files (.po files) as a result of executing mvn gettext:merge. ${project.build.sourceDirectory} is a predefined constant that points to the root folder of our project source code, usually src/main/java.
• targetBundle, java package where to place MO files as a result of executing mvn gettext:dist. Bear in mind that a MO files are binary files, or in case of working with Java, .class files. The name of resulting MO files have the following format: Messages_XX.class, and they are stored at target/classes directory.
• keywords, list of keywords used to mark texts in source code. In my case, I’ve used _ as a marker.

There’s even more parameters, but unfortunately I couldn’t find any piece of documentation which covered them. If you want to find out more, consider downloading the source code or take a look at it online. On the contrary, it’s well documented and easy to understand.

Notice that you don’t neccessary have to use gettext-maven-plugin to parse source code and generate MO files, you can still use GNU Gettext command-line utilities to do that. In addition, running mvn gettext:merge fails if target .po file doesn’t exist. Remember that:

msgmerge -U es.po keys.pot

just updates msgids from .pot file to .po file, it never creates a target .po file. For doing that you must run msginit (you can also copy .pot file to locale .po file, but you should remember to adjust the header to your needs, pay attention particularly to attribute charset).

So, basically in this post I’ve covered how to setup Gettext Commons library and Gettext Commons Maven plugin and how to use the latter in a Maven2 project. In my next post, I’ll focus exclusively on how to use Gettext Commons for i18n .java files.

It seems to be an old video, but I just discovered it recently and I though it could be interesting to linked it here.

A masterly lecture by Alex Wright, author of “Glut: Mastering Information Through the Ages”, where he reviews the works of the early precursos of the Web: Eugene Garfield, Vannevar Bush, Ted Nelson, etc  and how their works has inspired and somehow shape the Web as we know it today. It came to my mind a Paul Baran quote I read once where he recognized that technology progress was accumulative, “If you don’t take care”, he said, “you may end up thinking you have done a big contribution, but in fact, every contribution relies on some other previous work”. Or to put it in Isaac Newton’s words “If I have seen further it is by standing on the shoulders of giants“. Enjoy!

Recently I’ve been engaged addding internationalization support for a Java project running on top of ZK framework. It’s been a quite interesting experience, and I think it sharing my experience could be useful for other people facing similar problems. So this entry is planned to be a series of posts about internationalization (i18n hereafter). In this first post, I’ll explain what are the GNU gettext utilities and how to use them.

GNU gettext utilities are perhaps the most popular tools for i18n among free and open source projects. Basically, the idea is to mark text to be localized in the source code with some special tag. For instance:

Label lblName = new Label("First name");

Texts to be translated should be wrapped by gettext function which later will be called to retrieve a localized string.

Label lblName = new Label(gettext("First name"));

Generally, it could be convenient ro rename gettext as _ for shorten tags and make them easier to identify in the code.

public String gettext(String str) {
    return gettext(str);
}

There’s a basic set of steps or workflow when working with GNU gettex utilities. First, we mark our source code properly. Then we run command gettext command to parse those marks. The result will be a keys.pot file containing all texts prompt to be translated.

find ./src -name "*.java" -exec xgettext --from-code=utf-8 -k_ -o keys.pot '{}' \;

A keys.pot file has the following structure:

white-space
#  translator-comments
#. extracted-comments
#: reference...
#, flag...
#| msgid previous-untranslated-string
msgid untranslated-string
msgstr translated-string

A simple entry can look like this:

#: src/main/java/com/igalia/UnexpectedError.java:101
msgid "Run-time error"
msgstr "Error en tiempo de ejecución"

Every entry consists of its correspondant msgid, plus a list of comments containing  filename and line for every msgid that happened in the code. Considering that, it can be concluded that every msgid in a .pot file is in fact unique.

Once our keys.pot file has been generated, it’s time to localize it.You can simply copy keys.pot to your destination .po file (es.po, pt.po, jp.po, etc.), however the best way to do this is using msginit command:

msginit -l es_ES -o es.po -i keys.pot

In case a locale .po file already exists in our project, we can easily updated it by running the following command:

msgmerge -U es.po keys.pot

This will update all missing entries from .pot file to .po, so now you’re ready for translating new entries with poedit or your favorite text-editor.

Lastly, we need to turn the .po files into a machine-oriented format, which may yield efficient retrieval of translations by the programs of the package:

msgfmt --java2 -d . -r app.i18n.Messages -l es es.po

Please refer to GNU `gettext’ utilities: 10.1 Invoking the msgfmt Program for more details about parameters invocation.

In my next post, I’ll focus on how to use these tools to give i18n support to a Java project. Till then, have fun.

There’s a sentence from Douglas Crockford speech “Javascript: The Good Parts”, which I found very clever and gave me some food for thought about what Javascript does actually mean for the Web today:

“Javascript is succeeding very well in an environment were Java was a total failure”

When Java was launched it was promised to be the language of the Internet basically because of the way programs were compiled and distributed, being able to run in every client machine where a JVM was available. Applets were the mantra of Java on its early years. Java visioneers foresee a future were programs will be downloaded and run on the client-side, downplaying the role of the server. On these days the idea of network computer also emerged, which interestingly has recently come back in the form of netbooks.

Mostly due to the lack of performance of applets, Sun dropped her attempt soon afterwards, coming with the idea of servlets and large collections of technologies focused on the server side. Ironically, Javascript, a language named after Java but that has nothing to do with it (Crockford speak got more on this), has succeeded over time on the idea of applets, as a sort of client-side application which brings rich user experience and interaction. And certainly the fact that Javascript is shipped on every browser gives it a huge potential over other RIA technologies such as a Flex or Silverlight. It’s in fact the most widespread programming language in the world, meaning it has the potential to reach the largest number of users. Given these facts, and counting  the huge improvements the language has gone through over the years (XHR, Json, firebug), plus the huge adoption of open standards, it comes at no surprise why Javascript has turned into one of the most important languages in the world nowadays.

By the way, after seeing what the Webkit team are doing with CSS and 3D transformations, I’m more convinced than ever that the future of RIA means Javascript and open web technologies (not counting the huge effort Google is investing on making Javascript the lingua-franca of the Web).

Lastly I cannot finish this post without recommending Cockford speech at Google Techtalks. It’s not only smart and eye-opener but witty and fun (I’m eager to read his book). There’re too many memorable quotes to put them all down here, definitively worth seeing it.

A quick search on Google returns lots of links on how to implement this, one at the topmost is about writing a helping class called EnumUserType.

This solution is outdated as new versions of Hibernate support JPA annotation @Enumerated. Using this former method is much simpler and therefore recommended.

Using @Enumeration annotation:

package org.navalplanner.business.resources.entities;

public enum Status {
BUSY,
AVAILABLE;
}

package org.navalplanner.business.resources.entities;

class MyClass {
@Enumeration(EnumType.VALUE)
private Status status;
}

In the example above, Status and MyClass should be stored in two separate files. Please notice that when doing the Hibernate mapping of class MyClass, if status access policy is set to property, an extra pair of get/set methods should be defined for class MyClass:

@Enumeration(EnumType.VALUE)
public Status getStatus {
return status;
}

@Enumeration(EnumType.VALUE)
public void setStatus(Status status) {
this.status = status;
}

However, I couldn’t get this working using annotations. I don’t know why, maybe hbm.xml definition and annotations cannot be mixed. So, I ended up translated this annotation to a hbm.xml file:

<class name="MyClass">
<id name="id">
<generator class="native"/>
</id>

<property name="status">
<type name="org.hibernate.type.EnumType">
<param name="enumClass">com.igalia.Status</param>
</type>
</property>
</class>

And that’s it!

I have come up with this Firefox 3.X meme at Mike Beltzerns blog. It consists of evaluating a javascript expression that queries Firefox bookmarks database (sqlite3 moz_places), fetching the first result for every letter of the alphabet. Check Mike’s entry for instructions (there’s no point of copy&paste them here isn’t?).

I got many entries for intranet addresses which are pointles pasting them here, as cannot be linked to anything useful. I marked them as “localhost”.

a: localhost

b: Blogger: Escritorio

c: Software Craftsmanship 2009 : Programme

d: Iurgi de excursion: Dinero de la suerte y otras costumbres chinas

e: elmundo.es. Líder mundial de información en castellano

f: Faitic – Inicio

g: Global Voices Online

h: localhost

i: localhost

j: JavaBlackBelt – Training and Skills Management in Java Hibernate Spring

k: Kirai – Un geek en Japón by Héctor García

l: localhost

m: Marca.com

n: The pain in Spain … – Paul Krugman Blog – NYTimes.com

o: O’Reilly Radar – Insight, analysis, and research about emerging technologies

p: planet.igalia.com

q: QuinTV

r: RConversation

s:  localhost

t: the bandarlog

u: unweaving the web

v: voyporoporto.com

w: elmundo.es. Líder mundial de información en castellano

y: YouTube – pinowsky’s YouTube

Last Thursday I got the opportunity of attending Software Craftsmanship 2009, a conference about agile methodologies and test-driven development.

The conference was divided in three tracks, many of them happening at the same timing on different rooms. I attended the following:

Three paradigms: Taking An Extreme Position on Code Style in a Safe Environment by Keith Braithwaite

A series of three short exercises to be done on an existing base of source code, a small project comprised by a data provider, a controller and a data transfer object. For each exercise, Keith give the audience some time to bang their heads and reach to some kind of conclusion. After that, people discuss about the solutions proposed. Of course, none of these changes could break the logic of the program, that means basically, do not break the tests. It was my impression that the bottom line of the exercise was to show that unless you do not break the test whatever change you make in a program is still valid (for the better or for the worse). It was also my impression that there was not definitive answers.

TDD As If You Meant It by Keith Braithwaite
On this talk Keith proposed a small problem to be solved: considering the Japanese board game Go, write a program to determine when a stone is in an atari state. Perhaps the problem was the less interesting thing, here the point was solving the problem following a test-driven development way. First,write a test that fails, add some implementing code to make it run, later move that implementation code to a class when you have enough code. Here the audience was asked to paired with someone else. I got to recognize that my experience on TDD is very limited, but I was lucky to pair with someone with more experience than me. The interesting thing was that although most everyone recognized to do TDD on their daily basis, they found the task very complicated, since they tend somehow to approach testing in other ways, not starting from the root of the problem they want to solve. This was truly TDD as you meant it, an eye-opener seminar.

Empirical Experiences of Refactoring in Open Source  by Steve Counsell (Brunel University).

It was a lecture about refactoring in open source, a project carried at Brunei University. Firstly, Steve started defining the term of code smell. A term coined by Kent Beck to mean “a surface indication that usually corresponds to a deeper problem in the system”. Later Martin Fowler would pointed out several refactoring tasks you can do on a project to reduce code smell. Steve used this tasks as metrics and with the help of a tool developed at Brunei university, they managed to gather some statistics and conclusions on 10 to 15 java open-source projects. Someone in the audience asked if this tool was also open-source. It seems it was never freed but Steve seemed opened to doing it.

5 Reasons To Have A Coding Dojo At Your Company by Ivan Sanchez.

Ivan explained the concept of a coding Dojo, and immediately after we got a hands-on Dojo. We worked on an existing source code base, two people paired and carried the development of the program for over 5 minutes. The audience could suggest solutions, shorcuts, whatever. When the time expired, the driver left, and another member of the audience come in.

Unluckily I missed Programming In The Small by Ivan Moore & Mike Hill . It was my first time in London, and despite leaving early in the morning I found myself lost in the tube on my way to the BBC Media Center.
Anyway, it was a very good conference. The atmosphere was of excitement and at the end of the day it seemed people was waiting for more. I wish I had a deeper knowledge on TDD and could have taken more advantage of the seminars, but I could at least grasp the basics and participate. It has broaden my mind on TDD so my expectations are fulfilled on that sense.

Lastly, I would like to mention the splendid organization of the event (catering service included), the caring of Jason Gorman and colleagues who help him on the organization and the support of the BBC and its Backstage strategy. After the event, the BBC kindly invited us to a tour at the television center. Although I missed my chance of having myself spot close to Doctor Who’s tardis, I enjoyed very much the pints at the BBC Club and the chatting with the guys from Songkick and other BBC IT workers.

It seems very likely there will be a next edition next year, but perhaps bigger. Don’t miss out!

PS: In case you want to learn more check the summary video by Jason on YouTube!

Next Thursday 26th I will be attending Software Craftsmanship, hold at the BBC Media Center in London.

It’s a one-day event about how to build software the right way, as the own conference’s motto states. Most of the speakers, rather say all of them,  have an extensive background on eXtreme programming. Some of them, as Ivan Moore or Keith Braithwaite, are in fact well-known speakers at some of the most important conferences on Agile methodologies as Agile 2008, or Qcon.

What attracts me the most is the practical approach of the programme, where most of the speeches are actually seminars. In some of them, an exercise and a set of questions are proposed to the audience, then with the help of the speaker the audience should withdrawn their own conclusions and discuss about them. And there is always the chance to peer with somebody, what reinforces the importance of the role that Agile methodologies have in the whole conference.

Lastly, I wouldn’t like finishing this post without mentioning that Igalia has kindly covered most part of the expenses. As you may know, here at Igalia, every worker (from the second year on) got a pocket budget for training. This budget can be used to strengthen our knowledge on different areas we’d like to improve, ranging from technical things to business, finance, etc. On the other hand, accommodation will be fully covered by the brand-new apartment Igalia has rented in London. Thought in the beginning as a good way to help workers go abroad for improving their English, I think it can also be a good tool for many of us who would like to take advantage of the many conferences and events that are organize in London throughout the year.

This kind of policies show the high commitment of Igalia on training and education, something that I personally appreciate very much.