Wiktionary to XML

Wiktionary to XML

In GCompris it would be very usefull to have a large amount of words and definitions in a XML formatted form. This would allow us to create different kinds of activities around reading and writing skills.

In the early days of GCompris there was no such data available under an open license. But now things have changed and the Wiktionary dictionary is one of the Wikimedia projects.

Sadly, it is formatted as WikiText instead of XML so it is very hard for a computer to parse it and extract relevant informations.

I decided to make it a try and transform a Wiktionary dump in an XML structured format.

The primary goal is to provide content that is appropriate for children and this is another challenge because in Wiktionary:

• we find words definitions with sexual connotation,
• there are way too much words. For GCompris a list of 1000 or 2000 words and definition would be perfect,
• wikitags and formatting is specific for each language which makes life harder.

I created a tool called wiktio2xml, it can be found under gcomprixogoo branch in tool/witio2xml. To run it, you must pick the Wiktionary XML dump and a list of words to extract. Then just run:

• ./wiktio2xml.py frwiktionary-20100915-pages-articles.xml fr_words.txt -o miniwiktio.html

To get more information on what it does, just run ./wiktio2xml.py -h:

Usage: wiktio2xml.py [options] wiktionary_dump.xml word_list.txt
Options:
 -h, --help            show this help message and exit
 -o OUTPUT, --output=OUTPUT
                       write result to file or directory
 -q, --quiet           don't print in progress messages to stdout
 -d, --debug           print debug traces to stdout
 -s, --site            Creates a web site

Results of what the script creates at this stage

• in a single HTML-formatted page
• in a static html site
• (In the end an XML file have to be defined)