Keyboard Related Problems on Unix

Nov 4th, 2001

<tomas@frydrych.uklinux.net>

It appears that many of the out-of-the-box Linux distributions do not have the keyboard setup correctly if you are using a language other than English. When using AW, this manifests itself in two ways. (1) You are getting small circles instead of your characters, or you see correct characters on the screen but when you print the document you get small circles for some, many or even all characters. (2) If you try to open your document in AW running on Windows, you see garbage.

This is what the problems is: the XFree server supports a large number of characters, and it has a name for each of them. For instance the Unicode character u05d0, which is the Hebrew letter Aleph, is known to the XFree server as hebrew_aleph. When you want to type Hebrew Aleph, the keyboard has to send this specific character to the XFree server. The association of a particular key with a particular character is done via what is called xmodmap. This file specifies what character each key on the keyboard generates. For instance, it associates the key which would have ‘t’ on the English keyboard with the letter Aleph on Hebrew keyboard.

The proper way to do this is to assign to this key (key no. 28) the character name hebrew_aleph. When an 8-bit application, such as the terminal requests character input from the XFree server, the server translates the code hebrew_aleph into the character set the terminal is using, probably ISO-8859-8, and so it will send the terminal code 0x00e0. A Unicode application, in contrast, will request the keycode as is and translate it internally into a Unicode value. In our case it translates the code hebrew_aleph into the Unicode value 0x05d0.

Unfortunately, often instead of using the X character names, the xmodmap assigns the key a numerical value of the character from the 8-bit character set of the language, e.g., in our example the key 28 gets associated with value 0x00e0, i.e., the code for Aleph in ISO-8859-8 character set. The problem with this is that for the X server 0x00e0 is not an Aleph, but the character agrave. Now when an 8-bit application request input from the server, it will send it this numerical value, and everything will appear to be fine. However, when a Unicode application such as AW requests the input, it will also send it this numerical value, which will be internally translated into Unicode, leaving us with Unicode character 0x00e0. Since this character is not found in the ISO-8859-8 fonts which you are probably using if you are wanting Hebrew, you will see a small circle instead (sometimes, if your fonts are misconfigured as well, you will see your text on screen fine, but get only the small circles when trying to print).

There is a simple test to see whether your keyboard is misconfigured. Create a new AW document and type couple of letters from your language. Save the document in abw format and then open it with a plain text editor. At the start of the document there is a kind of a header, and then later on you will find your characters in between a pair of <c></c> tags. If AW is getting correct messages from the keyboard, each of these characters should be represented by a single byte. However, if you find instead that the characters are represented by &#xXXXX tokens (where XXXX is the Unicode value of the character), it means that AW is receiving characters from the keyboard that are not found in your 8-bit character set.

The solution: you have to fix your keyboard map.

How precisely you do this depends on your setup. If you are using some keyboard utility, such as the kikbd that comes with KDE, you will need to fix the keyboard definition files of the utility, if you are using xmod, you need to fix the xmodmap file that contains the keyboard definition.

As far as AW is concerned, you have two options; either you can use the character names as described above (you will find the names in the XFree server’s documentation), or you can assign the key a value 0x0100XXXX where XXXX is the Unicode code of the character (e.g., 0x010005d0 for Hebrew Aleph). This latter method has the advantage that you can assign even characters that XFree does not have names for, but you should be aware that not all applications are capable of handling keyboard defined in this way.