The presented idea is only work-in-progress. I made some small demo, but not the whole idea is implemented yet. (The following code uses UTF-8, so switch you browser).

The idea is to have some macro that to allow definitions of the string constants in several languages:


The strings are actually defined on the undefined data memory, so, initially they will be empty. Only place is allocated for every string, enough to contains the longest string from the language sets.
The different languages strings are separated in the defined data section in separate arrays for every language.
Then when you want to switch to some language, small procedure simply copy the whole language array to the real strings and the program switches to another language. For example, the below code will switch the whole program to English:



The attached file contains small demo application. It simply prints some text on a language depending on the command line argument. Execute it without argument to get the list of supported languages.
The program is compiled for Win32, Linux and KolibriOS (not tested).

Might work for small hobby projects, but definitely doesn't scale up for something larger - especially if you need other people to do translations. I'm also not a fan of having non-ASCII characters in source code files; for personal projects it's a matter of taste, when collaborating with a lot of people of different nationalities (and with a wide selection of operating systems, IDEs and text editors) it's simply too much hassle.

Externalizing string data does mean slightly more overhead when dealing with the strings (but this is so little it doesn't matter in practice), and it does require some up-front organizing consideration as well as discipline in key naming - but in practice it's worth it in the long run.

Yeah, it can be annoying that you can't directly see the contents of a string in source code, but if you have decent tooling support for your translations it's not a real issue. (You could also use a scheme where the English/whatever-master-language text is used as the lookup key, but that becomes clumsy for anything but small strings).

f0dder, I am not agree with your statement: "but this is so little it doesn't matter in practice". Using external strings and tools requires that the application must be designed as easy-translatable from the beginning. Also, it limits your programming methods.
The above approach makes the program translation easy, because only string constants have to be changed. It also does not adds overhead to the program. You simply use the strings as always.

The source code is UTF-8 - standard enough IMHO. You only need UTF-8 compatible editor.
The translation tool that I will make for Fresh IDE will allow all strings defined with "itext" macro in the project to be edited from one common place and then patched back to the source. So, we can have both - easy readable source and easy translation.

Rather than using a copy function how about using a single master language pointer that points to a set of pointers for each string. The advantages here are

1. changing languages only requires updating a single pointer
2. the strings can remain immutable within read only memory
3. each string length is native and doesn't need to conform to the longest one from each language

revolution, I though about this approach and rejected it, because it will need from the user to write the program in some special way (i.e. using pointers and indexes instead of simple pointers to the strings). I wanted to make the internationalization system as transparent as possible for the programmer.

Use macros to build the tables and pointers. The only thing you need to add is a final line to instantiate all the built tables at the end of the file.

I got nothing? More detailed?

Maybe this can help?

http://board.flatassembler.net/topic.php?t=12012

Revolution, I am using this technique all the time (including the macro library discussed here). But what exactly you suggest about the internationalization problem? Imagine one program written in English only (Fresh IDE or FASMW for example). Now, how to make it to support multiply languages?
My approach makes this task very easy - You only have to edit the strings to contain the proper translation and then to insert a call to "SetLanguage" in order to switch to the new language. (well in GUI applications you maybe have to restart the application)

Why not just load from an .ini file instead? Then when creating controls, for that specific application you call a function like so:




To simplify, you can then use lists to load the current locale setting in memory on start up like so:

typedef, read above. It is explained why not.

It's easier to handle if you design properly from the beginning, but it's not impossible to change at a later state - especially not if you have decent tooling support for locating and externalizing strings

Sure, it's a tradeoff - but it's not something that I feel limiting. Rather, I find that it makes everything a helluvalot easier in the long run.

With your method, you need to hunt through all your assembly files for strings, and have no easy way to get an overview of which strings have been translated to which languages etc. OK, you can do a tool that scans for the itext macro etc., but that's more complex and less foolproof than dealing with a purpose-built external format. Especially if you want to support external translators, which would mean import/export functionality.

To nit-pick, it adds memory overhead for the "current strings" buffer and a bit of CPU overhead for the copying - but of course this is pretty irrelevant, and there'd be overhead when dealing with external strings as well (though you can load just the languages you need, rather than your approach that has all languages in-memory). I've yet to see situations where localizable strings are involved in performance-sensitive code, anyway.

Not everybody uses that (yes, even in 2012), and you also have to guarantee that encoding isn't messed up in flight, etc. Easy enough if you're a single developer, a small team, or have very organized ways of working... that's not always the case in the real world, though

The project I'm working on probably has somewhat larger scope, though - I'm not sure how many languages we're currently translating to, but there's some forty languages on the target list, and we have somewhat more than 1000 strings, I believe (which is only that low because most other stuff is content-managed). But that's a web project (yes, I sold my soul for god-$).

For desktop applications, I still prefer externalized localization though. There's still the argument that it's plain easier to deal with that way, especially when you need external translators, but also because it makes it possible to deploy new translations without distributing new binaries, and it also means you can reload the localization without restarting your application. Oh, and it's easier for end-users to do their own localizations that way, too

I agree with external language strings file.
Very convenient.
But I want it to be a UNICODE file.

Define "UNICODE"?

I personally prefer UTF-8 for storage/exchange. It does mean you'll likely need to convert to an internal format, but it's (often) easier to deal with than whatever native format.

Also, JohnFound, I'm not trying to be a grumpy old flaming man - just offering some alternate perspectives. I do like the (programmer) ease of use, and it's convenient that you load strings to .bss so you don't need to deal with indirection or manual GetTranslatedString() calls. It's a decent enough thing for smaller projects, especially if you're going to handle all the translations yourself (I'm still not fond of non-ASCII in source code files, but for smaller projects/groups there's not many good reasons against it, so I'll write that down to personal prejudice ).

I am not against the external files with language packs. I know it is easier to deploy new languages and all other advantages. And when I found some way to make FASM to write files during compilation, I can make macros to directly export the string tables to the external language files, instead of data section of the program.
But still IMHO, the string definitions are better to stay in the source code. (I really hate tradeoffs).
I think this method can work on big projects as well.

UNICODE for me is 16-bits per every character.
I am not sure however, how to make these files.
Say, I need a file with some Japanese text -- how do I do this?

That's just one particular encoding (UCS-2), which can't even handle the entire unicode set. Windows is UTF-16.

Any reasonable text editor that can do handle various encodings - for free ones, Notepad++ can handle it. You need font support as well, of course, and some input method

You want to convince me that all characters known to man will not fit into 16-bit?

And this is just Japanese, Chinese, and whatever other languages using these characters (if any).

But your way uses too much time and code like just like Microsoft spent 90% of their time coding security features in MS-Word