Cyrillic font problem

Player
1

Hello! I’ve decided to try out EasyRPG and I must say I’m impressed by the amount of the work done.

However, I’ve tried to run my unfinished game, and I found out that the font being used is unreadable:

Compare this with how it should look (this is RPG_RT under Wine, without proper fonts; it should look even better in Windows with the fonts installed):

I.e. most of the text is lost, because font is too wide and non-Russian letters are missing (і, ў). The same problem exists in all games in Cyrillic script, including all the Russian games.

Due to tradition, all Japanese fonts have double-width Russian letters. They are double-width due to compatibility with some ages old Japanese encoding (back then the codepoint determined the width of character).

To make games in Cyrillic script playable, something has to be done with the Cyrillic font. Is there a way to change font? Maybe I could help by drawing nice Cyrillic letters?

Edited: I’ve just read in the blog that you use the Shinonome font. Unfotunately, editing Cyrillic in this font is not an option (it is kept there for compatilibily), but maybe it’s possible to fork this font for EasyRPG, and add normal Cyrillic there? Or add an option to change font on a per-game basis?

2

Thanks for your report, that is really a serious problem. We have a similiar problem with chinese games (some Kanjis not displayed).
We are planning to support loading of custom fonts but this is a low priority task.

3

I understand that loading different fonts is a complex problem, but maybe it is possible at least to have good default fonts?

I.e. it is unlikely that any Japanese will need that double-width Cyrillic (it is kept for compatibility if someone, e.g., uses them in pseudographic tables — but it’s unlikely to break any Japanese game). So, it would be nice if the default font had half-width Cyrillic letters.

If you’re concerned about Chinese fonts, is it possible to fill the missing font codepages with characters from 文泉驛 Unibit? It’s a GNU GPL’d Chinese font with a wide coverage. Of course, it won’t help with the cases when Chinese and Japanese characters are different but have the same codepoint (今 ＝ 人丶7 vs. 人一7), but at least it will be readable.

4

Well, if you really want you can alter the shinonome font to get half-width characters.
On our repository look in lib\shinonome\kanjic\font_src.bit and search for “STARTCHAR 2721”

The mapping is defined here: http://cpansearch.perl.org/src/GUS/Unicode-UTF8simple-1.06/gb2312.txt.
First char is “0x2721=>0x0410, # CYRILLIC CAPITAL LETTER A”, last char is “0x2771=>0x044F, # CYRILLIC SMALL LETTER YA”

You also have to change

DWIDTH 12 0
BBX 12 12 0 -2

to

DWIDTH 6 0
BBX 6 12 0 -2

And then upload the modified file and we will recompile the player for testing.
(or do it yourself by running “generate_cxx_font.rb” and then recompiling the Player)

GNU GPL is bad because we plan switching to a different license later.

Edit:
There are actually smaller cyrillic characters in kanjip\font_src_diff.bit. Not half width either, but around 8. Hm, using them will need a change in our font renderer because we hardcode on 6 or 12 pixel width and has already too many width to fit on the msgbox…

5

Thanks! I’ll try editing the font and the font renderer’s width code and write back once I come up with something useful. ^^"

6

I get an error when trying to run this script… Unfortunately, I don’t know Ruby at all and I don’t know what is going on.

When I run it with a shebang line, I get an error «file or directory doesn’t exist»:

$ cd ~/Software/easyRPG/pl/easyrpg-player-0.1.3/lib/shinonome $ ./generate_cxx_font.rb /usr/bin/env: ruby -Ku: Dosiero aü dosierujo ne ekzistas

When I run it with Ruby interpreter directly, I get an error about encoding:

$ ruby generate_cxx_font.rb /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': iconv will be deprecated in the future, use String#encode instead. Loading Lantin-1...done Loading Hankaku...generate_cxx_font.rb:61:in `conv': invalid encoding ("UTF-32LE", "JIS_X0201") (Iconv::InvalidEncoding) from generate_cxx_font.rb:61:in `read_font' from generate_cxx_font.rb:100:in `read_file' from generate_cxx_font.rb:134:in `<main>'

I’m using the source tarball I’ve loaded here: easy-rpg.org/jenkins/view/Playe … yer-linux/

I’m running Linux Mint Debian Edition which should be compatible with Debian Testing. ruby --version returns this:
ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]

Any ideas? Maybe I should install a different Ruby version? Or install some package that gives more encodings to iconv?

Edit: for the time being, I got away by commenting out all the Hankaku-related stuff in generate_cxx_font.rb. Obviously, this left me without halfwidth katakana, so it would be nice if someone helped me to install JIS_X0201 encoding into my Iconv — I don’t know how this is done. Actually, it seems GNU’s libiconv doesn’t provide support for JIS_X0201 encoding, at least it’s not listed in on its homepage…

7

Try running this version of the generate_cxx_font.rb script.
raw.githubusercontent.com/Zeger … xx_font.rb

It should give you around 20 “Invalid sequence” warnings.

8

[quote=“Zegeri”]Try running this version of the generate_cxx_font.rb script.
raw.githubusercontent.com/Zeger … xx_font.rb[/quote]
This version works, thanks!

9

I finally got to draw the Cyrillic font, and it turned out easier to do it than I expected. :slight_smile: I didn’t need to edit any game’s code, just generate_cxx_font.rb.

So…

  1. Create a directory lib/shinonome/cyrillic in your sources folder

  2. Download font_src.bit here: (edit: deleted this file from server as it is no longer relevant). Put it into lib/shinonome/cyrillic

  3. Edit generate_cxx_font.rb to merge Cyrillic fonts by adding these lines (edited parts are shown in boldface):

print “Loading Hankaku…”
hankaku = read_file(File.new(’./hankaku/font_src_diff.bit’, ‘r’), “CP932”, true)
print “done\n”

print “Loading Cyrillic…”
cyrillic = read_file(File.new(’./cyrillic/font_src.bit’, ‘r’), kanji_encoding, true)
print “done\n”

print “Loading Gothic…”
gothic = read_file(File.new(’./kanjic/font_src.bit’, ‘r’), kanji_encoding, false)
print “done\n”

and then these:

generating

print “Generating Gothic…”
gothic_final = gothic.merge(cyrillic).merge(hankaku).merge(latin)
code_max = write_all(File.new("./gothic.cxx", “w”), “SHINONOME_GOTHIC”, gothic_final)
print “done\n”

  1. Recompile (first by ruby generate_cxx_font.rb in lib/shinonome folder, then by using make).

Now, Russian text is readable:

Compare (menu from the beginning of DemonStory):

Some more examples (Russian translation of Ib and Don Miguel’s Sample Game):

There’s still no support for other languages besides Russian, unfortunately, because the codepage you’ve chosen doesn’t contain a whole range of Cyrillic… -_-" This can’t be easily fixed I believe…? Or can it?

All my changes are under CC0 (or in public domain, if that is possible in your jurisdiction), and I hope they can be incorporated into the project.

10

[quote=“Jikmok”]I finally got to draw the Cyrillic font, and it turned out easier to do it than I expected. :slight_smile:
(…)
All my changes are under CC0 (or in public domain, if that is possible in your jurisdiction), and I hope they can be incorporated into the project.[/quote]

Thank you very much for this! I hope this will be added for the upcoming release.

I don’t know the Cyrillic extended support for specific Windows on RPG Maker 2000/2003, as it is not Unicode and common Windows codepage was CP1251. Do you know games using them and which Windows encoding might being used if any?

11

Windows-1251 contains characters not only for Russian, but also for all the living Slavic languages in Cyrillic script: Belarusian (іў), Ukrainian (іїєґ), Serbian (јћђљњ), Macedonian (ѓќјѕџ).

I don’t know of any published games in these languages. If unpublished and unfinished games count, I was trying to make a Belarusian game once, but never finished it. ^^’

If EasyRPG supported Windows-1251, it would be great, but does it? As far as I can see from generate_cxx_font.rb, Cyrillic is stored in kanji_encoding, which is JIS_X0208. But maybe I’m missing something? If you can tell me what to change so that I could use cp1251, I could draw the missing cp1251 characters too.

Also, I think the expectations are different in 2014 and in 2000. In 2000, language support was an afterthought, and it was OK to neglect smaller languages. But now, most people expect something Unicodish… It would be sad if EasyRPG would not support languages like Tatar or Kazakh, which have no established encoding except Unicode. Of course, there are no games in these languages now… But how can they appear without a proper support?

[Off topic] By the way, about the Russian games… RM2k3 is going to be a problem because Russian games expect an unique RTP, Vlad’s RTP, which is expected to be installed alongside with the Don Miguel’s RTP. So e.g. it has two Basis chipsets (RM2k and RM2k3) available at the same time. I don’t know how Vlad come up with such a ‘brilliant’ idea, but it indeed enjoyed some use… And I don’t think these games can be handled in a compatible way.

12

Yeah, that was expected. Good that it really worked properly :slight_smile:

Cool. Thanks for your contribution. Will be integrated for sure!
At least for normal RPG Maker games this is not fixable because they are non-unicode Windows programs and are all depending on a codepage. There is no unicode version of the editor available, therefore there are no unicode rpg games in the wild.
This is fixable by using our own (still heavy under development) new editor and XML (all rpg files are converted from binary to xml). They will have complete unicode support (as you can see here: https://community.easyrpg.org/t/now-official-non-official-easyrpgs-editor/130/1#p968). Still highly experimental and not in the main codebase yet.

I havnt tested it but I guess you can just replace “kanji_encoding” with “UTF-32LE” and then specify the UTF-32 code under “STARTCHAR” for full unicode support (and add a if encoding == “UTF-32LE” around line 58 in the script because converting UTF-32LE to UTF-32LE will likely fail)

13

Thanks!

I believe this is fixable, albeit in a way incompatible with RPG_RT.exe. RM2k games can be encoded in encoding 65001, which is UTF-8.

Here is an example game with text encoded in UTF-8 (by manually editing the map file in the hex editor, but still…): utf8-test.tar.gz. EasyRPG seems to have no problem playing UTF-8 games:

Of course, RPG Maker 2000/2003 is not able to edit games in encoding 65001, but it may be useful anyway. An example of a use case when codepage 65001 can be useful: we could have a gettext-like program that extracts all the text from events and then puts them back translated (and having UTF-8 in translations makes it possible to support languages not supported by legacy code pages).

The only possible problem I can see are the string limits. UTF-8 stings usually require more place than strings in any other encoding. But RM2k seems to have a variable-size ints for all the numbers in the binary format, so I don’t think this will be a problem…

Thanks! I will try it and write back later.

14

Oh, nice… In that case you can try something else:

Use LCF2XML. To convert the files to XML. Then open the RPG_RT.ini and add.

[EasyRPG]
Encoding=65001

And convert the file back to binary via LCF2XML, it respects the RPG_RT.ini file (and autodetects the target format).

15

Your cyrillic font is now merged in the official player code. Thanks for your contribution!

16

Thanks!

As for other languages, changing kanji_encoding to UTF-32LE works like a charm and allows adding any Unicode characters.

I’m almost finished drawing most of the Cyrillic Unicode block, and I also plan to draw some Latin letters: Polish is definitely needed, as there are quite a few Polish games around (I haven’t tried playing them in EasyRPG yet, but I believe the current font doesn’t have ąćęłńśźż). I’ll post my other changes once they’re ready.