Problems with displaying chinese

[chenas]

Such as the title.

[paultech]

If you read the top of https://github.com/ocornut/imgui/blob/master/imgui.cpp you will see the answer is:

A: When loading a font, pass custom Unicode ranges to specify the glyphs to load. ImGui will support UTF-8 encoding across the board.

Character input depends on you passing the right character code to io.AddInputCharacter(). The example applications do that.

  io.Fonts->AddFontFromFileTTF("myfontfile.ttf", size_in_pixels, NULL, io.Fonts->GetGlyphRangesJapanese());  // Load Japanese characters
  io.Fonts->GetTexDataAsRGBA32() or GetTexDataAsAlpha8()
  io.ImeWindowHandle = MY_HWND;      // To input using Microsoft IME, give ImGui the hwnd of your application

`

[chenas]

I have tried that, it didn`t work. I try to display chinese characters, but just english.

io.Fonts->AddFontFromFileTTF("../../extra_fonts/Cousine-Regular.ttf", 16.0f, NULL, io.Fonts->GetGlyphRangesChinese());
ImGui::Text("\u6211\u662F\u4E2D\u6587");
ImGui::Text("fsdfsdad我是中文");

The demo example that I run in widows, also can`t display non-latin characters.

[ocornut]

The Cousine font doesn't have any Asian character. Use Arial Unicode or a font that support those characters.

[chenas]

I have tried fonts below, but it doesn`t work. It shows "?????" or "口口口" instead.
I also tried to save your file as 'UTF-8 without signature' with vs2010.
io.Fonts->AddFontFromFileTTF("../../extra_fonts/arial_unicode_ms.ttf", 10.0f);
io.Fonts->AddFontFromFileTTF("../../extra_fonts/Arial.ttf", 18.0f);
io.Fonts->AddFontFromFileTTF("c:\Windows\Fonts\ArialUni.ttf", 18.0f, NULL, io.Fonts->GetGlyphRangesChinese());

[ocornut]

Only the last one here has the correct parameter io.Fonts->GetGlyphRangesChinese().

Try to load only one font at a time - If you try to load more than one font you need to change font with PushFont/PopFont.

Make sure you use "c:\\Windows\\Fonts\\ArialUni.ttf" and the file exists (you can debug inside AddFontFromFileTTF and see if the file is correctly opened for you).

ImGui::Text("\u6211\u662F\u4E2D\u6587"); this are 16-bit unicode points, it won't work as UTF-8.
ImGui::Text("fsdfsdad我是中文"); should work if your source file is UTF-8 and the compiler accept it.

If you are not sure about your source code encoding, first try with ImGui::Text("Kanjis: \xe6\x97\xa5\xe6\x9c\xac\xe8\xaa\x9e (nihongo)"); If this work then your source code encoding isn't UTF-8 or the compiler doesn't understand it.

[chenas]

I tried what you said, that doesn`t work.
io.Fonts->AddFontFromFileTTF("../../extra_fonts/Arial.ttf", 18.0f, NULL, io.Fonts->GetGlyphRangesChinese());
ImGui::Text("fsdfsdad我是中文dfd");
ImGui::Text("Kanjis: \xe6\x97\xa5\xe6\x9c\xac\xe8\xaa\x9e (nihongo)");

[ocornut]

does "../../extra_fonts/Arial.ttf" exists? It is a full unicode font?
I don't provide any such font in extra_fonts/

[chenas]

I copy from C:\Windows\Fonts

[ocornut]

It works here. Are you copying ArialUni.ttf and NOT Arial.ttf ?

    ImGuiIO& io = ImGui::GetIO();
    io.Fonts->AddFontFromFileTTF("c:\\Windows\\Fonts\\ArialUni.ttf", 18.0f, NULL, io.Fonts->GetGlyphRangesChinese());

[...]

ImGui::Text("fsdfsdad我是中文dfd");
ImGui::Text("Kanjis: \xe6\x97\xa5\xe6\x9c\xac\xe8\xaa\x9e (nihongo)");

[chenas]

What！My develope env ： win7 64bit, vs2010.

[ocornut]

What's the size of your font file?
Are you copying ArialUni.ttf and NOT Arial.ttf ?

[ocornut]

No, we probably have the same one. You aren't answering all questions clearly when asked, making is harder to help you. Post your program code on pastebin.com or gist.

[chenas]

I dont think its the problem of the program code, I just downloaded from your git, and run in my machine. And I want to test if the Chinese characters can be displayed.

[chenas]

I downloaded ArialUni.ttf online, and added to the project.
ImGuiIO& io = ImGui::GetIO();
io.Fonts->AddFontFromFileTTF("../../extra_fonts/ArialUni.ttf", 18.0f, NULL, io.Fonts->GetGlyphRangesJapanese());
It display correctly!

But below, can`t display any characters, like the first picture I uploaded before.
io.Fonts->AddFontFromFileTTF("../../extra_fonts/ArialUni.ttf", 18.0f, NULL, io.Fonts->GetGlyphRangesChinese());

Anyway, thanks for helping!

[heroboy]

1. Use this font: "C:\windows\fonts\SimSun.ttc" or "msyh.ttc" or "simhei.ttf".
2. You should convert string to utf8 encoding.

[chenas]

I tried, even the English can`t be displayed correctly!
io.Fonts->AddFontFromFileTTF("c:\Windows\Fonts\msyh.ttc", 18.0f, NULL, io.Fonts->GetGlyphRangesChinese());

[ocornut]

Try this:

• AddFontDefault() first
• then add your font as second font
• call ImGui::ShowTestWindow() and look in "Window Options"->"Fonts"->Atlas Texture what do you see?

[chenas]

io.Fonts->AddFontDefault();
io.Fonts->AddFontFromFileTTF("../../extra_fonts/ArialUni.ttf", 18.0f, NULL, io.Fonts->GetGlyphRangesChinese());

io.Fonts->AddFontDefault();
io.Fonts->AddFontFromFileTTF("../../extra_fonts/ArialUni.ttf", 18.0f, NULL, io.Fonts->GetGlyphRangesJapanese());

[ocornut]

It looks like your problem is with uploading the texture. With the default oversampling settings the texture will be very big and it may not work with your drivers. What rendering backend are you using?

Lower quality:

    ImFontConfig font_config; font_config.OversampleH = 1; font_config.OversampleV = 1; font_config.PixelSnapH = 1;
    io.Fonts->AddFontFromFileTTF("c:\\Windows\\Fonts\\ArialUni.ttf", 18.0f, &font_config, io.Fonts->GetGlyphRangesChinese());

OversampleH is by default 3. You can try 1 or 2.

This is mainly because the atlas system is not currently dynamic..we're uploading all 22000 characters to a texture which is a big big waste here.

[chenas]

I run the opengl_exmple, suppose opengl rendering backend.
I didn`t use AddFontDefault(), some code like below：
ImGuiIO& io = ImGui::GetIO();
ImFontConfig font_config;
font_config.OversampleH = 1; //or 2 is the same
font_config.OversampleV = 1;
font_config.PixelSnapH = 1;
io.Fonts->AddFontFromFileTTF("../../extra_fonts/ArialUni.ttf", 18.0f, &font_config, io.Fonts->GetGlyphRangesChinese());

It seems Chinese characters can be displayed correctly, but some characters will be lost.
ImGui::Text("hello 菜单 在哪里文件我是中文a");
ImGui::Text("hello 菜单 在哪 里文件 我是中文a");

[ocornut]

The second line has spaces I wonder if perhaps they have incorrect width.
What happens if you add your 2 lines twice?

        ImGui::Text("hello 菜单 在哪里文件我是中文a");
        ImGui::Text("hello 菜单 在哪 里文件 我是中文a");
        ImGui::Text("hello 菜单 在哪里文件我是中文a");
        ImGui::Text("hello 菜单 在哪 里文件 我是中文a");

[chenas]

By the way, the ArialUni.ttf is too big, you are right, it`s a big big waste. Do I have another way?

[ocornut]

What happen if you remove the spacing between 在哪 and 里文 ?
Maybe it isn't a regular U+0020 space and it has a wrong width (need to investigate).

You can load a limited range, by copying and modifying this function

const ImWchar*  ImFontAtlas::GetGlyphRangesChinese()
{
    static const ImWchar ranges[] =
    {
        0x0020, 0x00FF, // Basic Latin + Latin Supplement
        0x3000, 0x30FF, // Punctuations, Hiragana, Katakana
        0x31F0, 0x31FF, // Katakana Phonetic Extensions
        0xFF00, 0xFFEF, // Half-width characters
        0x4e00, 0x9FAF, // CJK Ideograms
        0,
    };
    return &ranges[0];
}

If you know the ranges of characters you need.

Otherwise, unfortunately you can't yet. I need to rewrite ImFontAtlas to allocate characters dynamically as they are needed.

[chenas]

It displayed like the third line of the picture that I uploaded. The space is a Chinese character, too.

Unfortunately, I don't know the range of Chinese characters. I will try to find it on line later.

[ocornut]

I'd like to know which character it is (it appears like a normal space when pasted on Github).
Can you dump the string?
Something like

const char* p = "hello 菜单 在哪 里文件 我是中文a";
while (*p)
{
  printf("%02X\n", *p);
  p++;
}

[chenas]

I copied your code and changed the space to Chinese space.

Also, I find a very strange thing.
ImGui::Text("hello 菜单 在哪 里文件 我是中文a");
If I delete an "a" at the end of the Chinese characters, the compiler will say there is something wrong with this line, seems like missing bracket ')'.
But, it`s ok as below.
ImGui::Text("hello 菜单 在哪里文件我是中文a");
ImGui::Text("hello 菜单 在哪 里文件 我是中文");

[ocornut]

Sorry could you try this one

const char* p = "hello 菜单 在哪 里文件 我是中文a";
while (*p)
{
  printf("%02X\n", (unsigned char)*p);
  p++;
}

I'll look at it tomorrow :)
Thanks!

[chenas]

:)

[hypernewbie]

22K characters? so THAT'S where all the gpu vidmem went

[chenas]

@hypernewbie Sorry, I don`t understand what you said. Could you tell me more?

[ocornut]

\xE4\xBB\x3F looks like invalid UTF-8

This page doesn't decode it: https://mothereff.in/utf-8
And the code in ImGui doesn't decode it either. Investigating.

ImGui::Text("\x68\x65\x6C\x6C\x6F\x20" "\xE8\x8F\x9C\xE5\x8D\x95\x20" "\xE5\x9C\xA8\xE5\x93\xAA\x20" "a" "\xE9\x87\x8C\xE6\x96\x87" "\xE4\xBB\x3F\xE6\x88\x91\xE6\x98\xAF\xE4\xB8\xAD\xE6\x96\x87\x61");

"件" should be \xE4\xBB\xB6
"件 " (given a normal space) should be \xE4\xBB\xB6\x20
but your stream has \xE4\xBB\x3F

There's no way I can recover the 6 meaningful bits from B6 with 3F, I think it is an encoding bug with Visual Studio. Maybe you have invalid UTF8 data in your file and Visual Studio is being confused. If I paste your data here the data becomes correct (probably corrected/filtered by github).

Right after is \xE6\x88\x91 which is 我

[ocornut]

I have made a change to the UTF-8 decoder to avoid terminating a string on malformed input.

So it is easier to understand what's going on, but it is a bug in your data/software/visual studio. Maybe if you just delete the line and input it again it will be correct?

As for VRAM correct, it is in my task list to allocate and render memory only when characters are used, but I don't know when I can do it.

[BentleyBlanks]

It seems workd after I add prefix 「u8」to the string. Looks like this

ImGui::Text(u8"hello 菜单 在哪里文件我是中文a");  // success on Visual Studio 2015
ImGui::Text("hello 菜单 在哪里文件我是中文a");  // didn't workd

Maybe it is the problem of Visual Studio ?(unicode font have been loaded as texture data already)

[ocornut]

Using u8 if the standard c++11 way to request a string literal in utf-8. I've been starting to make that more explicit in the docs/examples.

[sgf]

@BentleyBlanks i think thats because ur code file is not unicode(u8 or gb2132?) format.

[mylylyl]

Facing this weird stuff today. The program just fine 2 minutes ago and I re-compile it then every character becomes white box
*edit: works fine after using font settings

[wollong]

How can I support to display multi-language using format like this
string str = u8"我是中文";
ImGui::Text("%s",str);

It's possible to make it?

[ocornut]

It's possible to make it?

Please read FAQ and misc/fonts/README.txt
(EDIT 2019: now in docs/FONTS.txt)

[wollong]

It's possible to make it?

Please read FAQ and misc/fonts/README.txt

thx a lot! figured out codecvt works

[sweihub]

It works for me with the code for latest imgui

// io.Fonts->AddFontDefault();  --- DON'T CALL THIS
ImFont* font = io.Fonts->AddFontFromFileTTF("c:\\Windows\\Fonts\\msyh.ttc", 
        16.0f, NULL, io.Fonts->GetGlyphRangesChineseFull());
...

ImGui::Begin(u8"Hello, world! 这是窗口标题"); 

[xfdang]

It has been a while, but I slove my problem by toggle 'Use Unicode UTF-8 for worldwide language support' in region settings(win10).

[PathogenDavid]

@xfdang: You really should not rely on the system codepage being UTF-8, doing so means your project won't compile correctly on other machines.

You're much better off either setting it explicitly, using UTF-8 BOM encoding for your files, or using the u8 prefix on strings.

[xiangfa7766]

Now everyone have a good solution