Regression: Certain US International keyboard layout functions broken #103119

New Issue

John · 2022-12-10T23:44:19+01:00

John commented

2022-12-10 23:44:19 +01:00

System Information
Operating System: Windows 10 Pro 64bit

Blender Version
Broken: 3.4.0
Worked: 3.3
Caused by f7027f2253

Description Of The Problem
The United States International Keyboard enables users to type characters such as á by pressing the keys ' followed by a. One types the ' character by pressing that key and then the spaceboard key. (Or the same key again, which outputs two of the characters)

In the Blender 3.3.1 Text Editor, this functions as expected, but in 3.4 I find that this functionality is completely broken. Pressing ' followed by a produces the a character, and pressing ' and then the spacebar produces a space, leaving me unable to type the ' character. Pressing ' repeatedly produces no characters at all. The same issue applies to other keys with similar uses in the layout such as " or ```.

**System Information** Operating System: Windows 10 Pro 64bit **Blender Version** Broken: 3.4.0 Worked: 3.3 Caused by f7027f2253 **Description Of The Problem** The United States International Keyboard enables users to type characters such as `á` by pressing the keys `'` followed by `a`. One types the `'` character by pressing that key and then the spaceboard key. (Or the same key again, which outputs two of the characters) In the Blender 3.3.1 Text Editor, this functions as expected, but in 3.4 I find that this functionality is completely broken. Pressing `'` followed by `a` produces the `a` character, and pressing `'` and then the spacebar produces a space, leaving me unable to type the `'` character. Pressing `'` repeatedly produces no characters at all. The same issue applies to other keys with similar uses in the layout such as `"` or ```.

John commented

2022-12-10 23:44:19 +01:00

Added subscriber: @Solstice245

Iliya Katushenock commented

2022-12-11 00:13:55 +01:00

Added subscriber: @mod_moder

Iliya Katushenock commented

2022-12-11 00:13:55 +01:00

#103098 (Text object doesn't work with accented letters.)

Iliya Katushenock commented

2022-12-11 00:22:34 +01:00

Changed status from 'Needs Triage' to: 'Archived'

Iliya Katushenock closed this issue

2022-12-11 00:22:34 +01:00

Iliya Katushenock commented

2022-12-11 00:22:34 +01:00

3.1:
{key ctrl alt `}
3.5:
{key ctrl alt ` '}

- 3.1: {key ctrl alt `} ![image.png](https://archive.blender.org/developer/F14051019/image.png) - 3.5: {key ctrl alt ` '} ![image.png](https://archive.blender.org/developer/F14051022/image.png)

Iliya Katushenock commented

2022-12-11 00:29:43 +01:00

Changed status from 'Archived' to: 'Needs Triage'

Iliya Katushenock reopened this issue

2022-12-11 00:29:43 +01:00

Iliya Katushenock commented

2022-12-11 00:29:43 +01:00

Though maybe I was too quick to close it. Can you tell if this works for you? Some explanation of this was given in the report mentioned above.

John commented

2022-12-11 00:30:03 +01:00

I don't care if you have some annoying work around, to me this represents a clear step back in terms of user friendliness and should be addressed.
I will also add that your instructions are unclear as I still find myself unable to produce the ' character itself given that information.

EDIT: It instead produces the ´ character.

I don't care if you have some annoying work around, to me this represents a clear step back in terms of user friendliness and should be addressed. I will also add that your instructions are unclear as I still find myself unable to produce the `'` character itself given that information. EDIT: It instead produces the `´` character.

Harley Acheson commented

2022-12-11 04:12:55 +01:00

Added subscriber: @Harley

Harley Acheson commented

2022-12-11 04:12:55 +01:00

I see it working as expected in 3.3.1 with US International Keyboard, but not working in master.

With this keyboard layout set, pressing single-quote then a gives you á, while pressing double-quote then e results in ë. 3.3.1 works as expected, master actually swallows both single and double-quote.

Nothing is jumping out in the change log for GHOST_SystemWin32.cpp. This might have to be bisected.

I see it working as expected in 3.3.1 with US International Keyboard, but not working in master. With this keyboard layout set, pressing single-quote then a gives you á, while pressing double-quote then e results in ë. 3.3.1 works as expected, master actually swallows both single and double-quote. Nothing is jumping out in the change log for GHOST_SystemWin32.cpp. This might have to be bisected.

Harley Acheson commented

2022-12-11 04:13:06 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Harley Acheson commented

2022-12-11 05:08:05 +01:00

I see it breaking between 3.3.2 (working) and 3.4.0 (not working)

Pratik Borhade commented

2022-12-12 07:31:15 +01:00

Added subscriber: @PratikPB2123

Pratik Borhade commented

2022-12-12 07:49:08 +01:00

Added subscriber: @ideasman42

Pratik Borhade commented

2022-12-12 07:49:08 +01:00

Caused by f7027f2253
@ideasman42 ^

Caused by f7027f2253 @ideasman42 ^

Harley Acheson commented

2022-12-12 18:16:36 +01:00

@ideasman42 - This is minimum changes that I can make that allow this dead key composition stuff. Just two small changes:

P3387: Fix #103119: Min Changes to Allow Dead Key Layouts

diff --git a/intern/ghost/intern/GHOST_SystemWin32.cpp b/intern/ghost/intern/GHOST_SystemWin32.cpp
index 7aef23b39b67..089b2bcec889 100644
--- a/intern/ghost/intern/GHOST_SystemWin32.cpp
+++ b/intern/ghost/intern/GHOST_SystemWin32.cpp
@@ -1174,102 +1174,102 @@ void GHOST_SystemWin32::processWheelEvent(GHOST_WindowWin32 *window, WPARAM wPar
 GHOST_EventKey *GHOST_SystemWin32::processKeyEvent(GHOST_WindowWin32 *window, RAWINPUT const &raw)
 {
   const char vk = raw.data.keyboard.VKey;
   bool key_down = false;
   GHOST_SystemWin32 *system = (GHOST_SystemWin32 *)getSystem();
   GHOST_TKey key = system->hardKey(raw, &key_down);
   GHOST_EventKey *event;
 
   /* NOTE(@campbellbarton): key repeat in WIN32 also applies to modifier-keys.
    * Check for this case and filter out modifier-repeat.
    * Typically keyboard events are *not* filtered as part of GHOST's event handling.
    * As other GHOST back-ends don't have the behavior, it's simplest not to send them through.
    * Ideally it would be possible to check the key-map for keys that repeat but this doesn't look
    * to be supported. */
   bool is_repeat = false;
   bool is_repeated_modifier = false;
   if (key_down) {
     if (system->m_keycode_last_repeat_key == vk) {
       is_repeat = true;
       is_repeated_modifier = GHOST_KEY_MODIFIER_CHECK(key);
     }
     system->m_keycode_last_repeat_key = vk;
   }
   else {
     if (system->m_keycode_last_repeat_key == vk) {
       system->m_keycode_last_repeat_key = 0;
     }
   }
 
   /* We used to check `if (key != GHOST_kKeyUnknown)`, but since the message
    * values `WM_SYSKEYUP`, `WM_KEYUP` and `WM_CHAR` are ignored, we capture
    * those events here as well. */
   if (!is_repeated_modifier) {
     char utf8_char- [x] = {0};
     BYTE state[256];
     const BOOL has_state = GetKeyboardState((PBYTE)state);
     const bool ctrl_pressed = has_state && state[VK_CONTROL] & 0x80;
     const bool alt_pressed = has_state && state[VK_MENU] & 0x80;
 
-    if (!key_down) {
-      /* Pass. */
-    }
     /* No text with control key pressed (Alt can be used to insert special characters though!). */
-    else if (ctrl_pressed && !alt_pressed) {
+    if (ctrl_pressed && !alt_pressed) {
       /* Pass. */
     }
     /* Don't call #ToUnicodeEx on dead keys as it clears the buffer and so won't allow diacritical
      * composition. */
     else if (MapVirtualKeyW(vk, 2) != 0) {
       wchar_t utf16- [x] = {0};
       int r;
       /* TODO: #ToUnicodeEx can respond with up to 4 utf16 chars (only 2 here).
        * Could be up to 24 utf8 bytes. */
       if ((r = ToUnicodeEx(
                vk, raw.data.keyboard.MakeCode, state, utf16, 2, 0, system->m_keylayout))) {
         if ((r > 0 && r < 3)) {
           utf16- [x] = 0;
           conv_utf_16_to_8(utf16, utf8_char, 6);
         }
         else if (r == -1) {
           utf8_char- [x] = '\0';
         }
       }
+      if (!key_down) {
+        utf8_char- [x] = '\0';
+      }
     }
 
 #ifdef WITH_INPUT_IME
     if (key_down && ((utf8_char- [x] & 0x80) == 0)) {
       const char ascii = utf8_char[0];
       if (window->getImeInput()->IsImeKeyEvent(ascii, key)) {
         return NULL;
       }
     }
 #endif /* WITH_INPUT_IME */
 
     event = new GHOST_EventKey(system->getMilliSeconds(),
                                key_down ? GHOST_kEventKeyDown : GHOST_kEventKeyUp,
                                window,
                                key,
                                is_repeat,
                                utf8_char);
 
 #if 0 /* we already get this info via EventPrinter. */
     GHOST_PRINTF("%c\n", ascii);
 #endif
   }
   else {
     event = NULL;
   }
 
   return event;
 }
 
 GHOST_Event *GHOST_SystemWin32::processWindowSizeEvent(GHOST_WindowWin32 *window)
 {
   GHOST_SystemWin32 *system = (GHOST_SystemWin32 *)getSystem();
   GHOST_Event *sizeEvent = new GHOST_Event(
       system->getMilliSeconds(), GHOST_kEventWindowSize, window);
 
   /* We get WM_SIZE before we fully init. Do not dispatch before we are continuously resizing. */
   if (window->m_inLiveResize) {
     system->pushEvent(sizeEvent);
     system->dispatchEvents();

The first part, not skipping if not key_down, is needed to get the dead key stuff to do anything. I'm a little fuzzy on this, but I imagine that the actual keys get swallowed then the composed character comes in without a key down or key press?

Without the second change, setting utf8_char- [x] to zero if not key_down, everything seems to work fine but there are console warnings that "wm_event_add_ghostevent: ghost on your platform is misbehaving, utf8 events on key up"

@ideasman42 - This is minimum changes that I can make that allow this dead key composition stuff. Just two small changes: [P3387: Fix #103119: Min Changes to Allow Dead Key Layouts](https://archive.blender.org/developer/P3387.txt) ``` diff --git a/intern/ghost/intern/GHOST_SystemWin32.cpp b/intern/ghost/intern/GHOST_SystemWin32.cpp index 7aef23b39b67..089b2bcec889 100644 --- a/intern/ghost/intern/GHOST_SystemWin32.cpp +++ b/intern/ghost/intern/GHOST_SystemWin32.cpp @@ -1174,102 +1174,102 @@ void GHOST_SystemWin32::processWheelEvent(GHOST_WindowWin32 *window, WPARAM wPar GHOST_EventKey *GHOST_SystemWin32::processKeyEvent(GHOST_WindowWin32 *window, RAWINPUT const &raw) { const char vk = raw.data.keyboard.VKey; bool key_down = false; GHOST_SystemWin32 *system = (GHOST_SystemWin32 *)getSystem(); GHOST_TKey key = system->hardKey(raw, &key_down); GHOST_EventKey *event; /* NOTE(@campbellbarton): key repeat in WIN32 also applies to modifier-keys. * Check for this case and filter out modifier-repeat. * Typically keyboard events are *not* filtered as part of GHOST's event handling. * As other GHOST back-ends don't have the behavior, it's simplest not to send them through. * Ideally it would be possible to check the key-map for keys that repeat but this doesn't look * to be supported. */ bool is_repeat = false; bool is_repeated_modifier = false; if (key_down) { if (system->m_keycode_last_repeat_key == vk) { is_repeat = true; is_repeated_modifier = GHOST_KEY_MODIFIER_CHECK(key); } system->m_keycode_last_repeat_key = vk; } else { if (system->m_keycode_last_repeat_key == vk) { system->m_keycode_last_repeat_key = 0; } } /* We used to check `if (key != GHOST_kKeyUnknown)`, but since the message * values `WM_SYSKEYUP`, `WM_KEYUP` and `WM_CHAR` are ignored, we capture * those events here as well. */ if (!is_repeated_modifier) { char utf8_char- [x] = {0}; BYTE state[256]; const BOOL has_state = GetKeyboardState((PBYTE)state); const bool ctrl_pressed = has_state && state[VK_CONTROL] & 0x80; const bool alt_pressed = has_state && state[VK_MENU] & 0x80; - if (!key_down) { - /* Pass. */ - } /* No text with control key pressed (Alt can be used to insert special characters though!). */ - else if (ctrl_pressed && !alt_pressed) { + if (ctrl_pressed && !alt_pressed) { /* Pass. */ } /* Don't call #ToUnicodeEx on dead keys as it clears the buffer and so won't allow diacritical * composition. */ else if (MapVirtualKeyW(vk, 2) != 0) { wchar_t utf16- [x] = {0}; int r; /* TODO: #ToUnicodeEx can respond with up to 4 utf16 chars (only 2 here). * Could be up to 24 utf8 bytes. */ if ((r = ToUnicodeEx( vk, raw.data.keyboard.MakeCode, state, utf16, 2, 0, system->m_keylayout))) { if ((r > 0 && r < 3)) { utf16- [x] = 0; conv_utf_16_to_8(utf16, utf8_char, 6); } else if (r == -1) { utf8_char- [x] = '\0'; } } + if (!key_down) { + utf8_char- [x] = '\0'; + } } #ifdef WITH_INPUT_IME if (key_down && ((utf8_char- [x] & 0x80) == 0)) { const char ascii = utf8_char[0]; if (window->getImeInput()->IsImeKeyEvent(ascii, key)) { return NULL; } } #endif /* WITH_INPUT_IME */ event = new GHOST_EventKey(system->getMilliSeconds(), key_down ? GHOST_kEventKeyDown : GHOST_kEventKeyUp, window, key, is_repeat, utf8_char); #if 0 /* we already get this info via EventPrinter. */ GHOST_PRINTF("%c\n", ascii); #endif } else { event = NULL; } return event; } GHOST_Event *GHOST_SystemWin32::processWindowSizeEvent(GHOST_WindowWin32 *window) { GHOST_SystemWin32 *system = (GHOST_SystemWin32 *)getSystem(); GHOST_Event *sizeEvent = new GHOST_Event( system->getMilliSeconds(), GHOST_kEventWindowSize, window); /* We get WM_SIZE before we fully init. Do not dispatch before we are continuously resizing. */ if (window->m_inLiveResize) { system->pushEvent(sizeEvent); system->dispatchEvents(); ``` The first part, not skipping if not key_down, is needed to get the dead key stuff to do anything. I'm a little fuzzy on this, but I imagine that the actual keys get swallowed then the composed character comes in without a key down or key press? Without the second change, setting utf8_char- [x] to zero if not key_down, everything seems to work fine but there are console warnings that "wm_event_add_ghostevent: ghost on your platform is misbehaving, utf8 events on key up"

Campbell Barton commented

2022-12-13 05:25:59 +01:00

@Harley thanks for looking into this, for 3.4x we could just revert however this is causing conflicts.

@Solstice245 could you see if P3387 works as expected? - this is a build from our built-bot:

https://builder.blender.org/download/experimental/blender-3.5.0-alpha+temp-win32-dead-key-fix.303216d3d3ab-windows.amd64-release.zip

@Harley thanks for looking into this, for 3.4x we could just revert however this is causing conflicts. @Solstice245 could you see if [P3387](https://archive.blender.org/developer/P3387.txt) works as expected? - this is a build from our built-bot: https://builder.blender.org/download/experimental/blender-3.5.0-alpha+temp-win32-dead-key-fix.303216d3d3ab-windows.amd64-release.zip

John commented

2022-12-13 09:23:58 +01:00

I will check that out when I can, which is hopefully in the next 24 hours.