error message update translation #74959

New Issue

Jozef Matta · 2020-03-20T09:21:23+01:00

Jozef Matta commented

2020-03-20 09:21:23 +01:00

System Information
Operating system: Windows-10-10.0.18362 64 Bits
Graphics card: GeForce GTX 960/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 441.12

Blender Version
Broken: version: 2.82 (sub 7), branch: master, commit date: 2020-02-12 16:20, hash: blender/blender@77d23b0bd7
Worked: (optional)

Short description of error
In use I18n Update Traslation/Update Trunk an error message is displayed:

Traceback (most recent call last):

File "C:\Program Files\Blender Foundation\Blender 2.82\2.82\scripts\addons\ui_translate\update_svn.py", line 197, in execute
  utils_languages_menu.gen_menu_file(stats, self.settings)
File "C:\Program Files\Blender Foundation\Blender 2.82\2.82\scripts\modules\bl_i18n_utils\utils_languages_menu.py", line 97, in gen_menu_file
  f.write("\n".join(data_lines))
File "C:\Program Files\Blender Foundation\Blender 2.82\2.82\python\lib\encodings\cp1250.py", line 19, in encode
  return codecs.charmap_encode(input,self.errors,encoding_table)[0]

UnicodeEncodeError: 'charmap' codec can't encode characters in position 544-546: character maps to

location: :-1

Exact steps for others to reproduce the error
[Please describe the exact steps needed to reproduce the issue]
[Based on the default startup or an attached .blend file (as simple as possible)]

**System Information** Operating system: Windows-10-10.0.18362 64 Bits Graphics card: GeForce GTX 960/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 441.12 **Blender Version** Broken: version: 2.82 (sub 7), branch: master, commit date: 2020-02-12 16:20, hash: `blender/blender@77d23b0bd7` Worked: (optional) **Short description of error** In use I18n Update Traslation/Update Trunk an error message is displayed: Traceback (most recent call last): ``` File "C:\Program Files\Blender Foundation\Blender 2.82\2.82\scripts\addons\ui_translate\update_svn.py", line 197, in execute utils_languages_menu.gen_menu_file(stats, self.settings) File "C:\Program Files\Blender Foundation\Blender 2.82\2.82\scripts\modules\bl_i18n_utils\utils_languages_menu.py", line 97, in gen_menu_file f.write("\n".join(data_lines)) File "C:\Program Files\Blender Foundation\Blender 2.82\2.82\python\lib\encodings\cp1250.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0] ``` UnicodeEncodeError: 'charmap' codec can't encode characters in position 544-546: character maps to <undefined> location: <unknown location>:-1 **Exact steps for others to reproduce the error** [Please describe the exact steps needed to reproduce the issue] [Based on the default startup or an attached .blend file (as simple as possible)]

Jozef Matta commented

2020-03-20 09:21:24 +01:00

Added subscriber: @pegas923

Germano Cavalcante commented

2020-03-20 16:08:42 +01:00

Added subscribers: @mont29, @mano-wii

Germano Cavalcante commented

2020-03-20 16:08:42 +01:00

This report does not have the steps to reproduce the problem, which is required for us to investigate the issue.

@mont29, is that something you are familiar with?

This report does not have the steps to reproduce the problem, which is required for us to investigate the issue. @mont29, is that something you are familiar with?

Bastien Montagne commented

2020-03-20 17:23:39 +01:00

Changed status from 'Needs Triage' to: 'Needs Developer To Reproduce'

Bastien Montagne self-assigned this 2020-03-20 17:23:39 +01:00

Bastien Montagne commented

2020-03-20 17:23:39 +01:00

Added subscriber: @LazyDodo

Bastien Montagne commented

2020-03-20 17:23:39 +01:00

The joys of windows and its non-UTF8 encodings… Am surprised though, @LazyDodo isn't our own "release python" supposed to enforce utf8 encoding as default? or is there no way to prevent it from using default OS encoding as default?

Can always explicitly force UTF8 encoding everywhere, but this is a bit painful and verbose…

The joys of windows and its non-UTF8 encodings… Am surprised though, @LazyDodo isn't our own "release python" supposed to enforce utf8 encoding as default? or is there no way to prevent it from using default OS encoding as default? Can always explicitly force UTF8 encoding everywhere, but this is a bit painful and verbose…

Ray molenkamp commented

2020-03-20 17:38:00 +01:00

Beats me, happy to look at this, but i'm unsure how to reproduce, the documentation i found does a great job on how to use the tools, but beyond 'All this makes references to the bf-translations svn repository.` it doesn't really help all that much on how to set it up.

So for me to look at this:

Assume i'm a 7 year old
Assume i'm not the brightest crayon in the box
Tell me how to reproduce this

Quick test seems to agree with you though that UTF8 is default

PYTHON INTERACTIVE CONSOLE 3.7.4 (default, Feb 17 2020, 16:23:28) [MSC v.1916 64 bit (AMD64)]

Builtin Modules:       bpy, bpy.data, bpy.ops, bpy.props, bpy.types, bpy.context, bpy.utils, bgl, blf, mathutils
Convenience Imports:   from mathutils import *; from math import *
Convenience Variables: C = bpy.context, D = bpy.data

>>> import sys; print(sys.getdefaultencoding())

utf-8

>>>

Beats me, happy to look at this, but i'm unsure how to reproduce, the [documentation i found ](https://wiki.blender.org/wiki/Process/Translate_Blender) does a great job on how to use the tools, but beyond 'All this makes references to the bf-translations svn repository.` it doesn't really help all that much on how to set it up. So for me to look at this: 1) Assume i'm a 7 year old 2) Assume i'm not the brightest crayon in the box 3) Tell me how to reproduce this Quick test seems to agree with you though that UTF8 is default ``` PYTHON INTERACTIVE CONSOLE 3.7.4 (default, Feb 17 2020, 16:23:28) [MSC v.1916 64 bit (AMD64)] Builtin Modules: bpy, bpy.data, bpy.ops, bpy.props, bpy.types, bpy.context, bpy.utils, bgl, blf, mathutils Convenience Imports: from mathutils import *; from math import * Convenience Variables: C = bpy.context, D = bpy.data >>> import sys; print(sys.getdefaultencoding()) utf-8 >>> ```

Bastien Montagne commented

2020-03-20 18:25:11 +01:00

@LazyDodo Don’t think you need to use the whole add-on? Just try to write a text file from blender's python, using complex characters (like e.g. some random Chinese ones), on a windows with non-UTF8 default encoding?

Something like that, e.g.:

import os
with open(os.path.join(bpy.app.tempdir, "test.txt"), 'w') as f:

f.write("yún 云/雲 – 'cloud'")

@LazyDodo Don’t think you need to use the whole add-on? Just try to write a text file from blender's python, using complex characters (like e.g. some random Chinese ones), on a windows with non-UTF8 default encoding? Something like that, e.g.: ```lang=python import os with open(os.path.join(bpy.app.tempdir, "test.txt"), 'w') as f: ``` f.write("yún 云/雲 – 'cloud'") ```

Bastien Montagne commented

2020-03-20 18:26:06 +01:00

Changed status from 'Needs Developer To Reproduce' to: 'Needs User Info'

Bastien Montagne commented

2020-03-20 18:26:06 +01:00

@pegas923 would be nice if you could run those three lines in your Blender py console as well, actually. ;)

Ray molenkamp commented

2020-03-20 18:50:28 +01:00

Out of the box doesn't work

>>> import os
>>> with open(os.path.join(bpy.app.tempdir, "test.txt"), 'w') as f:

...     f.write("yún 云/雲 – 'cloud'")
...     
Traceback (most recent call last):
  File "<blender_console>", line 2, in <module>
  File "k:\BlenderGit\build_windows_Full_x64_vc16_Release\bin\Release\2.83\python\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u4e91' in position 4: character maps to <undefined>

forcing it to utf8 does seem to do the trick though.

>>> import os
>>> with open(os.path.join(bpy.app.tempdir, "test.txt"), 'w', encoding="utf-8") as f:

...     f.write("yún 云/雲 – 'cloud'")
...     
17

>>>

I'm not python expert (as you noticed) but this seems to imply that files by default use platform specific encoding unless you specify otherwise.

Out of the box doesn't work ``` >>> import os >>> with open(os.path.join(bpy.app.tempdir, "test.txt"), 'w') as f: ... f.write("yún 云/雲 – 'cloud'") ... Traceback (most recent call last): File "<blender_console>", line 2, in <module> File "k:\BlenderGit\build_windows_Full_x64_vc16_Release\bin\Release\2.83\python\lib\encodings\cp1252.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0] UnicodeEncodeError: 'charmap' codec can't encode character '\u4e91' in position 4: character maps to <undefined> ``` forcing it to utf8 does seem to do the trick though. ``` >>> import os >>> with open(os.path.join(bpy.app.tempdir, "test.txt"), 'w', encoding="utf-8") as f: ... f.write("yún 云/雲 – 'cloud'") ... 17 >>> ``` I'm not python expert (as you noticed) but [this seems to imply ](http://python-notes.curiousefficiency.org/en/latest/python3/text_file_processing.html#files-in-a-typical-platform-specific-encoding) that files by default use platform specific encoding unless you specify otherwise.

Bastien Montagne commented

2020-03-20 20:41:54 +01:00

@LazyDodo oki thanks, then I have to put that explicit UTF8 encoding everywhere... Kind of assumed our own py in Blender was smart and enforced utf8 encoding by default. ;)

blender-admin commented

2020-03-20 20:49:48 +01:00

This issue was referenced by blender/blender@9607e54985

This issue was referenced by blender/blender@9607e54985838006ca5695ed88bdaf81dda29b4c

Bastien Montagne commented

2020-03-20 20:53:10 +01:00

Changed status from 'Needs User Info' to: 'Resolved'

Bastien Montagne closed this issue

2020-03-20 20:53:10 +01:00

Ray molenkamp commented

2020-03-20 20:56:16 +01:00

Looks like python is going to start warning in never versions when you not specify an encoding judging from PIP 597 , so it may not be a bad idea to be explicit everywhere (as annoying as it is)

For years we shipped a modified python on windows (mostly to deal with our old old compiler we used for releases, but i don't think we ever messed with the encoding stuff) but nowdays we are on a stock python,

Looks like python is going to start warning in never versions when you not specify an encoding judging from [PIP 597 ](https://discuss.python.org/t/pep-597-use-utf-8-for-default-text-file-encoding/1819) , so it may not be a bad idea to be explicit everywhere (as annoying as it is) For years we shipped a modified python on windows (mostly to deal with our old old compiler we used for releases, but i don't think we ever messed with the encoding stuff) but nowdays we are on a stock python,

Sign in to join this conversation.