DXFs with non-ascii characters make th edxf importer to crash. #48806

New Issue

mathieu menuet · 2016-07-07T18:26:44+02:00

mathieu menuet commented

2016-07-07 18:26:44 +02:00

System Information
WIn 7 x 64

Blender Version
Broken: 2.77a

Short description of error
The DXF importer crashes every time a DXF has non-ascii characters. For example if there is an ä.

Exact steps for others to reproduce the error
Try to import attached dxf.test nonascii bl 2.dxf
This dxf import successfully in Autocad, QGis and Accutrans.

**System Information** WIn 7 x 64 **Blender Version** Broken: 2.77a **Short description of error** The DXF importer crashes every time a DXF has non-ascii characters. For example if there is an ä. **Exact steps for others to reproduce the error** Try to import attached dxf.[test nonascii bl 2.dxf](https://archive.blender.org/developer/F320020/test_nonascii_bl_2.dxf) This dxf import successfully in Autocad, QGis and Accutrans.

mathieu menuet commented

2016-07-07 18:26:44 +02:00

Changed status to: 'Open'

mathieu menuet commented

2016-07-07 18:26:44 +02:00

Added subscriber: @bliblubli

Vuk Gardašević commented

2016-07-08 13:39:40 +02:00

Added subscriber: @VukGardasevic

Vuk Gardašević commented

2016-07-08 13:39:40 +02:00

OpenCad also opens the file.

The dxf entry inf.0 is not parsed properly and hits an ValueError check in pytags.py because the script tries to convert it to a float.

OpenCad also opens the file. The dxf entry inf.0 is not parsed properly and hits an ValueError check in pytags.py because the script tries to convert it to a float.

Bastien Montagne commented

2016-08-16 15:18:36 +02:00

Added subscribers: @cnd, @mont29

Lukas Treyer was assigned by Bastien Montagne

2016-08-16 15:18:36 +02:00

Bastien Montagne commented

2016-08-16 15:18:36 +02:00

@cnd, mind having a look at that one? Thanks.

Lukas Treyer commented

2016-08-16 18:26:31 +02:00

I looked at it. Forwarded the issue to the author of the dxfgrabber library: https://bitbucket.org/mozman/dxfgrabber/issues/13/inf0-values-not-parsed-properly

I also updated the dxfgrabber library to the latest version and added the authors patches to the dxf importer but it didn't help.

I looked at it. Forwarded the issue to the author of the dxfgrabber library: https://bitbucket.org/mozman/dxfgrabber/issues/13/inf0-values-not-parsed-properly I also updated the dxfgrabber library to the latest version and added the authors patches to the dxf importer but it didn't help.

mathieu menuet commented

2016-08-16 22:57:34 +02:00

Hi Lukas. Is omehow managed to create a file with 2 problems. This inf.0 problem is not the one I wanted to report here. There is really a problem with production files where manually changing all ä to ae (in our case some layer are named with such characters) is the only way to make the import successful.

Lukas Treyer commented

2016-08-17 00:01:16 +02:00

ok. but if I replace all inf.0 with inf and turn off all merging options (no clue yet what is wrong there) I can import your file in Blender. Did you attach the correct file?

mathieu menuet commented

2016-08-17 21:06:16 +02:00

You are right, wrong file. Here is a file with a "ä" in the layer name.
test umlaut.dxf

You are right, wrong file. Here is a file with a "ä" in the layer name. [test umlaut.dxf](https://archive.blender.org/developer/F338367/test_umlaut.dxf)

Lukas Treyer commented

2016-08-18 00:18:03 +02:00

Changed status from 'Open' to: 'Resolved'

Lukas Treyer closed this issue

2016-08-18 00:18:03 +02:00

Lukas Treyer commented

2016-08-18 00:18:03 +02:00

I tested it with the newest version. "ä" does not seem to be a problem (anymore?). But I needed to fix a problem with an extrusion value.

mathieu menuet commented

2016-08-18 11:00:29 +02:00

Using latest master (just compiled b10d005 ) I can't import any of the 2 files I attached in this thread. It seems Manfred solved the problem after you updated the lib https://bitbucket.org/mozman/dxfgrabber/issues/13/inf0-values-not-parsed-properly? I tested on Linux but I guess it's the same on all OSs. I used default options.

Error trying to import https://dev-files.blender.org/file/data/7ecvwzyzt3s2rie3tbpf/PHID-FILE-mvueofrgohmt7yxp25o3/test_umlaut.dxf

Traceback (most recent call last):
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/__init__.py", line 540, in execute
    proj_dxf, proj_scn, self.represent_thickness_and_width, self.import_atts, dxf_unit_scale)
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/__init__.py", line 117, in read
    projDXF, projSCN, thicknessWidth, but_group_by_att, dxf_unit_scale)
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfimport/do.py", line 104, in __init__
    self.dwg = dxfgrabber.readfile(dxf_filename, {"assure_3d_coords": True})
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 43, in readfile
    return readfile_as_asc(filename, options)
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 58, in readfile_as_asc
    return _read_encoded_file(filename, options, encoding=get_encoding())
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 62, in _read_encoded_file
    from .drawing import Drawing
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/drawing.py", line 9, in <module>
    from .sections import Sections
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/sections.py", line 12, in <module>
    from .entitysection import EntitySection, ObjectsSection
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/entitysection.py", line 12, in <module>
    from .dxfentities import entity_factory
ImportError: No module named 'io_import_dxf.dxfgrabber.dxfentities'

location: <unknown location>:-1

Error trying to import https://dev-files.blender.org/file/data/5op53o4ast4pucl5irrd/PHID-FILE-hspjsfaahoj3xbng32mw/test_nonascii_bl_2.dxf

Traceback (most recent call last):
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 43, in readfile
    return readfile_as_asc(filename, options)
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 58, in readfile_as_asc
    return _read_encoded_file(filename, options, encoding=get_encoding())
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 55, in get_encoding
    info = dxfinfo(fp)
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/tags.py", line 436, in dxfinfo
    tag = next(tagreader)
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/tags.py", line 134, in stream_tagger
    x = next_tag()
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/tags.py", line 120, in next_tag
    code = stream.readline()
  File "/home/test/source/blender/build_linux_full/bin/2.77/python/lib/python3.5/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 3284: invalid continuation byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/__init__.py", line 540, in execute
    proj_dxf, proj_scn, self.represent_thickness_and_width, self.import_atts, dxf_unit_scale)
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/__init__.py", line 117, in read
    projDXF, projSCN, thicknessWidth, but_group_by_att, dxf_unit_scale)
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfimport/do.py", line 104, in __init__
    self.dwg = dxfgrabber.readfile(dxf_filename, {"assure_3d_coords": True})
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 45, in readfile
    return readfile_as_utf8(filename, options, errors='ignore')
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 49, in readfile_as_utf8
    return _read_encoded_file(filename, options, encoding='utf-8', errors=errors)
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 62, in _read_encoded_file
    from .drawing import Drawing
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/drawing.py", line 9, in <module>
    from .sections import Sections
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/sections.py", line 12, in <module>
    from .entitysection import EntitySection, ObjectsSection
  File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/entitysection.py", line 12, in <module>
    from .dxfentities import entity_factory
ImportError: No module named 'io_import_dxf.dxfgrabber.dxfentities'

location: <unknown location>:-1

Using latest master (just compiled b10d005 ) I can't import any of the 2 files I attached in this thread. It seems Manfred solved the problem after you updated the lib https://bitbucket.org/mozman/dxfgrabber/issues/13/inf0-values-not-parsed-properly? I tested on Linux but I guess it's the same on all OSs. I used default options. Error trying to import https://dev-files.blender.org/file/data/7ecvwzyzt3s2rie3tbpf/PHID-FILE-mvueofrgohmt7yxp25o3/test_umlaut.dxf ``` Traceback (most recent call last): File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/__init__.py", line 540, in execute proj_dxf, proj_scn, self.represent_thickness_and_width, self.import_atts, dxf_unit_scale) File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/__init__.py", line 117, in read projDXF, projSCN, thicknessWidth, but_group_by_att, dxf_unit_scale) File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfimport/do.py", line 104, in __init__ self.dwg = dxfgrabber.readfile(dxf_filename, {"assure_3d_coords": True}) File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 43, in readfile return readfile_as_asc(filename, options) File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 58, in readfile_as_asc return _read_encoded_file(filename, options, encoding=get_encoding()) File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 62, in _read_encoded_file from .drawing import Drawing File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/drawing.py", line 9, in <module> from .sections import Sections File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/sections.py", line 12, in <module> from .entitysection import EntitySection, ObjectsSection File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/entitysection.py", line 12, in <module> from .dxfentities import entity_factory ImportError: No module named 'io_import_dxf.dxfgrabber.dxfentities' location: <unknown location>:-1 ``` Error trying to import https://dev-files.blender.org/file/data/5op53o4ast4pucl5irrd/PHID-FILE-hspjsfaahoj3xbng32mw/test_nonascii_bl_2.dxf ``` Traceback (most recent call last): File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 43, in readfile return readfile_as_asc(filename, options) File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 58, in readfile_as_asc return _read_encoded_file(filename, options, encoding=get_encoding()) File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 55, in get_encoding info = dxfinfo(fp) File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/tags.py", line 436, in dxfinfo tag = next(tagreader) File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/tags.py", line 134, in stream_tagger x = next_tag() File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/tags.py", line 120, in next_tag code = stream.readline() File "/home/test/source/blender/build_linux_full/bin/2.77/python/lib/python3.5/codecs.py", line 321, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 3284: invalid continuation byte During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/__init__.py", line 540, in execute proj_dxf, proj_scn, self.represent_thickness_and_width, self.import_atts, dxf_unit_scale) File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/__init__.py", line 117, in read projDXF, projSCN, thicknessWidth, but_group_by_att, dxf_unit_scale) File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfimport/do.py", line 104, in __init__ self.dwg = dxfgrabber.readfile(dxf_filename, {"assure_3d_coords": True}) File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 45, in readfile return readfile_as_utf8(filename, options, errors='ignore') File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 49, in readfile_as_utf8 return _read_encoded_file(filename, options, encoding='utf-8', errors=errors) File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/__init__.py", line 62, in _read_encoded_file from .drawing import Drawing File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/drawing.py", line 9, in <module> from .sections import Sections File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/sections.py", line 12, in <module> from .entitysection import EntitySection, ObjectsSection File "/home/test/source/blender/build_linux_full/bin/2.77/scripts/addons/io_import_dxf/dxfgrabber/entitysection.py", line 12, in <module> from .dxfentities import entity_factory ImportError: No module named 'io_import_dxf.dxfgrabber.dxfentities' location: <unknown location>:-1 ```

mathieu menuet commented

2016-08-18 11:02:00 +02:00

@cnd disabling all the import options doesn't solve the problem.

Lukas Treyer commented

2016-08-18 11:10:13 +02:00

As you can read from the error message ImportError: No module named 'io_import_dxf.dxfgrabber.dxfentities' your error does not have to anything with your files but with my second last erroneous commit. I Added the two missing files last night. So you might just need to update your build and try again.

And yes, the dxfgrabber lib now supports inf.0 but it's not standard and actually you should file a bug report at QGIS people for producing wrong dxf files. And the latest updates to dxfgrabber are also not in Blender yet.

As you can read from the error message `ImportError: No module named 'io_import_dxf.dxfgrabber.dxfentities'` your error does not have to anything with your files but with my second last erroneous commit. I Added the two missing files last night. So you might just need to update your build and try again. And yes, the dxfgrabber lib now supports inf.0 but it's not standard and actually you should file a bug report at QGIS people for producing wrong dxf files. And the latest updates to dxfgrabber are also not in Blender yet.

Lukas Treyer commented

2016-08-18 11:17:07 +02:00

and just to avoid an extra round of posts here:

it does not matter what build version of blender you build for addons. what matters is that you update the addons repository:

cd ~/blender-build/blender
git pull --rebase
git submodule foreach git pull --rebase origin master

as described in https://wiki.blender.org/index.php/Dev:Doc/Building_Blender/Mac (or windows or linux)

and just to avoid an extra round of posts here: it does not matter what build version of blender you build for addons. what matters is that you update the addons repository: `cd ~/blender-build/blender` `git pull --rebase` `git submodule foreach git pull --rebase origin master` as described in https://wiki.blender.org/index.php/Dev:Doc/Building_Blender/Mac (or windows or linux)

mathieu menuet commented

2016-08-18 11:17:22 +02:00

updating dxfgrabber to latest version still doesn't resolves the problem.

mathieu menuet commented

2016-08-18 11:19:35 +02:00

@cnd when you call "make update" in the source dir on linux, it also updates all addons. So I have the latest version including your changes normally. But I'll check.

mathieu menuet commented

2016-08-18 11:25:59 +02:00

@cnd Just checked, I have the latest version. In the commit log, you only did one commit in the last days. Maybe your second commit for missing files didn't work somehow?

mathieu menuet commented

2016-08-20 15:03:27 +02:00

@cnd just tried latest buildbot on Windows and Linux, it seems some files in the addon are still missing.

ImportError: No module named 'io_import_dxf.dxfgrabber.dxfentities'

@cnd just tried latest buildbot on Windows and Linux, it seems some files in the addon are still missing. ``` ImportError: No module named 'io_import_dxf.dxfgrabber.dxfentities' ```

mathieu menuet commented

2016-08-20 15:03:48 +02:00

Changed status from 'Resolved' to: 'Open'

mathieu menuet reopened this issue

2016-08-20 15:03:48 +02:00

Lukas Treyer commented

2016-08-20 15:13:22 +02:00

ok. forgot to push. pushed now.

mathieu menuet commented

2016-08-20 16:40:41 +02:00

@cnd thanks, It imports now files with "umlauts" but they are displayed strangely:
Renaming this groups to ä and Ä works so Blender supports it, it seems something wrong happens in the conversion process.

@cnd thanks, It imports now files with "umlauts" but they are displayed strangely: ![umlaut.jpg](https://archive.blender.org/developer/F339869/umlaut.jpg) Renaming this groups to ä and Ä works so Blender supports it, it seems something wrong happens in the conversion process.

Brendon Murphy commented

2019-06-11 11:06:35 +02:00

Added subscriber: @BrendonMurphy