X3D import is broken #61052

New Issue

kursad k · 2019-01-31T05:40:00+01:00

kursad k commented

2019-01-31 05:40:00 +01:00

System Information
Operating system: Win10 x64
Graphics card: Nvidia 1070

Blender Version
Broken:
version: 2.79 (sub 6), branch: blender2.8, commit date: 2018-12-14 16:39, hash: 4e3562f45a, type: Release
build date: 14/12/2018, 17:17

Short description of error
Exporting Suzanne (or any other object) and importing Suzanne.x3d back results in broken import process. I tried this many times. I cant tell if this is an issue with the export or the import. The file format looks fine.

Exact steps for others to reproduce the error

Add Suzanne or any other object
Open export menu (file) and choose x3d
Choose selected (defaults produce the same result)
Give and name and export
Import the exported .x3d from the "import" menu.

And you should get something like this


location: <unknown location>:-1
Info: starting X3D export to 'T:\\tmo\\_tmp_local\\untitled.x3d'...
Info: finished X3D export to 'T:\\tmo\\_tmp_local\\untitled.x3d'
Traceback (most recent call last):
  File "C:\dev\blender-nightly\2.79\python\lib\urllib\request.py", line 1317, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "C:\dev\blender-nightly\2.79\python\lib\http\client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\dev\blender-nightly\2.79\python\lib\http\client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\dev\blender-nightly\2.79\python\lib\http\client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\dev\blender-nightly\2.79\python\lib\http\client.py", line 1016, in _send_output
    self.send(msg)
  File "C:\dev\blender-nightly\2.79\python\lib\http\client.py", line 956, in send
    self.connect()
  File "C:\dev\blender-nightly\2.79\python\lib\http\client.py", line 928, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "C:\dev\blender-nightly\2.79\python\lib\socket.py", line 727, in create_connection
    raise err
  File "C:\dev\blender-nightly\2.79\python\lib\socket.py", line 716, in create_connection
    sock.connect(sa)
OSError: [WinError 10013] An attempt was made to access a socket in a way forbidden by its access permissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\dev\blender-nightly\2.79\scripts\addons\io_scene_x3d\__init__.py", line 81, in execute
    return import_x3d.load(context, **keywords)
  File "C:\dev\blender-nightly\2.79\scripts\addons\io_scene_x3d\import_x3d.py", line 3601, in load
    global_matrix=global_matrix,
  File "C:\dev\blender-nightly\2.79\scripts\addons\io_scene_x3d\import_x3d.py", line 3468, in load_web3d
    root_node, msg = x3d_parse(filepath)
  File "C:\dev\blender-nightly\2.79\scripts\addons\io_scene_x3d\import_x3d.py", line 1443, in x3d_parse
    doc = xml.dom.minidom.parseString(data, parser)
  File "C:\dev\blender-nightly\2.79\python\lib\xml\dom\minidom.py", line 1972, in parseString
    {'parser': parser})
  File "C:\dev\blender-nightly\2.79\python\lib\xml\dom\minidom.py", line 1949, in _do_pulldom_parse
    toktype, rootNode = events.getEvent()
  File "C:\dev\blender-nightly\2.79\python\lib\xml\dom\pulldom.py", line 259, in getEvent
    self.parser.feed(buf)
  File "C:\dev\blender-nightly\2.79\python\lib\xml\sax\expatreader.py", line 217, in feed
    self._parser.Parse(data, isFinal)
  File "..\Modules\pyexpat.c", line 666, in ExternalEntityRef
  File "C:\dev\blender-nightly\2.79\python\lib\xml\sax\expatreader.py", line 413, in external_entity_ref
    "")
  File "C:\dev\blender-nightly\2.79\python\lib\xml\sax\saxutils.py", line 364, in prepare_input_source
    f = urllib.request.urlopen(source.getSystemId())
  File "C:\dev\blender-nightly\2.79\python\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\dev\blender-nightly\2.79\python\lib\urllib\request.py", line 525, in open
    response = self._open(req, data)
  File "C:\dev\blender-nightly\2.79\python\lib\urllib\request.py", line 543, in _open
    '_open', req)
  File "C:\dev\blender-nightly\2.79\python\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "C:\dev\blender-nightly\2.79\python\lib\urllib\request.py", line 1345, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\dev\blender-nightly\2.79\python\lib\urllib\request.py", line 1319, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10013] An attempt was made to access a socket in a way forbidden by its access permissions>

**System Information** Operating system: Win10 x64 Graphics card: Nvidia 1070 **Blender Version** Broken: version: 2.79 (sub 6), branch: blender2.8, commit date: 2018-12-14 16:39, hash: 4e3562f45ad, type: Release build date: 14/12/2018, 17:17 **Short description of error** Exporting Suzanne (or any other object) and importing Suzanne.x3d back results in broken import process. I tried this many times. I cant tell if this is an issue with the export or the import. The file format looks fine. **Exact steps for others to reproduce the error** - Add Suzanne or any other object - Open export menu (file) and choose x3d - Choose selected (defaults produce the same result) - Give and name and export - Import the exported .x3d from the "import" menu. And you should get something like this ``` location: <unknown location>:-1 Info: starting X3D export to 'T:\\tmo\\_tmp_local\\untitled.x3d'... Info: finished X3D export to 'T:\\tmo\\_tmp_local\\untitled.x3d' Traceback (most recent call last): File "C:\dev\blender-nightly\2.79\python\lib\urllib\request.py", line 1317, in do_open encode_chunked=req.has_header('Transfer-encoding')) File "C:\dev\blender-nightly\2.79\python\lib\http\client.py", line 1229, in request self._send_request(method, url, body, headers, encode_chunked) File "C:\dev\blender-nightly\2.79\python\lib\http\client.py", line 1275, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "C:\dev\blender-nightly\2.79\python\lib\http\client.py", line 1224, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "C:\dev\blender-nightly\2.79\python\lib\http\client.py", line 1016, in _send_output self.send(msg) File "C:\dev\blender-nightly\2.79\python\lib\http\client.py", line 956, in send self.connect() File "C:\dev\blender-nightly\2.79\python\lib\http\client.py", line 928, in connect (self.host,self.port), self.timeout, self.source_address) File "C:\dev\blender-nightly\2.79\python\lib\socket.py", line 727, in create_connection raise err File "C:\dev\blender-nightly\2.79\python\lib\socket.py", line 716, in create_connection sock.connect(sa) OSError: [WinError 10013] An attempt was made to access a socket in a way forbidden by its access permissions During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\dev\blender-nightly\2.79\scripts\addons\io_scene_x3d\__init__.py", line 81, in execute return import_x3d.load(context, **keywords) File "C:\dev\blender-nightly\2.79\scripts\addons\io_scene_x3d\import_x3d.py", line 3601, in load global_matrix=global_matrix, File "C:\dev\blender-nightly\2.79\scripts\addons\io_scene_x3d\import_x3d.py", line 3468, in load_web3d root_node, msg = x3d_parse(filepath) File "C:\dev\blender-nightly\2.79\scripts\addons\io_scene_x3d\import_x3d.py", line 1443, in x3d_parse doc = xml.dom.minidom.parseString(data, parser) File "C:\dev\blender-nightly\2.79\python\lib\xml\dom\minidom.py", line 1972, in parseString {'parser': parser}) File "C:\dev\blender-nightly\2.79\python\lib\xml\dom\minidom.py", line 1949, in _do_pulldom_parse toktype, rootNode = events.getEvent() File "C:\dev\blender-nightly\2.79\python\lib\xml\dom\pulldom.py", line 259, in getEvent self.parser.feed(buf) File "C:\dev\blender-nightly\2.79\python\lib\xml\sax\expatreader.py", line 217, in feed self._parser.Parse(data, isFinal) File "..\Modules\pyexpat.c", line 666, in ExternalEntityRef File "C:\dev\blender-nightly\2.79\python\lib\xml\sax\expatreader.py", line 413, in external_entity_ref "") File "C:\dev\blender-nightly\2.79\python\lib\xml\sax\saxutils.py", line 364, in prepare_input_source f = urllib.request.urlopen(source.getSystemId()) File "C:\dev\blender-nightly\2.79\python\lib\urllib\request.py", line 222, in urlopen return opener.open(url, data, timeout) File "C:\dev\blender-nightly\2.79\python\lib\urllib\request.py", line 525, in open response = self._open(req, data) File "C:\dev\blender-nightly\2.79\python\lib\urllib\request.py", line 543, in _open '_open', req) File "C:\dev\blender-nightly\2.79\python\lib\urllib\request.py", line 503, in _call_chain result = func(*args) File "C:\dev\blender-nightly\2.79\python\lib\urllib\request.py", line 1345, in http_open return self.do_open(http.client.HTTPConnection, req) File "C:\dev\blender-nightly\2.79\python\lib\urllib\request.py", line 1319, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [WinError 10013] An attempt was made to access a socket in a way forbidden by its access permissions> ```

kursad k commented

2019-01-31 05:40:00 +01:00

Added subscriber: @kursadk

kursad k changed title from ~~X3D is broken~~ to X3D import is broken

2019-01-31 05:41:24 +01:00

Philipp Oeser commented

2019-01-31 10:24:05 +01:00

Added subscriber: @lichtwerk

Philipp Oeser commented

2019-01-31 10:24:05 +01:00

Cannot reproduce here.

Havent looked at the code, but as it seems to have to do with permissions [or another process using that port already]

Could you try running blender with "Run as administrator"?
Try (temporarily) disabling antivirus?
any other known process blocking a port possibly?

Marking as incomplete until we have those tried...

Cannot reproduce here. Havent looked at the code, but as it seems to have to do with permissions [or another process using that port already] - Could you try running blender with "Run as administrator"? - Try (temporarily) disabling antivirus? - any other known process blocking a port possibly? Marking as incomplete until we have those tried...

Brecht Van Lommel commented

2019-01-31 13:13:46 +01:00

Added subscriber: @brecht

Brecht Van Lommel commented

2019-01-31 13:13:46 +01:00

It can be reproduced by disabling your network connection. Any code that does network access needs to handle exceptions.

Maybe we don't really need network access to import an X3D file? Not sure if the DTD's are needed for parsing or just some validation that we don't care about.

It can be reproduced by disabling your network connection. Any code that does network access needs to handle exceptions. Maybe we don't really need network access to import an X3D file? Not sure if the DTD's are needed for parsing or just some validation that we don't care about.

Philipp Oeser commented

2019-01-31 14:31:43 +01:00

Seems like this is only done for reporting the linenumbers... [without this x3d import is heaps quicker btw...]
Assume this is taken from here https://stackoverflow.com/questions/4789171/python-reporting-line-column-of-origin-of-xml-node

Not sure if this parser = xml.sax.make_parser() can be tweaked to not require a network connection though...

we could encapsulate with try/except urllib.error.URLError?
(and in that case just parse without this special parser?)
(that would require some tweaks in the code to not hadle/report linenumbers then...)

Seems like this is only done for reporting the linenumbers... [without this x3d import is heaps quicker btw...] Assume this is taken from here https://stackoverflow.com/questions/4789171/python-reporting-line-column-of-origin-of-xml-node Not sure if this `parser = xml.sax.make_parser()` can be tweaked to not require a network connection though... we could encapsulate with try/except urllib.error.URLError? (and in that case just parse without this special parser?) (that would require some tweaks in the code to not hadle/report linenumbers then...)

Philipp Oeser commented

2019-01-31 15:11:36 +01:00

This seems to do the trick:
P900: #61052 snippet



diff --git a/io_scene_x3d/import_x3d.py b/io_scene_x3d/import_x3d.py
index 903222cf..efcbb64e 100644
--- a/io_scene_x3d/import_x3d.py
+++ b/io_scene_x3d/import_x3d.py
@@ -1413,6 +1413,7 @@ def x3d_parse(path):
     """
     import xml.dom.minidom
     import xml.sax
+    from xml.sax import handler
 
     '''
     try:    doc = xml.dom.minidom.parse(path)
@@ -1438,6 +1439,8 @@ def x3d_parse(path):
 
     parser = xml.sax.make_parser()
     orig_set_content_handler = parser.setContentHandler
+    parser.setFeature(handler.feature_external_ges, False)
+    parser.setFeature(handler.feature_external_pes, False)
     parser.setContentHandler = set_content_handler
 
     doc = xml.dom.minidom.parseString(data, parser)

This seems to do the trick: [P900: #61052 snippet](https://archive.blender.org/developer/P900.txt) ``` diff --git a/io_scene_x3d/import_x3d.py b/io_scene_x3d/import_x3d.py index 903222cf..efcbb64e 100644 --- a/io_scene_x3d/import_x3d.py +++ b/io_scene_x3d/import_x3d.py @@ -1413,6 +1413,7 @@ def x3d_parse(path): """ import xml.dom.minidom import xml.sax + from xml.sax import handler ''' try: doc = xml.dom.minidom.parse(path) @@ -1438,6 +1439,8 @@ def x3d_parse(path): parser = xml.sax.make_parser() orig_set_content_handler = parser.setContentHandler + parser.setFeature(handler.feature_external_ges, False) + parser.setFeature(handler.feature_external_pes, False) parser.setContentHandler = set_content_handler doc = xml.dom.minidom.parseString(data, parser) ```

Brecht Van Lommel commented

2019-01-31 15:18:06 +01:00

Looks good to me.

kursad k commented

2019-01-31 17:15:44 +01:00

In #61052#610922, @brecht wrote:
It can be reproduced by disabling your network connection. Any code that does network access needs to handle exceptions.

Really good catch! My desktop indeed does not have any internet connection, well rather has very restricted firewall, so Blender or any other spawn processes can`t make non Lan connections

I did not think about mentioning it in the original report.

> In #61052#610922, @brecht wrote: > It can be reproduced by disabling your network connection. Any code that does network access needs to handle exceptions. > Really good catch! My desktop indeed does not have any internet connection, well rather has very restricted firewall, so Blender or any other spawn processes can`t make non Lan connections I did not think about mentioning it in the original report.

blender-admin commented

2019-02-01 09:21:22 +01:00

This issue was referenced by blender/blender-addons@d410029833

This issue was referenced by blender/blender-addons@d410029833365916b2d3a1fee25d0e02c26c648a

Philipp Oeser commented

2019-02-01 10:05:21 +01:00

Changed status from 'Open' to: 'Resolved'

Philipp Oeser closed this issue

2019-02-01 10:05:21 +01:00

John W Carlson commented

2019-02-01 10:53:12 +01:00

Added subscriber: @yottzumm