Page MenuHome

Netrender: errno 32: Broken pipe
Closed, ResolvedPublic

Description

System Information
Fedora 28

Blender Version
2.79b, Python version 3.6.6

Short description of error

I am setting up the network renderer, but when I send tasks to the master, I get "[Errno 32] Broken pipe".

On the master, I see:

Fra:1 Mem:16.03M (0.00M, Peak 16.03M) | Time:00:52.00 | Version check                                                                                                                                              
Fra:1 Mem:16.03M (0.00M, Peak 16.03M) | Time:00:52.01 | New job, missing files (1 total)                                                                                                                           
Fra:1 Mem:16.03M (0.00M, Peak 16.03M) | Time:00:52.01 | Receiving job file  
----------------------------------------
Exception happened during processing of request from ('172.18.0.1', 40556)
Traceback (most recent call last):
  File "/usr/lib64/python3.6/socketserver.py", line 651, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib64/python3.6/socketserver.py", line 361, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib64/python3.6/socketserver.py", line 721, in __init__
    self.handle()
  File "/usr/lib64/python3.6/http/server.py", line 418, in handle
    self.handle_one_request()
  File "/usr/lib64/python3.6/http/server.py", line 406, in handle_one_request
    method()
  File "/usr/share/blender/scripts/addons/netrender/master.py", line 794, in do_PUT
    self.write_file(file_path)
  File "/usr/share/blender/scripts/addons/netrender/master.py", line 236, in write_file
    length = int(self.headers['content-length'])
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

Exact steps for others to reproduce the error
Set up a master and a slave, send a task from client to master.

My investigation

The problem seems to happen when the client tries to send the job files. This is done in line 338 of client.py, using the request() method of HTTPConnection. The argument is a file handle, no headers are specified. The documentation of this method says:

If headers contains neither Content-Length nor Transfer-Encoding, but there is a request body, one of those header fields will be added automatically. If body is None, the Content-Length header is set to 0 for methods that expect a body (PUT, POST, and PATCH). If body is a string or a bytes-like object that is not also a file, the Content-Length header is set to its length. Any other type of body (files and iterables in general) will be chunk-encoded, and the Transfer-Encoding header will automatically be set instead of Content-Length.

(see https://docs.python.org/3/library/http.client.html)

So, no content-length when using a file object. It is visible from the stack trace (length = int(self.headers['content-length'])) that the master expects the content-length to be set. This leads to the exception.

I have not yet tried whether explicitly setting the content length header solves the problem.

Related Objects

Event Timeline

A K (campino) added a comment.EditedSep 27 2018, 2:22 PM

Explicitly setting the content-length indeed solves the problem.

I'll check how to get the fix upstream later. However, the buggy code is copied to several locations within the Netrender code, so I need to check for duplicates and fix all of them. Also, I believe that several people use netrender without being affected, so I am afraid that the fix breaks things for them. Is there any advice on how to verify that it does not?

Also, I hit https://developer.blender.org/T54222 as soon as I fixed the first problem. I think I would prefer to fix this as well first.

I've submitted code: https://developer.blender.org/D3741

However, this lacks a reviewer and possibly some tags. I would like it if someone could fill in the missing information.