Page MenuHome

Saving blend-files straight to server nets 0.5% speed of a windows file copy to the same location
Closed, InvalidPublic

Description

System Information
Operating system: Windows 10 (client), Ubuntu 18.04 (client), Windows Server 2019 (server)
Graphics card: GTX 1080 TI, Intel integrated Graphics 4000

Blender Version
Broken: version: 2.80 (sub 75), branch: master, commit date: 2019-07-29 14:47, hash: rBf6cb5f54494e

Short description of error
This is a crosspost of a question I asked at Blender Stackexchange. More specific information and context can be found there.

In short, copying a 2.2GB blend file to a windows share through explorer takes 23s (765Mbps) on our Gigabit network.
Saving the same file to the same share location directly from blender takes +5000s (<3.5Mbps).
The process of saving in itself when writing to the internal SSD takes 4 seconds.

I tried this to multiple shares (albeit on the same server) and from different (wired) clients.

Technically it works, it does save the file, it's just not usable as it is and prevents us from having a better workflow.

Exact steps for others to reproduce the error

  • Take any blendfile with a considerable size
  • Try to save to a Windows share
  • Wait for the process to complete

Event Timeline

Some comments on this: I don't think it is 2.8 related, 2.79 and previous were equally slow.

I noticed in the past that saving directly to a USB connected thumb drive can also be very slow. My guess is this has something to do with the way Blender saves files, not necessarily to network or 2.8 specifically.

Indeed, I just tested in 2.8 to confirm the issue is still there.

Bastien Montagne (mont29) changed the task status from Unknown Status to Invalid.Aug 29 2019, 11:27 AM
Bastien Montagne (mont29) claimed this task.

that is almost certainly because copying the file from windows file browser will result in a single write operation of 2.2GB. Blender writes small chunks of memory at a time (among other reasons, to avoid having to store GB's of binary file in RAM...), so on such a big file it most likely will result in thousands of write commands. there are most certainly ways to setup your remote shared drive in a way that those get packed together, but that is really not a Blender bug or issue. This is sysadmin work to find a proper solution here, if you want direct random read/write on a remote storage.

Wouter Vandenneucker (woutervddn) changed the task status from Invalid to Unknown Status.Aug 29 2019, 1:34 PM

I wouldn't say that this is for sysadmin to work out; the 2.2GB file was to even out network delays. Writing any blender project bigger than that standard cube to the network is problematic at this point.

If 2.8 is supposed to be used in production by real studios, direct writing to external filesystems is a real use case in a normal workflow.
Take any other professional software suite, be it an Adobe program, an Autodesk program, Solidworks,... they all handle writing to the network as it's supposed to. Even for +GB filesizes.
It's not a job of a sysadmin to find a way around the programs poor handling of write ops; at least not if the aim is to be a professional software suite.

Even with linking and a proper setup, writing a file of 25MB takes an excessive amount of time to be practical for any professional work.
I can solve this easily by having an rsync tool in the background that uploads everything to right folder, that is besides the point.

Blender should either handle writes to external sources properly or prohibit writing to it as a whole. Having the UI locked for +10 minutes is a definite no-no whichever way you look at it!

Bastien Montagne (mont29) changed the task status from Unknown Status to Invalid.Aug 29 2019, 2:25 PM

We do not have control over where you write a file> Improvements are always possible, but that is a feature request then, not a bug, and it has nothing to do on this tracker... Also please do not reopen tasks, triaging is to be done by developers, not users.