Page MenuHome

Mesh import/export with high vertex count is very slow
Closed, ArchivedPublic

Description

System Information
Win 7 AMD HD 78xx

Blender Version
Broken: always

Short description of error
Exporting or importing meshes from Blender which have a high number of vertives (> 1 million) take ages to import or export.

Exact steps for others to reproduce the error

  • Create Suzanne
  • Use Subsurf until the polycount is in the millions ( 5 -10 million)
  • Export the mesh (obj or fbx - without UVs/normals will be faster, naturally)
  • Wait forever ;)

Importing the mesh will take an equally long time.

This is a giant blocker to sculpting workflow in Blender as it makes exporting to for example xNormal extremely time consuming.

Details

Type
Bug

Event Timeline

Peter Boos (PGTART) closed this task as Invalid.
Peter Boos (PGTART) claimed this task.

The art of sculpting isnt super small polygons... Its knowing where to put details and where to remove them.
So if your bad ad sculpting then that comes with a cost.
Its not Blender's fault that processing lots of faces take more time, its mostly the speed of your hardware, you could buy SSD's and a faster computer, or better learn to sculpt.

Like for example suzane use subdiv 2 and use smooth surfaces, a final render would look the same.

I don't understand your charge that I am bad at sculpting. For highly detailed sculpts resolutions of well over 10 million points are common.

That this is not Blender's fault is also wrong. Other software (take zBrush for example) handles large mesh exports in reasonable timeframes of under 30 seconds, not up to 15 minutes.

And of course I'm aware that making a 10 mil vert suzanne is pointless. It was just an example.

Sculpting is also not the only area that the long load times pose an issue. It happens with photogrammetry generated meshes as well.

Please reopen the task, this should be adressed.

If this is your regular job, then i stronlgy recommend you a SSD, this is not a bug in code; the waiting time you have is simply caused by the IO speed of your harddisk.
That suszane of 667MB took me about 2 to 3 secconds to load on SSD disk. So we realy cannot blame Blender here, but if it takes you 15 minutes upgrade your hardware, because your Disk IO speed is the bottle neck here.

I have a SSD and when I export the model to the SSD as .obj it takes several minutes. So that can not be the problem.

I don't want to overreach but I would like to point out that the file size of the blend file is 667 kilobytes, not 667 megabyte (if by "that Suzanne of 667MB" you mean my example file). So it would load very quickly, the reason being that the geometry is created at runtime and not saved in the file since it is generated by a subsurf modifier.

Germano Cavalcante (mano-wii) changed the task status from Invalid to Archived.

The code for exporting obj is written in python and python is usually slow.
By analyzing the export_obj.py code, there are some parts that could be improved using the numpy package.

One of the reasons for the slow code is the line with:

face_index_pairs.sort (key = sort_func)

That is called when the Keep Vertex Order option is checked.

Other reasons for slowness are the loops:

  • for v in me_verts:;
  • for f, f_index in face_index_pairs: (2x)
  • and possibly others, but I got tired of waiting;

Actually this slowness is quite inconvenient, but the code is not broken so the priority is not high.

Currently the developers are focusing their time on the Blender 2.8 Project. So I have no expectations that this problem will be corrected so soon.

In any case, it is good that it is taken into account.

@Peter Boos (PGTART) as much fun as yelling at users is that they can't model and their hardware sucks, I decided to sit down and do some tests.

Decided to give it a whirl, exporting to an SSD drive.

testing procedure:

  1. Export from blender .obj which is a common format that virtually all applications know how to read
  1. Import into meshlab
  1. Import into blender
  1. Export from meshlab

With the file from this ticket as is (subdiv level 7):

exporting .obj from blender     : 5m05s (950mb file)
Meshlab export to .obj          : crashed couldn't test. 

Meshlab import of the .obj file : 1m24s 
Blender Import of the .obj file : had to kill it after 5 minutes after it consumed 18GB of ram

changed subdiv level to 5 and payed attention to ram usage:

exporting .obj from blender     : 18s (76mb file) 
Meshlab export to .obj          : 4s 

Meshlab import of the .obj file : 3s / 375mb ram
Blender Import of the .obj file : 18s / 1.3gb ram

changed subdiv level to 6 and payed attention to ram usage:

exporting .obj from blender     : 78s (243mb file) 
Meshlab export to .obj          : 22s  

Meshlab import of the .obj file : 22s / 2.0gb ram
Blender Import of the .obj file : 75s / 4.3gb ram

So in all instances blender was *MUCH* slower, and used *MUCH* *MUCH* more ram compared to other applications while performing the same tasks.

@Peter Boos (PGTART) i'm not sure if you just had a bad day or something else, but generally we try not to be this abusive to users, please try to be a tad more civil in the future?

@LazyDodo (LazyDodo) @Germano Cavalcante (mano-wii) Thank you for taking the issue seriously, the initial response was infuriating. It's sad that the issue won't be resolved soon but thank you for the hint with the vertex order, I'll keep that in mind :)

If you can, try exporting to Collada (dae) using the Collada (Default) exporter.
On a X3 455 (an old AMD 3 core):

  • it takes around 10-15 seconds for a subdivision 6 Suzanne (~ 2M polygons).
  • 57 seconds for a subdiv 7 Suzanne (~ 8M polygons) ( around 4 GB used in total as the subdivision was applied with the base of 2GB just for the model). The DAE file is 986 MB in size.

@Vuk Gardašević (lijenstina) Thank you for the tip! My target platform (xnormal) even supports dae.

Peter Boos (PGTART) reopened this task as Open.Nov 2 2017, 10:14 PM
This comment was removed by Peter Boos (PGTART).

i am civil,
Let me give you some more numbers
Apply modifiers then save the file to a Blend file thats 667Mb of raw (but also compressed) blender data,
Without blend saving only applying the modifier, that raw data inside blender its about 1.7 Gig what you see on screen.
that's to be converted to obj since the formats are not the same it takes around 7Gig of data to be processed to become a obj file of more then 900Mb.
well it depends on the horsepower of your machine, over here its about 60 seconds.
And i'm kinda amazed that such a huge file is still processed that fast, for the amounts of data that it has to process.
Maybe add more Ram, or a faster processor (more cores) might help as well. (60 sec tested on Asus K95v a laptop 4 years old now).

So to say blender has a bug its slow, with these data numbers to me seams a rather bold statement.

Though its also subjective "there is no standard for fast" here, upgrading your hardware for sure would help.
You claim 15 minutes what i do in a minute, well i reopen because i only wonder what a python dev might think of this bold claim.
If you reconsider these facts and my earlier advice you might as well close it yourself.

Maybe as i'm not english my language was a bit to formal, i would advice people who start to use blender to go to blender artist.
People there are helping people out with problems like these, this however is a bug report area, not wishing area.
Essentially its working, and it depends on your hardware how fast it works.
And it depends on your workflow how much data you generate.

@Peter Boos (PGTART), There is no point adding more juice to the CPU when python export/import works on single core. Converting import/export scripts to multithreading would help a lot. That's why this is really valid ticket.

Let's start using! http://assimp.sourceforge.net

LazyDodo (LazyDodo) closed this task as Archived.Nov 2 2017, 10:33 PM

it's not a valid ticket, the import/export is working as designed, so technically this would be a feature request which doesn't belong on the tracker (so sadly i'm gonna have to abide by tracker rules and close the ticket).

That being said, there's nothing with how formal you were, but you straight up told an end user he didn't know how to do his job, no matter how you phrase that, that's not behavior that is appropriate on the tracker.

ah i see now, at first i responded quickly, i wasnt judging him, but the line was a general statement.
i should have written that poor sculpts, result in poor meshes, i was trying to explain that in general there (like a saying), but not to judge his specific level.

My excuses to him as, i had no intend to blame him there, bad tutor choice of words from my side.