Bloated mesh data example (multires CD_MDISPS hanging around without a multires modifier) #98249

New Issue

Malcolm Reed · 2022-05-19T10:41:25+02:00

Malcolm Reed commented

2022-05-19 10:41:25 +02:00

System Information
Operating system: Windows-10-10.0.19043-SP0 64 Bits
Graphics card: NVIDIA GeForce RTX 2070 SUPER/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 471.68

Blender Version
Broken: version: 3.2.0 Beta, branch: master, commit date: 2022-05-16 16:16, hash: a2dacefb46
Worked: Unknown

Short description of error
Mesh data seems bloated when saving a blend file. I have an example file attached with one mesh object with 400 faces that is around 6mb when saved to disk. This is a super simple example.

I noticed inconsistencies when saving out lots of different character mesh combinations with compression, some files on disk were ~700kb and some as high as ~48,000kb for essentially the same face counts.

I did some tests (with the attached scenes), by exporting the mesh as an OBJ and bringing it back in again and saving as a new file. This saves around 5mb.

I can't see the difference between the two the meshes, so I can't account for the 5mb of bloat. Is this a Bug? Is there a work around?
For now I think I'll have to write a tool to 'clean' the meshes by exporting and re-importing them.

Exact steps for others to reproduce the error
See the attached files; bloated mesh.blend, mesh.obj, imported obj mesh.blend.

Not much to reproduce, I guess it is a case of why is the 'bloated mesh.blend' file so large.

Much appreciated, Mal
example_files.zip

**System Information** Operating system: Windows-10-10.0.19043-SP0 64 Bits Graphics card: NVIDIA GeForce RTX 2070 SUPER/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 471.68 **Blender Version** Broken: version: 3.2.0 Beta, branch: master, commit date: 2022-05-16 16:16, hash: `a2dacefb46` Worked: Unknown **Short description of error** Mesh data seems bloated when saving a blend file. I have an example file attached with one mesh object with 400 faces that is around 6mb when saved to disk. This is a super simple example. I noticed inconsistencies when saving out lots of different character mesh combinations with compression, some files on disk were ~700kb and some as high as ~48,000kb for essentially the same face counts. I did some tests (with the attached scenes), by exporting the mesh as an OBJ and bringing it back in again and saving as a new file. This saves around 5mb. I can't see the difference between the two the meshes, so I can't account for the 5mb of bloat. Is this a Bug? Is there a work around? For now I think I'll have to write a tool to 'clean' the meshes by exporting and re-importing them. **Exact steps for others to reproduce the error** See the attached files; bloated mesh.blend, mesh.obj, imported obj mesh.blend. Not much to reproduce, I guess it is a case of why is the 'bloated mesh.blend' file so large. Much appreciated, Mal [example_files.zip](https://archive.blender.org/developer/F13088677/example_files.zip)

Malcolm Reed commented

2022-05-19 10:41:26 +02:00

Added subscriber: @reed2000

Philipp Oeser commented

2022-05-19 10:59:25 +02:00

Added subscriber: @lichtwerk

Philipp Oeser commented

2022-05-19 10:59:25 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Philipp Oeser commented

2022-05-19 10:59:25 +02:00

Hm, can confirm, will check

Philipp Oeser commented

2022-05-19 12:50:55 +02:00

Hm, there is an unnamed customdata layer on the loops (seems to be multires data).

If I exclude this from file writing, the filesize goes down again:
P2962: T98249_snippet



diff --git a/source/blender/blenkernel/intern/customdata.cc b/source/blender/blenkernel/intern/customdata.cc
index 623c3bb698c..47583ee5e4e 100644
--- a/source/blender/blenkernel/intern/customdata.cc
+++ b/source/blender/blenkernel/intern/customdata.cc
@@ -4361,6 +4361,10 @@ void CustomData_blend_write_prepare(CustomData *data,
       data->totlayer--;
       // CLOG_WARN(&LOG, "skipping layer %p (%s)", layer, layer->name);
     }
+    else if (layer->name- [x] == '\0'){
+      printf("poop\n");
+      data->totlayer--;
+    }
     else {
       if (UNLIKELY((size_t)j >= write_layers_size)) {
         if (write_layers == write_layers_buff) {

Not sure how that ended up there (dont think this was created from python? -- those customdata layers usually have a name enforced...)
@reed2000: any idea how this happened? got a way to reproduce from scratch? Is this an old file?

EDIT: there is a way to get rid of the multires data again, just add a new multires modifier and remove it again

Hm, there is an unnamed customdata layer on the loops (seems to be multires data). If I exclude this from file writing, the filesize goes down again: [P2962: T98249_snippet](https://archive.blender.org/developer/P2962.txt) ``` diff --git a/source/blender/blenkernel/intern/customdata.cc b/source/blender/blenkernel/intern/customdata.cc index 623c3bb698c..47583ee5e4e 100644 --- a/source/blender/blenkernel/intern/customdata.cc +++ b/source/blender/blenkernel/intern/customdata.cc @@ -4361,6 +4361,10 @@ void CustomData_blend_write_prepare(CustomData *data, data->totlayer--; // CLOG_WARN(&LOG, "skipping layer %p (%s)", layer, layer->name); } + else if (layer->name- [x] == '\0'){ + printf("poop\n"); + data->totlayer--; + } else { if (UNLIKELY((size_t)j >= write_layers_size)) { if (write_layers == write_layers_buff) { ``` Not sure how that ended up there (dont think this was created from python? -- those customdata layers usually have a name enforced...) @reed2000: any idea how this happened? got a way to reproduce from scratch? Is this an old file? EDIT: there is a way to get rid of the multires data again, just add a new multires modifier and remove it again

Philipp Oeser commented

2022-05-19 12:51:49 +02:00

Changed status from 'Confirmed' to: 'Needs User Info'

Philipp Oeser commented

2022-05-19 12:51:49 +02:00

So we will need to find a way to reproduce this borked multires data hanging around

Philipp Oeser changed title from ~~Bloated mesh data example~~ to Bloated mesh data example (multires CD_MDISPS hanging around without a multires modifier)

2022-05-19 12:52:34 +02:00

Philipp Oeser commented

2022-05-19 13:02:47 +02:00

Ah, we had this before...

In #88144#1225502, @lichtwerk wrote:
Found a way to reproduce this:

add Multires to the Default Cube object, subdivide twice

add a Monkey

change the mesh of the Monkey to the Cube mesh under Object Data Properties

(you can delete the default cube object now if you like)

add Multires modifier to the Monkey object
--> it already has 2 multires levels (without subdividing even once), if you now subdivide, it will jump right to level 3

note: this behavior does not occur if one removes the Multires modifier from the Default Cube object prior to deletion in step 4

So opposed to any other modifier (afaict), Multires seems to store info in mesh that is kept around.
It does get removed from the mesh when the modifier gets removed, but it does not get removed from the (possibly shared) mesh when the object gets deleted.

Upon further inspection, these are stored in a CD_MDISPS custom data layer on the mesh.
Now when ob->data is changed (step - [x]), or when a new modifer is added, the levels of the modifer will be made matching (see BKE_mesh_assign_object & BKE_modifiers_test_object & multiresModifier_set_levels_from_disps)
This can also be observed when for example a second multires modifier is added on top of an existing mutires modifier (levels will also be made matching).

Now I assume that in the end this has its reasons (and it might very well turn out to not be a bug), but will leave up to module devs to decide.
(I think it would be reasonable to remove CD_MDISPS initially when adding a new multires modifier?)

So, it is a duplicate of #88144 (Multires modifier data (CD_MDISPS) can remain on a (once shared) mesh, bloats filesize, adding a new multires modifer will already have its levels then), will merge reports...

Ah, we had this before... > In #88144#1225502, @lichtwerk wrote: > Found a way to reproduce this: > > - - [x] add Multires to the Default Cube object, subdivide twice > - - [x] add a Monkey > - - [x] change the mesh of the Monkey to the Cube mesh under `Object Data Properties` > - - [x] (you can delete the default cube object now if you like) > - - [x] add Multires modifier to the Monkey object > --> it already has 2 multires levels (without subdividing even once), if you now subdivide, it will jump right to level 3 > > note: this behavior does not occur if one removes the Multires modifier from the Default Cube object prior to deletion in step 4 > > So opposed to any other modifier (afaict), Multires seems to store info in mesh that is kept around. > It does get removed from the mesh when the modifier gets removed, but it does not get removed from the (possibly shared) mesh when the object gets deleted. > > Upon further inspection, these are stored in a `CD_MDISPS` custom data layer on the mesh. > Now when `ob->data` is changed (step - [x]), or when a new modifer is added, the levels of the modifer will be made matching (see `BKE_mesh_assign_object` & `BKE_modifiers_test_object` & `multiresModifier_set_levels_from_disps`) > This can also be observed when for example a second multires modifier is added on top of an existing mutires modifier (levels will also be made matching). > > Now I assume that in the end this has its reasons (and it might very well turn out to not be a bug), but will leave up to module devs to decide. > (I think it would be reasonable to remove CD_MDISPS initially when adding a **new** multires modifier?) So, it is a duplicate of #88144 (Multires modifier data (CD_MDISPS) can remain on a (once shared) mesh, bloats filesize, adding a new multires modifer will already have its levels then), will merge reports...

Philipp Oeser commented

2022-05-19 13:03:02 +02:00

Closed as duplicate of #88144

Philipp Oeser closed this issue

2022-05-19 13:03:02 +02:00

Malcolm Reed commented

2022-05-19 15:03:57 +02:00

Hi Guys,
Multires is part of my workflow, I have a mesh I setup and UV'd, then add a multires and sculpt. Sometimes the multires is applied at a high level to capture the sculpt data then a new one added and unsubdivided. I'm fairly lose with the application of this method of working.

Whatever data is left around, could we not have an actionable step to remove that data once all modifiers are removed and the artists wants a clean mesh for export?

Hi Guys, Multires is part of my workflow, I have a mesh I setup and UV'd, then add a multires and sculpt. Sometimes the multires is applied at a high level to capture the sculpt data then a new one added and unsubdivided. I'm fairly lose with the application of this method of working. Whatever data is left around, could we not have an actionable step to remove that data once all modifiers are removed and the artists wants a clean mesh for export?

Philipp Oeser commented

2022-05-19 15:10:50 +02:00

The data is usually removed alongside the modifier.

But there is this special case when you switch an object's mesh to a mesh that has a multires modifier.
In this case you end up with the data on the mesh (even though you dont even have a modifier on the object).
That is the limitation of the current design, just keep that in mind.
Does that answer your question?

The data is usually removed alongside the modifier. But there is this special case when you switch an object's mesh to a mesh that has a multires modifier. In this case you end up with the data on the mesh (even though you dont even have a modifier on the object). That is the limitation of the current design, just keep that in mind. Does that answer your question?

Malcolm Reed commented

2022-05-19 15:21:23 +02:00

Yeah it does, thank you, that is really useful to know how to fix it.

Just wondering if the mesh cleanup tools or similar should offer a fix for example, as it's easy to get a twist.

Cheers.

For anyone else who finds themselves here:

import bpy

objects = bpy.context.selected_objects
for obj in objects:
    bpy.context.view_layer.objects.active = obj
    bpy.ops.object.modifier_add(type='MULTIRES')
    bpy.ops.object.modifier_remove(modifier="Multires")

Yeah it does, thank you, that is really useful to know how to fix it. Just wondering if the mesh cleanup tools or similar should offer a fix for example, as it's easy to get a twist. Cheers. For anyone else who finds themselves here: ``` import bpy objects = bpy.context.selected_objects for obj in objects: bpy.context.view_layer.objects.active = obj bpy.ops.object.modifier_add(type='MULTIRES') bpy.ops.object.modifier_remove(modifier="Multires") ```

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Bloated mesh data example (multires CD_MDISPS hanging around without a multires modifier) #98249