Dyntopo warning message won't go away after entering vertex paint mode once [lurking CD_MDEFORMVERT layer in 'corrupt' mesh] #57178

New Issue

Nam-Quang Tran · 2018-10-12T12:24:41+02:00

Nam-Quang Tran commented

2018-10-12 12:24:41 +02:00

System Information
Ubuntu 14.04 LTS, some on-board graphics card

Blender Version
Blender 2.79b

Short description of error
When sculpting with dyntopo, if you enter vertex paint mode once, dyntopo will from then on always be disabled upon entering sculpt mode, and enabling it will always give a vertex data warning message.

Exact steps for others to reproduce the error

Open a new scene with the default cube selected.
Go to sculpt mode.
Enable dyntopo in the tool shelf.
Go to object mode, then back to sculpt mode. Dyntopo is still enabled, as expected.
Go to vertex paint mode, then go back to sculpt mode. Dyntopo has now been automatically disabled with a vertex data warning.
From now on, after going out of and back into sculpt mode, dyntopo will always be disabled, and enabling it will always give the vertex data warning message.

As far as I can tell, there's no way on the UI to remove the vertex color data that was apparently added when vertex paint mode was entered. Thus, it becomes impossible to make the vertex data warning message go away for good, or to keep dyntopo enabled when leaving and reentering sculpt mode.

Additonal note: I discovered this issue during a sculpting project when I accidentally entered vertex paint mode by clicking on the wrong button in the pie menu. This could easily happen to others!

**System Information** Ubuntu 14.04 LTS, some on-board graphics card **Blender Version** Blender 2.79b **Short description of error** When sculpting with dyntopo, if you enter vertex paint mode once, dyntopo will from then on always be disabled upon entering sculpt mode, and enabling it will always give a vertex data warning message. **Exact steps for others to reproduce the error** - Open a new scene with the default cube selected. - Go to sculpt mode. - Enable dyntopo in the tool shelf. - Go to object mode, then back to sculpt mode. Dyntopo is still enabled, as expected. - Go to vertex paint mode, then go back to sculpt mode. Dyntopo has now been automatically disabled with a vertex data warning. - From now on, after going out of and back into sculpt mode, dyntopo will always be disabled, and enabling it will always give the vertex data warning message. # As far as I can tell, there's no way on the UI to remove the vertex color data that was apparently added when vertex paint mode was entered. Thus, it becomes impossible to make the vertex data warning message go away for good, or to keep dyntopo enabled when leaving and reentering sculpt mode. Additonal note: I discovered this issue during a sculpting project when I accidentally entered vertex paint mode by clicking on the wrong button in the pie menu. This could easily happen to others!

Nam-Quang Tran commented

2018-10-12 12:24:41 +02:00

Added subscriber: @qforce

Adam Preisler commented

2018-10-12 19:01:02 +02:00

Added subscriber: @AdamPreisler

Adam Preisler commented

2018-10-12 19:01:02 +02:00

Unfortunately these 2 features contradict each other on a fundamental level so this is the way it has to be...

Unless maybe if (combo of speculation) enabling the dyntopo again would instead of removing vertex color information completely keep it in memory and paint it from a point cloud of vertices each having it's vertex color and always looking at colors of closest point (vertex) around to determine gradients..

However this would have to run on GPU to be viable for dense meshes.

I suppose it can be done but the workload for such a bugfix to work properly is not trivial and also not the focus right now, especially when there's a reasonable alternative that is to use the multires modifier instead (possibly one that already further subdivides an already denser mesh that is vertex painted)

If you refer to the warning always appearing again and dyntopo being automaticaly disabled then maybe it would make sense to not show it again and keep it enabled but this happens because vertex color data was detected.

I checked and simply enabling dyntopo doesn't actually delete your vertex color data and only replaces them with white on parts of the mesh that you sculpt on.

Then maybe the warning doesn't have to appear multiple times for the same object but always only once and then never again?

Unfortunately these 2 features contradict each other on a fundamental level so this is the way it has to be... Unless maybe if (combo of speculation) enabling the dyntopo again would instead of removing vertex color information completely keep it in memory and paint it from a point cloud of vertices each having it's vertex color and always looking at colors of closest point (vertex) around to determine gradients.. However this would have to run on GPU to be viable for dense meshes. I suppose it can be done but the workload for such a bugfix to work properly is not trivial and also not the focus right now, especially when there's a reasonable alternative that is to use the multires modifier instead (possibly one that already further subdivides an already denser mesh that is vertex painted) If you refer to the warning always appearing again and dyntopo being automaticaly disabled then maybe it would make sense to not show it again and keep it enabled but this happens because vertex color data was detected. **I checked and simply enabling dyntopo doesn't actually delete your vertex color data and only replaces them with white on parts of the mesh that you sculpt on.** Then maybe the warning doesn't have to appear multiple times for the same object but always only once and then never again?

Nam-Quang Tran commented

2018-10-12 22:09:57 +02:00

As far as I'm concerned, the only two issues are (1) the annoying warning message that just keeps popping up, and (2) the fact that now with vertex color data added I have to keep enabling dyntopo all the time. Personally, I've never really needed vertex painting, and I feel that with the modern PBR workflow that everybody uses these days, vertex painting is becoming less and less important.

So, in my opinion, a simple, hopefully low-effort solution like the following two would already be plenty:

When the warning message pops up, give the user the option to nuke all vertex color data on the object. This way the warning message won't show up again, and the dyntopo state will then also be remembered when leaving and reentering sculpt mode. In addition, should the hapless user accidentally enter vertex paint mode again, he or she can simply opt to nuke the newly added vertex color data again.
When the warning message pops up, point out to the user that he or she has to manually remove the vertex color data. Then add a "Clear vertex color data" button somewhere on the UI, e.g.:

in vertex paint mode in the tool shelf and/or header menu
on the Data tab under "Geometry Data" right next to the other "Clear" buttons, such as "Clear Sculpt-Mask Data" and "Clear Skin Data".

Solution #1 would be more convenient, but also more dangerous, because all of a sudden the user is given the option to wipe out all vertex color data. The two solutions could also be combined.

As far as I'm concerned, the only two issues are (1) the annoying warning message that just keeps popping up, and (2) the fact that now with vertex color data added I have to keep enabling dyntopo all the time. Personally, I've never really needed vertex painting, and I feel that with the modern PBR workflow that everybody uses these days, vertex painting is becoming less and less important. So, in my opinion, a simple, hopefully low-effort solution like the following two would already be plenty: 1) When the warning message pops up, give the user the option to nuke all vertex color data on the object. This way the warning message won't show up again, and the dyntopo state will then also be remembered when leaving and reentering sculpt mode. In addition, should the hapless user accidentally enter vertex paint mode again, he or she can simply opt to nuke the newly added vertex color data again. 2) When the warning message pops up, point out to the user that he or she has to manually remove the vertex color data. Then add a "Clear vertex color data" button somewhere on the UI, e.g.: - in vertex paint mode in the tool shelf and/or header menu - on the Data tab under "Geometry Data" right next to the other "Clear" buttons, such as "Clear Sculpt-Mask Data" and "Clear Skin Data". Solution #1 would be more convenient, but also more dangerous, because all of a sudden the user is given the option to wipe out all vertex color data. The two solutions could also be combined.

Adam Preisler commented

2018-10-12 22:44:20 +02:00

Just click on the minus next to vertex color data and that deletes it.

Seems a little too elaborate to add an isolated option to delete vertex data from inside the sculpt mode. The warning message is there for a reason but maybe it could only appear once after Blender startup so that it doesn't pop up again and again and again during work.

I personally use Vertex Color for a lot of stuff and find it very valuable. It is great for masking textures for example. (and much much more^^)

Just click on the minus next to vertex color data and that deletes it. ![Screen Shot 10-12-18 at 09.39 PM.JPG](https://archive.blender.org/developer/F5042553/Screen_Shot_10-12-18_at_09.39_PM.JPG) Seems a little too elaborate to add an isolated option to delete vertex data from inside the sculpt mode. The warning message is there for a reason but maybe it could only appear once after Blender startup so that it doesn't pop up again and again and again during work. I personally use Vertex Color for a lot of stuff and find it very valuable. It is great for masking textures for example. (and much much more^^)

Nam-Quang Tran commented

2018-10-12 23:03:08 +02:00

Ah, sorry, looks like my initial bug report was misleading.

Indeed, if you follow the steps to reproduce listed above and then remove the vertex colors from the list, the dyntopo warning message disappears and the dyntopo state is also remembered.

But! I have this test blend file here where the dyntopo issue still occurs. The test file is based on my original sculpt file, and I simplified it down to a single vertex at the origin. I removed all object data that I was able to remove, including the vertex colors, and yet the warning message still appears, and the dyntopo enabled state is not remembered.

So now it looks like some unidentified custom data is lurking around in there and causing the dyntopo issue. Thus, what would resolve the issue for me would be a more informative warning message, one that specifically says what kind of custom data is causing the problem, so that I can go about removing it. An option to remove the custom data would be even better.

Dyntopo_Test.01.blend

Ah, sorry, looks like my initial bug report was misleading. Indeed, if you follow the steps to reproduce listed above and then remove the vertex colors from the list, the dyntopo warning message disappears and the dyntopo state is also remembered. But! I have this test blend file here where the dyntopo issue still occurs. The test file is based on my original sculpt file, and I simplified it down to a single vertex at the origin. I removed all object data that I was able to remove, including the vertex colors, and yet the warning message still appears, and the dyntopo enabled state is not remembered. So now it looks like some unidentified custom data is lurking around in there and causing the dyntopo issue. Thus, what would resolve the issue for me would be a more informative warning message, one that specifically says what kind of custom data is causing the problem, so that I can go about removing it. An option to remove the custom data would be even better. [Dyntopo_Test.01.blend](https://archive.blender.org/developer/F5042769/Dyntopo_Test.01.blend)

-- commented

2019-07-04 15:24:12 +02:00

Added subscriber: @zgorg

-- commented

2019-07-04 15:24:12 +02:00

how to get off of this warning forever?

Philipp Oeser commented

2019-11-07 11:52:49 +01:00

Added subscriber: @lichtwerk

Philipp Oeser commented

2019-11-07 11:52:49 +01:00

CD_MDEFORMVERT layer (vertex group) is detected here, not sure where this comes from [not sure how to clean the mesh from it even tbh...]

`CD_MDEFORMVERT` layer (vertex group) is detected here, not sure where this comes from [not sure how to clean the mesh from it even tbh...]

Philipp Oeser commented

2019-11-07 12:45:56 +01:00

Changed status from 'Open' to: 'Archived'

Philipp Oeser closed this issue

2019-11-07 12:45:56 +01:00

Philipp Oeser self-assigned this 2019-11-07 12:45:56 +01:00

Philipp Oeser commented

2019-11-07 12:45:56 +01:00

Added subscribers: @dfelinto, @jesterking

Philipp Oeser commented

2019-11-07 12:45:56 +01:00

CD_MDEFORMVERT layer (vertex group) is detected here, not sure where this comes from [not sure how to clean the mesh from it even tbh...]

At this point, I would probably choose the workaround of a roundtrip blender > OBJ > blender (or similar)

Thus, what would resolve the issue for me would be a more informative warning message, one that specifically says what kind of custom data is causing the problem, so that I can go about removing it. An option to remove the custom data would be even better.

That sounds like a good idea, but I think this is outside the scope of this bugtracker.

But! I have this test blend file here where the dyntopo issue still occurs. The test file is based on my original sculpt file, and I simplified it down to a single vertex at the origin. I removed all object data that I was able to remove, including the vertex colors, and yet the warning message still appears, and the dyntopo enabled state is not remembered.

At this point I would not know how to proceed with this report.
Unless we have a reproducable way of achieving such a "corrupted" mesh, I guess there is not much we can do?

I would vote for archiving this report (we can always reopen once we found the cause of this...).
I assume safeguarding all of our tools against mesh "corruption" is out of scope (and would imply performance hits in many cases)?
@dfelinto, @jesterking : since mesh corruption comes up once in a while [and sometimes the cause (be it blender modeling tools, having done something in a development version of blender or caused by some importer) is not easy/possible to track down], it might be good to find a general "rule" for proceeding with such cases? Put on https://wiki.blender.org/wiki/Process/Bug_Reports/Triaging_Playbook?

So unless there are objections, will archive this (@qforce, @zgorg: feel free though to comment again to reopen if we've found a way to reproduce here...)

> CD_MDEFORMVERT layer (vertex group) is detected here, not sure where this comes from [not sure how to clean the mesh from it even tbh...] At this point, I would probably choose the workaround of a roundtrip `blender` > `OBJ` > `blender` (or similar) > Thus, what would resolve the issue for me would be a more informative warning message, one that specifically says what kind of custom data is causing the problem, so that I can go about removing it. An option to remove the custom data would be even better. That sounds like a good idea, but I think this is outside the scope of this bugtracker. > But! I have this test blend file here where the dyntopo issue still occurs. The test file is based on my original sculpt file, and I simplified it down to a single vertex at the origin. I removed all object data that I was able to remove, including the vertex colors, and yet the warning message still appears, and the dyntopo enabled state is not remembered. At this point I would not know how to proceed with this report. Unless we have a reproducable way of achieving such a "corrupted" mesh, I guess there is not much we can do? I would vote for archiving this report (we can always reopen once we found the cause of this...). I assume safeguarding all of our tools against mesh "corruption" is out of scope (and would imply performance hits in many cases)? @dfelinto, @jesterking : since mesh corruption comes up once in a while [and sometimes the cause (be it blender modeling tools, having done something in a development version of blender or caused by some importer) is not easy/possible to track down], it might be good to find a general "rule" for proceeding with such cases? Put on https://wiki.blender.org/wiki/Process/Bug_Reports/Triaging_Playbook? So unless there are objections, will archive this (@qforce, @zgorg: feel free though to comment again to reopen if we've found a way to reproduce here...)

Philipp Oeser changed title from ~~Dyntopo warning message won't go away after entering vertex paint mode once~~ to Dyntopo warning message won't go away after entering vertex paint mode once [lurking CD_MDEFORMVERT layer in 'corrupt' mesh]

2019-11-07 12:46:50 +01:00

Dalai Felinto commented

2019-11-07 17:21:57 +01:00

For the records, the scenario "Complete report, but corrupted mesh" was added to the bug triaging playbook.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Dyntopo warning message won't go away after entering vertex paint mode once [lurking CD_MDEFORMVERT layer in 'corrupt' mesh] #57178

As far as I can tell, there's no way on the UI to remove the vertex color data that was apparently added when vertex paint mode was entered. Thus, it becomes impossible to make the vertex data warning message go away for good, or to keep dyntopo enabled when leaving and reentering sculpt mode.