Page MenuHome

Fix T68159: Mesh > Normals > Point to Target crashes

Authored by Philipp Oeser (lichtwerk) on Sep 11 2019, 12:48 AM.



Would crash when switching out of editmode while still in modal.
I consider this fix a bandaid, should probably handled differently?

Diff Detail

rB Blender
T68159 (branched from master)
Build Status
Buildable 4893
Build 4893: arc lint + arc unit

Event Timeline

For some operators it might make sense to be able to switch modes. So killing all modals is probably not an option. Or maybe all modals whose poll function returns false should be killed.

Shouldn't it be possible to cancel the operator without a memory leak?

It'd need to call point_normals_free before returning at the very least… But I don’t think such specific code is valid, pretty sure that kind of issue will also affect other modal tools like knife etc.

I’d say best solution would be to forbid mode switching when a modal operator is running, although that might indeed be an issue for some (I don’t have any example off hand of such modal operator that would require mode switching though?). Otherwise, we may do what @Jacques Lucke (JacquesLucke) suggests too, but then when shall we call the poll functions? before or after mode switching? Since after leaving Edit mode e.g. all references kept to bmesh & co would be invalid, not sure a proper clean exit would be possible?

I cant find another modal tool that lets me switch modes, I'll investigate some more and try to figure out why there is OPERATOR_PASS_THROUGH in edbm_point_normals_modal at all (assume this has to do with it)...

Checking a poll function while a modal operator runs could back-fire, since we cant guarantee the state a modal operator enters will be valid a valid state once the operator is running.

It also doesn't fix the root cause which is that memory this operator points to may be freed while the operator runs.

For example, this operator can be started, then a UV layer can be added (re-allocating custom data), or vertices deleted... etc.

We should really not be using OPERATOR_RUNNING_MODAL here, since it allows for many operations to run which could crash this operator.

Committed alternate fix rBffe599b4bdbe: Fix T68159: Normals point target crashes when exiting edit-mode it frees the data if events are passed through, as we cant trust the data in that case.

I think we could consider replacing this with the tool-system, as it seems like an awkward use of a modal operator which only exposes keys to access other tools.