Depsgraph 'object_instances' showing incorrect list of objects #57760

Closed
opened 2018-11-10 07:10:30 +01:00 by Jonathan Dent · 9 comments

Can be reproduced with a default Blender 2.8 scene:

Set up the Python bindings to print out the "depsgraph.scene.objects" and the "depsgraph,object_instances" during a render (or at any time)

The first printout will show all objects correctly (Camera, Cube, Lamp)

The object_instance list prints out the Lamp and then the Camera twice. The Cube is not listed.

This happens with other configurations: linked objects, dupli objects, particles........ In all cases one object is always listed twice and one object will be missing from the object_instances list.

Can be reproduced with a default Blender 2.8 scene: Set up the Python bindings to print out the "depsgraph.scene.objects" and the "depsgraph,object_instances" during a render (or at any time) The first printout will show all objects correctly (Camera, Cube, Lamp) The object_instance list prints out the Lamp and then the Camera twice. The Cube is not listed. This happens with other configurations: linked objects, dupli objects, particles........ In all cases one object is always listed twice and one object will be missing from the object_instances list.
Author

Added subscriber: @jdent02

Added subscriber: @jdent02
Member

Added subscribers: @mont29, @Sergey, @dfelinto, @lichtwerk

Added subscribers: @mont29, @Sergey, @dfelinto, @lichtwerk
Bastien Montagne was assigned by Philipp Oeser 2018-11-12 10:35:05 +01:00
Member

Can confirm, check with:

import bpy

C = bpy.context

for o in C.depsgraph.objects:
    print(o)
    
for o in C.depsgraph.object_instances:
    print(o.object)

this might be related to studio/blender-studio#57558?

Not sure if @mont29 can have a look? Also adding @Sergey and @dfelinto here...

Can confirm, check with: ``` import bpy C = bpy.context for o in C.depsgraph.objects: print(o) for o in C.depsgraph.object_instances: print(o.object) ``` this might be related to studio/blender-studio#57558? Not sure if @mont29 can have a look? Also adding @Sergey and @dfelinto here...

If i rememebr correctly, this is due to some temp memory is stored in the iterator. This works fine for C++ RNA used by Cycles, but in Python things become more complicated since it seems python first stores next iteration (which does change temp fields) and only then gives a control to the loop body. This makes the loop body to see "wrong" things.

Not sure how to solve this though.

If i rememebr correctly, this is due to some temp memory is stored in the iterator. This works fine for C++ RNA used by Cycles, but in Python things become more complicated since it seems python first stores next iteration (which does change temp fields) and only then gives a control to the loop body. This makes the loop body to see "wrong" things. Not sure how to solve this though.

Yes, depsgraph instance iterator is totally dynamic (that why you shall never store any of its items, only get whatever data you need from it immediately).

@Sergey where do you get that info about python's iterator behavior? Could not find anything about it from a quick web search…

Anyway, went ahead and made an ugly work-around that basically switches between two set of iterating data, such that previous iteration keeps valid data:
P823: Proposed fix for #57760 Depsgraph 'object_instances' showing incorrect list of objects

diff --git a/source/blender/makesrna/intern/rna_depsgraph.c b/source/blender/makesrna/intern/rna_depsgraph.c
index 7bfd3f679d8..660e3753903 100644
--- a/source/blender/makesrna/intern/rna_depsgraph.c
+++ b/source/blender/makesrna/intern/rna_depsgraph.c
@@ -280,38 +280,52 @@ static PointerRNA rna_Depsgraph_objects_get(CollectionPropertyIterator *iter)
  * Contains extra information about duplicator and persistent ID.
  */
 
+typedef struct RNA_Depsgraph_Instances_Iterator
+{
+	BLI_Iterator iterators[2];
+	DEGObjectIterData deg_data[2];
+	int counter;
+} RNA_Depsgraph_Instances_Iterator;
+
 static void rna_Depsgraph_object_instances_begin(CollectionPropertyIterator *iter, PointerRNA *ptr)
 {
-	iter->internal.custom = MEM_callocN(sizeof(BLI_Iterator), __func__);
-	DEGObjectIterData *data = MEM_callocN(sizeof(DEGObjectIterData), __func__);
+	RNA_Depsgraph_Instances_Iterator *di_it = iter->internal.custom = MEM_callocN(sizeof(*di_it), __func__);
 
+	DEGObjectIterData *data = &di_it->deg_data[di_it->counter % 2];
 	data->graph = (Depsgraph *)ptr->data;
 	data->flag = DEG_ITER_OBJECT_FLAG_LINKED_DIRECTLY |
 	             DEG_ITER_OBJECT_FLAG_LINKED_VIA_SET |
 	             DEG_ITER_OBJECT_FLAG_VISIBLE |
 	             DEG_ITER_OBJECT_FLAG_DUPLI;
 
-	((BLI_Iterator *)iter->internal.custom)->valid = true;
-	DEG_iterator_objects_begin(iter->internal.custom, data);
-	iter->valid = ((BLI_Iterator *)iter->internal.custom)->valid;
+	di_it->iterators[di_it->counter % 2].valid = true;
+	di_it->iterators[di_it->counter % 2].data = data;
+	DEG_iterator_objects_begin(&di_it->iterators[di_it->counter % 2], data);
+	iter->valid = di_it->iterators[di_it->counter % 2].valid;
 }
 
 static void rna_Depsgraph_object_instances_next(CollectionPropertyIterator *iter)
 {
-	DEG_iterator_objects_next(iter->internal.custom);
-	iter->valid = ((BLI_Iterator *)iter->internal.custom)->valid;
+	RNA_Depsgraph_Instances_Iterator *di_it = (RNA_Depsgraph_Instances_Iterator *)iter->internal.custom;
+	di_it->iterators[(di_it->counter + 1) % 2] = di_it->iterators[di_it->counter % 2];
+	di_it->deg_data[(di_it->counter + 1) % 2] = di_it->deg_data[di_it->counter % 2];
+	di_it->counter++;
+	di_it->iterators[di_it->counter % 2].data = &di_it->deg_data[di_it->counter % 2];
+	DEG_iterator_objects_next(&di_it->iterators[di_it->counter % 2]);
+	iter->valid = di_it->iterators[di_it->counter % 2].valid;
 }
 
 static void rna_Depsgraph_object_instances_end(CollectionPropertyIterator *iter)
 {
-	DEG_iterator_objects_end(iter->internal.custom);
-	MEM_freeN(((BLI_Iterator *)iter->internal.custom)->data);
-	MEM_freeN(iter->internal.custom);
+	RNA_Depsgraph_Instances_Iterator *di_it = (RNA_Depsgraph_Instances_Iterator *)iter->internal.custom;
+	DEG_iterator_objects_end(&di_it->iterators[di_it->counter % 2]);
+	MEM_freeN(di_it);
 }
 
 static PointerRNA rna_Depsgraph_object_instances_get(CollectionPropertyIterator *iter)
 {
-	BLI_Iterator *iterator = (BLI_Iterator *)iter->internal.custom;
+	RNA_Depsgraph_Instances_Iterator *di_it = (RNA_Depsgraph_Instances_Iterator *)iter->internal.custom;
+	BLI_Iterator *iterator = &di_it->iterators[di_it->counter % 2];
 	return rna_pointer_inherit_refine(&iter->parent, &RNA_DepsgraphObjectInstance, iterator);
 }
 

Seems to work from quick testing.

Yes, depsgraph instance iterator is totally dynamic (that why you shall never store any of its items, only get whatever data you need from it immediately). @Sergey where do you get that info about python's iterator behavior? Could not find anything about it from a quick web search… Anyway, went ahead and made an ugly work-around that basically switches between two set of iterating data, such that previous iteration keeps valid data: [P823: Proposed fix for #57760 Depsgraph 'object_instances' showing incorrect list of objects](https://archive.blender.org/developer/P823.txt) ``` diff --git a/source/blender/makesrna/intern/rna_depsgraph.c b/source/blender/makesrna/intern/rna_depsgraph.c index 7bfd3f679d8..660e3753903 100644 --- a/source/blender/makesrna/intern/rna_depsgraph.c +++ b/source/blender/makesrna/intern/rna_depsgraph.c @@ -280,38 +280,52 @@ static PointerRNA rna_Depsgraph_objects_get(CollectionPropertyIterator *iter) * Contains extra information about duplicator and persistent ID. */ +typedef struct RNA_Depsgraph_Instances_Iterator +{ + BLI_Iterator iterators[2]; + DEGObjectIterData deg_data[2]; + int counter; +} RNA_Depsgraph_Instances_Iterator; + static void rna_Depsgraph_object_instances_begin(CollectionPropertyIterator *iter, PointerRNA *ptr) { - iter->internal.custom = MEM_callocN(sizeof(BLI_Iterator), __func__); - DEGObjectIterData *data = MEM_callocN(sizeof(DEGObjectIterData), __func__); + RNA_Depsgraph_Instances_Iterator *di_it = iter->internal.custom = MEM_callocN(sizeof(*di_it), __func__); + DEGObjectIterData *data = &di_it->deg_data[di_it->counter % 2]; data->graph = (Depsgraph *)ptr->data; data->flag = DEG_ITER_OBJECT_FLAG_LINKED_DIRECTLY | DEG_ITER_OBJECT_FLAG_LINKED_VIA_SET | DEG_ITER_OBJECT_FLAG_VISIBLE | DEG_ITER_OBJECT_FLAG_DUPLI; - ((BLI_Iterator *)iter->internal.custom)->valid = true; - DEG_iterator_objects_begin(iter->internal.custom, data); - iter->valid = ((BLI_Iterator *)iter->internal.custom)->valid; + di_it->iterators[di_it->counter % 2].valid = true; + di_it->iterators[di_it->counter % 2].data = data; + DEG_iterator_objects_begin(&di_it->iterators[di_it->counter % 2], data); + iter->valid = di_it->iterators[di_it->counter % 2].valid; } static void rna_Depsgraph_object_instances_next(CollectionPropertyIterator *iter) { - DEG_iterator_objects_next(iter->internal.custom); - iter->valid = ((BLI_Iterator *)iter->internal.custom)->valid; + RNA_Depsgraph_Instances_Iterator *di_it = (RNA_Depsgraph_Instances_Iterator *)iter->internal.custom; + di_it->iterators[(di_it->counter + 1) % 2] = di_it->iterators[di_it->counter % 2]; + di_it->deg_data[(di_it->counter + 1) % 2] = di_it->deg_data[di_it->counter % 2]; + di_it->counter++; + di_it->iterators[di_it->counter % 2].data = &di_it->deg_data[di_it->counter % 2]; + DEG_iterator_objects_next(&di_it->iterators[di_it->counter % 2]); + iter->valid = di_it->iterators[di_it->counter % 2].valid; } static void rna_Depsgraph_object_instances_end(CollectionPropertyIterator *iter) { - DEG_iterator_objects_end(iter->internal.custom); - MEM_freeN(((BLI_Iterator *)iter->internal.custom)->data); - MEM_freeN(iter->internal.custom); + RNA_Depsgraph_Instances_Iterator *di_it = (RNA_Depsgraph_Instances_Iterator *)iter->internal.custom; + DEG_iterator_objects_end(&di_it->iterators[di_it->counter % 2]); + MEM_freeN(di_it); } static PointerRNA rna_Depsgraph_object_instances_get(CollectionPropertyIterator *iter) { - BLI_Iterator *iterator = (BLI_Iterator *)iter->internal.custom; + RNA_Depsgraph_Instances_Iterator *di_it = (RNA_Depsgraph_Instances_Iterator *)iter->internal.custom; + BLI_Iterator *iterator = &di_it->iterators[di_it->counter % 2]; return rna_pointer_inherit_refine(&iter->parent, &RNA_DepsgraphObjectInstance, iterator); } ``` Seems to work from quick testing.

@mont29, from a debugger mainly, and doing sync prints in py code in RNA. Not sure how it's expected to work or anything, but observation kinda makes sense: store the next value, to allow destroying current iterator value. But all that is a guess work, can't back up by any official doc or anything.

@mont29, from a debugger mainly, and doing sync prints in py code in RNA. Not sure how it's expected to work or anything, but observation kinda makes sense: store the next value, to allow destroying current iterator value. But all that is a guess work, can't back up by any official doc or anything.

@Sergey oki… then I guess I’ll just go with my patch for now, seems to be working and would not expect it to break anything. Just not very happy with that kind of ugly workaround thing :|

@Sergey oki… then I guess I’ll just go with my patch for now, seems to be working and would not expect it to break anything. Just not very happy with that kind of ugly workaround thing :|

This issue was referenced by b44e6f2b3d

This issue was referenced by b44e6f2b3d32e604c3030eaf61b15e0e2a20d520

Changed status from 'Open' to: 'Resolved'

Changed status from 'Open' to: 'Resolved'
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
5 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#57760
No description provided.