Page MenuHome

PyAPI: Object.ray_cast returns wrong face indices with some modifier combinations
Confirmed, NormalPublicKNOWN ISSUE

Description

System Information
Operating system: Windows-7-6.1.7601-SP1 64 Bits
Graphics card: AMD Radeon HD 6900 Series ATI Technologies Inc. 4.5.13399 Core Profile Context 15.200.1062.1003

Blender Version
Broken: version: 2.82 (sub 3), branch: master, commit date: 2019-11-30 17:33, hash: rBf1ac64921b49
Worked: (optional)

Short description of error
With some modifier combinations, Object.ray_cast returns face indices that don't match the indices the object would have if the modifiers were applied or when using Object.evaluated_get. I tested this also in 2.79 and it doesn't work there either.

Attached is a blend file which includes a slightly modified version of the operator_modal_view3d_raycast.py template. It now prints the indices into the console. I included two different test scenarios just in case.

Note, the only changes in the code are the debug print on line 63 and the change to execute the operator only on mouse press on line 84.

Exact steps for others to reproduce the error

  1. Open the attached blend file
  2. Open System Console
  3. Run the "operator_modal_view3d_raycast.py" script

Case 1:

  1. Select "Bevel + Decimate" object
  2. See that the Decimate modifier's face count is 24
  3. Search for "RayCast View Operator" and press Enter
  4. Click on the object's faces and check the printed indices inside the Console
  5. Confirm that they go above 23

Case 2:

  1. Enable Developer Extras in Preferences > Interface
  2. Select "Boolean + Bevel Applied" object
  3. Switch into edit mode
  4. Inside Viewport Overlays popover, enable indices
  5. Select the top face and check the index
  6. Switch into object mode
  7. Search for "RayCast View Operator" and press Enter
  8. Click on "Boolean + Bevel" object's top face and check the printed index inside the Console
  9. Confirm that it doesn't match the index of the object which has the modifiers applied

Event Timeline

Germano Cavalcante (mano-wii) lowered the priority of this task from 90 to 30.Dec 3 2019, 4:48 PM

If I'm not mistaken, ray_cast now returns the loop_triangle index.
If I'm correct, you can get the poligon index like this:

me.calc_loop_triangles()
polygon_index = me.loop_triangles[0].polygon_index

I don't think this is a bug, can you confirm if this solves your problem?

No, unfortunately that doesn't solve it. :(

This is from the API docs:

Return (result, location, normal, index):

result, Whether the ray successfully hit the geometry, boolean

location, The hit location of this ray cast, float array of 3 items in [-inf, inf]

normal, The face normal at the ray cast hit location, float array of 3 items in [-inf, inf]

index, The face index, -1 when original data isn’t available, int in [-inf, inf]

And this is from the template:

success, location, normal, face_index = obj.ray_cast(ray_origin_obj, ray_direction_obj)

No mention of loop triangles.

And it doesn't always behave wrong. A demonstration with my addon which has face highlighting:

As we can see, only when I move the Wireframe modifier before the Solidify modifier, ray_cast starts to return wrong indices and the highlighting stops working.

Germano Cavalcante (mano-wii) raised the priority of this task from 30 to 50.EditedDec 3 2019, 6:58 PM

Upon further investigation, I realized that this method always tries to return the original mesh index. (The index of the mesh without modifiers).
See: https://developer.blender.org/diffusion/B/browse/master/source/blender/makesrna/intern/rna_object_api.c$480

But apparently the CustomData_get_layer(&me_eval->pdata, CD_ORIGINDEX) of the Bevel modifier does not work that way.
@Howard Trickey (howardt), do you know why this happens?

We could return the actual mesh poly index or the looptri index.
This hasn't been done yet to keep the backward compatibility.

CC @Campbell Barton (campbellbarton)

Sybren A. Stüvel (sybren) changed the task status from Confirmed to Needs Information from User.Jan 20 2020, 2:29 PM

@Antti Tikka (symstract) Please provide a blend file that shows this issue without depending on a reimplementation of Blender code. As it is now it requires a developer to carefully read through your Python code, compare it to the current code in Blender, see if there are any differences, etc.

@Sybren A. Stüvel (sybren) I edited the description to include a mention of the minimal changes done to the template (debug print + execute only on mouse press). I also added comments for them in the code. I don't think redoing the code is necessary as the template is perfect for showcasing the issue?

Sybren A. Stüvel (sybren) changed the subtype of this task from "Report" to "Known Issue".Jan 30 2020, 1:31 PM

Upon further investigation, I realized that this method always tries to return the original mesh index.

I think the biggest issue here is that the function can return the index of either the original or the evaluated mesh, without telling the caller which one it is. It certainly makes the function's result hard to use.

I'm marking this as a Known Issue, as it's not a bug in Blender but working as designed. Design could (and IMO should) be better, but that doesn't make this a bug.

Aaron Carlisle (Blendify) changed the task status from Needs Information from User to Confirmed.Feb 10 2020, 6:01 PM

it's not a bug in Blender but working as designed. Design could (and IMO should) be better, but that doesn't make this a bug.

Could you ellaborate on the very design it is responding to? Because I feel totally unable to use this function since:

  1. It may return an face index that is not a valid polygon index, even on an original mesh.
_, _, _, poly_index = obj.original.ray_cast(ray_origin, ray_direction)
print(poly_index) # 8
print(len(obj.original.data.polygons))  # 6
  1. It returns a different face index than scene.ray_cast.
_, _, _, poly_index1, obj, _ = scene.ray_cast(depsgraph, ray_origin, ray_direction)
_, _, _, poly_index2 = obj.original.ray_cast(ray_origin, ray_direction)
_, _, _, poly_index3 = obj.evaluated_get(depsgraph).ray_cast(ray_origin, ray_direction)
print(poly_index1) # 24
print(poly_index2) # 8
print(poly_index3) # 8
  1. Worst: the behavior changes during runtime. Unfortunately I don't know how to reproduce the bug reliably but on more complicated scenes I've noticed that it often returns correct indices at start-up, and after a few viewport operations that I could not identify for sure it suddenly changes the returned indices.

Doing further tests show that scene.ray_cast returns an index which is the correct polygon index of obj.evaluated_get(depsgraph), but the index of obj.ray_cast is valid neither as a polygon index nor as a loop_triangle index, nor as an index in the original object. So what is it?

Here is the sample scene I've used: blend. It is simply the default cube with a triangulate modifier and a subsurf modifier.

(Tested in 2.90 and 2.91)

I wasn't able to even get this working at all, in 2.93. Although maybe I'm not fully understanding how it works. But wonder if it's more broken now after some API changes?