Python API support for element picking (matching selection picking) #86445

New Issue

Campbell Barton · 2021-03-09T23:55:34+01:00

Campbell Barton commented

2021-03-09 23:55:34 +01:00

Currently there is no convenient way to pick the object under the mouse cursor.

Motivation

Picking an item (typically an object), under the cursor is useful for interactive tools,
for selecting a target... etc. as is already done in Blender's C code.

Currently methods of doing this are insufficient.

Ray-cast only works for mesh, geometry types and isn't useful for mesh elements, bones... etc.
Selection picking changes the selection, is awkward to access the result and doesn't allow to filter elements (if you only want certain object types for example).

This limits the kinds of interactive tools that can be written (at least easily).

Proposal

For space types that support picking elements, add a method for picking items, example usage:

location = event.mouse_region_x, event.mouse_region_y
ob = context.region.find_on_screen_object(location)

Since it's common to filter items based on type, or other properties, a filter function should be supported.

ob = context.region.find_on_screen_object(location, filter=lambda ob: ob.type == 'GPENCIL')

BMesh elements could be supported too.

Note that returning multiple values is needed in some cases if the element on it's own doesn't provide enough information. For example, a bone or vertex doesn't give access to the object, which is needed for multi-edit mode support.

result = context.region.find_on_screen_mesh_edge(
    location,
    filter=lambda candidate: candidate.object.get("my_property") == 1 and candidate.vert.co.x > 1.0,
)
if result:
    print(result.object, result.vert)

Other space types can support this too.

node = context.region.find_on_screen_node(location, filter=lambda node: node.type != 'OUTPUT_MATERIAL')

Open Topics

Would we support a returning multiple hits (callers might be able to use the filter function for this if they really wanted).
Should this be a generic function for all types of objects:

Types could be passed in as an argument. e.g.

`context.region.find_on_screen_item(location, type={'EDIT_MESH_EDGE', 'EDIT_MESH_FACE'})`

Why limit this to picking? Why not support passing in a bounding box & return it's result, or a polygon and return the lasso result?

This could be handled as a separate step, mentioning it here to note that it's a possibility but not part of this proposal.
We could return an object that stores multiple values, this means if we need to add additional context to the result, it wont break scripts.

e.g:

  result = context.region.find_on_screen_node(location)
  if result:
      print("Found", result.node, result.node_group)

Scope

While this need not implement all data-types initially, these are data-types that could be supported.

3D View
- Objects (all types)
- Armature Edit Mode: Bone, Head/Tail
- Armature PoseBones: Bone (only)
- MetaBall: Inner / Outer rings.
- Mesh Elements: (Vert/Edge/Face)
- Curve: Vertices & Handles
- Lattice: Vertices
- Grease Pencil: Vertices
- Particle Edit Mode: Vertices
- Stroke path bezier handles.
- Gizmos? (not sure about this one)
Image View
- UV Elements: Vertex, Loop, Face
- Mask: Vertices & Handles
Node Editor
- Nodes
- Node Sockets
- Node Links? (not sure about this one)
Sequencer
- Sequence: strip, handles
Movie clip editor
- Track: Center, corners, rotation handle.
- Mask: Vertices & Handles
Dope Sheet
- Channels: ID's & individual channels
- Keyframes.
- Markers
Graph Editor
- Channels: ID & individual channels.
- F-Curves: Whole curves & bezier handles
NLA
- Channels: ID & individual tracks.
- NLA strips.
- Markers.
Text Editor
- Line and cursor offset (low priority).
Python Console
- Line and cursor offset (low priority).
Info
- Report (low priority).
Outliner:
- Scene, ID data-block + data (needs investigation, the hierarchy of data could be returned: e.g. scene, view_layer, collection, object, mesh, modifier)
File browser
- File name.

Implementation Details

This should share functionality with selection picking (for predictable results and to avoid code duplication).
This would need to be implemented in the C/Python API directly (not the RNA API), as it uses callbacks & non RNA types (in the case of BMesh).

Currently there is no convenient way to pick the object under the mouse cursor. ### Motivation Picking an item (typically an object), under the cursor is useful for interactive tools, for selecting a target... etc. as is already done in Blender's C code. Currently methods of doing this are insufficient. - Ray-cast only works for mesh, geometry types and isn't useful for mesh elements, bones... etc. - Selection picking changes the selection, is awkward to access the result and doesn't allow to filter elements (if you only want certain object types for example). This limits the kinds of interactive tools that can be written (at least easily). ### Proposal For space types that support picking elements, add a method for picking items, example usage: ``` location = event.mouse_region_x, event.mouse_region_y ob = context.region.find_on_screen_object(location) ``` Since it's common to filter items based on type, or other properties, a filter function should be supported. ``` ob = context.region.find_on_screen_object(location, filter=lambda ob: ob.type == 'GPENCIL') ``` BMesh elements could be supported too. Note that returning multiple values is needed in some cases if the element on it's own doesn't provide enough information. For example, a bone or vertex doesn't give access to the object, which is needed for multi-edit mode support. ``` result = context.region.find_on_screen_mesh_edge( location, filter=lambda candidate: candidate.object.get("my_property") == 1 and candidate.vert.co.x > 1.0, ) if result: print(result.object, result.vert) ``` Other space types can support this too. ``` node = context.region.find_on_screen_node(location, filter=lambda node: node.type != 'OUTPUT_MATERIAL') ``` ### Open Topics - Would we support a returning multiple hits (callers might be able to use the filter function for this if they really wanted). - Should this be a generic function for all types of objects: ``` Types could be passed in as an argument. e.g. ``` ``` `context.region.find_on_screen_item(location, type={'EDIT_MESH_EDGE', 'EDIT_MESH_FACE'})` ``` - Why limit this to picking? Why not support passing in a bounding box & return it's result, or a polygon and return the lasso result? *This could be handled as a separate step, mentioning it here to note that it's a possibility but not part of this proposal.* - We could return an object that stores multiple values, this means if we need to add additional context to the result, it wont break scripts. ``` e.g: ``` ``` result = context.region.find_on_screen_node(location) if result: print("Found", result.node, result.node_group) ``` ### Scope While this need not implement all data-types initially, these are data-types that could be supported. - 3D View - Objects (all types) - Armature Edit Mode: Bone, Head/Tail - Armature PoseBones: Bone (only) - MetaBall: Inner / Outer rings. - Mesh Elements: (Vert/Edge/Face) - Curve: Vertices & Handles - Lattice: Vertices - Grease Pencil: Vertices - Particle Edit Mode: Vertices - Stroke path bezier handles. - Gizmos? *(not sure about this one)* - Image View - UV Elements: Vertex, Loop, Face - Mask: Vertices & Handles - Node Editor - Nodes - Node Sockets - Node Links? (not sure about this one) - Sequencer - Sequence: strip, handles - Movie clip editor - Track: Center, corners, rotation handle. - Mask: Vertices & Handles - Dope Sheet - Channels: ID's & individual channels - Keyframes. - Markers - Graph Editor - Channels: ID & individual channels. - F-Curves: Whole curves & bezier handles - NLA - Channels: ID & individual tracks. - NLA strips. - Markers. - Text Editor - Line and cursor offset (low priority). - Python Console - Line and cursor offset (low priority). - Info - Report (low priority). - Outliner: - Scene, ID data-block + data *(needs investigation, the hierarchy of data could be returned: e.g. scene, view_layer, collection, object, mesh, modifier)* - File browser - File name. ---- ### Implementation Details - This should share functionality with selection picking (for predictable results and to avoid code duplication). - This would need to be implemented in the C/Python API directly (not the RNA API), as it uses callbacks & non RNA types (in the case of BMesh).

Campbell Barton commented

2021-03-09 23:55:34 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Campbell Barton commented

2021-03-09 23:55:34 +01:00

Added subscriber: @ideasman42

Chris Kohl commented

2021-03-10 00:23:45 +01:00

Added subscriber: @ckohl_art

Eugene Du commented

2021-03-10 09:12:16 +01:00

Added subscriber: @APEC

Robert Guetzkow commented

2021-03-10 10:09:18 +01:00

Added subscriber: @rjg

Jaume Bellet commented

2021-03-12 13:50:00 +01:00

Added subscriber: @JaumeBellet

Jaume Bellet commented

2021-03-12 16:45:04 +01:00

As you may noticed I'm interested on allowing add-ons being accessible allowing a good enhancement to blender functionality. So maybe I have a look a this Only issue is having time for all things!

Wilsimar commented

2021-03-15 15:09:28 +01:00

Added subscriber: @wilBr

Wilsimar commented

2021-03-15 15:12:20 +01:00

Curve: Vertices & Handles

picking curve samples points beetwen vertices (or 'edge' curve) would be very usefull too.

> Curve: Vertices & Handles picking curve samples points beetwen vertices (or 'edge' curve) would be very usefull too.

yoshiki nosaka commented

2021-12-27 08:21:04 +01:00

Added subscriber: @nosaka

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Python API support for element picking (matching selection picking) #86445

Motivation

Proposal

Open Topics

Scope

Implementation Details