Spreadsheet Editor #86279

New Issue

Dalai Felinto · 2021-03-04T17:29:06+01:00

Dalai Felinto commented

2021-03-04 17:29:06 +01:00

The spreadsheet editor is an editor to debug and navigate the data inside the blender file.

A prototype can be found on the temp-spreasheet-editor branch.

The initial plan is to tackle this for geometry nodes focusing on the geometry data (points, edges, ...). The design already takes into consideration other future uses cases though.

{F9866674, size=full}

The spreadsheet editor is an editor to debug and navigate the data inside the blender file. A prototype can be found on the `temp-spreasheet-editor` branch. The initial plan is to tackle this for geometry nodes focusing on the geometry data (points, edges, ...). The design already takes into consideration other future uses cases though. ---- {[F9866674](https://archive.blender.org/developer/F9866674/spreadsheet-presentation.png), size=full}

Dalai Felinto self-assigned this 2021-03-04 17:29:06 +01:00

Dalai Felinto commented

2021-03-04 17:29:06 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Dalai Felinto commented

2021-03-04 17:29:06 +01:00

Added subscriber: @dfelinto

Duarte Farrajota Ramos commented

2021-03-04 18:08:39 +01:00

Added subscriber: @DuarteRamos

Jaume Bellet commented

2021-03-04 18:12:30 +01:00

Added subscriber: @JaumeBellet

Demeter Dzadik commented

2021-03-04 19:18:10 +01:00

Added subscriber: @Mets

Demeter Dzadik commented

2021-03-04 19:18:10 +01:00

So this is what you guys have been filling those whiteboards with!

I think a more likely rigging use case would be simply to list bones. Bones have so many properties, listing the same property of many bones or filtering based on property values would be useful in many cases:

Find bones that have the Deform checkbox enabled when they shouldn't
Find bones that are set to non-selectable
Find bones with a certain rigify type
Find all children of a bone (aka all bones with the same parent)
Select all (bones)things matching the current filter

This could also provide a great way to add the "Copy Attributes" addon's features to core blender, which imo belongs in core blender.

I think the filter settings cannot be complex enough. I really see the usefulness of this scaling linearly with how complex filters can be applied. For example, finding bones who have constraints whose subtarget is a specific other bone, is a pretty damn specific case that is probably hard to design a filter for, but it would nonetheless be ridiculously useful for exploring other people's rigs.

The possibilities seem quite endless, so if you need ideas I'm here, but I'm curious now where would you limit the initial scope for this.

So this is what you guys have been filling those whiteboards with! I think a more likely rigging use case would be simply to list bones. Bones have so many properties, listing the same property of many bones or filtering based on property values would be useful in many cases: - Find bones that have the Deform checkbox enabled when they shouldn't - Find bones that are set to non-selectable - Find bones with a certain rigify type - Find all children of a bone (aka all bones with the same parent) - Select all (bones)things matching the current filter This could also provide a great way to add the "Copy Attributes" addon's features to core blender, which imo belongs in core blender. I think the filter settings cannot be complex enough. I really see the usefulness of this scaling linearly with how complex filters can be applied. For example, finding bones who have constraints whose subtarget is a specific other bone, is a pretty damn specific case that is probably hard to design a filter for, but it would nonetheless be ridiculously useful for exploring other people's rigs. The possibilities seem quite endless, so if you need ideas I'm here, but I'm curious now where would you limit the initial scope for this.

Hans Goudey commented

2021-03-04 19:36:33 +01:00

Added subscriber: @HooglyBoogly

Hans Goudey commented

2021-03-04 19:36:33 +01:00

One thing I would like to see in a complete design is a comparison of this and the outliner. There are clearly different use cases, but there are also apparently many overlapping use cases, especially as the scope we imagine for the spreadsheet editor increases.
Just for some examples, the outliner already has display modes, filters, a tree layout, search, sorting, and the ability to change data properties.
So I think it's essential to consider the design differences between this and the outliner, and list them clearly.

Another possibility is that the spreadsheet editor basically ends up becoming a better designed, more flexible outliner, which I actually think could be awesome! It's just important to acknowledge that in the design.

One thing I would like to see in a complete design is a comparison of this and the outliner. There are clearly different use cases, but there are also apparently many overlapping use cases, especially as the scope we imagine for the spreadsheet editor increases. Just for some examples, the outliner already has display modes, filters, a tree layout, search, sorting, and the ability to change data properties. So I think it's essential to consider the design differences between this and the outliner, and list them clearly. Another possibility is that the spreadsheet editor basically ends up becoming a better designed, more flexible outliner, which I actually think could be awesome! It's just important to acknowledge that in the design.

Antonio Vazquez commented

2021-03-04 22:13:33 +01:00

Added subscriber: @antoniov

Antonio Vazquez commented

2021-03-04 22:17:09 +01:00

I would like to make grease pencil able to use spreadsheet. A lot of possible uses.

I have several questions.

Is any plan to create a python API to read/write in the spreadsheet?
Have you considered how powerful this can be to use as an Import/Export gateway?
Any plan to allow "custom" spreadsheet? For example, a python API creates a list of used materials, or markers or any data to be exported.
A CSV export/import tool would be included as core feature.

I would like to make grease pencil able to use spreadsheet. A lot of possible uses. I have several questions. * Is any plan to create a python API to read/write in the spreadsheet? * Have you considered how powerful this can be to use as an Import/Export gateway? * Any plan to allow "custom" spreadsheet? For example, a python API creates a list of used materials, or markers or any data to be exported. * A CSV export/import tool would be included as core feature.

Swann Martinez commented

2021-03-05 14:00:59 +01:00

Added subscriber: @slumber

Dalai Felinto commented

2021-03-05 18:38:23 +01:00

@antoniov for me the most obvious benefit for I/O would be once we get collection exporting/importing and then being able to see all the collections I/O settings in a single place.

To support Python to read/write the data seems a bit feature creep to me. Can you tell me a use case where this would be used?

@antoniov for me the most obvious benefit for I/O would be once we get collection exporting/importing and then being able to see all the collections I/O settings in a single place. To support Python to read/write the data seems a bit feature creep to me. Can you tell me a use case where this would be used?

Antonio Vazquez commented

2021-03-05 19:01:25 +01:00

User case for python and custom data could be:

Design a kitchen, machinery or anything that has parts and be able to export the list of components, for example number of bolts required.
Generate with python a set of data, based on geometry or any other blender data, export in CVS/JSON and use in other tools outside blender. I know you can do now directly the export, but it's great to be able to see/edit the data and use it as common solution.
Generate outside Blender any data and be able to import in the spreadsheet to view/edit and maybe run an script to do anything.
Generate list of data relative to animation pipeline... markers position, keyframe position, details of VSE editor strips, etc, etc.
Almost all database engines can generate CSV, so this data could be imported easily.
Generate tabular list of frames to render for Render farms.

Basically, the spreadsheet can be used as gateway for pipeline integration in multiple situations. In business database applications this is very common.

Maybe this is not the initial goal of the design, but this can open a huge number of possibilities and I think it's worth to keep in mind, if not to do now, to prepare the pieces to do it in the future.

User case for python and custom data could be: * Design a kitchen, machinery or anything that has parts and be able to export the list of components, for example number of bolts required. * Generate with python a set of data, based on geometry or any other blender data, export in CVS/JSON and use in other tools outside blender. I know you can do now directly the export, but it's great to be able to see/edit the data and use it as common solution. * Generate outside Blender any data and be able to import in the spreadsheet to view/edit and maybe run an script to do anything. * Generate list of data relative to animation pipeline... markers position, keyframe position, details of VSE editor strips, etc, etc. * Almost all database engines can generate CSV, so this data could be imported easily. * Generate tabular list of frames to render for Render farms. Basically, the spreadsheet can be used as gateway for pipeline integration in multiple situations. In business database applications this is very common. Maybe this is not the initial goal of the design, but this can open a huge number of possibilities and I think it's worth to keep in mind, if not to do now, to prepare the pieces to do it in the future.

JT Nelson commented

2021-03-06 07:15:49 +01:00

Added subscriber: @jta

Mateusz Grzeliński commented

2021-03-06 11:26:44 +01:00

Added subscriber: @grzelins

Mateusz Grzeliński commented

2021-03-06 11:26:44 +01:00

As secondary goal consider that also file browser has a spreadsheet like view and many filtera.

Python api for spreadsheet view is needed IMHO. see for example elegant table view in addon VFX Grace https://gumroad.com/l/QgGUq

Consider use of table view in log viewer (gsoc project) , info editor or as debug feature in blender python console (variable viewer)

As secondary goal consider that also file browser has a spreadsheet like view and many filtera. Python api for spreadsheet view is needed IMHO. see for example elegant table view in addon VFX Grace https://gumroad.com/l/QgGUq Consider use of table view in log viewer (gsoc project) , info editor or as debug feature in blender python console (variable viewer)

Ermins Ermanis commented

2021-03-06 11:32:46 +01:00

Added subscriber: @erminos

Sybren A. Stüvel commented

2021-03-08 11:40:09 +01:00

Added subscriber: @dr.sybren

Sybren A. Stüvel commented

2021-03-08 11:40:09 +01:00

In #86279#1124846, @dfelinto wrote:
To support Python to read/write the data seems a bit feature creep to me. Can you tell me a use case where this would be used?

I agree with @dfelinto here. If the spreadsheet is just a different view on Blender data, and these data already have an API to modify them. The fact that it's shown in a grid instead of the regular way Blender shows that data is irrelevant when it comes to data manipulation from code. To make this more concrete: to modify mesh vertex positions, a script should modify those vertex positions directly, and not go via some spreadsheet view.

There is no need for any spreadsheet API just to read data from a CSV file, you can do that fine without such a view. Of course I see the appeal of having a generic CSV importer that you can use to interactively map columns in the CSV file to columns in the spreadsheet view. That's a different project from exposing existing data in a speadsheet UI, though.

> In #86279#1124846, @dfelinto wrote: > To support Python to read/write the data seems a bit feature creep to me. Can you tell me a use case where this would be used? I agree with @dfelinto here. If the spreadsheet is just a different view on Blender data, and these data already have an API to modify them. The fact that it's shown in a grid instead of the regular way Blender shows that data is irrelevant when it comes to data manipulation from code. To make this more concrete: to modify mesh vertex positions, a script should modify those vertex positions directly, and not go via some spreadsheet view. There is no need for any spreadsheet API just to read data from a CSV file, you can do that fine without such a view. Of course I see the appeal of having a generic CSV importer that you can use to interactively map columns in the CSV file to columns in the spreadsheet view. That's a different project from exposing existing data in a speadsheet UI, though.

blender-admin commented

2021-03-08 16:23:21 +01:00

This issue was referenced by 9cb5f0a228

This issue was referenced by 9cb5f0a2282a7a84f7f8636b43a32bdc04b51cd5

Dimitar P. commented

2021-03-08 16:31:33 +01:00

Added subscriber: @Dimitar

Dimitar P. commented

2021-03-08 16:31:33 +01:00

One of the features that's been missing for me is a comprehensive material editor. This new spreadsheet seems like it could be a good place to edit materials globally without having to click an object that contains the material.

Antonio Vazquez commented

2021-03-08 17:06:49 +01:00

My comment about API or for CVS was not whether or not it can be done currently, of course it can already be done in python.

Including a simple tabulated data import / export system, in CVS or JSON, could be very useful in countless cases and would not require any programming on the part of the user. If the user does not have to create their own exporter, that would add a standardization of this type of process that requires tabulated data, both at the I/O and visualization level.

Anyway, maybe this project is not the one for this idea, although IMHO it is a pity to miss this opportunity to do something really useful.

My comment about API or for CVS was not whether or not it can be done currently, of course it can already be done in python. Including a simple tabulated data import / export system, in CVS or JSON, could be very useful in countless cases and would not require any programming on the part of the user. If the user does not have to create their own exporter, that would add a standardization of this type of process that requires tabulated data, both at the I/O and visualization level. Anyway, maybe this project is not the one for this idea, although IMHO it is a pity to miss this opportunity to do something really useful.

Kenzie commented

2021-03-08 21:49:57 +01:00

Added subscriber: @KenzieMac130

Kenzie commented

2021-03-08 21:49:57 +01:00

In #86279#1126292, @antoniov wrote:
My comment about API or for CVS was not whether or not it can be done currently, of course it can already be done in python.

Including a simple tabulated data import / export system, in CVS or JSON, could be very useful in countless cases and would not require any programming on the part of the user. If the user does not have to create their own exporter, that would add a standardization of this type of process that requires tabulated data, both at the I/O and visualization level.

Anyway, maybe this project is not the one for this idea, although IMHO it is a pity to miss this opportunity to do something really useful.

Exporting of the current spreadsheet view/selection to CSV (even to just the clipboard) would be useful for analysis, sharing information for troubleshooting, etc. CSV import however I feel like should be implemented elsewhere such as its own node.

> In #86279#1126292, @antoniov wrote: > My comment about API or for CVS was not whether or not it can be done currently, of course it can already be done in python. > > Including a simple tabulated data import / export system, in CVS or JSON, could be very useful in countless cases and would not require any programming on the part of the user. If the user does not have to create their own exporter, that would add a standardization of this type of process that requires tabulated data, both at the I/O and visualization level. > > Anyway, maybe this project is not the one for this idea, although IMHO it is a pity to miss this opportunity to do something really useful. Exporting of the current spreadsheet view/selection to CSV (even to just the clipboard) would be useful for analysis, sharing information for troubleshooting, etc. CSV import however I feel like should be implemented elsewhere such as its own node.

Sergey commented

2021-03-09 10:09:29 +01:00

Added subscriber: @randum

Sergey commented

2021-03-09 10:09:29 +01:00

One of the use cases for using spreadsheets would be to show output content of nodes in custom node tree editor. But in case of Sverchok add-on I don't see how this could be done without low level API to spreadsheet editor because output of most nodes is just Python data. Sverchok generates Blender data blocks only to show result of whole node tree.

Here is little mock-up how spreadsheet editor could look like for Sverchok purposes.

One of the use cases for using spreadsheets would be to show output content of nodes in custom node tree editor. But in case of Sverchok add-on I don't see how this could be done without low level API to spreadsheet editor because output of most nodes is just Python data. Sverchok generates Blender data blocks only to show result of whole node tree. Here is little mock-up how spreadsheet editor could look like for Sverchok purposes. ![Evup5aUWEAUhbLq.png](https://archive.blender.org/developer/F9880766/Evup5aUWEAUhbLq.png)

Ulrich Büch commented

2021-03-09 11:08:23 +01:00

Added subscriber: @Ulrich-Buch

Ulrich Büch commented

2021-03-09 11:08:23 +01:00

Would it be granular enough to allow e.g. showing bone or object transforms at different evaluation steps? Like e.g. pre-Constraint transforms? (I know for this you could mute the constraints, but it's just an example and you get the picture.)
What I refer to here is how the final transform may be influenced by a number of things (parent transforms, delta transforms, drivers, constraints, animation curves etc.). And those are evaluated in some specific order by blender, afaik.
Here's a representation from a (somewhat dated) blogpost by aligorith which illustrates what I mean:

On another note: Is support for image-datablocks planned? The mockup suggests inspecting volume-objects down to individual voxels is planned, if so would this extend to 2D-images or would that reach too far into the image-editor's domain?

Would it be granular enough to allow e.g. showing bone or object transforms at different evaluation steps? Like e.g. pre-Constraint transforms? (I know for this you could mute the constraints, but it's just an example and you get the picture.) What I refer to here is how the final transform may be influenced by a number of things (parent transforms, delta transforms, drivers, constraints, animation curves etc.). And those are evaluated in some specific order by blender, afaik. Here's a representation from a (somewhat dated) blogpost by aligorith which illustrates what I mean: ![transform_pipeline.png](https://archive.blender.org/developer/F9880818/transform_pipeline.png) On another note: Is support for image-datablocks planned? The mockup suggests inspecting volume-objects down to individual voxels is planned, if so would this extend to 2D-images or would that reach too far into the image-editor's domain?

Brecht Van Lommel commented

2021-03-09 15:21:00 +01:00

Added subscriber: @brecht

Brecht Van Lommel commented

2021-03-09 15:21:00 +01:00

Reposting my comment from #85654 about the visual design.

Like @HooglyBoogly I prefer the header row and reduced vertical lines for vector attributes as in my original patch.
I think there should be more space after the index number. I find it difficult to read the first attribute value, it blends together with the index number too much. Center or right aligning the numbers would be an alternative solution.
Personally I think a header, two sidebars and a statusbar in this spreadsheet editor is too much. In general I think we're adding too many additional bars and overlays in the Blender UI. For example the "dataset region" could just be an enum menu in the header.

Reposting my comment from #85654 about the visual design. * Like @HooglyBoogly I prefer the header row and reduced vertical lines for vector attributes as in my original patch. * I think there should be more space after the index number. I find it difficult to read the first attribute value, it blends together with the index number too much. Center or right aligning the numbers would be an alternative solution. * Personally I think a header, two sidebars and a statusbar in this spreadsheet editor is too much. In general I think we're adding too many additional bars and overlays in the Blender UI. For example the "dataset region" could just be an enum menu in the header.

John Cox commented

2021-03-10 12:17:35 +01:00

Added subscriber: @JohnCox-3

John Cox commented

2021-03-10 12:17:35 +01:00

Can we get some clarity about terminology?

In the mock-up the domains are called
Point
Edge
Face
Corner

In the implementation that's just been committed,
only point data is displayed but the normals are labelled vertex_normal

So my question is, does vertex mean Point or Corner?
If it means Point, why not call the normals point_normal for consistency.
If it means Corner, why not just use Vertex in the interface?

I foresee confusion if vertex isn't clearly defined and consistently used.

Can we get some clarity about terminology? In the mock-up the domains are called Point Edge Face Corner In the implementation that's just been committed, only point data is displayed but the normals are labelled vertex_normal So my question is, does vertex mean Point or Corner? If it means Point, why not call the normals point_normal for consistency. If it means Corner, why not just use Vertex in the interface? I foresee confusion if vertex isn't clearly defined and consistently used.

Paul Kotelevets commented

2021-03-11 13:45:22 +01:00

Added subscriber: @1D_Inc

Paul Kotelevets commented

2021-03-11 13:45:22 +01:00

Table editor? Nice. A step closer to BIM. A lot of questions though.
For example:

The spreadsheet editor is an editor to debug and navigate the data inside the blender file.

Will it be possible to view things like distribution textures per materials in table form, to investigate where exactly some textures used? (Will there be access to such datablocks?)

Table editor? Nice. A step closer to BIM. A lot of questions though. For example: > The spreadsheet editor is an editor to debug and navigate the data inside the blender file. Will it be possible to view things like distribution textures per materials in table form, to investigate where exactly some textures used? (Will there be access to such datablocks?)

Daniele De Luca commented

2021-03-11 14:14:41 +01:00

Added subscriber: @Sparazza

Dalai Felinto commented

2021-03-12 19:28:27 +01:00

Changed status from 'Confirmed' to: 'Resolved'

Dalai Felinto closed this issue

2021-03-12 19:28:27 +01:00

Dalai Felinto commented

2021-03-12 19:28:27 +01:00

Closing this now since it was moved to: https://wiki.blender.org/wiki/Source/Editors/Spreadsheet#Spreadsheet_Editor

@JohnCox-3 vertex means point. Corner is something else. Other datasets (corner included) are coming soon.
@1D_Inc probably not.

Closing this now since it was moved to: https://wiki.blender.org/wiki/Source/Editors/Spreadsheet#Spreadsheet_Editor @JohnCox-3 vertex means point. Corner is something else. Other datasets (corner included) are coming soon. @1D_Inc probably not.

Paul Kotelevets commented

2021-03-13 00:00:15 +01:00

In #86279#1128940, @dfelinto wrote:
@1D_Inc probably not.

Okay, will keep using addons with output to blender's text file for task like that.
Thank you for the answer.

> In #86279#1128940, @dfelinto wrote: > @1D_Inc probably not. Okay, will keep using addons with output to blender's text file for task like that. Thank you for the answer. ![image.png](https://archive.blender.org/developer/F9888027/image.png)

John Cox commented

2021-03-13 12:14:50 +01:00

@dfelinto But "Vertex Color" is really Corner data no? Will that be renamed?

Dalai Felinto commented

2021-03-15 17:29:07 +01:00

@JohnCox-3 we are not planning to change the name of the any built-in attribute. If vertex color is stored per corner it will be in the corner domain, otherwise in point.

Dalai Felinto commented

2021-03-15 17:32:44 +01:00

Find bones that have the Deform checkbox enabled when they shouldn't
Find bones that are set to non-selectable
Find bones with a certain rigify type
Find all children of a bone (aka all bones with the same parent)
Select all (bones)things matching the current filter

@Mets most of those are interesting use cases. I would recommend you reaching out to the animation & rigging module to see if they can implement support for bones in the spreadsheet. The use case I showed was related to weight painting mostly.

Keep in mind that the "finding all children" probably belongs to the outliner though. I don't think it belongs here. All the other use cases are solid.

> Find bones that have the Deform checkbox enabled when they shouldn't > Find bones that are set to non-selectable > Find bones with a certain rigify type > Find all children of a bone (aka all bones with the same parent) > Select all (bones)things matching the current filter @Mets most of those are interesting use cases. I would recommend you reaching out to the animation & rigging module to see if they can implement support for bones in the spreadsheet. The use case I showed was related to weight painting mostly. Keep in mind that the "finding all children" probably belongs to the outliner though. I don't think it belongs here. All the other use cases are solid.

Paul Kotelevets commented

2021-03-16 13:18:53 +01:00

Well, yes, exploring blend file content better fits spreadsheet format than custom hierarchical structures, such as bones.
File content data structure is way more predictable.

Well, yes, exploring blend file content better fits spreadsheet format than custom hierarchical structures, such as bones. File content data structure is way more predictable.

Sybren A. Stüvel commented

2021-03-16 14:33:46 +01:00

Removed subscriber: @dr.sybren

Chris Kohl commented

2021-06-03 11:41:51 +02:00

Added subscriber: @ckohl_art

jeremy bepoix commented

2022-04-22 09:36:55 +02:00

Added subscriber: @jeremybep

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Spreadsheet Editor #86279