2D Vector Socket Type #92765

New Issue

Hans Goudey · 2021-11-02T23:57:51+01:00

Hans Goudey commented

2021-11-02 23:57:51 +01:00

Many nodes, and UVs are more correct and intuitive with 2D vector sockets.
Using a more specific data type can make node networks more intuitive, and faster.

Though changing the math nodes is a larger topic/design challenge, initially many nodes can be changed to support this socket type:

Attribute statistic
Capture attribute
Transfer attribute
Image texture
Random value
Compare node (#92729)

2D vector attributes will have to be supported in many places that they aren't currently as well.

Other texture nodes can be supported when shader nodes is updated to support 2D vectors.

Many nodes, and UVs are more correct and intuitive with 2D vector sockets. Using a more specific data type can make node networks more intuitive, and faster. Though changing the math nodes is a larger topic/design challenge, initially many nodes can be changed to support this socket type: - Attribute statistic - Capture attribute - Transfer attribute - Image texture - Random value - Compare node (#92729) 2D vector attributes will have to be supported in many places that they aren't currently as well. Other texture nodes can be supported when shader nodes is updated to support 2D vectors.

Hans Goudey commented

2021-11-02 23:57:51 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Hans Goudey commented

2021-11-02 23:57:51 +01:00

Added subscriber: @HooglyBoogly

Paul Larson commented

2021-11-03 01:38:52 +01:00

Added subscriber: @GeorgiaPacific

Yuro commented

2021-11-03 22:19:32 +01:00

Added subscriber: @Yuro

Miro Horváth commented

2021-11-08 15:27:41 +01:00

Added subscriber: @MiroHorvath

Charlie Jolly commented

2021-11-08 23:11:55 +01:00

Added subscriber: @CharlieJolly

Charlie Jolly commented

2021-11-08 23:11:55 +01:00

Textures like noise already provide 2D/3D data using a vector socket. This has advantages for the user so that when changing mode this does not change the links. I think 2D sockets could be implemented as 3D/vector sockets but with a 2D UI. This UI flag would hide the Z channel in the user interface. This would also make implementing dynamic nodes easier as only the socket UI flag would need to be set to show that the socket is processing 2D data. Internally the node will take the XY channels of any connected 3D vector, This also raises the idea of implementing this as a dynamic data socket with support for 4D data too. This could potentially make it easier to implement overloaded functions in nodes. Currently there is a lot of boilerplate to handle sockets that could be avoided.

Hans Goudey commented

2021-11-09 18:06:46 +01:00

In #92765#1250660, @CharlieJolly wrote:
This has advantages for the user so that when changing mode this does not change the links.

I think this is a separate issue honestly. Reusing links for sockets when changing types could be improved, even without this new socket type.

I think 2D sockets could be implemented as 3D/vector sockets but with a 2D UI. This UI flag would hide the Z channel in the user interface. This would also make implementing dynamic nodes easier as only the socket UI flag would need to be set to show that the socket is processing 2D data. Internally the node will take the XY channels of any connected 3D vector, This also raises the idea of implementing this as a dynamic data socket with support for 4D data too. This could potentially make it easier to implement overloaded functions in nodes. Currently there is a lot of boilerplate to handle sockets that could be avoided.

I agree the amount of boilerplate code should be improved, but I don't agree that the best way to do that is to actually pass around 3D vectors. This makes code more complicated for a few reasons:

Type inference becomes more complicated, because a 3D vector socket also has a length to pass around.
Areas that interact with node sockets also have to consider this, instead of just supporting another socket type.
Correspondence between socket types and the stored data is no longer simple.

Better ways to reduce boilerplate code, in my opinion:

Templated functions that can work on vectors of multiple lengths.
Dynamic type sockets in node declarations.
Improvements to multi-function signatures and API (less clear on the details here, but I think some improvements would be possible).

The part of your argument I find most convincing is that we could support arbitrary length (2,3,4,etc.) vectors somehow. I think in practice this would be more complicated though.

> In #92765#1250660, @CharlieJolly wrote: > This has advantages for the user so that when changing mode this does not change the links. I think this is a separate issue honestly. Reusing links for sockets when changing types could be improved, even without this new socket type. > I think 2D sockets could be implemented as 3D/vector sockets but with a 2D UI. This UI flag would hide the Z channel in the user interface. This would also make implementing dynamic nodes easier as only the socket UI flag would need to be set to show that the socket is processing 2D data. Internally the node will take the XY channels of any connected 3D vector, This also raises the idea of implementing this as a dynamic data socket with support for 4D data too. This could potentially make it easier to implement overloaded functions in nodes. Currently there is a lot of boilerplate to handle sockets that could be avoided. I agree the amount of boilerplate code should be improved, but I don't agree that the best way to do that is to actually pass around 3D vectors. This makes code more complicated for a few reasons: - Type inference becomes more complicated, because a 3D vector socket also has a length to pass around. - Areas that interact with node sockets also have to consider this, instead of just supporting another socket type. - Correspondence between socket types and the stored data is no longer simple. Better ways to reduce boilerplate code, in my opinion: - Templated functions that can work on vectors of multiple lengths. - Dynamic type sockets in node declarations. - Improvements to multi-function signatures and API (less clear on the details here, but I think some improvements would be possible). The part of your argument I find most convincing is that we could support arbitrary length (2,3,4,etc.) vectors somehow. I think in practice this would be more complicated though.

Charlie Jolly commented

2021-11-11 12:27:12 +01:00

Adding a 2D socket is straight forward as it can basically copy the existing 3D socket.

It would be beneficial to do this in parallel with a reference implementation porting a node such as the Vector Math Node for Cycles/OSL/Eevee/GN to explore boilerplate and UX. E.g. now with coloured wires, type is much more visible, should there be an Auto data type mode? Some of this was implemented in the legacy Attribute nodes but not in the shader node implementations.

Adding a 2D socket is straight forward as it can basically copy the existing 3D socket. It would be beneficial to do this in parallel with a reference implementation porting a node such as the Vector Math Node for Cycles/OSL/Eevee/GN to explore boilerplate and UX. E.g. now with coloured wires, type is much more visible, should there be an Auto data type mode? Some of this was implemented in the legacy Attribute nodes but not in the shader node implementations.

Hans Goudey commented

2021-11-11 21:23:42 +01:00

I think it's okay as a first step to add the socket type and use it in some geometry nodes.
Supporting 2D vectors in shader nodes is a separate project that should also be done, but I think it increases the scope of this task a little too much.

We should support type inferencing in the future (automatically changing node types when connecting them to different sockets. I think we should design with that feature in mind.

I think it's okay as a first step to add the socket type and use it in some geometry nodes. Supporting 2D vectors in shader nodes is a separate project that should also be done, but I think it increases the scope of this task a little too much. We should support type inferencing in the future (automatically changing node types when connecting them to different sockets. I think we should design with that feature in mind.

Charlie Jolly commented

2021-11-11 23:22:32 +01:00

In #92765#1252587, @HooglyBoogly wrote:
We should support type inferencing in the future (automatically changing node types when connecting them to different sockets. I think we should design with that feature in mind.

Yes totally!

> In #92765#1252587, @HooglyBoogly wrote: > We should support type inferencing in the future (automatically changing node types when connecting them to different sockets. I think we should design with that feature in mind. Yes totally!

Vladislav Macíček commented

2021-11-15 17:45:34 +01:00

Added subscriber: @Aeraglyx

Brecht Van Lommel commented

2022-02-03 16:07:12 +01:00

Added subscriber: @brecht

Brecht Van Lommel commented

2022-02-03 16:07:12 +01:00

I only saw this task late, but personally I'd really rather not do this. I don't know the amount of work this is for the geometry nodes team, but if we want to maintain consistency across node systems in Blender this will also lead to a bunch of work for shader and texture nodes, that I don't think is worth committing the time to.

I think in terms of usability this is a really minor issue, that is not worth adding a lot of complexity across Blender for.

I only saw this task late, but personally I'd really rather not do this. I don't know the amount of work this is for the geometry nodes team, but if we want to maintain consistency across node systems in Blender this will also lead to a bunch of work for shader and texture nodes, that I don't think is worth committing the time to. I think in terms of usability this is a really minor issue, that is not worth adding a lot of complexity across Blender for.

Hans Goudey commented

2022-02-04 18:12:57 +01:00

The amount of work for geometry nodes beyond this initial patch adding the socket type is very small. Almost all math is templated, so usually the most that's necessary is adding cases to switch statements, etc. This will get better when we have multi-type sockets implemented (#95448).
Code should be written so that this is easy, and indeed, where we've used templates and other concepts, it is. For texture nodes, since the implementation would be started from scratch, hopefully in a rather type-agnostic way, I don't think this would be a problem at all.

But I'm guessing you're really referring to the current state of the code, particularly in shader nodes. There I have less experience, so I definitely won't try to tell you "it's simple!".
However, at the cost of a bit of performance, 2D vectors could just be handled as 3D vectors internally. Then it's just a few UI changes if we want to use the socket type somewhere, which IMO are pretty trivial.

In terms of usability, it's not the most important thing, but I think there are many places it acts as a teaching tool, saves effort, and simplifies node<->attribute exchange.
Generally I've noticed that this "wrong data type" has more of a UX impact than I expected, that's only anecdotal though. Some areas that it would help:

The image texture node input becomes clearer, helpful as a teaching tool.
Maybe most importantly, we can still have a near 1-1 mapping between node types and socket types. Currently when writing to a 2D vector attribute, the type is changed to 3D, which can make working with UVs harder
- Supporting 2D vectors properly avoids having to add a "write 3D as 2D" option, which would complicate code and UX.
Any node that operates on or creates UVs would become clearer, easier to understand immediately.

Also, we're planning on adding more data types in the future anyway: rotations (#92967), and we've even discussed a transform data type, which may be especially useful for things like rigging nodes and nested instances.
So if the aim is consistency across editors, code will have to become more type agnostic anyway. Compared to those types, 2D vectors are trivial and can be easily emulated by 3D vectors internally if necessary.

We've also been having discussions about generalizing math nodes to work on more than just floats, see #95448, and #94899. Adding 2D vectors works well with those designs.

The amount of work for geometry nodes beyond this initial patch adding the socket type is very small. Almost all math is templated, so usually the most that's necessary is adding cases to switch statements, etc. This will get better when we have multi-type sockets implemented (#95448). Code should be written so that this is easy, and indeed, where we've used templates and other concepts, it is. For texture nodes, since the implementation would be started from scratch, hopefully in a rather type-agnostic way, I don't think this would be a problem at all. But I'm guessing you're really referring to the *current state* of the code, particularly in shader nodes. There I have less experience, so I definitely won't try to tell you "it's simple!". However, at the cost of a bit of performance, 2D vectors could just be handled as 3D vectors internally. Then it's just a few UI changes if we want to use the socket type somewhere, which IMO are pretty trivial. In terms of usability, it's not the most important thing, but I think there are many places it acts as a teaching tool, saves effort, and simplifies node<->attribute exchange. Generally I've noticed that this "wrong data type" has more of a UX impact than I expected, that's only anecdotal though. Some areas that it would help: - The image texture node input becomes clearer, helpful as a teaching tool. - Maybe most importantly, we can still have a near 1-1 mapping between node types and socket types. Currently when writing to a 2D vector attribute, the type is changed to 3D, which can make working with UVs harder - Supporting 2D vectors properly avoids having to add a "write 3D as 2D" option, which would complicate code and UX. - Any node that operates on or creates UVs would become clearer, easier to understand immediately. Also, we're planning on adding more data types in the future anyway: rotations (#92967), and we've even discussed a transform data type, which may be especially useful for things like rigging nodes and nested instances. So if the aim is consistency across editors, code will have to become more type agnostic anyway. Compared to those types, 2D vectors are trivial and can be easily emulated by 3D vectors internally if necessary. We've also been having discussions about generalizing math nodes to work on more than just floats, see #95448, and #94899. Adding 2D vectors works well with those designs.

Charlie Jolly commented

2022-02-04 19:28:50 +01:00

After working on adding vector support to the Map Range shader nodes I can see the benefits of having 2D as a subtype of a vector socket rather than a completely separate socket type. This flag can be used in the UI to show less channels and by the back end to determine which code to call. This has benefits of less sockets and node links to manage.

Hans Goudey commented

2022-02-04 19:34:31 +01:00

the benefits of having 2D as a subtype of a vector socket rather than a completely separate socket type.

This seems more complicated to me honestly. In one case you're extending an existing concept-- socket types-- to a add a new value. In the other hand you're adding something new, and instead of just checking the socket type, code that wants to use 2D vectors now has to check two things.
Also, the changes to map range was adding support for vectors entirely, which seems different than adding supporting vectors with a smaller size.

This has benefits of less sockets and node links to manage.

Can you talk more about this? Do you mean adding the 2D socket type to the declaration? This should be handled relatively soon by allowing declaring multi-type sockets in declarations.
And what do you mean by "fewer links"? You mean how the old links stay around when you change the type? That's also a similar problem, and unrelated to this task IMO.

> the benefits of having 2D as a subtype of a vector socket rather than a completely separate socket type. This seems *more* complicated to me honestly. In one case you're extending an existing concept-- socket types-- to a add a new value. In the other hand you're adding something new, and instead of just checking the socket type, code that wants to use 2D vectors now has to check two things. Also, the changes to map range was adding support for vectors entirely, which seems different than adding supporting vectors with a smaller size. > This has benefits of less sockets and node links to manage. Can you talk more about this? Do you mean adding the 2D socket type to the declaration? This should be handled relatively soon by allowing declaring multi-type sockets in declarations. And what do you mean by "fewer links"? You mean how the old links stay around when you change the type? That's also a similar problem, and unrelated to this task IMO.

Brecht Van Lommel commented

2022-02-04 20:24:14 +01:00

For texture nodes, I think we would want to share as many nodes as possible with shader nodes, as well as the GPU implementation with Eevee shader nodes. So in that sense they are not being written from scratch.

Handling 2D as 3D consistently in shading requires more than UI changes. Nodes may fill in the 3rd component with a non-zero value (e.g. noise texture) or casting 3D to 2D may not set the 3rd component to zero, and then an operation like vector length gives a different result. I guess it requires automatically inserting a bunch of nodes that set the 3rd component to zero.

The image texture node in shading accepts 3D coordinates depending on the chosen projection. It's common in shading to map UV coordinates into 3D in some way, and this seems less convenient if you need to explicitly convert it to a 3D vector first before using e.g. vector math or mapping nodes to map it into 3D. In general users would now have to do explicit casting if type inference starts automatically adapting e.g. math nodes to 2D or 3D. This kind of inferencing can have unexpected consequences if it even affects nodes inside of a group for example and changes a bunch of math nodes from 2D to 3D, though I don't know if that's the plan.

For UV maps, I also wonder if users don't already have to specify that they want to output a UV map? Depending on the use case it may not make a difference, but for the UI in the UV editor, exporting to various file formats, you still need to know what is a UV coordinate or some other arbitrary 2D float attribute.

An argument in favor of 2D vectors is that MaterialX has them also, although I don't like that design.

For texture nodes, I think we would want to share as many nodes as possible with shader nodes, as well as the GPU implementation with Eevee shader nodes. So in that sense they are not being written from scratch. Handling 2D as 3D consistently in shading requires more than UI changes. Nodes may fill in the 3rd component with a non-zero value (e.g. noise texture) or casting 3D to 2D may not set the 3rd component to zero, and then an operation like vector length gives a different result. I guess it requires automatically inserting a bunch of nodes that set the 3rd component to zero. The image texture node in shading accepts 3D coordinates depending on the chosen projection. It's common in shading to map UV coordinates into 3D in some way, and this seems less convenient if you need to explicitly convert it to a 3D vector first before using e.g. vector math or mapping nodes to map it into 3D. In general users would now have to do explicit casting if type inference starts automatically adapting e.g. math nodes to 2D or 3D. This kind of inferencing can have unexpected consequences if it even affects nodes inside of a group for example and changes a bunch of math nodes from 2D to 3D, though I don't know if that's the plan. For UV maps, I also wonder if users don't already have to specify that they want to output a UV map? Depending on the use case it may not make a difference, but for the UI in the UV editor, exporting to various file formats, you still need to know what is a UV coordinate or some other arbitrary 2D float attribute. An argument in favor of 2D vectors is that MaterialX has them also, although I don't like that design.

Charlie Jolly commented

2022-02-04 20:40:25 +01:00

@HooglyBoogly I'm not clear on the difference between declaring multi-type sockets and actual 2D sockets. Maybe that's causing confusion on my part. For example, I'm thinking to support 2D in the map range node means adding 2D sockets in addition to the float and 3D vector sockets and the additional boilerplate code. For users, this would mean links and values would also change when switching mode etc. That's why I was suggesting 2D as a UI flag for vector sockets to solve those problems.