Page MenuHome

Update functions not working when changing socket values in Node bpy_struct
Closed, ResolvedPublic

Description

System Information
Operating system: Arch Linux
Graphics card: RTX 2080

Blender Version
Broken: (All, latest check: 2.80 beta, 2019-07-08 22:16)

Short description of error
The update callback is not being called when the values coming into the node's sockets are changed, be it directly or via other connected nodes. It gets called only when a new link is established/removed with the node's sockets. The socket value update doesn't react to anything.

Exact steps for others to reproduce the error
Having something like this in a script:

class MyNode (bpy.types.CompositorNodeCustomGroup):

    bl_name='Name'
    bl_label='Label'
    update = lambda _: print("I like turtles!")
    socket_value_update = lambda c: print("Hakuna matata!")
    ....

Event Timeline

Jacques Lucke (JacquesLucke) lowered the priority of this task from 90 to Low.Aug 7 2019, 1:51 PM

Looks like this function is not actually a callback currently. Maybe it was supposed to be called by the script, I don't know. I can't find any code in Blender that uses the function in the first place, so guess this is just a leftover...
Should probably be removed.

Hitokage added a comment.EditedAug 7 2019, 1:54 PM

Should probably be removed.

Thanks for the information. Well I would actually vote for making it work instead of removing. It is related to this problem. This is the functionality that is missing and fixing that would allow us to create fully usable custom compositor nodes add-ons.

I see, unfortunately I never created new nodes for existing node systems, so I'm not sure if there are other ways to solve this, probably not...
This has low priority at the moment. I do not know how hard it is to fix this. A fix of this would probably be part of some larger feature like D2917.

I see, I would really appreciate this feature, it is quite crucial cause this seems to be the only way to make the socket value update work when working with custom nodes. Without it custom nodes lack an important ability to be the part of the compositor nodetree. Thanks for the cooperation.

I also vote for this feature. I'm designing an addon, and I prefer to have a clean separation between model and view (think MVC), where the model should work without Blender and includes a graph, so I want to represent it with Blender nodes but these are only a facade. What I want to achieve is whenever the user changes a value of a node (internal property, or input socket value, or input socket connection), the model is notified, and could lead to some computations. However it is currently impossible to have this notification. My fallback relies on the application handler depsgraph_update_pre, and this isn't great.

My fallback relies on the application handler depsgraph_update_pre, and this isn't great.

Thanks for sharing this tip. It is a good workaround. Did you find any way to read the current socket value? I know we can use default_value property but it doesn't work when the socket is connected to another node. I thought of following the link to that node but the actual value might travel through the whole tree.

The trick comes from studying Sverchok source code :)

That's exactly the point of MVC. The graph is only a representation (V for View) of how your model is organized as a graph. What your model really is, is up to you, and Blender is agnostic of that. "Reading the socket value" depends on the model definition of what are nodes and what they do.

For instance, say you're making a graph of math operations. This example is so simple that it could be implemented only on the view side, but imagine a more complex use case. Whenever you instantiate a node on Blender (say node Add), you'd have the information (through the missing callback that this thread is about) that the user wants to create an Add node *on the model graph*. So in the callback (C for Controller) you'd add a node in the model. In the MVC theory, this model change triggers the update of the view, i.e. creates a node in Blender. Currently, Blender creates the node for you, which isn't great if there are some logics in your model (i.e. Blender only checks for compatible socket types, but you may want to add extra checks). Anyway. Whenever you connect two nodes on Blender, you'd affect connections on your model nodes.

Essentially, the model would be like:

class Node:
    def evaluate(self):
        raise NotImplementedError("!")

class BinaryNode(Node):
    def __init__(self):
        super()
        self.lhs_node = None
        self.rhs_node = None

class AddNode(BinaryNode):
    def evaluate(self):
        if not self.lhs_node or not self.rhs_node:
            raise MissingNodeConnectionError()
        return self.lhs_node.evaluate() + self.rhs_node.evaluate()

Note that there are no NodeLink in the model. Indeed, Blender needs a link representation (e.g. you can select it in the editor), but from the model point of view, there's no need for it, nodes simply have references to other nodes.

In my case, I want the graph to be exportable (JSON file I guess) and runnable both inside or outside Blender, so my model would essentially be a data-only graph, and there are two views, Blender's node editor and the runtime version graph. The downside is that for every node, there are three different classes, so it ends up making a lot of code.

Back to the thread. I'm claiming the above example is a stereotypical use case for Blender's node trees, and I'd like that Blender allows for it by giving callbacks on editor events. In order to keep the current behavior (e.g. creating a NodeLink when the user connected two sockets), the callbacks may be initialized with default functions which just do that, and therefore by setting custom callbacks, the current behavior won't happen by default anymore.

I see. Anyway I'd like to point out that with the socket update callback the current value of the socket (not just the default_value but also the one coming from the connected node) should be available in some way.

If you're talking about the manually set value for a non-connected input socket, it's surprisingly available as the socket instance's default_value (whose doc is Input value used for unconnected socket) for already available sockets for primitive types (NodeSocketFloat, NodeSocketVector, ...). Try that out: change a socket value in the editor, and display its default_value in Python (D.node_groups[...].nodes[...].inputs[...].default_value). For custom sockets, there are properties defined in the respective classes.

As I wrote in the comment above, I'd be interested in any kind of value, not just the non-connected one. I think that a unified interface for this would be helpful or a way to get the current value from link.

Then I already explained how to achieve that. Let's not flood this thread; if you still don't get it, contact me. It's good that Blender is in charge of the view only, the model is somewhat free. If you want to do without separating the model from the view, i.e. your node classes inherited from bpy.types.Node are also responsible for computing stuffs, then you need to traverse the graph by yourself, which will be actually more painful, both to implement and to maintain.

Yea the traversing idea is what I don't like actually :D. What you described feels far too complex for someone who just needs to add a new node type. It would be far more clean to be able to access the socket value directly from the basic API. It'd be more addon-friendly :D, focusing just on the particular functionality without the need to go outside your own node checking the graph state. But maybe I'm not seeing all the advantages and disadvantages of both approaches.

It may sound complex, but actually I believe MVC leads to a cleaner code, easier to implement and to maintain, and therefore saves time overall. It pays off even for only a few node types. You may be afraid if you never experienced it before... well if you only have a few node types, I can be a good opportunity to try, so that you'll grasp the concept faster, and you'll be ready for bigger projects later :)

I understand that MVC leads to a clean code but for some reason I think that programming a node should not cover stuff like analyzing the nodes outside and the whole graph. I'd think of node as a unit with interface (sockets and properties) and logic inside, just like a class in C++ for example. But yes in current API situation you are right about that being the best approach. Thanks for sharing that! It helped me a lot. :-)

Germano Cavalcante (mano-wii) changed the task status from Confirmed to Needs Information from User.Wed, Feb 12, 3:59 PM

I cannot reproduce this with either blender 2.80rc1 or current development versions of Blender through the Custom Nodes template.

Please try the latest daily build: https://builder.blender.org/download/

If the problem persists, please give us more clear instructions on how to reproduce it from scratch.

Hitokage added a comment.EditedThu, Feb 13, 9:58 AM

I cannot reproduce this with either blender 2.80rc1 or current development versions of Blender through the Custom Nodes template.

Adding an example script tested in 2.83 alpha:


Also included a commented workaround as suggested by @Jonathan Giroux (Koltes) which kinda works.
I think the problem probably lies deeper since python scripting allows us to work with the already created node tree model and for example the value coming to the socket via links from other sockets apparently cannot be obtained without traversing the connected nodes and tracing back the source of the value.

Thanks for the reply!

depsgraph_update is not a workaround, it is the correct solution.

The entire node graph should be thought of as something edited by the user, not what gets evaluated. The actual input value for evaluation, how data flows through the graph and gets evaluated is separate from that. Values may be animated or driven, the compositor may transform the node graph for optimization purposes, the graph may get evaluated on the GPU, etc.

We should either remove the update callback, or define it to be called on updates like linking/unlinking sockets. But for direct or indirect changes to input values, that is not going to happen.

I understand that, either way the two update callback functions in the API are confusing and as you said should be removed if not implemented or redefined. However the inability to get the updated socket value is a drawback when creating a custom node. Traversing the tree manually is quite annoying since many possible things can happen and the original value might be transformed along the way. It feels like the custom nodes via Python API functionality is still very limited.

Yes, the API is limited. There are basically only two supported ways to use it:

  • Define custom node groups, that are built from existing nodes
  • Create a completely separate node system with own evaluation

Everything else is unsupported basically.

We should either remove the update callback, or define it to be called on updates like linking/unlinking sockets.

If I understand correctly, this is already happening? That is what the original report said:

The update callback is not being called when .... It gets called only when a new link is established/removed with the node's sockets.

So you're essentially saying that the model should ask the view about values? I admit values can be animated so any model depends on Blender's animation features, or one would have to reproduce it to be independent of Blender.

BTW what is the animation model in Blender? If an animated value depends on an other (eg driver), how would Blender knows the order of evaluation? Especially if the driver is a complicated, scripted expression. I'm asking because I'm guessing this is a push model (hence order of evaluation), but it may be a pull model (in this case, order is meaningless).

Everything else is unsupported basically.

I see, that's a pity though :D. Just this option to access the actual values in connected sockets would solve many problems and basically would allow the developer to do anything with the input data just using Python script which breaks all the current limitations of the model. Please consider this. Blender is highly adjustable and extensible via scripting which I really love but this is really something that is a big obstacle when doing anything with nodes.

I've updated the descriptions in the API now.

It would be good to support writing custom nodes in existing node systems, and with the new function nodes system under development that will likely happen at some point.

But for now it's really not possible to have it working like this. Remember that a node may be used in multiple instances of a node group in the compositor node graph, or separately for each eye in a stereo render, or it may get executed millions of times on the GPU with different values.

and with the new function nodes system under development that will likely happen at some point.

That sounds interesting! Thanks for the feedback!

So you're essentially saying that the model should ask the view about values? I admit values can be animated so any model depends on Blender's animation features, or one would have to reproduce it to be independent of Blender.

The model here is the user-editable node graph data structure, and that can be viewed and edited in the node editor. But the evaluation is not part of MVC cycle. Think of it like creating a program binary from user-editable source code, and then running that an arbitrary number of times with different input data. The source code is not affected by running the program.

BTW what is the animation model in Blender? If an animated value depends on an other (eg driver), how would Blender knows the order of evaluation? Especially if the driver is a complicated, scripted expression. I'm asking because I'm guessing this is a push model (hence order of evaluation), but it may be a pull model (in this case, order is meaningless).

See here:
https://wiki.blender.org/wiki/Source/Depsgraph

I can't find any page related to a new node system, can you please give a link?

I agree on that. But then socket values which are not linked are constants or come from the animation system, right? I mean, you said a couple of minutes ago that the graph could be evaluated on the GPU, but how is it achievable if at every node we'd had to query CPU-side for values?

Thanks for the links.

The dependency graph ensure things are done in the right order. Animation is evaluated on the CPU, then the render engine or compositor is notified about updated values and re-evaluates things as necessary.

Ha! I think we're talking from the start about getting notified about updated values, but of course our conceptions differ from your knowledge.

So is there such a system? Or do you mean the graph is traversed on Blender side (at every pre-render), so that all values are collected, whatever they come from, then the result of this collection is sent to the GPU which is therefore agnostic of any animated value system?

The dependency graph is a complicated system, there are different types of nodes and different renderers that all work differently.

Unless you need this information to implement something specific, I'd rather not spend more time explaining this.

Indeed I'm on a specific project: a custom renderer :) So I'm interested in all info you can give.

I think these info would benefit to a lot of people. Currently my only source of info is studying Sverchok, and I'm even not sure if the code can be trusted at 100%. I understand that the whole system is complicated. But please, make at least an example about how we should use it. There's a template 'Python > Custom Nodes' which shows how to create the nodes, but doesn't say anything about evaluation. I'm willing to enhance this template with evaluation if you don't have time, but I'm still unconfident about the proper way to do it.

Please check the render engine API docs then:
https://docs.blender.org/api/master/bpy.types.RenderEngine.html

The material or node tree datablock will be marked as updated by the dependency graph, and then you can re-export it. There is no granular information about which nodes or sockets changed.

Ok so just to be sure to understand clearly: are you suggesting to build a runtime model based on the node graph, only at startup and later on when the datablock is updated, and then we can process this runtime model the way we want? And there's no other choice than to entirely rebuild the runtime model from zero, when the dadtablock is updated?