Page MenuHome

Cycles/Blender API redesign
Confirmed, NormalPublicDESIGN

Description

This design task is to propose and discuss ideas related to improvements to the Cycles API. Proposed changes will affect the Blender session and scene synchronization.

Problem

The current API to create a Cycles scene is too specific to the data types and exposes internal behaviors and class members to API users (clients hereafter). There exists a system in place, using the concept of Nodes and Sockets, to unify Cycles objects, and to generically manipulate them. The Nodes represent the objects (like a Mesh or a Light) and the Sockets represent their properties (like their vertices or intensity). However, it is not used throughout the API and clients have to directly manipulate certain properties in an unsafe manner. This direct manipulation has also two drawbacks related to scene updates. On one hand, we have to manually tag the objects or the system for an update; forgetting this can lead to missing objects or shaders in the render. On the other hand, there is no real way of telling what exactly has changed, so we may resort to making unnecessary work: simply modifying the vertex positions, without using displacement, should not free the device memory, but could simply copy the new positions in place (granularity of updates and fast scene rebuild shall be discussed in a separate task).

Ideally, the API should be simple, as generic as possible (i.e. to add the same shader to different nodes sharing a common property) while still allowing for strict typing (meaning we should not be able to set vertices on a camera if we know it is a Camera). Tagging for an update, and other bookkeeping behaviors if any, should become automatic. For this, we have to generalize the concept of Nodes and Sockets and use it throughout the public API. This will encapsulate internal behaviors for individual Nodes and the entire scene.

Abstraction

Every object accessible through the public API should become a Node, and public members of C++ classes in the current API, should be marked private.

Each node will have publicly accessible members exposed as sockets through setters/getters. Other member variables will be private and not be exposed as sockets.

Getters/Setters

Accessing sockets should be done via the various methods of the Node class. The different subclasses (Mesh, Curve, Light, etc.) could have more specific APIs to access and modify data.

For example, the vertices socket on the Mesh Node could be either accessed though Node::get_float3_array("vertices") or Mesh::get_vertices(). The specific APIs can get huge if done for every property, so it could either be constrained to only commonly used properties, or its generation could be automated somehow.

The reason to have Mesh::get_vertices() is for type safety, if the Cycles sockets change then there will be a compile error rather than a runtime error.

Updates

The X::tag_update(scene) will become automatic on socket (and attribute) changes. Each socket will have an associated bitflag, and when the socket value set function is called and the value is different, that bitflag will be set on the node.

Current logic in tag_update for indirectly tagging other nodes for update will be done later, as part of scene update.

Internally, some optimizations might be done to immediately also tag managers for update, or add updated nodes to a set so it is not required to loop over all nodes in scene update. This kind of optimization can be done as a second step.

D8644: Cycles: add update flags to Node and SocketType

Attributes

Geometry attributes are similar to sockets, but we prefer to keep them separate. This is to ensure there are no name collisions with sockets, and to support additional features and slightly different behavior where needed.

Attributes would use the same update flag mechanism. For performance reasons, comparing the old and new values detect if there was an actual changes would be optional. If the caller knows that the values has changed, it would be a waste of time to compare memory on a large array.

Further, we should support attributes taking ownership of array pointers rather than copying the array contents. For example for a system that already has vertex locations cached in memory for the entire animation, no expensive copy should be required.

Similarly for motion blur, we can have an efficient way to shift the motion blur steps, removing the first step and adding a new step at the end. For this each step would need to be stored in a separate array, rather than a single array with all time steps as is done now.

Creating Nodes and Ownership

Creating (and deleting) Nodes of various types should be done via an API (perhaps Scene::create_*_node()), instead of naively calling new and delete.

Nodes will optionally have an owner, to support T79174: Cycles Procedural API and faster scene update. This ensures that when Alembic or USD procedurals create nodes, e.g. the Blender exporter will not try to delete them. Rather they would be owned by a procedural node, and deleted along with it.

D8540: Cycles: introduce an ownership system to define if nodes should be removed from the scene.

Node Types

The available node types would be the same as the existing ones, plus a few additional ones to make the whole system more consistent.

Serialization

If the entire data structure is node based, serializing entire scenes becomes possible. However defining our own supported file format is not a goal, and we will likely just remove the XML support for now. Instead we should document the API well with examples.

Example

Here's an (incomplete) example on how rendering a simple scene would look like. The idea is to render something with the minimum amount of setup. This is using an approach where data is accessed using a specific API for each Node Type. A more complete example for the Mesh Node can be made.

int main()
{
    // data for a unit cube
    auto vertices_array = ccl::array<ccl::float3>({
            make_float3(-1.0f, -1.0f, -1.0f),
            make_float3( 1.0f, -1.0f, -1.0f),
            make_float3( 1.0f,  1.0f, -1.0f),
            make_float3(-1.0f,  1.0f, -1.0f),
            make_float3(-1.0f, -1.0f,  1.0f),
            make_float3( 1.0f, -1.0f,  1.0f),
            make_float3( 1.0f,  1.0f,  1.0f),
            make_float3(-1.0f,  1.0f,  1.0f)
    });

    auto triangles_array = ccl::array<ccl::int3>({ 
           make_int3(4, 0, 3),
           make_int3(4, 7, 3),
           make_int3(1, 5, 6),
           make_int3(1, 2, 6),
           make_int3(4, 5, 1),
           make_int3(4, 0, 1),
           make_int3(3, 2, 6),
           make_int3(3, 7, 6),
           make_int3(0, 1, 2),
           make_int3(0, 3, 2),
           make_int3(5, 4, 7),
           make_int3(5, 6, 7)
    });

    auto normals_array = ccl::array<ccl::float3>({ 
        make_float3(-1.0f,  0.0f,  0.0f),
        make_float3( 1.0f,  0.0f,  0.0f),
        make_float3( 0.0f, -1.0f,  0.0f),
        make_float3( 0.0f,  1.0f,  0.0f),
        make_float3( 0.0f,  0.0f, -1.0f),
        make_float3( 0.0f,  0.0f,  1.0f)
    });

    // set up the render session
    auto session = ccl::create_session()
    auto scene = session.get_scene();

    // create a mesh
    auto mesh = scene->create_mesh_node("my_mesh_name");
#if 0 // case for a generic API using the Node's methods
    mesh->set_float3_array("vertices", vertices_array);
    mesh->set_in3_array("triangles", triangles_array);
    mesh->set_float3_array("face_normals", normals_array);
#else // case for a specific Mesh API
    mesh->set_vertices(vertices_array);
    mesh->set_triangles(triangles_array);
    mesh->set_face_normals(normals_array);
#endif

    // create a shader
    auto shader = scene->create_shader_node("my_shader_name");
    // ... setup the shader network, can have scene->get_default_diffuse_shader();, etc.

    // add the shader to the list of shaders used by the mesh
    // we could tell which triangles this shader affects, by default we assume the shader affects every triangles
    mesh->shaders().add(shader);

    auto camera = scene->create_camera_node("my_camera_name");
    // ... setup the camera data

    // create device for the render
    auto device = scene->create_device_node();
    // .... setup the device data (requested feature, number of threads, etc.), can have multiple devices

    // do the render
    session.render();
}

Event Timeline

First of all I love the approach of starting from a hypothetical client using the API.

auto triangles_array = ccl::array<ccl::float3>({ 
       make_int3(4, 0, 3),

This seems somewhat counter intuitive, perhaps ccl::array<ccl::int3> was meant here?

auto device = scene->create_device_node();

This bit will need a bit of work, as the example is currently structured this would probably be more intuitive in session rather than scene , scene ideally is a scene description, it ought to not concern it self with hardware enumeration/configuration.

This seems somewhat counter intuitive, perhaps ccl::array<ccl::int3> was meant here?

Ah yes, it's a typo.

auto device = scene->create_device_node();

This bit will need a bit of work, as the example is currently structured this would probably be more intuitive in session rather than scene , scene ideally is a scene description, it ought to not concern it self with hardware enumeration/configuration.

I agree, the idea is to make everything a node and to centralize a bit node creation, since most nodes are in the scene it made sense to have the scene create all of them. The Scene already contains a pointer to the Device, however it does not create it.

Hi Guys. My first time here on the forum.

I would like to put in a vote to keep xml parsing compiling if it's not causing specific problems.

I am currently porting the latest cycles into a new version of Poser and have been fleshing out the xml for our use. I could share back to the community any parts of it that would be commonly useful if you keep the xml files in place.

Brecht Van Lommel (brecht) changed the task status from Needs Triage to Confirmed.Oct 28 2020, 2:57 PM

So @Kévin Dietrich (kevindietrich) has done a bunch of work to get us closer to the code sample in the original post, but we're not there yet.

We still have the concept of SceneParams, SessionParams which are not exactly like nodes. The Session needs to be recreated when adding devices or enabled denoising. And it's not obvious when you can edit a Session's scene, how to mutex luck, when to reset, etc. There's still a bunch of tag_update calls that need to be made when changing certain data. And for denoising the user needs to manually sync some data between various Cycles data structures.

Ideally we can get towards a state where the scene and it's nodes can just be updated, and then the Session can figure out what to do in response to that. Restart the render if needed, recreate devices, etc.

Following the above example code, dynamically updating the scene in an interactive session should ideally be something like this.

// Acquire the scene for editing, pausing any render operations that might be accessing the scene data structures.
// The scene pointer is only available here, and should not be stored permanently.
Scene& scene = session.begin_scene_edit();

// Make arbitrary edits to the scene. There is no tag_update() here.

// Session responds to scene changes.
// - Flushing update tags between nodes
// - (Re)creating devices
// - Scene device update
// - Restarting the render from sample 0
// 
// These may also be deferred to a running render thread if there is one, or until one is created.
session.end_scene_edit(scene);

I did some work towards this in a branch, and will post some WIP patches soon. Adding the basic begin/end scene edit mechanism, eliminating the need to call tag_update(), and did some work towards making denoising and passes be handled more automatically.

However there's still some significant roadblocks towards getting it this simple, and I'm not sure it's entirely practical. For example recreating devices (also for denoising) or changing the shading system between SVM/OSL seems impractical. That's because they persistently modify scene data and do not keep around the original for memory efficiency and ease of implementation. So I think there are a few high-level parameters that we may not allow changing dynamically, and that will continue to require recreating the session in an interactive session. But we should be able to minimize and document them.