Improve Cycles standalone #38279

New Issue

Thomas Dinges · 2014-01-18T20:22:55+01:00

Thomas Dinges commented

2014-01-18 20:22:55 +01:00

Hi,
I create this task here so we can talk about a design/implementation for a decent API and improvements for Cycles standalone.

Image: The Cycles standalone app
Standalone code: https://developer.blender.org/diffusion/C/

Current API
At the moment we have a basic xml api, which was only intended for testing.
Examples can be found here: https://developer.blender.org/diffusion/C/browse/master/examples/

New API
I think we need an api which is easier to read/write, while still staying simple and manually editable.
We'd like to get feedback from people, who are interested in integrating Cycles into other applications.

Reasons to make this happen
I still see a great potential in the standalone, and as most code is already there, it shouldn't be too hard to make this usable.
Benefits:

Cycles would become usable for other 3D applications, and therefore we could also get more potential developers.
Even for Blender, it would help as benchmark and testing tool for the engine itself. We could have a automated test suite then.

Hi, I create this task here so we can talk about a design/implementation for a decent API and improvements for Cycles standalone. Image: [The Cycles standalone app ](http://blender.dingto.org/cycles_standalone/standalone.png) Standalone code: https://developer.blender.org/diffusion/C/ **Current API** At the moment we have a basic xml api, which was only intended for testing. Examples can be found here: https://developer.blender.org/diffusion/C/browse/master/examples/ **New API** I think we need an api which is easier to read/write, while still staying simple and manually editable. We'd like to get feedback from people, who are interested in integrating Cycles into other applications. **Reasons to make this happen** I still see a great potential in the standalone, and as most code is already there, it shouldn't be too hard to make this usable. Benefits: - Cycles would become usable for other 3D applications, and therefore we could also get more potential developers. - Even for Blender, it would help as benchmark and testing tool for the engine itself. We could have a automated test suite then.

Thomas Dinges commented

2014-01-18 20:22:55 +01:00

Changed status to: 'Open'

Thomas Dinges commented

2014-01-18 20:22:55 +01:00

Added subscribers: @ThomasDinges, @brecht, @MartijnBerger, @MatthewHeimlich

andreas atteneder commented

2014-01-22 16:13:43 +01:00

Added subscriber: @andreas.atteneder

Mert commented

2014-01-23 21:00:10 +01:00

Added subscriber: @mrt3d

Martijn Berger commented

2014-01-25 21:10:04 +01:00

To me there are 2 distinctly different but related things.

The API as in the c/c++ calls I have to make to get cycles to do anything

For this I would like the following

at least python bindings so we can test things easy / fast
ability to build a libcycles.dll/so/dylib

The scene description file format. This should be as close to the API as possible.

For this I would like the following:

ASCII/UTF-8 and binary representation that are both interchangeable and convertible both ways
the text variant should be somewhat understandable / hand editable

I think we can rip out the XML and replace it with a bison / flex based parser like luxrender currently has.
And add support for loading obj and ply meshes.

To me there are 2 distinctly different but related things. The API as in the c/c++ calls I have to make to get cycles to do anything For this I would like the following - at least python bindings so we can test things easy / fast - ability to build a libcycles.dll/so/dylib The scene description file format. This should be as close to the API as possible. For this I would like the following: - ASCII/UTF-8 and binary representation that are both interchangeable and convertible both ways - the text variant should be somewhat understandable / hand editable I think we can rip out the XML and replace it with a bison / flex based parser like luxrender currently has. And add support for loading obj and ply meshes.

Brecht Van Lommel commented

2014-01-26 13:31:54 +01:00

That all seems reasonable. I'm not sure if we need bison / flex based parser though, guess it depends on the file format. I wouldn't use a RIB like file format, it's not a great fit semantically and I don't see much point using the syntax. I'll paste what I wrote on the mailing list some time ago here:

FILE SYNTAX

XML and RIB syntax are well known. JSON is perhaps more readable and less verbose due to not requiring start+end elements, but also seems a bit out of place. XML would be easiest as we already have a parser from OpenImageIO, though perhaps it's not particularly hip to use XML :)

It's mostly a matter of taste I guess, maybe someone can come up with a good argument for one choice or the other.

SEMANTICS

We do not need an attribute stacks like Renderman does. That's nice if you really want to construct more complex scenes with just a text editor, but I don't think that's our target. I rather have a flat list of data with a unique ID that can reference each other, so instancing is natural and not conditional on the attribute stack state. The list of data types I imagine would be:

output settings
integrator settings
camera
geometry (mesh, curves, .. in external file)
object (either a geometry instance or parent object with child objects)
material
light
shader

We should make it so that we can have RIB like archives that we can delay loading once we support that in the core. We have to be a bit careful here in defining the semantics, so we don't just assume the whole archive is included in place.

GEOMETRY

Geometry does not need to be in the scene file format, we can reference external files in some standard format(s) instead. Alembic is quite nice for this, it supports an object hierarchy and is designed for efficient loading.

Main problem is that it is a big dependency, and we don't have Blender export here yet? Perhaps basic .obj and later alembic support is a reasonable compromise.

SHADERS

Shaders are node based in Cycles and OSL. You would have 3 types of shaders that all look the same from the outside, as a node with inputs and outputs:

Builtin node
OSL shader
Shader group

A material or light would reference such a shader and only have a few extra properties, most of its behavior is defined by the shader. A shader group is like a Blender node group and basically a more general type of shader instance as in Renderman. If we support archives this could be stored in a separate file but use the same syntax as the scene file format.

That all seems reasonable. I'm not sure if we need bison / flex based parser though, guess it depends on the file format. I wouldn't use a RIB like file format, it's not a great fit semantically and I don't see much point using the syntax. I'll paste what I wrote on the mailing list some time ago here: **FILE SYNTAX** XML and RIB syntax are well known. JSON is perhaps more readable and less verbose due to not requiring start+end elements, but also seems a bit out of place. XML would be easiest as we already have a parser from OpenImageIO, though perhaps it's not particularly hip to use XML :) It's mostly a matter of taste I guess, maybe someone can come up with a good argument for one choice or the other. **SEMANTICS** We do not need an attribute stacks like Renderman does. That's nice if you really want to construct more complex scenes with just a text editor, but I don't think that's our target. I rather have a flat list of data with a unique ID that can reference each other, so instancing is natural and not conditional on the attribute stack state. The list of data types I imagine would be: * output settings * integrator settings * camera * geometry (mesh, curves, .. in external file) * object (either a geometry instance or parent object with child objects) * material * light * shader We should make it so that we can have RIB like archives that we can delay loading once we support that in the core. We have to be a bit careful here in defining the semantics, so we don't just assume the whole archive is included in place. **GEOMETRY** Geometry does not need to be in the scene file format, we can reference external files in some standard format(s) instead. Alembic is quite nice for this, it supports an object hierarchy and is designed for efficient loading. Main problem is that it is a big dependency, and we don't have Blender export here yet? Perhaps basic .obj and later alembic support is a reasonable compromise. **SHADERS** Shaders are node based in Cycles and OSL. You would have 3 types of shaders that all look the same from the outside, as a node with inputs and outputs: * Builtin node * OSL shader * Shader group A material or light would reference such a shader and only have a few extra properties, most of its behavior is defined by the shader. A shader group is like a Blender node group and basically a more general type of shader instance as in Renderman. If we support archives this could be stored in a separate file but use the same syntax as the scene file format.

andreas atteneder commented

2014-01-26 15:43:37 +01:00

Allow me to point out an additional aspect:

One of Cycles benefits is instant feedback. Noisy, but good for quick iteration. That's what makes it interesting for some cloud use cases.

For example take a look at what Lagoa is doing ( http://home.lagoa.com/ go for "Try Lagoa" ).

So apart from the classic create scene->render->finished workflow it would be great to be able to:

be able to change the scene during render (like the camera position)
have callbacks, that inform you about the status.
have access to the current renderbuffers

Well, it comes down to being able to build an interactive rendered view, just like Blender has it and outsource the actual render to servers.

I know that some of those things may already exist. I created a prototype last week, that renders one of dingto's demo XMLs and saves away a JPG every 100 samples. I had troubles though since I found no documentation.

I just want to point out these use cases, so they may be considered in Cycles standalone API.

thanks

Allow me to point out an additional aspect: One of Cycles benefits is instant feedback. Noisy, but good for quick iteration. That's what makes it interesting for some cloud use cases. For example take a look at what Lagoa is doing ( http://home.lagoa.com/ go for "Try Lagoa" ). So apart from the classic create scene->render->finished workflow it would be great to be able to: - be able to change the scene during render (like the camera position) - have callbacks, that inform you about the status. - have access to the current renderbuffers Well, it comes down to being able to build an interactive rendered view, just like Blender has it and outsource the actual render to servers. I know that some of those things may already exist. I created a prototype last week, that renders one of dingto's demo XMLs and saves away a JPG every 100 samples. I had troubles though since I found no documentation. I just want to point out these use cases, so they may be considered in Cycles standalone API. thanks

Brecht Van Lommel commented

2014-01-26 16:19:14 +01:00

Yes, that's a good point, if we make an API it should support incremental updates. Most of it is there in the Blender integration, just needs to be wrapped a bit nicer.

Tuft commented

2014-01-29 11:33:33 +01:00

(Migrated from localhost:3001)

Added subscriber: @Tuft

Tuft commented

2014-01-29 11:37:30 +01:00

(Migrated from localhost:3001)

Hey guys,

I'm using 3DS Max because of school and work, but I miss that badass Blender, so I'd be very pleased to provide any help for a Cycles integration with 3DS Max. I'll watch the progress of the project and maybe try a basic export to Cycles maxscript if needed.

I know Max users would love Cycles.

Hey guys, I'm using 3DS Max because of school and work, but I miss that badass Blender, so I'd be very pleased to provide any help for a Cycles integration with 3DS Max. I'll watch the progress of the project and maybe try a basic export to Cycles maxscript if needed. I know Max users would love Cycles.

Florian Richter commented

2014-01-30 16:21:55 +01:00

Added subscriber: @FlorianRichter

Florian Richter commented

2014-01-30 16:25:59 +01:00

A Max to Cycles Standalone exporter would be possible for me, too.

At work I often use Maxscript, so adding an XML-exporter of standard materials and maybe geometry could be possible with enough time and if the standalone of Cycles gets better (more features and better documentation).

More interesting would be a real dlu-plugin to support animations and the real render pipeline in Max. :)

A Max to Cycles Standalone exporter would be possible for me, too. At work I often use Maxscript, so adding an XML-exporter of standard materials and maybe geometry could be possible with enough time and if the standalone of Cycles gets better (more features and better documentation). More interesting would be a real dlu-plugin to support animations and the real render pipeline in Max. :)

Florian Richter commented

2014-01-31 21:53:50 +01:00

To push the API discussion forward.
Creating a Max-XML-Exporter for the cycles scene in general should be possible and not that difficult (but of course time consuming). But before I start coding and then it gets changed, we should make sure if we use our own XML format or use a predefined format and improve the importer of the Standalone.
The main advantage of XML would be, that its easy to parse and understand.
Like brecht wrote, the differentiation between the scene in general and the objects might be good. But for example shaders, should they then be a part of the object file or the scene file. Theoretical it would be better if they were a part of the object, but then using the same material on more then one objects would be more difficult then needed. So:
If we use our own XML-format, it should be carefully and future proofed designed and be useful for future updates and extensions.

EDIT: what about animations?

To push the API discussion forward. Creating a Max-XML-Exporter for the cycles scene in general should be possible and not that difficult (but of course time consuming). But before I start coding and then it gets changed, we should make sure if we use our own XML format or use a predefined format and improve the importer of the Standalone. The main advantage of XML would be, that its easy to parse and understand. Like brecht wrote, the differentiation between the scene in general and the objects might be good. But for example shaders, should they then be a part of the object file or the scene file. Theoretical it would be better if they were a part of the object, but then using the same material on more then one objects would be more difficult then needed. So: If we use our own XML-format, it should be carefully and future proofed designed and be useful for future updates and extensions. EDIT: what about animations?

Thomas Dinges commented

2014-02-15 00:15:45 +01:00

Hi,
I worked on some updates the past few days:

XML:

Nodes are almost completely wrapped (missing are RGB Curves and Color Ramp...)
Lights (Point, Area, Spot, Sun) are supported now.

GUI:

The camera is now properly refreshed, when changing the window size or passing --width --height overwrites on startup.
Render can be paused with "P" key.
Added an "Interactive Mode", which can be enabled with the "I" key.
** Left/Right mouse can be used for Camera Move/Rotate, additionally W/A/S/D keys for camera movement.

So I think we have a pretty nice basis here now, when it comes to the GUI and XML.

ToDo:

API: Create a design proposal and implement it.
The current XML API has limitations and is not 100% complete yet. Stick to XML or use JSON is the main question still, I don't have a strong feeling here. Whatever we will decide on, we should then check on current limitations and consistency. Some things have been mentioned already in this thread.
Object Import: .obj or .abc (Alembic) would be nice here, I started looking into it, but not much progress yet.

Hi, I worked on some updates the past few days: **XML:** * Nodes are almost completely wrapped (missing are RGB Curves and Color Ramp...) * Lights (Point, Area, Spot, Sun) are supported now. **GUI:** * The camera is now properly refreshed, when changing the window size or passing --width --height overwrites on startup. * Render can be paused with "P" key. * Added an "Interactive Mode", which can be enabled with the "I" key. ** Left/Right mouse can be used for Camera Move/Rotate, additionally W/A/S/D keys for camera movement. So I think we have a pretty nice basis here now, when it comes to the GUI and XML. **ToDo:** * **API**: Create a design proposal and implement it. The current XML API has limitations and is not 100% complete yet. Stick to XML or use JSON is the main question still, I don't have a strong feeling here. Whatever we will decide on, we should then check on current limitations and consistency. Some things have been mentioned already in this thread. * Object Import: .obj or .abc (Alembic) would be nice here, I started looking into it, but not much progress yet.

Pedro A. commented

2014-02-17 17:22:47 +01:00

Added subscriber: @povmaniac

Pedro A. commented

2014-02-17 17:22:47 +01:00

Well .. this project moves. It seems quite attractive, not very complicated, and it is C++. :)

Kristian Sons commented

2014-03-17 10:58:07 +01:00

Added subscriber: @ksons

haggi krey commented

2014-03-29 23:49:37 +01:00

Added subscriber: @haggi

haggi krey commented

2014-03-29 23:49:37 +01:00

Hi,
this is very exciting. I'd like to implement cycles into maya as soon as possible. But it looks as if it will take some time before we have an API. And I fear my programming skills are far away from being helpful in creating the API itself. But as soon as you have some basic code, it would be a pleasure for me to implement and test it in maya.

Hi, this is very exciting. I'd like to implement cycles into maya as soon as possible. But it looks as if it will take some time before we have an API. And I fear my programming skills are far away from being helpful in creating the API itself. But as soon as you have some basic code, it would be a pleasure for me to implement and test it in maya.

luke frisken commented

2014-04-08 08:40:05 +02:00

Added subscriber: @kellpossible

luke frisken commented

2014-04-08 08:40:05 +02:00

Very nice! Seems like --output to image file doesn't work when --background is enabled, because there is no display I'm guessing from the code in session.cpp. Am trialing using cycles standalone for a product rendering application, this option would be very useful to us. Is it possible to have a display without actually displaying it? (to only use for writing out to a file).

Francis Pimenta commented

2014-04-13 19:53:06 +02:00

Added subscriber: @mandrake0

Francis Pimenta commented

2014-04-13 19:53:06 +02:00

hi,

i have got some questions about the API. is the definition about the Format fix? because i read some time ago a post from linus torvalds which i find a pretty interesting idea that i want to share.

He used for the diving Software a XML File because he didn't like it he used the git object database format and that i find is a interesting idea. with this concept you will get all the benefits from git itself history, diff's, .... what do you think would that be a option as a format for cycles render?

it's better explained in his post: Linus Torvalds post on google+

hi, i have got some questions about the API. is the definition about the Format fix? because i read some time ago a post from linus torvalds which i find a pretty interesting idea that i want to share. He used for the diving Software a XML File because he didn't like it he used the git object database format and that i find is a interesting idea. with this concept you will get all the benefits from git itself history, diff's, .... what do you think would that be a option as a format for cycles render? it's better explained in his post: [Linus Torvalds post on google+ ](https://plus.google.com/+LinusTorvalds/posts/X2XVf9Q7MfV)

Uwe Schmidt commented

2014-04-29 14:21:46 +02:00

Added subscriber: @Bantha

Erel Herzog commented

2014-05-05 20:17:32 +02:00

Added subscriber: @erel96-2

Artan Sadiku commented

2014-05-08 23:03:47 +02:00

Added subscriber: @sadikuartan

Uwe Schmidt commented

2014-05-20 14:35:02 +02:00

Hi,
the actual xml interface works quite well for defining materials. The main weakness atm seems to be the missing possibility to import geometry with uv mapping, the xml does not seem to support that.

A version with OBJ import would be great, I think the renderer would already be very usable once we can use proper geometry.

Hi, the actual xml interface works quite well for defining materials. The main weakness atm seems to be the missing possibility to import geometry with uv mapping, the xml does not seem to support that. A version with OBJ import would be great, I think the renderer would already be very usable once we can use proper geometry.

Francis Pimenta commented

2014-05-20 15:12:09 +02:00

hi bantha,

for me the question comes up how i should integrate the cycles shader nodes into the other dcc program mesh export is simple. there i'm not sure what's the best way (Cycles Nodes or OSL).
i have made shortly just a fun work to learn a bit python so i have made a exporter (bad code and very simple) for houdini.
if you want to try: Cycles render in houdini

hi bantha, for me the question comes up how i should integrate the cycles shader nodes into the other dcc program mesh export is simple. there i'm not sure what's the best way (Cycles Nodes or OSL). i have made shortly just a fun work to learn a bit python so i have made a exporter (bad code and very simple) for houdini. if you want to try: [Cycles render in houdini](http://www.sidefx.com/index.php?option=com_forum&Itemid=172&page=viewtopic&p=147592#147592)

Bastien Montagne commented

2014-05-20 16:12:17 +02:00

Added subscriber: @mont29

Fábio Santos commented

2014-07-04 19:58:19 +02:00

Added subscriber: @fabio-1

Fábio Santos commented

2014-07-04 19:58:19 +02:00

I'm currently improving the current blender > cycles standalone XML exporter and I hit a problem which I feel needs solving: the tag (inside the tag) currently identifies the nodes involved in the connection in a less-than-optimal way: "nodeName (SPACE character) socketName". This is problematic because node names will not be able to have spaces this way.

I'm going to change cycles_xml.cpp so as to change the format from:

    <connect from="nodeName socketName" to="node2Name socket2Name">

To:

  <connect from_node="nodeName" from_socket="socketName" to_node="node2Name" to_socket="socket2Name">

Unless anyone has a better idea or is against this.

I'm currently improving the current blender > cycles standalone XML exporter and I hit a problem which I feel needs solving: the <connect> tag (inside the <shader> tag) currently identifies the nodes involved in the connection in a less-than-optimal way: "nodeName (SPACE character) socketName". This is problematic because node names will not be able to have spaces this way. I'm going to change cycles_xml.cpp so as to change the format from: ``` <connect from="nodeName socketName" to="node2Name socket2Name"> ``` To: ``` <connect from_node="nodeName" from_socket="socketName" to_node="node2Name" to_socket="socket2Name"> ``` Unless anyone has a better idea or is against this.

Thomas Dinges commented

2014-07-04 20:02:04 +02:00

I am fine with the change, but let's wait a few days for possible further feedback.

Patrick Rosendahl commented

2014-07-05 23:09:10 +02:00

Added subscriber: @PRosendahl

Patrick Rosendahl commented

2014-07-05 23:09:10 +02:00

Much cleaner. So we can also get rid of the node's socket-name conversion where spaces are eliminated (ie "is camera ray" -> "iscameraray") in xml_socket_name().

Where is the blender-cycles standalone XML exporter located at?

Much cleaner. So we can also get rid of the node's socket-name conversion where spaces are eliminated (ie "is camera ray" -> "iscameraray") in xml_socket_name(). Where is the blender-cycles standalone XML exporter located at?

Fábio Santos commented

2014-07-06 23:03:19 +02:00

The blender-cycles standalone XML exporter is only capable of exporting meshes, so I'm enhancing it.

The original exporter is in intern/cycles/app/io_export_cycles_xml.py

My take on the exporter is currently here: https://github.com/fabiosantoscode/cycles-xml-exporter

The blender-cycles standalone XML exporter is only capable of exporting meshes, so I'm enhancing it. The original exporter is in intern/cycles/app/io_export_cycles_xml.py My take on the exporter is currently here: https://github.com/fabiosantoscode/cycles-xml-exporter

Patrick Rosendahl commented

2014-07-07 12:58:48 +02:00

Very nice! Here is my shot at materials export:
https://developer.blender.org/T40929
Will request closing that topic, so let's continue with your exporter.

Very nice! Here is my shot at materials export: https://developer.blender.org/T40929 Will request closing that topic, so let's continue with your exporter.

Patrick Rosendahl commented

2014-07-07 23:16:07 +02:00

Here is a patch that adds normals and embedded files (textures) to the XML:
https://developer.blender.org/T40773
Maybe that task can be closed/merged.

Here is a patch that adds normals and embedded files (textures) to the XML: https://developer.blender.org/T40773 Maybe that task can be closed/merged.

Fábio Santos commented

2014-07-08 00:23:56 +02:00

Really nice to know that shaders are somewhat taken care of already :) I'm trying to get Cycles working on the web browser, actually, through the CrowdProcess platform.

Patrick Rosendahl commented

2014-07-08 00:44:39 +02:00

I think it is very important to export/import materials, this will make people share stuff. Right now it is impossible to transfer materials from file->file.
People could paste simple text files to blendswap...

Step one is exporting. Textual representation comes in handy and can be recycled for XML-export. (Or the other way around ;-) ) That way we have one task for both purposes.

Step two would be importing the text back as nodes.

I'm not sure how to handle groups. It is a neat feature and must be preserved. However it is very versatile, not sure how it fits into rather "static" XML.

I think it is very important to export/import materials, this will make people share stuff. Right now it is impossible to transfer materials from file->file. People could paste simple text files to blendswap... Step one is exporting. Textual representation comes in handy and can be recycled for XML-export. (Or the other way around ;-) ) That way we have one task for both purposes. Step two would be importing the text back as nodes. I'm not sure how to handle groups. It is a neat feature and must be preserved. However it is very versatile, not sure how it fits into rather "static" XML.

Patrick Rosendahl commented

2014-07-09 14:21:31 +02:00

@fabio-1: pls contact me at BA, so I can help you w/ the cycles stuff: http://blenderartists.org/forum/member.php?225611-PRosendahl

imsky commented

2014-07-19 11:12:39 +02:00

(Migrated from localhost:3001)

Added subscriber: @imsky

Thomas Dinges commented

2014-09-15 18:09:18 +02:00

Hi everyone,
I have some news here. We want to switch from XML to a JSON format, which is easier to read and write in my opinion.

Are you guys ok with it? I don't want to break your work on im-/exporters, but maintaining both XML and JSON might be a bit overkill.

Hi everyone, I have some news here. We want to switch from XML to a JSON format, which is easier to read and write in my opinion. Are you guys ok with it? I don't want to break your work on im-/exporters, but maintaining both XML and JSON might be a bit overkill.

Gabriel SELLAM commented

2014-12-06 23:20:30 +01:00

Added subscriber: @GabrielSELLAM

dav commented

2015-01-16 22:27:42 +01:00

(Migrated from localhost:3001)

Added subscriber: @dav

Lord Odin commented

2015-01-17 02:30:07 +01:00

Added subscriber: @derekbarker

Aaron Carlisle commented

2015-01-18 00:28:18 +01:00

Added subscriber: @Blendify

Mohamed Sakr commented

2015-04-09 13:24:43 +02:00

Added subscriber: @MohamedSakr-2

Mohamed Sakr commented

2015-04-09 13:24:43 +02:00

Hi dingto,

I would like to create a commercial render engine based on Cycles, tell me what should I "do and don't" with Apache2 License please.

Hi dingto, I would like to create a commercial render engine based on Cycles, tell me what should I "do and don't" with Apache2 License please.

Neal Alexander commented

2015-05-06 17:36:46 +02:00

Added subscriber: @NHA

Neal Alexander commented

2015-05-06 17:36:46 +02:00

Regarding data format:

Whats the ratio of actual-data vs markup-tags when using XML? XML can easily undermine itself when structuring small bits and pieces of numerical data.

JSON is probably better, or you could even use plain S-EXPRS.

XML vs S-EXPR debate:
http://c2.com/cgi/wiki?XmlIsaPoorCopyOfEssExpressions

JSON vs S-EXPR:
http://irreal.org/blog/?p=713

Making a utility for converting an S-Expression layout to JSON should be extremely trivial in lisp/scheme/racket.

Regarding data format: Whats the ratio of actual-data vs markup-tags when using XML? XML can easily undermine itself when structuring small bits and pieces of numerical data. JSON is probably better, or you could even use plain S-EXPRS. XML vs S-EXPR debate: http://c2.com/cgi/wiki?XmlIsaPoorCopyOfEssExpressions JSON vs S-EXPR: http://irreal.org/blog/?p=713 Making a utility for converting an S-Expression layout to JSON should be extremely trivial in lisp/scheme/racket.

dsman commented

2015-05-17 11:33:18 +02:00

Added subscriber: @dsman

dsman commented

2015-05-17 11:33:18 +02:00

Can someone tell me how do I build standalone binaries so that I can run cycles using command prompt ? Cycles is great thing . Someone need to create documentation for Cycles XML api. We are integrating it in our own small 3D application.

Thomas Dinges commented

2015-05-17 11:35:41 +02:00

You can find some build instructions in the standalone repository: https://developer.blender.org/diffusion/C/

Bill D Strong commented

2015-06-27 21:47:20 +02:00

Added subscriber: @BillDStrong

Bill D Strong commented

2015-06-27 21:47:20 +02:00

On the issue of changing the interchange format, have you thought about OpenGEX (Open Game Exchange)? It is opensource and was designed to exchange game scene/level data.

It does have features meant specifically for real-time games, so you may think it isn't a great fit, but there is another option. The same folks that brought us OpenGEX created a superset that can be customized for your needs. It is called OpenDDL, adn you can find out more about it here.

http://openddl.org/

While I like json better than xml, I would think there are benefits to using a domain specific exchange format.

Another option might be the Pixar Universal Scene Descriptor. The idea seems to be to put everything any application might need for a scene as a list into the file, and let the application decide the pieces it needs. They are working to make this an industry standard in the same vein as OpenSubDiv and OIIO.

http://graphics.pixar.com/usd/

Edit: Put in correct links.

On the issue of changing the interchange format, have you thought about OpenGEX (Open Game Exchange)? It is opensource and was designed to exchange game scene/level data. It does have features meant specifically for real-time games, so you may think it isn't a great fit, but there is another option. The same folks that brought us OpenGEX created a superset that can be customized for your needs. It is called OpenDDL, adn you can find out more about it here. http://openddl.org/ While I like json better than xml, I would think there are benefits to using a domain specific exchange format. Another option might be the Pixar Universal Scene Descriptor. The idea seems to be to put everything any application might need for a scene as a list into the file, and let the application decide the pieces it needs. They are working to make this an industry standard in the same vein as OpenSubDiv and OIIO. http://graphics.pixar.com/usd/ Edit: Put in correct links.

Stefan Werner commented

2015-07-15 13:13:04 +02:00

Added subscriber: @Stefan_Werner

Stefan Werner commented

2015-07-15 13:13:04 +02:00

JSON or XML should be irrelevant, both are going to be equally easy to read or write with a decent library. I hope nobody has intentions of reinventing the wheel by writing yet another XML parser!

What's more important, in my opinion, would be the ability from Cycles to dump an entire scene/session to XML. That way, an application that wants to support standalone Cycles as well as link against Cycles directly doesn't have to implement everything twice. It can simply setup the scene as usual, and at the very end either let it render or write it out to a file. Roughly how the RenderMan API does it with RiBegin().

JSON or XML should be irrelevant, both are going to be equally easy to read or write with a decent library. I hope nobody has intentions of reinventing the wheel by writing yet another XML parser! What's more important, in my opinion, would be the ability from Cycles to dump an entire scene/session to XML. That way, an application that wants to support standalone Cycles as well as link against Cycles directly doesn't have to implement everything twice. It can simply setup the scene as usual, and at the very end either let it render or write it out to a file. Roughly how the RenderMan API does it with RiBegin().

haggi krey commented

2015-10-27 23:25:25 +01:00

Hi,
I'll try to start with a maya implementation of Cycles. I'll have a closer look at the way it works in the standalone. If I have questions concerning the implementation is this the correct forum to ask or should I ask elsewhere?

Hi, I'll try to start with a maya implementation of Cycles. I'll have a closer look at the way it works in the standalone. If I have questions concerning the implementation is this the correct forum to ask or should I ask elsewhere?

Lin commented

2016-03-10 15:05:54 +01:00

Added subscriber: @nilram

Lin commented

2016-03-10 15:05:54 +01:00

Hi, I'm a student who wants to contribute to Cycles standalone project for this year GSoC. I have compiled Cycles and run the examples but I'm not sure where I should go on.

I set some prerequisites as preparations.

Get to know the xml API as suggested here http://wiki.blender.org/index.php/Dev:Source/Render/Cycles/Standalone.Part of the required improvements is to change the exchange file format.
Get to know how other mature render engine works in a production pipeline (I'm not sure where I should start for this, reading source code of RenderMan for example?)

Sergey suggested to see how it could be integrated into a production pipeline and list areas which are currently weak and should be improved, but I think I probably get some understanding about no.2 right?

Hi, I'm a student who wants to contribute to Cycles standalone project for this year GSoC. I have compiled Cycles and run the examples but I'm not sure where I should go on. I set some prerequisites as preparations. 1. Get to know the xml API as suggested here http://wiki.blender.org/index.php/Dev:Source/Render/Cycles/Standalone.Part of the required improvements is to change the exchange file format. 2. Get to know how other mature render engine works in a production pipeline (I'm not sure where I should start for this, reading source code of RenderMan for example?) Sergey suggested to see how it could be integrated into a production pipeline and list areas which are currently weak and should be improved, but I think I probably get some understanding about no.2 right?

Alex Fuller commented

2016-04-14 03:19:06 +02:00

Added subscriber: @boberfly

Alex Fuller commented

2016-04-14 03:19:06 +02:00

Hi,

Something which would be great would be a procedural API in the same vein as Renderman and Arnold, where one can dynamically load geometry to the renderer when the renderer hits a bounding box ray hit for the first time.

https://support.solidangle.com/display/AFCUG/Arnold+Procedural
https://renderman.pixar.com/view/appnote23

That way integrating things like Alembic or Cortex objects makes it a lot more straightforward and leaves the geometry out of the XML or whatever readable text format will be used. I'd like to see better integration of Cycles in Gaffer, for example (which supplies procedurals for both 3Delight and Arnold via Cortex).

http://imageengine.github.io/gaffer/

One final note, can the relatively new baking system of Cycles also be exposed in standalone? Generating baked passes would make for a great lightmapper or baking high-res details to low-res models. If this is relatively trivial to implement, I would be happy to implement and supply patches back if someone could guide on where to look.

Cheers

Hi, Something which would be great would be a procedural API in the same vein as Renderman and Arnold, where one can dynamically load geometry to the renderer when the renderer hits a bounding box ray hit for the first time. https://support.solidangle.com/display/AFCUG/Arnold+Procedural https://renderman.pixar.com/view/appnote23 That way integrating things like Alembic or Cortex objects makes it a lot more straightforward and leaves the geometry out of the XML or whatever readable text format will be used. I'd like to see better integration of Cycles in Gaffer, for example (which supplies procedurals for both 3Delight and Arnold via Cortex). http://imageengine.github.io/gaffer/ One final note, can the relatively new baking system of Cycles also be exposed in standalone? Generating baked passes would make for a great lightmapper or baking high-res details to low-res models. If this is relatively trivial to implement, I would be happy to implement and supply patches back if someone could guide on where to look. Cheers

James Brad Barnette commented

2016-08-24 18:08:35 +02:00

Added subscriber: @calmasacow

James Brad Barnette commented

2016-08-24 18:08:35 +02:00

I would have to wholeheartedly agree with What of the earlier posters stated. Pixar's USD should be implemented. It is also Open source and is already has exporters for all for the High-end production Applications. It is battle tested and Pixar's Main format for going between all of their applications and renderers. After that I would say that the second Most important would be the ability to import and render Alembic Sequences. These are Opensource Battle tested solutions with clear documented API that are already supported by all of the Proven Production pipelines. if the Goal of this project is to Turn cycles standalone in to a proven application for high-end production Implementing those to will get you there the fast with the lest amount of having to meddle with the Softwares them selves as the plugins are already made and /or integrated.

Everything you need to know.

Pixar OpenSources USD

I would have to wholeheartedly agree with What of the earlier posters stated. Pixar's USD should be implemented. It is also Open source and is already has exporters for all for the High-end production Applications. It is battle tested and Pixar's Main format for going between all of their applications and renderers. After that I would say that the second Most important would be the ability to import and render Alembic Sequences. These are Opensource Battle tested solutions with clear documented API that are already supported by all of the Proven Production pipelines. if the Goal of this project is to Turn cycles standalone in to a proven application for high-end production Implementing those to will get you there the fast with the lest amount of having to meddle with the Softwares them selves as the plugins are already made and /or integrated. Everything you need to know. [Pixar OpenSources USD ](http://graphics.pixar.com/usd/docs/Introduction-to-USD.html)

Brecht Van Lommel commented

2016-08-24 20:28:39 +02:00

USD support could be good, maybe even as the native Cycles file format, but we'd be adding a big required dependency.

Adopting USD is not going to give us automatic integration into applications though, it only helps a bit. You still need to make your own plugin to integrate into the UI, make all the shaders, lights, render lays, AOVs and other settings available, probably still have a code path outside of USD for fast IPR, etc.

USD support could be good, maybe even as the native Cycles file format, but we'd be adding a big required dependency. Adopting USD is not going to give us automatic integration into applications though, it only helps a bit. You still need to make your own plugin to integrate into the UI, make all the shaders, lights, render lays, AOVs and other settings available, probably still have a code path outside of USD for fast IPR, etc.

James Brad Barnette commented

2016-08-25 01:09:35 +02:00

This comment was removed by @calmasacow

*This comment was removed by @calmasacow*

Mohamed Sakr commented

2016-08-26 00:00:21 +02:00

Added subscriber: @MohamedSakr

Mohamed Sakr commented

2016-08-26 00:00:21 +02:00

This comment was removed by @MohamedSakr-2

*This comment was removed by @MohamedSakr-2*

Thomas Dinges commented

2016-08-26 00:02:37 +02:00

Mohamed, please don't post random questions in here, that has nothing to do with Cycles standalone. Use the Mailing list or IRC for such questions please.

(And no, we don't have real AOVs, only the predefined passes that you probably already know).

Mohamed, please don't post random questions in here, that has nothing to do with Cycles standalone. Use the Mailing list or IRC for such questions please. (And no, we don't have real AOVs, only the predefined passes that you probably already know).

Bob Mercier commented

2016-09-30 08:38:57 +02:00

Added subscriber: @bobmercier

Bob Mercier commented

2016-09-30 08:41:48 +02:00

Is there any activity on this? I know alternatives, C apis, other file formats, etc., have been discussed. I've made some small additions to the xml handling to support motion blur as well as a simple dso loader to experiment with rendering alembic caches. How can I make these available for others to play with?

Thomas Dinges commented

2016-09-30 09:04:01 +02:00

Hi Bob, these sound great. You can upload patches here https://developer.blender.org/differential/diff/create/ then we can check on the work and possibly include it.

Giovanni Remigi commented

2016-11-17 13:52:56 +01:00

Added subscriber: @GiovanniRemigi

Giovanni Remigi commented

2016-11-17 13:52:56 +01:00

This comment was removed by @GiovanniRemigi

*This comment was removed by @GiovanniRemigi*

Giovanni Remigi commented

2016-12-04 22:51:28 +01:00

For anybody interested I successfully compiled and integrated Cycles into the Qt framework. Cycles is compiled as a dynamic library.

The project is available here:

https://bitbucket.org/gremigi/qt-cycles/overview

It includes a small demo application which creates a cube with a chosen image and allows the use of the mouse to zoom into the scene or drag the cube around.
Scene and object are created programmatically to give a better example of integration.

Currently I only compiled with CPU support for easy of life. It perfectly works on Mac, Linux and also Windows using MinGW64.

For anybody interested I successfully compiled and integrated Cycles into the Qt framework. Cycles is compiled as a dynamic library. The project is available here: https://bitbucket.org/gremigi/qt-cycles/overview It includes a small demo application which creates a cube with a chosen image and allows the use of the mouse to zoom into the scene or drag the cube around. Scene and object are created programmatically to give a better example of integration. Currently I only compiled with CPU support for easy of life. It perfectly works on Mac, Linux and also Windows using MinGW64.

Brecht Van Lommel commented

2016-12-06 20:30:47 +01:00

Nice work, always cool to see Cycles used in different ways.

Satish Goda commented

2016-12-12 09:08:34 +01:00

Added subscriber: @satishgoda1

Vincent Olivier commented

2016-12-14 01:36:37 +01:00

Added subscriber: @up4

Rombout Versluijs commented

2017-03-06 18:12:06 +01:00

Added subscriber: @RomboutVersluijs

Rombout Versluijs commented

2017-03-06 18:12:07 +01:00

have you guys scene this? http://insydium.uk/cycles4d/
The've got a full working version of cycles for Cinema4d

have you guys scene this? http://insydium.uk/cycles4d/ The've got a full working version of cycles for Cinema4d

Brecht Van Lommel commented

2017-03-06 23:22:52 +01:00

Yes, @MohamedSakr-2 implemented that :)

Mohamed Sakr commented

2017-03-07 03:06:24 +01:00

I'm just following the great folks :) , Brecht, Sergey, Lukas, DingTo, Juicyfruit

Matti Kaihola commented

2017-03-26 12:15:02 +02:00

Added subscriber: @MattiKaihola

Aaron Carlisle commented

2017-05-24 19:05:51 +02:00

Removed subscriber: @Blendify

Van hentenryck Alain commented

2017-06-05 13:41:17 +02:00

Added subscriber: @Qyko

Brian M Dube commented

2017-09-27 05:05:13 +02:00

Added subscriber: @bdube-1

Charles Wardlaw commented

2017-11-10 15:41:29 +01:00

Added subscriber: @CharlesWardlaw

Charles Wardlaw commented

2017-11-10 15:41:29 +01:00

@bobmercier Did you ever post your patch anywhere? Both the XML motion blur as well as the dso alembic loader are very relevant to something I'm tinkering on right now.

Lucas Alberto commented

2018-02-23 02:45:34 +01:00

Added subscriber: @lucasa

feng xie commented

2019-08-21 02:00:46 +02:00

Added subscriber: @fx

Grische commented

2020-09-01 01:09:48 +02:00

Added subscriber: @grische

Atomic commented

2021-06-13 15:51:54 +02:00

Added subscriber: @AtomP

Atomic commented

2021-06-13 15:51:54 +02:00

I'm playing around with a version of the Cycles Stand Alone .exe.
I'm having trouble getting any kind of image map to show up. Is that implemented yet?

When I specify a filename, it never loads, but I do get the pink file not found for the world?

<!-- Background Shader -->
<background>
	<sky_texture name="sky" filename="G:/Maps/HDRi/HDRI_Studio001.hdr"  />
	<background name="bg" strength="1.0" color="1.0, 1.0, 1.0" />
	<connect from="sky color" to="bg color" />
	<connect from="bg background" to="output surface" />
</background>

I also tried the inline image that Fabio's posted XML export generates, and that did not work either.

<shader name="Material"><image_texture inline="iVBORw0KGg......" .>

Another pink plane.

If it is possible, can someone post an XML file that demonstrates image map capabilities or syntax?

I'm playing around with a version of the Cycles Stand Alone .exe. I'm having trouble getting any kind of image map to show up. Is that implemented yet? When I specify a filename, it never loads, but I do get the pink file not found for the world? ```  <background> <sky_texture name="sky" filename="G:/Maps/HDRi/HDRI_Studio001.hdr" /> <background name="bg" strength="1.0" color="1.0, 1.0, 1.0" /> <connect from="sky color" to="bg color" /> <connect from="bg background" to="output surface" /> </background> ``` I also tried the inline image that Fabio's posted XML export generates, and that did not work either. ``` <shader name="Material"><image_texture inline="iVBORw0KGg......" .> ``` Another pink plane. If it is possible, can someone post an XML file that demonstrates image map capabilities or syntax?

Brecht Van Lommel commented

2021-06-13 17:21:31 +02:00

sky_texture is the wrong node, it's environment_texture.

`sky_texture` is the wrong node, it's `environment_texture`.

Atomic commented

2021-06-13 21:28:03 +02:00

Thanks for the tip.
Through trial and error I have managed to get a background to load. There seems to be a minor bug that image maps on another drive can't be referenced. When I copied them to a subfolder under the Scene1.XML, and referenced them relatively, it started working.

<background name="World">
<texture_coordinate name="tex_coord" />
	<environment_texture name="hdri" filename="./maps/HDRI-Sunny.hdr" vector="0.0 1.0 0.0"  />
	<background name="bg" strength="1.0" color="1.0 1.0 1.0" />
	<connect from="hdri color" to="bg color" />
	<connect from="tex_coord Generated" to="hdri Vector" />
	<connect from="bg background" to="output surface" />
</background>

However, the orientation is wrong, and when I specify a vector, it is ignore. I also added a Texture Coordinate node and connected the Generated output to the Vector with no change.

There also seems to be a bug assigning basecolor to the principled shader. It always remained at the default grey that it was compiled with. (Shown On The Horse)
After a lot of guessing, I finally figured out that BaseColor is actually Base_Color, which is not in the .oso. Is there some cross-mapping in the code taking place? Or is it assumed that camel case input names auto convert to underscore delimited?

<shader name="box1_shader">
	<principled_bsdf name="box1" roughness="0.06" Base_Color="0.300000011921 0.1875 0.0750000029802"/>
	<image_texture name="map_diffuse" filename="./maps/Horse-Texture.png" />
	<connect from="map_diffuse color" to="box1 Base_Color" />
	<connect from="box1 bsdf" to="output surface" />
</shader>

By guessing the "UV" token, and supplying data from an OBJ file, I was able to get a texture map to show up on a plane.

<?xml version="1.0" ?>
<cycles>
	<mesh
		P="-2 0 -2  2 0 -2  -2 0 2  2 0 2  "
		nverts="4 "
		verts="2 3 1  0 "
		UV="0.000100 0.000100  0.999900 0.000100  0.999900 0.999900  0.000100 0.999900  "
	/>
</cycles>

I just wonder what other XML tokens are supported, if any, and what they are? (i.e. Normals, Vertex Colors, Alternate UVs)

Thanks for the tip. Through trial and error I have managed to get a background to load. There seems to be a minor bug that image maps on another drive can't be referenced. When I copied them to a subfolder under the Scene1.XML, and referenced them relatively, it started working. ``` <background name="World"> <texture_coordinate name="tex_coord" /> <environment_texture name="hdri" filename="./maps/HDRI-Sunny.hdr" vector="0.0 1.0 0.0" /> <background name="bg" strength="1.0" color="1.0 1.0 1.0" /> <connect from="hdri color" to="bg color" /> <connect from="tex_coord Generated" to="hdri Vector" /> <connect from="bg background" to="output surface" /> </background> ``` ![Untitled-1.jpg](https://archive.blender.org/developer/F10169859/Untitled-1.jpg) However, the orientation is wrong, and when I specify a vector, it is ignore. I also added a Texture Coordinate node and connected the Generated output to the Vector with no change. There also seems to be a bug assigning basecolor to the principled shader. It always remained at the default grey that it was compiled with. (Shown On The Horse) After a lot of guessing, I finally figured out that *BaseColor* is actually *Base_Color*, which is not in the .oso. Is there some cross-mapping in the code taking place? Or is it assumed that camel case input names auto convert to underscore delimited? ``` <shader name="box1_shader"> <principled_bsdf name="box1" roughness="0.06" Base_Color="0.300000011921 0.1875 0.0750000029802"/> <image_texture name="map_diffuse" filename="./maps/Horse-Texture.png" /> <connect from="map_diffuse color" to="box1 Base_Color" /> <connect from="box1 bsdf" to="output surface" /> </shader> ``` By guessing the "UV" token, and supplying data from an OBJ file, I was able to get a texture map to show up on a plane. ``` <?xml version="1.0" ?> <cycles> <mesh P="-2 0 -2 2 0 -2 -2 0 2 2 0 2 " nverts="4 " verts="2 3 1 0 " UV="0.000100 0.000100 0.999900 0.000100 0.999900 0.999900 0.000100 0.999900 " /> </cycles> ``` ![Untitled-1.jpg](https://archive.blender.org/developer/F10172841/Untitled-1.jpg) I just wonder what other XML tokens are supported, if any, and what they are? (i.e. Normals, Vertex Colors, Alternate UVs)

Nicolas Lelong commented

2022-03-17 16:02:47 +01:00

Added subscriber: @rotoglup

Brecht Van Lommel added this to the Render & Cycles project 2023-02-07 19:08:35 +01:00

Philipp Oeser removed the

Interest

Render & Cycles

label 2023-02-09 13:56:50 +01:00

Sign in to join this conversation.