Cycles: Add Support for IES files as textures for light strength
ClosedPublic

Authored by Lukas Stockner (lukasstockner97) on Oct 7 2015, 4:45 PM.
Tags
None
Tokens
"Love" token, awarded by SteffenD."Yellow Medal" token, awarded by OgDEV."The World Burns" token, awarded by cardboard."Love" token, awarded by pisuke."Like" token, awarded by disnel."Love" token, awarded by duarteframos."Like" token, awarded by vklidu."Like" token, awarded by bliblubli."Like" token, awarded by sterlingroth."Love" token, awarded by marcog.

Details

Summary

This patch adds support for IES files, a file format that is commonly used to store the directional intensity distribution of light sources.
The new "IES Light" node is supposed to be plugged into the Strength input of the Emission node of the lamp.

Since people generating IES files do not really seem to care about the standard, the parser is flexible enough to accept all test files I have tried.
Some common weirdnesses are distributing values over multiple lines that should go into one line, using commas instead of spaces as delimiters and adding an END marker at the end of the file.

The user interface of the node is similar to the script node, the user can either select an internal Text or load a file.
Internally, IES files are handled similar to Image textures: They are stored by the ImageManager and each unique IES is assigned to one slot.

The local coordinate system of the lamp is used, so that the direction of the light can be changed. For UI reasons, it's usually best to add an area light,
rotate it and then change its type, since especially the point light does not immediately show its local coordinate system in the viewport.

Diff Detail

Repository
rB Blender
There are a very large number of changes, so older changes are hidden. Show Older Changes
intern/cycles/blender/blender_shader.cpp
824

Guess we don't really need to try getting both of absolute path and text datablock context.

intern/cycles/blender/blender_util.h
471

get_text_datablock_content? text is a bit vague imo.

intern/cycles/kernel/kernel_shader.h
286 ↗(On Diff #5125)

Think we'd rather always consider transform is always a full affine one. Doesn't cost that much more memory

This revision now requires changes to proceed.Oct 7 2015, 6:45 PM

patch works really well. Best implementation I have seen until now in an open source renderer. UI-wise, I would add a new lamp type. I mean, actually point, hemi, spot and area lights are particular cases of what IES lights can be. For new users (and also old ones like me :D ), it's really not intuitive to make a point-light (some lights are very long or are big squares so far away from a point) and then add a texture to it that is plugged not in color like usual but strength. For an artists, it would be much better to have a IES light type.

I would like to test this new IES lamp
so wondering if there is a build available for win 64 ?

I have a lot of IES files and can test it

thanks for any feedback
happy bl

any minimum doc for this new patch
giving general data's for these IES specs ?

IES Std

there are 2 IES - one for US and one for EU

note: In the USA many large manufacturers of fixtures still have old IES std from 1990
so just hope it can be read too!

so just wondering which IES files can be read here!

Thanks
happy bl

Would be good to have it early in master to have a good testing before 2.77 :)

I finally got to updating this patch, the IES handling code is now split into an IESLight class (in util/) that handles parsing and some code in the ShaderManager that handles the IESLight slots (like the ImageManager did before).
Most of the first review points should be covered, the main issue that remains is the Light transform: The "Lamp primitive" seems strange, and the question remains how to store the Transform: IIRC, @Thomas Dinges (dingto) wanted to save more storage and @Sergey Sharybin (sergey) wanted to store both full transforms. Personally, I'd go with the full transforms: That's 128 additional bytes per lamp, compared to meshes, images and the BVH it's probably negligible...

I think you are right and this per light overhead is worth the price.

Thomas Dinges (dingto) requested changes to this revision.Nov 15 2015, 3:43 PM

Looks quite good already. I am fine with the additional memory usage per light. Some comments inline.

intern/cycles/kernel/kernel_textures.h
82

This will take away one more texture from sm_2x GPUs. Not so much of an issue, but we need to reduce the amount of max textures for these cards then. (see changes in svm_image.h and image.h in rB0ce3a755f83b)

intern/cycles/kernel/kernel_types.h
524 ↗(On Diff #5426)

Not so happy with this approach either, other solutions?

This revision now requires changes to proceed.Nov 15 2015, 3:43 PM
Brecht Van Lommel (brecht) added inline comments.
intern/cycles/kernel/kernel_textures.h
82

The lookup_table texture can probably be used, check how other code uses scene->lookup_tables->add_table(...).

intern/cycles/kernel/kernel_types.h
524 ↗(On Diff #5426)

For faster lamp MIS it would make sense to eventually include lamps in the BVH, so then they would be traceable. Still somewhat confusing now, but a lamp primitive is not a bad concept to have I think.

Some general questions:

  • Why IES was decided to be in the ShaderManager?
  • Don't see why can't we move to LUT for both SVM and OSL?
  • Perhaps it makes sense to add parsing facilities to the RNA API, similar to the PontDensity? So we don't need to re-implement this code for all the render engines (upcoming OpenGL PBR?). This is something where @Campbell Barton (campbellbarton) could have storng opinion as well.
intern/cycles/app/cycles_xml.cpp
519 ↗(On Diff #5426)

Don't you miss readinf path to the IES file here or so?

intern/cycles/blender/blender_shader.cpp
827

Use mustage brackets here, plus consider adding assert(b_ies_node.mode() == BL::ShaderNodeIESLight::mode_INTERNAL) to the else branch.

intern/cycles/kernel/kernel_shader.h
291 ↗(On Diff #5426)

Consider for the future: move it to the utility function like lamp_fetch_transform().

intern/cycles/kernel/kernel_textures.h
82

Yes, using lookup table will be the proper way to deal with this. Think i've mentioned it in the comment from a while ago..

intern/cycles/kernel/kernel_types.h
957 ↗(On Diff #5426)

If IES table is stored in lookup_table we can remove this?

intern/cycles/kernel/osl/osl_services.cpp
967

This is something really weak, can't we interpolate LUT from a dedicated OSL node, similar to SVM?

intern/cycles/kernel/svm/svm_ies.h
25

This is more like a generig 2D LUT interpolation which will actually belong to table routines?

intern/cycles/render/light.h
55

We should start being more verbose on documentation. Mention of which axis is aligned with the light direction and which is an "up" direction wouldn't hurt here at all.

intern/cycles/render/nodes.cpp
1000

Split into two lines.

intern/cycles/render/shader.cpp
568 ↗(On Diff #5426)

Why it's part of ShaderManager?

source/blender/makesdna/DNA_node_types.h
899

char filepath[1024]; /* 1024 = FILE_MAX */ so it's simplier to nail all the filepaths down when/if we'll be ding extra filepath length bump.

I'm working on improving this patch right now, but I've got no idea how to improve the OSL handling. I could duplicate the interpolation into OSL, sure, but in the end it will always need a way to access the LUT data and therefore need a way to call code "outside" of OSL.
However, I don't see a way to to implement this without using the texture name hack. Also, considering that it's used already for image slots, there probably is no better way to implement this.

So, I'd be open to suggestions here, but I don't really expect that there's a better way to do it...

Ok, will have a look of how to make it all nice for OSL.

could someone please revise it, i have no problems using it in experimental build & would be very welcomed if it was included in the 2.79 release

Hi everyone.
In my opinion this should be a standard feature. Not only ies but also ldt.
And should go, if possible, in a nightly build.
Is there some way to test it?
can't complie it succesfully on debian9.

Okay, after over two years here's an updated version of the patch. Not really any functional changes, just some internal cleanup and improvements.

I'm still not sure how to handle the OSL problem, but considering that we're also using the texture filename trick for PTex and the bevel shader now, I guess it's agreed that there is no simple way to do it better?

Minor style and comment tweaks.

I don't know of a better way to handle the OSL problem, the current method seems ok to me.

From a quick review of the patch, it seems fine to me. I did a few tiny tweaks.

Making IES parsing available to Eevee as well would be nice, but someone can always do it later.

I saw some of @Sergey Sharybin (sergey)'s comments were addressed, but did not check if it was all of them.

Thanks for the cleanup, I forgot to check the older code for style. I think I missed some comments, I'll do another pass over the patch later today/tomorrow.

Guys,i am a lighting designer and i have been waiting for so long for this feature. I tried the patch and it really worked well. I mean there was so called addon but not giving the correct result, but this patch really works like in lighting calculation softwares. Do you have any idea when it can be implemented to Blender natively? 2.8 release?

Thanks,
and great job!

Glad to hear that it works!

I think this will end up in master soon, I've just finished rewriting the processing code to be a lot cleaner and also support more IES features now (such as 90-270 plane symmetry and photometry type B files), I'll post that later today.

The next release will be 2.8, and it definitely should be done in time for that.

Awesome to hear it reaches 2.8! I am crossing my fingers!

Here is a road lighting visualization i made by using ies datas thanks to your efforts!
https://www.youtube.com/watch?v=5WhTcjNXb4U

And a piece of advise for future developments, i mean i would definitely like to do if i could do coding; light levels = lux levels
False color display would be the best i think. (please google “lighting calculation false colors”)

Great job!
Looking forward a native implement!

Here's the updated version.

Parsing and processing were changed significantly, now the process is more robust and handle more corner cases. I tested the parser using this IES collection, and almost all 70k files (not counting duplicates) passed. The handful that
failed is completely messed up (one contains angles from -400 to +500 degrees, for example), so I'm happy with that result.

I also cleaned up the kernel interpolation code a bit, it should now be much clearer what actually happens there.
The ies_stride is also gone, the data is now packed densely with a offset table at the start.

All comments on the earlier versions should be applied.

Oh, and regarding false-color display etc.: Just go to the color management options and set the view to False Color. If you want control over the colors, use the compositor - F3444053 shows one way to do it, with each color being one power of ten (blue = 0.001, white=10).

I should also point out that you have to be careful with units - IES files are designed for photometry, but Cycles uses radiometric units. Conversion in the IES node is currently hardcoded for a D65 illuminant (see the comment in util_ies.cpp line 162 for details). Generally, you should probably not rely on absolute intensity values.

Oh, and regarding false-color display etc.: Just go to the color management options and set the view to False Color.

@Lukas Stockner (lukasstockner97) Can we get an AOV that store the amount of incoming light per pixel? That would also be an important information to be displayed as false colors (for example for architects and everyone who is in lighting simulation): http://blog.irayrender.com/post/50640565041/irradiance-buffer

The feature is also sometimes called "Illuminance".

@Orkun (OgDEV) I did the Google search you proposed and it seems like most lighting software offers both luminance and illuminance for false colour display. Cycles currently can show only luminance.

Looks fine to commit.

intern/cycles/kernel/svm/svm_ies.h
22

{ on new line.

intern/cycles/util/util_ies.cpp
206

Any reason you didn't use vector<vector<float> > here? It would make things a bit simpler & safer I think.

Ah yes lighting calculation softwares can show both luminance and illuminance values. Illuminance values are about the quantity of light that reaches to surface and luminance is about how bright a material is seen under that light; meaning under 500lx of illuminance a light blue material can give 30cd/m2 and a dark blue one 4cd/m2.
I advised this feature since values are important at engineering and designing worlds. Because any scene can be presented at a satisfying light level by changing some parameters, like exposure for instance or ISO. However, if i am doing a design projecting real world, i need values to understand what is going on, if i could achieve the effects on my computer screen at real life.
I have been using the lighting calculation softwares for this purpose up to now (Dialux, Relux ...). After seeing your patch that giving light beams similar to these lighting calculation softwares, i got really excited about the results and the progress you have made, and i thought that "values" can be the next target.
I will check out the compositor solution, thank you very much Lukas for the tip and advises.

Looks good to me. Thanks for updating this patch!

This revision was not accepted when it landed; it landed in state Needs Review.May 27 2018, 1:25 AM
This revision was automatically updated to reflect the committed changes.