Option to configure ffmpeg YCbCr transform #25249

New Issue

Xavier Thomas · 2010-12-16T00:18:30+01:00

Xavier Thomas commented

2010-12-16 00:18:30 +01:00

%%%This patch add an option to control the ffmpeg swscale YCbCr->RGB conversion.

The RAW option (to keep YCbCR data in the final image) is not working yet.
I plan on adding an option for loading directly to 32bit float rgb to avoid losses.

This works in the image editor and the node compositor only for the moment.
Changing the parametr in the compositor does not trigger an node update yet :(
%%%

%%%This patch add an option to control the ffmpeg swscale YCbCr->RGB conversion. The RAW option (to keep YCbCR data in the final image) is not working yet. I plan on adding an option for loading directly to 32bit float rgb to avoid losses. This works in the image editor and the node compositor only for the moment. Changing the parametr in the compositor does not trigger an node update yet :( %%%

Xavier Thomas commented

2010-12-16 00:18:30 +01:00

Changed status to: 'Open'

Xavier Thomas commented

2011-01-07 19:44:53 +01:00

%%%Forgot the patch :-/
Tanks to Troy for signaling.%%%

%%%Forgot the patch :-/ Tanks to Troy for signaling.%%%

Xavier Thomas commented

2011-01-07 19:50:19 +01:00

%%%Raw YCbCr option working now. I still misses correct color management (gamma) to permit easy functional uses.%%%

Bastien Montagne commented

2014-09-23 12:29:27 +02:00

Added subscriber: @mont29

Bastien Montagne commented

2014-09-23 12:29:27 +02:00

Is this patch still alive/useful?

Xavier Thomas commented

2014-09-25 16:03:23 +02:00

Hi,

Unfortunatey, I do not have time to develop blender anymore. This path is not maintained and probably does not apply/compile cleanly on a recent source tree, but i is pretty self contained to the ffmpeg decoding pipeline and could be ported/actualised easily.

The patch never really recieved much attention but my opinion is that it is verry usefull and that the ocio colormanagement pipeline in blender is not comlete nor correct (for most videos anyway) without this patch.

Hi, Unfortunatey, I do not have time to develop blender anymore. This path is not maintained and probably does not apply/compile cleanly on a recent source tree, but i is pretty self contained to the ffmpeg decoding pipeline and could be ported/actualised easily. The patch never really recieved much attention but my opinion is that it is verry usefull and that the ocio colormanagement pipeline in blender is not comlete nor correct (for most videos anyway) without this patch.

Sergey Sharybin commented

2015-02-26 11:10:58 +01:00

Added subscriber: @Sergey

Sergey Sharybin commented

2015-02-26 11:10:58 +01:00

main question is: is it even still relevant to have after OCIO integration where you can choose input color space already? And in any case, making it more intgrated into CM code (rather than a separate argument) seems to be a way to go?

Xavier Thomas commented

2015-02-26 13:10:36 +01:00

Hi Sergey,

Yes it is relevant even now that OCIO is integrated. The current workflow is:

ffmpeg decodes a frame but this frame is most probably in YCbCr colorspace because most compression formats require this.
ffmpeg does the colorspace transformation from YCbCr to sRGB.
ocio is used to convert form sRGB to bender internal.

The problem is that the YcbCr colorspace is not always exactly the same (it can be rec602 rec709 or rec2020) and ffmpeg does not always detect the good one (last time I checked it was always using rec601 so very bad for HD content).

My idea for this patch was to add a option to configure this YCbCR colorpsace conversion :

To stay in ffmpeg auto-detect
To force one of the conversion the user select
To not do any conversion and get the RAW YCbCr fram inside Blender. Advanced users can use this and then do the conversion themselves in Blender (with the YCbCr compositor nodes or by adding YCbCr colorspace to the OCIO config). This would prevent some data clipping also, if the later conversion is done with float.

Xavier

Hi Sergey, Yes it is relevant even now that OCIO is integrated. The current workflow is: - ffmpeg decodes a frame but this frame is most probably in YCbCr colorspace because most compression formats require this. - ffmpeg does the colorspace transformation from YCbCr to sRGB. - ocio is used to convert form sRGB to bender internal. The problem is that the YcbCr colorspace is not always exactly the same (it can be rec602 rec709 or rec2020) and ffmpeg does not always detect the good one (last time I checked it was always using rec601 so very bad for HD content). My idea for this patch was to add a option to configure this YCbCR colorpsace conversion : - To stay in ffmpeg auto-detect - To force one of the conversion the user select - To not do any conversion and get the RAW YCbCr fram inside Blender. Advanced users can use this and then do the conversion themselves in Blender (with the YCbCr compositor nodes or by adding YCbCr colorspace to the OCIO config). This would prevent some data clipping also, if the later conversion is done with float. Xavier

Xavier Thomas commented

2015-02-26 17:12:40 +01:00

After discussing this on IRC with Sergey we concluded that the best would be:

To add YCbCr colorspaces to the ocio config
Handle subsampling conversion in the ffmpeg image input code and not handle color-space conversion at this step
Only have one colorspace setting (the OCIO) for the user to worry about. And this can be auto-detected to sane default (at least as sane as before)

On the image input side, this would result in the most simple and clear solution for users. It would be possible and easy to convert old files and stay mostly compatible. The only difference would be that some "out of gamut" colors that are clipped by the ffmpeg conversion now would not be clipped (and thus be outside the 0 to 1 range) by the OCIO conversion. But that is a good think in my opinion, and clipping values can also easily be done afterward if really needed.

The output/encoding side needs more thinking though because a lot of codec/color-space combination would not be valid, and of course sub-sampling will not make things easier.

After discussing this on IRC with Sergey we concluded that the best would be: - To add YCbCr colorspaces to the ocio config - Handle subsampling conversion in the ffmpeg image input code and not handle color-space conversion at this step - Only have one colorspace setting (the OCIO) for the user to worry about. And this can be auto-detected to sane default (at least as sane as before) On the image input side, this would result in the most simple and clear solution for users. It would be possible and easy to convert old files and stay mostly compatible. The only difference would be that some "out of gamut" colors that are clipped by the ffmpeg conversion now would not be clipped (and thus be outside the 0 to 1 range) by the OCIO conversion. But that is a good think in my opinion, and clipping values can also easily be done afterward if really needed. The output/encoding side needs more thinking though because a lot of codec/color-space combination would not be valid, and of course sub-sampling will not make things easier.

Sergey Sharybin commented

2015-02-26 17:19:46 +01:00

Think it's possible to make YCrCb conversion LUT a bit wider than 0..1 range? Plus, currently videos are all stored as bytes, so they're clipped anyway. So seems it's not so huge a regression?

Xavier Thomas commented

2015-02-26 18:34:34 +01:00

I was thinking of Matrix transform instead of LUT, they seems more appropriate and not limited to a range of input.

YCbCr darta is byte coded and thus in the 0-1 range but converting it to sRGB result in value out of the 0-1 range, currently those are clipped by the ffmpeg colorspace conversion. If we use an OCIO Matrix tranform they will not be anymore. This is good, but my concerns was that it might "brake" old files.

I was thinking of Matrix transform instead of LUT, they seems more appropriate and not limited to a range of input. YCbCr darta is byte coded and thus in the 0-1 range but converting it to sRGB result in value out of the 0-1 range, currently those are clipped by the ffmpeg colorspace conversion. If we use an OCIO Matrix tranform they will not be anymore. This is good, but my concerns was that it might "brake" old files.

Sergey Sharybin commented

2015-02-26 18:57:38 +01:00

I didn't check if it could be a matrix in all the cases. If it could -- for sure matrix is the way to go.

If users will get all of a sudden deeper dynamic range that's kinda nice :)

I didn't check if it could be a matrix in all the cases. If it could -- for sure matrix is the way to go. If users will get all of a sudden deeper dynamic range that's kinda nice :)

Troy Sobotka commented

2015-02-26 22:38:33 +01:00

It can be a matrix and I have both broadcast and full range versions. I will look into this possibly after I fix the XYZ hard coded paths.

Probably needs a bit more thought for the higher bit depth varieties.

It can be a matrix and I have both broadcast and full range versions. I will look into this possibly after I fix the XYZ hard coded paths. Probably needs a bit more thought for the higher bit depth varieties.

Bastien Montagne commented

2015-05-22 11:57:36 +02:00

Changed status from 'Open' to: 'Archived'

Bastien Montagne closed this issue

2015-05-22 11:57:36 +02:00

Bastien Montagne self-assigned this 2015-05-22 11:57:36 +02:00

Bastien Montagne commented

2015-05-22 11:57:36 +02:00

No news here since months… archiving for now, we can always reopen should the raised points be addressed.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Option to configure ffmpeg YCbCr transform #25249