Page MenuHome

Camera Background Images
ClosedPublic

Authored by Jeroen Bakker (jbakker) on Apr 3 2019, 3:08 PM.

Details

Summary

Migrate old legacy code to the draw mamager/object mode. The old legacy
version did not work with wireframe. By migrating the code
to modern draw manager code we have mode control on the drawing process.

Still background images do not work with OIT, the cause seems to be that the transparent pixels are treated as background pixels. The next picture makes this clear.

Fix: T62876, T63237

Diff Detail

Repository
rB Blender

Event Timeline

WIP: Refactor Camera Background Images

Migrating legacy drawing code for camera background images
to be compatible with draw manager.

two questions still needs to be validated:

  1. Is there a better way to calculate the matrices. I simulate a screen/viewrect space transformation.
  2. GPU textures of the MovieClip are not freed when scrubbing this leads to memory issues. Need a clever way for freeing unused images.
Jeroen Bakker (jbakker) retitled this revision from Camera Background Images to WIP: Camera Background Images.Apr 3 2019, 3:17 PM
Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)
source/blender/draw/modes/object_mode.c
1182

update comment

1354

remove line

source/blender/draw/modes/shaders/object_camera_image_frag.glsl
7

Add depth to the gl_Position?

source/blender/draw/modes/shaders/object_camera_image_vert.glsl
14

Better names for pos_s, pos2d

19

in stead of booleans we can use a vec2.

source/blender/draw/modes/object_mode.c
1289

Change to vec2

source/blender/gpu/intern/gpu_draw.c
247

Merge with gpu_get_image_gputexture code

Use premul for in front rendering and use OIT blending for rendering in back.
As rendering in back is additive the visual results differ from what users
might expect when changing the alpha value.

Issue is related that IOT transparent objects have an alpha value of 0.0 in the DST buffer.
This will give other artifacts (transparent pixels are seens as background and overwritten at all.

This is a shortcoming of having all these options to help the user and how we render them in the pipeline.

Set background images to use premul blending.

Jeroen Bakker (jbakker) retitled this revision from WIP: Camera Background Images to Camera Background Images.Apr 10 2019, 12:56 PM

Rebased with master (clang-format)

Rebased with latest master. Due to recent changes the color managament of the bg pics needs to be reviewed.
The GPU_textures are scene_linear. So we need to perform the view transform, before performing the blending
to get similar results as before.

This might get us one step back.

I think this would just copy the code from object_empty_image_frag.glsl for color management?

Added color management

source/blender/draw/modes/object_mode.c
1282

Revisit this order, rotation and translation do not work together.

Jeroen Bakker (jbakker) edited the summary of this revision. (Show Details)Jun 18 2019, 3:45 PM

Fixed matices

source/blender/draw/modes/object_mode.c
1282

Also scaling does not work as expected.

@Jeroen Bakker (jbakker) Holdout in combination with transparent film doesn't seem to work. When scene as Film Transparent enabled and I create a viewlayer that is set to holdout, the Cycles viewport rendering shows the transparent area created by the holdout as grey world background, but not the Background images. You can try this file:


Just hit shift+z to go to rendered viewport to see the issue.
It does work fine in 2.79 (using mask layer in renderlayers)

@Jeroen Bakker (jbakker) DOF in Cycles over background images doesn't work at all:

I see that holdout is not working as expected. You can say it was not working before as holdout influences the Z-buffer and therefore also influences the grid rendering and viewport rendering on multiple places.

This is current master, showing that the grid lines is hidden by the holdout.


When film is not transparent background images are also not drawn

The new way of camera background rendering uses Z-buffer to test how to integrate the camera background images, hence the holdout effects the image rendering.
The depth buffer needs to be correct, otherwise the viewport compositing will not work.

Hah, this patch supports camera background image rendering in cycles without film transparency enabled. You win some, you loose some.

A possible solution might be to use an Alpha test before depth test, will experiment with this.

Hmmm seems that we don't have actual alpha values to play with so there will always be artifacts....

I think we can go ahead and commit this change, assuming @Clément Foucault (fclem) approves. It's better than not showing background images at all.

Ideally it would probably first draw with the render engine with proper alpha channel, then alpha under the background images, and then alpha under the checkerboard. This may be a bit too big of a change to do before the 2.80 release though, I don't personally have insight into that.

source/blender/makesrna/intern/rna_camera.c
234

I'm not sure why -10..10 limits were added here. If the intent was only to get a better step size and precision then I think "-FLT_MAX" and "FLT_MAX" would be better as soft limits.

Jeroen Bakker (jbakker) marked an inline comment as done.Jun 20 2019, 4:14 PM
Jeroen Bakker (jbakker) updated this revision to Diff 15980.

Fixed matrices: currently a bit dump as every step in the evaluation is done as a separate matrix.

Fine as it is.

Ideally it would probably first draw with the render engine with proper alpha channel, then alpha under the background images, and then alpha under the checkerboard. This may be a bit too big of a change to do before the 2.80 release though, I don't personally have insight into that.

Exactly my though. Shouldn't be too hard in practice but this can go after this diff is committed.

source/blender/draw/modes/object_mode.c
1525

This is correct. BLEND_ALPHA is the old BLEND

This revision is now accepted and ready to land.Jun 20 2019, 11:04 PM
Jeroen Bakker (jbakker) marked an inline comment as done.Jun 21 2019, 9:57 AM
This revision was automatically updated to reflect the committed changes.