Page MenuHome

world_to_camera_view broken for off-axis projection
Confirmed, NormalPublicBUG

Description

System Information
Operating system: Windows 10
Graphics card: GTX 1070

Blender Version
Broken: 2.82

Short description of error
When a camera has shift_x or shift_y != 0, world_to_camera_view does not work.

We can test this by comparing points projected manually with camera.calc_matrix_camera(), and with the utility world_to_camera_view. When there is no shift_x or shift_y, they line up. But when we change shift_x or shift_y, they are no longer equal.

Projection using the matrix from camera.calc_matrix_camera() seems to be correct. So I think world_to_camera_view() has a bug. This is suprising since the docs suggest that world_to_camera_view() "Takes shift-x/y ... into account"

Exact steps for others to reproduce the error

  1. Open new blend file with default camera and cube
  2. Run this script
import bpy
import numpy as np
from bpy_extras.object_utils import world_to_camera_view

D, C = bpy.data, bpy.context

camera = C.scene.camera
cube = D.objects["Cube"]

def project_with_matrix():

    width, height = C.scene.render.resolution_x, C.scene.render.resolution_y
    projection_matrix = camera.calc_matrix_camera(C.evaluated_depsgraph_get(), x=width, y=height)
    projection_matrix = np.array([list(row) for row in projection_matrix])

    # Cube vertices in camera space
    verts_camspace = np.array([list(camera.matrix_world.inverted() @ v.co) for v in cube.data.vertices])
    
    # Homogenize
    verts_camspace_h = np.hstack([verts_camspace, np.ones((len(verts_camspace), 1))]) 
    
    # Project
    projected = verts_camspace_h.dot(projection_matrix.T)
    
    # Dehomogenize
    projected = projected[:, :2] / projected[:, 3, None] 
    
    # [-1, 1] to [0, 1]
    projected = (projected + 1.0) / 2.0
    
    return projected


def project_with_world_to_camera_view():

    projected = []
    for v in cube.data.vertices:
        projected.append(list(world_to_camera_view(C.scene, camera, v.co).xy))
    
    return np.array(projected)


if __name__ == "__main__":

    camera.data.shift_y = 0.0
    camera.data.shift_x = 0.0
    print("shift_x=0.0, shift_y=0.0:", np.allclose(project_with_matrix(), project_with_world_to_camera_view()))

    camera.data.shift_x = 0.1
    camera.data.shift_y = 0.0
    print("shift_x=0.1, shift_y=0.0:", np.allclose(project_with_matrix(), project_with_world_to_camera_view()))

    camera.data.shift_x = 0.0
    camera.data.shift_y = 0.1
    print("shift_x=0.0, shift_y=0.1:", np.allclose(project_with_matrix(), project_with_world_to_camera_view()))

See this output:

shift_x=0.0, shift_y=0.0: True
shift_x=0.1, shift_y=0.0: False
shift_x=0.0, shift_y=0.1: False

Note: this assumes the default cube has identity transform, so we skip applying it's world matrix

Event Timeline

Erroll Wood (errollw) updated the task description. (Show Details)
Germano Cavalcante (mano-wii) changed the task status from Needs Triage to Confirmed.Wed, Mar 11, 12:20 PM
Germano Cavalcante (mano-wii) changed the subtype of this task from "Report" to "Bug".

I can confirm.
I tested it with this script and the result is as if the shift-x/y had not been applied:

import bpy
from bpy_extras.object_utils import world_to_camera_view

def project_with_matrix(depsgraph, scene, camera, location):
    from mathutils import Vector

    width, height = scene.render.resolution_x, scene.render.resolution_y
    persp_mat = camera.calc_matrix_camera(depsgraph, x=width, y=height) @ camera.matrix_world.inverted()
    loc = persp_mat @ location.to_4d()
    
    return (loc.xyz / (2 * loc.w)) + Vector((0.5, 0.5, 0.5))

scene = bpy.context.scene
cursor_loc = scene.cursor.location
ob_cam = bpy.context.object
depsgraph = bpy.context.evaluated_depsgraph_get()

print('----------------')
print(world_to_camera_view(scene, ob_cam, cursor_loc))
print(project_with_matrix(depsgraph, scene, ob_cam, cursor_loc))