Page MenuHome

Vertex colour baking
Closed, ResolvedPublic


This patch adds the ability to bake directly to vertex colours.The advantage of this is that it doesn't require a UV map, however the quality of the bake will depend on the resolution of the mesh. See attached screenshot.

Change summary:
- Added a button to enable vertex colour baking in render buttons > bake panel.
- Added function shade_verts to compliment shade_tface and zspan_scanconvert. It does the same sort of thing as those functions, but only calls do_bake_shade once for each vertex.
- Changed bake_shade and bake_displacement to write to MCol instead of pixel buffer if MCol is present (not NULL).
- Disabled code for creating zspan and image buffers when baking to vertices.
- Added new table 'origindex' to VlakTableNode, on Brecht's advice. This maps from VlakRen back to the face in the source mesh. It is only populated when doing a vertex colour bake.

- Baking textures, displacement and normals.
- Baking from selected to active.

Partially working:
- Full render, AO (kind of works, but produces some artefacts).

Not working:
- Baking meshes with modifiers (e.g. subdivision).

Any feedback on this patch would be welcome.

By making a contribution to this project, I certify that the contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file.

Signed-off-by: Alex Fraser <> on behalf of VPAC Ltd.



Event Timeline

Further testing of baking modes:

Working: Emit, alpha, mirror intensity, mirror colors, specular colors, specular intensity.

Partially working: Shadows (same as full render)

shadows on subsurf 3 & applied suzzane are giving odd artifacts (as if the points are in shadow).
re shade_verts:
- float vec[4][2] is used un-initialized.

- colors are being written to more than once. set all vcolors to black and do...
BLI_assert(basevcol->b < 10);
If this fails it means the color is being written to multiple times since it should always start as black.

Other than this artifact the patch work nice.

Update: I *think* one reason for artifacts is that verte colors are written to twice could be that the faces are triangulated on render so 2 tris will share 2 corners when split from a quad.

This cant be the only reason tho because triangulating the mesh the problem is still there.

Thanks for the reviews! Attached is a new version of the patch (vcolbake_2.patch) that only processes each MCol once. Also, as discussed in IRC with Campbell, `vec` wasn't really necessary at all so it has been removed. vcolbake_1to2.patch shows only the new changes.

Even though each face vertex is only processed once, the shadowing error remains. Need to look into this further.

uploaded vcolbake_3_bmesh.patch

This only works with triangles, since a bmesh polygon can have many corrasponding VlakRen (which are tessellated) while loop colors are not.

It only happens to work for triangles because the colors and the faces happen to line up.

AFAICS - we would need to store the polygon (or a least the MLoop array and its length), along with the MLoopCol in the BakeShade struct, as well as having an origindex lookoup in the VertRen (as is done for render face).

This way with a poly index and an original vertex - the MLoopCol can be found which the render vert/face came from.

Uploaded vcolbake_4_bmesh.patch. This version works with tris, quads and ngons. The technique is as described by Campbell in the previous comment.

Remaining issues:
- Doesn't work with modifiers like subsurf, because the origindex is taken from the derived mesh instead of the base.
- Some shadow artefacts.
- 3D view doesn't update after the bake finishes: you need to toggle edit mode for the new colours to be displayed.

Uploaded vcolbake_5_origindex.patch. This version works with objects with modifiers such as subsurf! Key difference:

- int *origindex_poly = need_origindex ? dm->getTessFaceDataArray(dm, CD_POLYINDEX) : NULL;
+ int *baseindex_poly = need_origindex ? CustomData_get_layer(&dm->faceData, CD_ORIGINDEX) : NULL;

It works with the other modifiers too, with the exception of Array, which seems to fill in origindex incorrectly.

Uploaded vcolbake_6.patch. This version:
- Uses a two-step lookup for the original mpoly index.
- Re-evaluates (re-draws) the mesh after baking.

It also *doesn't crash* with Array and Mirror, however it doesn't really work either. The problem is that the origindex returned when using those modifiers is not really the original index in the base mesh. So for now, any index above me->totpoly is ignored. This is not correct behaviour.

Attached a new version that applies against r53102.

The patch now works with the modifiers I have tested (array, subsurf, solidify, triangulate, ...). For most cases the results are good - particularly with subsurf. Some modifiers seem to change the order of the loops, but in general this type of bake is still useful.

The only rendering artefacts are with shadow, and they are minor (a small percentage of vertices will be incorrectly marked as shadowed).

This feature is quite stable; even in the presence of many modifiers and in animations it doesn't cause crashes.

Hi Alex, tested +1 to commit to trunk.

BUT - I did find some artifact that only happens with full render, perhaps self shadowing, see attached file.

Thanks Campbell! Committed, r53494.

Yes, those artefacts are annoying. I think you're right that it is self-shadowing, because they also appear in shadow-baking mode. I looked into it previously but couldn't figure out it: I tried shifting the coordinates and it didn't seem to help. I guess I should open a bug report about it :/

Alex Fraser (z0r) closed this task as Resolved.Jan 2 2013, 1:17 AM