Blender writes .abc color information wrong
Closed, ResolvedPublic

Description

System Information
All
nVidia

Blender Version
2.78c and forward.

Short description of error
Blender writes .abc color information wrong.

Exact steps for others to reproduce the error
Create some vertex colors on a cube and export it to Houdini. You will notice Houdini can't read the .abc Color information because the indices are written incorrectly.

Here is the log from the SideFX bug team...

Side Effects Support Ticket: #56695

Hello,

Houdini does not show the "Col" attribute because it is corrupt. "Col" is stored in the Alembic archive as an indexed attribute, however, the indices are corrupt.

Cheers,

Silvina


In response to:

Even if I rename Col to Cd inside of Blender when I export the .abc Houdini
never reveals the attribute.

Inside Houdini I have an Alembic node followed by an Unpack turned all the
way up to 10 levels of unpack recursion. When I inspect the attributes
there is no Col or Cd. I only see P and N attributes.

How were you able to inspect the Col attribute?
It would actually be nice to see any named attribute transferred.

ap

On Fri, Jan 5, 2018 at 3:54 PM, SideFX Support wrote:

Side Effects Support Ticket: #56695

Hello,

How are you exporting it? It is being saved as Col. Please try exporting
it as Cd.

Cheers,

Silvina


In response to:


Hello SESI,

I am trying to transfer vertex color information from Blender to Houdini,
via Alembic file format. I have attached the painted_cube.abc file that I
created and exported in Blender.

When I reference the .abc file in Houdini, I can not locate a @Cd
attribute, even after unpacking.
When I load the .abc file in Blender I can clearly see the painted vertex
information.

This implies, to me, that the color information is stored in the .abc file
but Houdini is not reading it correctly.

Can you look into this bug?

Thanks,
ap

Is the Side Effects bugtracker public? If so, could you share a link?

Dont have access to Houdini atm, but looking at this further I made a very simple Cube with vertexcolors

This reads back into blender just fine (including vertex colors).

As Houdini support was talking about a corrupt indexed attribute I had a look at the abc in abcecho

/opt/lib/alembic/bin/abcecho /tmp/VCol.abc 
AbcEcho for Alembic 1.7.1 (built Jan  8 2018 19:09:47)
  file written by: Blender
  using Alembic : Alembic 1.7.1 (built Jan  8 2018 19:09:47)
  written on : Tue Jan  9 17:07:26 2018
  user description : untitled

  ScalarProperty name=.childBnds;interpretation=box;datatype=float64_t[6];arraysize=6;numsamps=250
Object name=/Cube
  CompoundProperty name=.xform;schema=AbcGeom_Xform_v3
    ScalarProperty name=.inherits;interpretation=;datatype=bool_t;arraysize=1;numsamps=250
    ScalarProperty name=.ops;interpretation=;datatype=uint8_t;arraysize=1;numsamps=250
    ScalarProperty name=.vals;interpretation=;datatype=float64_t[16];arraysize=16;numsamps=250
  ScalarProperty name=visible;interpretation=;datatype=int8_t;arraysize=1;numsamps=250
Object name=/Cube/CubeShape
  CompoundProperty name=.geom;schema=AbcGeom_PolyMesh_v1
    ScalarProperty name=.selfBnds;interpretation=box;datatype=float64_t[6];arraysize=6;numsamps=1
    ArrayProperty name=P;interpretation=point;datatype=float32_t[3];arraysize=98;numsamps=1
    ArrayProperty name=.faceIndices;interpretation=;datatype=int32_t;arraysize=384;numsamps=1
    ArrayProperty name=.faceCounts;interpretation=;datatype=int32_t;arraysize=96;numsamps=1
    CompoundProperty name=.userProperties;schema=
      ScalarProperty name=meshtype;interpretation=;datatype=bool_t;arraysize=1;numsamps=1
    CompoundProperty name=.arbGeomParams;schema=
      CompoundProperty name=Col;schema=
        ArrayProperty name=.vals;interpretation=rgba;datatype=float32_t[4];arraysize=98;numsamps=1
        ArrayProperty name=.indices;interpretation=;datatype=uint32_t;arraysize=0;numsamps=1
    ArrayProperty name=N;interpretation=normal;datatype=float32_t[3];arraysize=98;numsamps=1

now the line reading ArrayProperty name=.indices;interpretation=;datatype=uint32_t;arraysize=0;numsamps=1 makes me wonder if we actually miss writing the indices? why is this arraysize zero?

On the other hand this could be totally fine [havent looked at the alembic specification further...]

@Sybren A. Stüvel (sybren) : could you have a look?

Atomic (atomic) added a comment.EditedJan 11 2018, 5:49 PM

I can verify that Philipp's cube export does not transfer color to Houdini.

Philipp Oeser (lichtwerk) triaged this task as Confirmed priority.Jan 12 2018, 9:08 PM

I've tried a change in how Blender exports the vertex colour, writing it non-indexed but directly as an ArrayProperty.
@Atomic (atomic) & @Philipp Oeser (lichtwerk) How does this file import for you?


If this works well, I'll commit it to master.

Note how the Col property is written now:

Object name=/Suzanne/SuzanneShape
  CompoundProperty name=.geom;schema=AbcGeom_PolyMesh_v1
    ScalarProperty name=.selfBnds;interpretation=box;datatype=float64_t[6];arraysize=6;numsamps=1
    ArrayProperty name=P;interpretation=point;datatype=float32_t[3];arraysize=507;numsamps=1
    ArrayProperty name=.faceIndices;interpretation=;datatype=int32_t;arraysize=1968;numsamps=1
    ArrayProperty name=.faceCounts;interpretation=;datatype=int32_t;arraysize=500;numsamps=1
    CompoundProperty name=.userProperties;schema=
      ScalarProperty name=meshtype;interpretation=;datatype=bool_t;arraysize=1;numsamps=1
    CompoundProperty name=.arbGeomParams;schema=
      ArrayProperty name=Col;interpretation=rgba;datatype=float32_t[4];arraysize=507;numsamps=1
    ArrayProperty name=N;interpretation=normal;datatype=float32_t[3];arraysize=507;numsamps=1

It should show like this:

The blend file i used to create this:

Atomic (atomic) added a comment.EditedJan 18 2018, 9:17 PM

This file's geometry does import into Houdini, but the color information is still missing. Even with an Unpack following up the Alembic import node.

This file's geometry does import into Houdini, but the color information is still missing.

@Atomic (atomic) does Houdini tell you anything as to *why*?

Even with an Unpack following up the Alembic import node.

I have no idea what that means.

Looks like vertexcolors are missing in maya as well...
(and interestingly enough maya doesnt give you a single line of output - even in debug AbcImport -mode import -debug "vertex-colours.abc")

Perhaps it would be useful to import the file into Maya/Houdini, add vertex colors, export, and see how it's different.

OK, this is from maya (also writes them indexed)

Object name=/Suzanne/SuzanneShape
  CompoundProperty name=.geom;schema=AbcGeom_PolyMesh_v1
    ScalarProperty name=.selfBnds;interpretation=box;datatype=float64_t[6];arraysize=6;numsamps=1
    ArrayProperty name=P;interpretation=point;datatype=float32_t[3];arraysize=507;numsamps=1
    ArrayProperty name=.faceIndices;interpretation=;datatype=int32_t;arraysize=1968;numsamps=1
    ArrayProperty name=.faceCounts;interpretation=;datatype=int32_t;arraysize=500;numsamps=1
    ArrayProperty name=N;interpretation=normal;datatype=float32_t[3];arraysize=1968;numsamps=1
    CompoundProperty name=.arbGeomParams;schema=
      CompoundProperty name=colorSet1;schema=
        ArrayProperty name=.vals;interpretation=rgba;datatype=float32_t[4];arraysize=397;numsamps=1
        ArrayProperty name=.indices;interpretation=;datatype=uint32_t;arraysize=1968;numsamps=1

looks like indexed is the way to go (and not directly as an ArrayProperty)?

quoting myself from before:

now the line reading ArrayProperty name=.indices;interpretation=;datatype=uint32_t;arraysize=0;numsamps=1 makes me wonder if we actually miss writing the indices? why is this arraysize zero?

So are just the indices missing?
But judging @Sybren A. Stüvel (sybren) 's comments in rB6db0fd65abc6e6d0fb75f35771eee5ca8f113147 and T53711: Alembic don´t import vertex colors correctly he is already aware of this?

looks like indexed is the way to go (and not directly as an ArrayProperty)?

Shame, would have been so much easier to dump the data in there and have it over with. Thanks for testing it.

So are just the indices missing?

As Blender does it now (compared to my personal branch) it tells the Alembic library that it'll be writing indexed, but then it doesn't provide the indices. Of course this is a bug, which can be fixed in two ways: 1) don't tell Alembic we're doing indexed, or 2) actually write the indices. Since 1) is the simplest one (just change true to false in the right API call) I wanted to test that. I guess it doesn't work, so next try will be solution 2); that includes me having to discover how to write indexed stuff to begin with. I'm guessing it writes a value for each face (or maybe even each unique colour), and then for each vertex indices the colour number.

Here is some more info just reported via IRC (pasting here cause user doesnt have an account). It's from a private bug report in UE4

Alembic file with vertex colors make UE4 to crash
Trying to import an alembic file with vertex colors will result in UE4 crashing.

To reproduce the problem make in your 3d program (I'm using blender 2.79b) a simple plane with a transform animation (2 keyframes is enough) and add a vertex color channel (or use the one attached). Export that as alembic. Import it then from Unreal and set the alembic options as "skeletal" and UE4 will crash.

Unreal staff answer:
It seems that Blender outputs the colours slightly incorrect, when reading your file the color property is set to be indexed. However there are no colour indices available (I'm not sure if this adheres to the Alembic spec?) though with shelved CL 3983495 you should be able to import them. In this case it will reuse the vertex position indices to sample the colours.

I've confirmed these observations with Maya, Houdini, UE4. One clue that might be useful--Unity's open-source Alembic importer _does_ accept Blender Alembic files with vertex colors (as long as you use their hard-coded attribute name "rgba").
https://github.com/unity3d-jp/AlembicForUnity

You have to write your own vertex color shader to test this, so I've made an example here:
https://github.com/n1ckfg/AlembicAudioTest

Nick

Unreal staff answer:
It seems that Blender outputs the colours slightly incorrect, when reading your file the color property is set to be indexed. However there are no colour indices available (I'm not sure if this adheres to the Alembic spec?)

This is already known & described in the comments in this task. I hoped we could go without indexing altogether, but it seems we'll have to index the colours. To be continued.

Hi Sybren! I've been following this thread with great interest. I've had reports that in addition to the software already mentioned above, Clarisse and Cinema 4D also don't import the vertex colors written by Blender. I have confirmed that with my own tests.

I've been digging around the web and, although it is awfully hard to find info on Alembic, I suddenly stumbled upon this little gem:

https://help.autodesk.com/view/MAYAUL/2017/ENU/?guid=__cpp_ref__abc_export_2_maya_mesh_writer_8cpp_example_html

Look for the writeColor() function, which looks a lot like Blender's write_mcol() function, but there's clearly indexing going on in there. I'm hoping this will shed some light on "how to write indexed stuff to begin with" :)

I went ahead and took the plunge into the Blender source and sure enough I fixed it! I submitted a patch at https://developer.blender.org/D3704 along with some more info on indexed writing.
I attached an Alembic file that contains a tree model with several vertex color layers. I tested it extensively in Houdini. Please test it and post back your success or failure. I hope it fixes this bug for at least Houdini and Maya.