Mesh normals handling in Alembic #71246

New Issue

Sybren A. Stüvel · 2019-10-31T16:57:13+01:00

Sybren A. Stüvel commented

2019-10-31 16:57:13 +01:00

How Blender currently handles normals in the Alembic import/export causes various issues and confusion. Examples of previous issues are #56792 (Alembic doesn't export custom normals correctly), #69182 (Auto-Smooth does not work on Alembic meshes without normals), and #71130 (Alembic split normal export issue). The goal of this design task is to document the current behaviour, describe issues, and propose a new approach.

Current behaviour

Current behaviour on export:

Exporting normals can be globally turned off by an exporter option.
Per mesh
- Auto Smooth is turned on → loop normals are exported.
- One or more polys are marked smooth → loop normals are exported.
- Otherwise, vertex normals are exported.

Current behaviour on import:

An Alembic mesh can contain various kinds of normals, but only one at a time. So it only contains loop normals, or vertex normals (or other normals that Blender completely ignores).

The Alembic mesh contains vertex normals → mark mesh as smooth.
The Alembic mesh contains loop normals → use as custom loop normals, and enble Auto Smooth to have Blender actually use them.
The Alembic mesh contains no normals → mark mesh as flat.

Issues with the Current Behaviour

The current behaviour is hard to handle, mostly due to the fairly hidden and unintuitive use of the Auto Smooth checkbox. Furthermore, the handling of absent normals differs between Blender and other software; for example, the Alembic plugin for Maya assumes "no normals" means "smooth shaded", whereas Blender does the opposite.

Exporting normals can also have a significant effect on the file size. Even though Alembic is not aiming to be the smallest file format ever, it's still beneficial when Blender avoids producing overly large files when this can be avoided. I'm assuming that smooth shading is more common than flat shading.

Proposed Behaviour

The proposed behaviour aims to be easier to understand, produce smaller files in general cases, and be more compatible with other DCC suites.

Proposed behaviour on export:

Exporting normals can be globally turned off by an exporter option.
Per mesh
- Custom loop normals are defined → loop normals are exported.
- One or more polys are marked flat → loop normals are exported.
- Otherwise, no normals are exported.

Proposed behaviour on import:

An Alembic mesh can contain various kinds of normals, but only one at a time. So it only contains loop normals, or vertex normals (or other normals that Blender completely ignores).

The Alembic mesh contains loop normals → use as custom loop normals, and enble Auto Smooth to have Blender actually use them.
The Alembic mesh contains vertex normals → convert to loop normals, and handle as above.
The Alembic mesh contains no normals → mark mesh as smooth.

Changes & Issues

The major changes in behaviour are:

"No normals" used to be interpreted as "flat", and I propose to change this to "smooth". This of course is a backward-incompatible change.
Assuming smooth shading is more common (it is in exporting animated characters), exporting will be faster and use less disk space.
Vertex normals are never written to Alembic any more.

To Be Discusses

Please let me know what you think of this proposal, mostly whether this is indeed as nice as I think it is (I think it's rather splendid) or whether there are downsides that I haven't seen/described yet.

How Blender currently handles normals in the Alembic import/export causes various issues and confusion. Examples of previous issues are #56792 (Alembic doesn't export custom normals correctly), #69182 (Auto-Smooth does not work on Alembic meshes without normals), and #71130 (Alembic split normal export issue). The goal of this design task is to document the current behaviour, describe issues, and propose a new approach. ## Current behaviour ### Current behaviour on export: - Exporting normals can be globally turned off by an exporter option. - Per mesh - Auto Smooth is turned on → **loop** normals are exported. - One or more polys are marked smooth → **loop** normals are exported. - Otherwise, **vertex** normals are exported. ### Current behaviour on import: An Alembic mesh can contain various kinds of normals, but only one at a time. So it only contains loop normals, or vertex normals (or other normals that Blender completely ignores). - The Alembic mesh contains **vertex** normals → mark mesh as smooth. - The Alembic mesh contains **loop** normals → use as custom loop normals, and enble Auto Smooth to have Blender actually use them. - The Alembic mesh contains **no** normals → mark mesh as flat. ## Issues with the Current Behaviour The current behaviour is hard to handle, mostly due to the fairly hidden and unintuitive use of the Auto Smooth checkbox. Furthermore, the handling of absent normals differs between Blender and other software; for example, [the Alembic plugin for Maya](https://github.com/alembic/alembic/blob/8641f34/maya/AbcExport/MayaMeshWriter.cpp#L659) assumes "no normals" means "smooth shaded", whereas Blender does the opposite. Exporting normals can also [have a significant effect on the file size](https://groups.google.com/forum/#!msg/alembic-discussion/xmP2QMlIWKM/-chYFSPWU_UJ). Even though Alembic is not aiming to be the smallest file format ever, it's still beneficial when Blender avoids producing overly large files when this can be avoided. I'm assuming that smooth shading is more common than flat shading. ## Proposed Behaviour The proposed behaviour aims to be easier to understand, produce smaller files in general cases, and be more compatible with other DCC suites. ### Proposed behaviour on export: - Exporting normals can be globally turned off by an exporter option. - Per mesh - Custom loop normals are defined → **loop** normals are exported. - One or more polys are marked flat → **loop** normals are exported. - Otherwise, **no** normals are exported. ### Proposed behaviour on import: An Alembic mesh can contain various kinds of normals, but only one at a time. So it only contains loop normals, or vertex normals (or other normals that Blender completely ignores). - The Alembic mesh contains **loop** normals → use as custom loop normals, and enble Auto Smooth to have Blender actually use them. - The Alembic mesh contains **vertex** normals → convert to loop normals, and handle as above. - The Alembic mesh contains **no** normals → mark mesh as **smooth**. ### Changes & Issues The major changes in behaviour are: - "No normals" used to be interpreted as "flat", and I propose to change this to "smooth". This of course is a backward-incompatible change. - Assuming smooth shading is more common (it is in exporting animated characters), exporting will be faster and use less disk space. - Vertex normals are never written to Alembic any more. ## To Be Discusses Please let me know what you think of this proposal, mostly whether this is indeed as nice as I think it is (I think it's rather splendid) or whether there are downsides that I haven't seen/described yet.

Sybren A. Stüvel self-assigned this 2019-10-31 16:57:13 +01:00

Sybren A. Stüvel commented

2019-10-31 16:57:13 +01:00

Added subscribers: @dr.sybren, @mont29, @eyecandy, @ajohnson223, @NikoMaisuradze, @Ruben-1

Fin O'Riordan commented

2019-10-31 17:27:25 +01:00

Added subscriber: @FinbarrORiordan

Julien Leménicier commented

2019-10-31 17:30:55 +01:00

Added subscriber: @lemenicier_julien

Michael Hermann commented

2019-10-31 17:33:05 +01:00

Added subscriber: @MichaelHermann

Maciej Jutrzenka commented

2019-10-31 18:46:13 +01:00

Added subscriber: @MaciejJutrzenka

Maciej Jutrzenka commented

2019-10-31 18:46:13 +01:00

would it be hard to make blender handle normals like vertex groups? So in the tab where we have normals. we simply choose the set? same like it is with UV.. and other stuff? and we simply have multiple flags where we can.. toggle what normal set is for viewport and what for rendering as defult... and also would be awesome if you could read those in shader useing.. attrbiute node for example.

Conrad Dueck commented

2019-10-31 19:14:11 +01:00

Added subscriber: @ConradDueck

Sybren A. Stüvel commented

2019-10-31 21:50:11 +01:00

In #71246#803956, @MaciejJutrzenka wrote:
would it be hard to make blender handle normals like vertex groups?

Yes, that would require much more work in Blender itself, and is beyond the scope of saving Blender meshes (as they are currently) to Alembic.

> In #71246#803956, @MaciejJutrzenka wrote: > would it be hard to make blender handle normals like vertex groups? Yes, that would require much more work in Blender itself, and is beyond the scope of saving Blender meshes (as they are currently) to Alembic.

Jayraj Kharvadi commented

2019-11-01 06:06:21 +01:00

Added subscriber: @JayrajKharvadi

Bastien Montagne commented

2019-11-01 09:13:06 +01:00

I would change that point on new import behavior:

The Alembic mesh contains vertex normals → mark mesh as smooth.

… to The Alembic mesh contains vertex normals → use as custom loop normals, and enble Auto Smooth to have Blender actually use them.

Think you will find some cases where custom normals (e.g. for toon shading effects) have been stored as vertex normals, because no 'hard edge' was desired, but normal directions others than default ones were used. Note that BKE has an API to set custom normals from per-vertex values as well. That’s what FBX is doing at least.

I would change that point on new import behavior: > The Alembic mesh contains vertex normals → mark mesh as smooth. … to `The Alembic mesh contains vertex normals → use as custom loop normals, and enble Auto Smooth to have Blender actually use them.` Think you will find some cases where custom normals (e.g. for toon shading effects) have been stored as vertex normals, because no 'hard edge' was desired, but normal directions others than default ones were used. Note that BKE has an API to set custom normals from per-vertex values as well. That’s what FBX is doing at least.

Andrew commented

2019-11-01 09:17:41 +01:00

In #71246#804089, @mont29 wrote:
I would change that point on new import behavior:

The Alembic mesh contains vertex normals → mark mesh as smooth.

… to The Alembic mesh contains vertex normals → use as custom loop normals, and enble Auto Smooth to have Blender actually use them.

Think you will find some cases where custom normals (e.g. for toon shading effects) have been stored as vertex normals, because no 'hard edge' was desired, but normal directions others than default ones were used. Note that BKE has an API to set custom normals from per-vertex values as well. That’s what FBX is doing at least.

Another alternative:
Change:
The Alembic mesh contains vertex normals → mark mesh as smooth.
To:
The Alembic mesh contains vertex normals → determine if the provided vertex normals are identical to automatically calculated "smooth" mesh normals, and if they are, mark mesh as smooth. Otherwise, use as custom vertex normals, and enble Auto Smooth to have Blender actually use them.

However, this leads me to a question: Does blender have a concept of "custom vertex normals", or must any kind of custom normals be loop normals?

> In #71246#804089, @mont29 wrote: > I would change that point on new import behavior: > >> The Alembic mesh contains vertex normals → mark mesh as smooth. > … to `The Alembic mesh contains vertex normals → use as custom loop normals, and enble Auto Smooth to have Blender actually use them.` > > Think you will find some cases where custom normals (e.g. for toon shading effects) have been stored as vertex normals, because no 'hard edge' was desired, but normal directions others than default ones were used. Note that BKE has an API to set custom normals from per-vertex values as well. That’s what FBX is doing at least. Another alternative: Change: The Alembic mesh contains vertex normals → mark mesh as smooth. To: The Alembic mesh contains vertex normals → determine if the provided vertex normals are identical to automatically calculated "smooth" mesh normals, and if they are, mark mesh as smooth. Otherwise, use as custom vertex normals, and enble Auto Smooth to have Blender actually use them. However, this leads me to a question: Does blender have a concept of "custom vertex normals", or must any kind of custom normals be loop normals?

Bastien Montagne commented

2019-11-01 09:21:34 +01:00

Blender has no concept of custom vertex normals, only loops (aka face corners) ones. Checking for 'equal to auto normals' might be possible, but won't be a cheap process, and not so sure it would be worth is… If you export your normals, even as vnors, then one can expect them to have some importance?

Andrew commented

2019-11-01 09:28:08 +01:00

In #71246#804091, @mont29 wrote:
Blender has no concept of custom vertex normals, only loops (aka face corners) ones. Checking for 'equal to auto normals' might be possible, but won't be a cheap process, and not so sure it would be worth is… If you export your normals, even as vnors, then one can expect them to have some importance?

As to whether there's always importance, it's difficult for me to say without having surveyed all of the DCC exporters, but I think your approach is a good one in the near term especially since the comparison method can always be added later without harming backward compatibility if it's found that other many exporters unnecessarily include vertex normals.

> In #71246#804091, @mont29 wrote: > Blender has no concept of custom vertex normals, only loops (aka face corners) ones. Checking for 'equal to auto normals' might be possible, but won't be a cheap process, and not so sure it would be worth is… If you export your normals, even as vnors, then one can expect them to have some importance? As to whether there's always importance, it's difficult for me to say without having surveyed all of the DCC exporters, but I think your approach is a good one in the near term especially since the comparison method can always be added later without harming backward compatibility if it's found that other many exporters unnecessarily include vertex normals.

Sybren A. Stüvel commented

2019-11-01 09:43:53 +01:00

FWIW, so far Blender has had the "vertex normals → shade smooth" approach, and I haven't seen any bug report about this behaviour as of yet.

@mont29 How expensive/cheap is your proposed behaviour? If it's relatively cheap (compared to loading loop normals when they are stored in Alembic) I don't see any reason we shouldn't do this. Of course meshes can be deforming, so this process would happen every frame in such a case.

FWIW, so far Blender has had the "vertex normals → shade smooth" approach, and I haven't seen any bug report about this behaviour as of yet. @mont29 How expensive/cheap is your proposed behaviour? If it's relatively cheap (compared to loading loop normals when they are stored in Alembic) I don't see any reason we shouldn't do this. Of course meshes can be deforming, so this process would happen every frame in such a case.

Andrew commented

2019-11-01 09:46:16 +01:00

FWIW, so far Blender has had the "vertex normals → shade smooth" approach, and I haven't seen any bug report about this behaviour as of yet.

I can probably construct a failing testcase for this, if it's useful.

> FWIW, so far Blender has had the "vertex normals → shade smooth" approach, and I haven't seen any bug report about this behaviour as of yet. I can probably construct a failing testcase for this, if it's useful.

Bastien Montagne commented

2019-11-01 09:48:36 +01:00

Reading vnors into custom lnors is about as expensive as reading real lnors.

Sybren A. Stüvel commented

2019-11-01 11:08:28 +01:00

In #71246#804098, @mont29 wrote:
Reading vnors into custom lnors is about as expensive as reading real lnors.

Ok. Let's just do that then.

In #71246#804097, @ajohnson223 wrote:

FWIW, so far Blender has had the "vertex normals → shade smooth" approach, and I haven't seen any bug report about this behaviour as of yet.

I can probably construct a failing testcase for this, if it's useful.

Thanks, but with the change @mont29 suggests we won't have that behaviour anyway.
Having a test case where custom loop normals are stored as vertex normals, that would be handy.

> In #71246#804098, @mont29 wrote: > Reading vnors into custom lnors is about as expensive as reading real lnors. Ok. Let's just do that then. > In #71246#804097, @ajohnson223 wrote: >> FWIW, so far Blender has had the "vertex normals → shade smooth" approach, and I haven't seen any bug report about this behaviour as of yet. > > I can probably construct a failing testcase for this, if it's useful. Thanks, but with the change @mont29 suggests we won't have that behaviour anyway. Having a test case where custom loop normals are stored as vertex normals, that would be handy.

Sybren A. Stüvel commented

2019-11-05 18:25:51 +01:00

I've added an implementation in D6197, but I need an Alembic file to test it. If someone could share with me an Alembic mesh that contains explicit vertex normals + a screenshot so that I can verify that they're loaded correctly, that would be great.

I've added an implementation in [D6197](https://archive.blender.org/developer/D6197), but I need an Alembic file to test it. If someone could share with me an Alembic mesh that contains explicit vertex normals + a screenshot so that I can verify that they're loaded correctly, that would be great.

Fin O'Riordan commented

2019-11-05 19:03:01 +01:00

This comment was removed by @FinbarrORiordan

*This comment was removed by @FinbarrORiordan*

Daniel Bystedt commented

2019-11-05 21:01:00 +01:00

Added subscriber: @DanielBystedt

Daniel Bystedt commented

2019-11-05 21:01:00 +01:00

Hi Sybren!

I did a quick setup in Houdini and exported 2 alembics. One where normals are assigned to vertecies and one on points. There is also a csv spreadsheet with point positions and normals per vertex

customVertexNormalsForSybren.rar

Hi Sybren! I did a quick setup in Houdini and exported 2 alembics. One where normals are assigned to vertecies and one on points. There is also a csv spreadsheet with point positions and normals per vertex ![custom normals in houdini.png](https://archive.blender.org/developer/F7951439/custom_normals_in_houdini.png) [customVertexNormalsForSybren.rar](https://archive.blender.org/developer/F7951441/customVertexNormalsForSybren.rar)

Fin O'Riordan commented

2019-11-05 22:07:57 +01:00

This comment was removed by @FinbarrORiordan

*This comment was removed by @FinbarrORiordan*

Sybren A. Stüvel commented

2019-11-06 10:15:45 +01:00

Thanks @DanielBystedt , your files confirmed the code is doing its thing properly.

It's indeed a bit confusing how different applications name things:

Alembic	Blender	Houdini
kFacevaryingScope	Loop Normals	Vertex Normals
kVaryingScope	Vertex Normals	Point Normals

![image.png](https://archive.blender.org/developer/F7958395/image.png) Thanks @DanielBystedt , your files confirmed the code is doing its thing properly. It's indeed a bit confusing how different applications name things: | Alembic | Blender | Houdini | | ------------------ | -------------- | -------------- | | kFacevaryingScope | Loop Normals | Vertex Normals | | kVaryingScope | Vertex Normals | Point Normals |

Andrew commented

2019-11-06 10:29:05 +01:00

@dr.sybren Hmm, I actually thought Alembic kVertexScope == Blender Vertex Normals, rather than kVaryingScope.

Do you know offhand the difference between kVaryingScope and kVertexScope in alembic-world?

edit for reference, here's the Geometry Scope enum from Blender:
a9efabe336/lib/Alembic/AbcGeom/GeometryScope.h (L120)

@dr.sybren Hmm, I actually thought Alembic kVertexScope == Blender Vertex Normals, rather than kVaryingScope. Do you know offhand the difference between kVaryingScope and kVertexScope in alembic-world? *edit* for reference, here's the Geometry Scope enum from Blender: https://github.com/alembic/alembic/blob/a9efabe336b707f7f0b326bc1d9fb1e7a38d922d/lib/Alembic/AbcGeom/GeometryScope.h#L120

Sybren A. Stüvel commented

2019-11-06 10:50:09 +01:00

In #71246#806519, @ajohnson223 wrote:
@dr.sybren Hmm, I actually thought Alembic kVertexScope == Blender Vertex Normals, rather than kVaryingScope.

Both are interpreted by Blender as vertex scope, see process_normals() in abc_mesh.cc: https://developer.blender.org/diffusion/B/browse/master/source/blender/alembic/intern/abc_mesh.cc$902

Do you know offhand the difference between kVaryingScope and kVertexScope in alembic-world?

Nope.

> In #71246#806519, @ajohnson223 wrote: > @dr.sybren Hmm, I actually thought Alembic kVertexScope == Blender Vertex Normals, rather than kVaryingScope. Both are interpreted by Blender as vertex scope, see `process_normals()` in `abc_mesh.cc`: https://developer.blender.org/diffusion/B/browse/master/source/blender/alembic/intern/abc_mesh.cc$902 > Do you know offhand the difference between kVaryingScope and kVertexScope in alembic-world? Nope.

Andrew commented

2019-11-06 10:50:33 +01:00

I got some clarification on kVaryingScope vs. kVertexScope here:

https://groups.google.com/d/msg/alembic-discussion/m6mVeR90bZY/iBPX6it2AboJ

quote:

kVertexScope = (for a mesh, this is a synonym of kVaryingScope. For other primitives, it can mean something distinct)

I got some clarification on kVaryingScope vs. kVertexScope here: https://groups.google.com/d/msg/alembic-discussion/m6mVeR90bZY/iBPX6it2AboJ quote: > kVertexScope = (for a mesh, this is a synonym of kVaryingScope. For other primitives, it can mean something distinct)

Sybren A. Stüvel commented

2019-11-06 10:53:25 +01:00

Changed status from 'Open' to: 'Resolved'

Sybren A. Stüvel closed this issue

2019-11-06 10:53:25 +01:00

Sybren A. Stüvel commented

2019-11-06 10:53:25 +01:00

Thanks @ajohnson223

The changes in this task were implemented in b0e7a1d4b4, and as such will be part of Blender 2.82.

Thanks @ajohnson223 The changes in this task were implemented in b0e7a1d4b4, and as such will be part of Blender 2.82.

Andrew commented

2019-11-06 11:03:26 +01:00

Thanks again @dr.sybren, I'm sure it's working nicely but i'll put it through its paces soon.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Mesh normals handling in Alembic #71246

Current behaviour

Current behaviour on export:

Current behaviour on import:

Issues with the Current Behaviour

Proposed Behaviour

Proposed behaviour on export:

Proposed behaviour on import:

Changes & Issues

To Be Discusses