OBJ: support edge-split option for compatibility with software that has incomplete support for normals #80177
Labels
No Label
Interest
Animation & Rigging
Interest
Blender Cloud
Interest
Collada
Interest
Core
Interest
Documentation
Interest
Eevee & Viewport
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
Import and Export
Interest
Modeling
Interest
Modifiers
Interest
Nodes & Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds, Tests & Devices
Interest
Python API
Interest
Rendering & Cycles
Interest
Sculpt, Paint & Texture
Interest
Translations
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Meta
Good First Issue
Meta
Papercut
Module
Add-ons (BF-Blender)
Module
Add-ons (Community)
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
5 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender-addons#80177
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
System Information
Operating system: Windows-10-10.0.18362-SP0 64 Bits
Graphics card: GeForce GTX 1070/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 452.06
Blender Version
Broken: version: 2.91.0 Alpha, branch: master, commit date: 2020-08-24 10:28, hash:
blender/blender@cb8da6efceWorked: (newest version of Blender that worked as expected)
Hi, I've just set up a new website 3d-illusions.co.uk to start selling some of my Blender work, and I've run into a bit of a problem trying to export the files to different formats so I can sell to a wider audience. FBX files seem to have completely mangled normals and badly formed ngons when opening in 3ds max 2017, and obj files have the same problem unless choosing to import into max as a single object, which is not an option as the end users wont be able to separate without generating more problems.
I've taken the obj file from blender into Houdini and then after amending some of the primitive path attributes to point to the correct material in the mtl file, saved it straight back out, and that seems to correct the problem. I compared the two obj files in VS CODE, and I think the problem with Blenders output is an insufficient number of floating points, or perhaps it's the different order?
It's a bit problematic because I can't save the file to other formats in Houdini unless I upgrade my licence, which I'd prefer not to do just for this functionality. Is this something that you're able to fix, or is this just an insurmountable limitation of Blender at it's core?
.blend, and both obj files below:
separate by loose parts breaks custom normals of resulting pieces.blend (ignore the name of the file, its the same file I used for a different bug report)
from blender.mtl
from blender.obj
from houdini.obj
ps USD seems to be exporting to cm rather than metres even if units are set correctly in Blender's scene/units tab......although this could be an issue with Houdini (i'll investigate that further).
Cheers :)
Added subscriber: @3di
Added subscriber: @KenzieMac130
Seems like it might be a bug in the other software, Blender appears to be generating valid .obj's at-least..
The normals are slightly different but not corrupted, doubt floating point precision is the issue given the ngons. If you look in the face section you will notice that Houdini forces all "vertices" (face corners in blender) to be a unique entry. While blender is making more efficient use of storage by differentiating between "vertex corners" (vertices) and "vertices" (points) in the file contents. It seems that might be causing confusion in that version of 3DSMax (which might not know how to deal with that).
Example:
Blender:
Houdini:
See face elements:
https://en.wikipedia.org/wiki/Wavefront_.obj_file/Face
In the meantime if necessary you might be able to fix that in MeshLab.
Added subscriber: @Raimund58
Thanks Alex. Could you modify blender's exporter to provide the same data as Houdini's? I'm assuming it's possible as Houdini is generating that additional data from the obj file provided by blender.
Tried meshlab but it appears it has the same problems with the normals as 3ds max does. Perhaps Blender's more efficient approach is actually problematic when it comes to cross application compatibility? I'd definitely rather have a slightly larger obj file than worry the files i'm selling aren't going to work in various applications.
Not very familiar with that area of blender code, just looking to help out the export issues to see if it can be addressed. (Edit, just found myself fixing up the obj .mtl code :P)
I was able to successfully import the model you supplied in the description into the latest version of MeshLab with no visible errors. MeshLab's obj exporter generated similar face elements to Houdini. But interestingly enough it seems that it is the only one that's ngons isn't mangled in Windows' 3D Viewer. For normal issues in the FBX exporter you can try different smoothing group options (normals only/edge/face) and see if that solves normal issues, but if the ngons are giving you issues you can also provide a triangulated/quads and tri version of your model as well if needed. Different software have different algorithms for triangulating ngons, you might be running up against that issue.
This kind of asset distribution is not an unusual practice as game pipelines prefer pre-triangulated meshes for flawless engine import while VFX prefers quaded meshes for deformation and sculpt, ngons can be dangerous when multiple apps are involved.
Hi @astrand130, great news that you're fixing up the .mtl code :)
Regarding meshlab, here's a different part of the model that shows the shading issues more clearly in meshlab. Be sure to turn on smooth shading in meshlab to see the problem.
problem with normals after export.obj
Regarding FBX, I've tried every concievable combination of settings to no avail, so i'm guessing fbx is even more susceptible to blender's more efficient but less robust geometry storage.
OBJ, It is very strange that in 3ds max the problem can be solved by importing all obj objects as a single mesh, perhaps that would be a clue for someone with more knowledge of the obj format? Users can't really import as single objects though, because it's not possible to keep the normals in max when detaching elements (that I know of).
Ah, Thats a rather interesting series of artifacts... Did not see that when inspecting the handle in MeshLab. Almost looks like tangent space mismatch but without the tangent normal map. I will take a look at it later. My PC is in maintainence for a bit. (HDD replacement likely coming up) Are the shading issues fixed when re-calculating the normals in 3DSMax? What about triangulating before export? I have a couple more theories on what that shading error might be caused by.
Since I started adding PBR support to mtl I could probably just fix this task if it turns out to be a bug on Blender's end.
Ouch, good excuse to get an m.2 (they're very cheap at the moment apparently). Retriangulation makes the issue even worse, recalculating the normals doesnt help either. The only thing that helps is to import the obj as a single object rather than keeping all the individual pieces. FBX just breaks completely where there are ngons.
Alright. Let me try to separate the two issues.
FBX is broken with Ngons: In my experience that is usually a mismatch between different software's handling of ngon tessellation. (Might be mistaken but I would need more information/screenshots of shading, triangulated wireframes comparing Blender with 3DSMax).
Weird Obj faceting... I have a feeling that (if it is a blender issue) it could be related to either non-unit normal vectors or the loading of that vertex data. If I could have access to the blend files (or parts of them) that you used to make "problems with normals after import.obj" that will be helpful. I will try loading that into meshlab or (a trial of max if necessary) to replicate the issue.
I cannot guarantee any fixes soon, I need to work on my PC and submit other code before 2.91 goes to bug fix only on Sep 16. But I will try my best when in the clear.
In the meantime keep searching for any potential viable workarounds or fixes in your pipeline. I know a lot of posts on forums like Polycount and Art Station have good information on normal/tangent space/topology issues when going between applications. Best wishes.
Added subscriber: @ideasman42
This seems more like a bug in 3DS MAX, did you report a bug to them?
How can it be a bug in 3ds max if it’s happening in multiple softwares, but not if the obj is recreated in Houdini?
It's not out of the question that the geometry is perfectly valid but there is a bug in the importers by other software. I will try to verify whether or not that is the case in the coming weeks.
exported obj files can't be opened in 3ds maxto Obj: Exported normals compatibility issuesAdded subscriber: @ankitm
Changed status from 'Needs Triage' to: 'Confirmed'
After going between software packages a bit I can confirm that the issues with mangled ngons stems from a mismatch in tessellation algorithms and can be fixed with the "triangulate faces" option.
As for the normals I was able to replicate the faceting issue with the model and in a couple of tests in MeshLab and TopoGun3. Windows 3D Viewer and Blender were both able to load the Obj with perfect normals. The obj file is completely valid, the other importers are the issue.
However when deleting the normal information from the file the results pointed to something more clear... The results in MeshLab and TopoGun3 remained the same... Blender loaded the model with flat shading and Windows 3D Viewer said the file had been corrupted. This indicates that some software when loading an Obj file does not load normals and instead generates them using their own algorithms. This creates broken results with auto-smooth (which works on a continuous set of vertices with normal discontinuities between splits stored in face corners). Any software that bothers to actually load the normal data will produce correct results while software that doesn't will have inevitable artifacts. Production 3D models often don't have edge splits or the need of auto-smoothing as they primarily focus on creating quad control surfaces for subdivision surfaces, while often most models for 3D Design are created in packages that primarily use smoothing groups, low poly assets are still a nieche category as such it makes sense why many 3D packages would opt not to load normals from Obj (or other formats)... (although it would certainly help to ask them on their forums)
Solution: Substitute Auto-Smooth for the Edge Split Modifier to create vertex discontinuities to help software that generates normals better understand and resolve edge splits. This could be done by the user or by adding such an option to the exporter.
Todo: Add Split Edges option to Obj Exporter
Obj: Exported normals compatibility issuesto Obj: Export needs edge split option for compatibilityPlease open a new issue for this if you can confirm it. Thank you!
@astrand130
@astrand130 I'm getting the normals issue when importing and choosing to use the files normals (not recalculating). Triangulating resolves the broken ngons, but makes shading broken (I believe even if loading the file back into blender).
Don't worry if you can't find the problem though, it's no problem to just keep regenerating the obj in Houdini to ensure maximum compatibility across softwares.
Yes, Blender is creating valid OBJ files. OBJ however is a format with a rather non-standardized ecosystem of files/exporters/importers.
Yes, there is software that loads normal information from OBJ, but there are others that don't and generate normals from OBJ smoothing groups.
Yes, software that doesn't load normals loads the geometry. However auto-smoothing has a tendency to break their calculation of normals. (wasn't able to figure out why, but edge splits worked as a more correct substitute in this scenerio)
In addition to 3DSMax other software (I tested) that has this issue includes MeshLab and Topogun3.
I was initially considering marking this invalid but upon reflection this seems like it should be a compatibility option to add to the exporter, thus I strongly suggest adding the ability to apply edge splits replicating auto-smooth at export time to maximize compatibility with other software.
Okay, so when you sent me the picture of the normal faceting in MeshLab I assumed that is the error you are referring to having in Max as well. When testing in a variety of software I isolated that issue to what I previously described. But it appears you are having more than one issue in your workflow (uniquely specific to Max?) that you are describing in this bug report?
Triangulating NGONs with different algorithms will usually result in different barycentric interpolation patterns, this is likely the issue you are experiencing. This is the reason that most production environments avoid NGONS. (I'm going to drop the NGON subject as there is not much you can do about that).
I am trying to isolate this issue to a specific cause, it would help if we could narrow this down to a specific issue.
Hi Alex. the visual appearance of the normals in max and meshlab appeared to be identically incorrect. Max is able to read the split normal data because if the option to import as a single object is checked, then the file normals appear fine (i'm selecting to use the file normals and not to recalculate them at import). If I resave the obj file in Houdini then max can import as multiple objects without the normal issue, so it seems that Houdini is producing more widely compatible obj files. I think you mentioned it was because Houdini adds additional vertex normal data whereas blender does it in a more efficient way?
It's no biggy anyway, close it down if you like. This is probably something I can code myself when things quiet down with client work.
@astrand130, thanks for the feedback, setting as a paper-cut.
Obj: Export needs edge split option for compatibilityto OBJ: support edge-split option for compatibility with software that has incomplete support for normalsDon't worry about it. Your' report actually highlighted an important issue that needs to be addressed in obj I/O.
If you want to investigate further: Try deleting the normal data in the file in a text editor and re-importing it in MAX. If the issue persists it should validate my assumption, but If it fails to load in MAX or produces different results that indicates that there is something else going on. If so edge split modifier in Blender will create similar normals/topology continuity as in the houdini example, no need to buy an expensive license to another software.
Awesome, thanks dude.
in my models I'm generating custom normals from the data transfer modifier to correct normals after boolean or knife project. Are you saying if I set the edge angle to something really low in the edge split modifier, like 0.00001, then all edges will be split and the custom normals will be re-assigned to each of the corner vertices...so still appearing smooth where necessary even though they're split?
@astrand130 I decided to have a go at the test you mentioned. stripping the normals resulted in the model importing without the shading artefacts that were being caused by Blender's obj file but not Houdini's obj file, this is because max was forced to recalculate it's own normals rather than use those Blender had created in the file. Here's the file if you want to give it a try:
problem with normals after export STRIPPED NORMALS.obj
If i'm understanding correctly, blender writes out point normals (in houdini language) because there only exists one vertex at corners unless they're purposefully split either using mark as sharp, edge split modifier, or weighed normal modifier, whereas houdini writes out vertex normals? So basically I just need to use Bmesh to create one corner vertex per connected face, and transfer the value of the point normal to each of the new vertices? sorry, haven't really looked into Bmesh and how blender stores geometry yet, I think it uses just one corner vertex for all connected faces?
No, it writes face corner normals (vertex in houdini language)
I don't think the weighted normal modifier creates vertex discontinuities (might be mistaken) but you can tell by applying the modifier and seeing if there are "doubles". (we need to create those double vertices for compatibility)
"point" (vertex in blender language) normals (I believe? looked that way in the files)
Whatever code you write should replicate the results of the applied edge split modifier. (haven't worked much with bmesh, I can't help you there)
Ah ok, i'll just try splitting the entire model up into seperate faces and then using the data transfer modifier to project the original normals back onto the new vertices.
That will make every face faceted as other programs generate their normals based on topology continuity, by doing that you just made every face discontinuous. You need to substitute the edge split modifier for whenever you use use auto-smooth. Normals of any kind will not be loaded in these applications.