New 3.1 OBJ exporter writes incorrect polygon/vertex groups in some cases #96824

New Issue

thoennes · 2022-03-28T04:08:09+02:00

thoennes commented

2022-03-28 04:08:09 +02:00

System Information
Operating system: Mojave 10.14.6
Graphics card: NVIDIA GeForce GT 750M 2 GB

Blender Version
Broken: Version 3.1.0 (3.1.0 2022-03-09)
Worked: Version 2.93.4

Summary
Blender OBJ import has the option to "keep vertex order". When exporting the Blender Object back out, the v's, g's and f's should be in the same order as they were imported.

Details
The bug is triggered if the input file spreads a single group's faces over multiple g labels. If a group's faces are in a single g label, then the OBJ exports correctly. the OBJ format allows a group's faces to be declared in multiple separate g labels. They are logically connected into a single group. Some applications export OBJs with faces in single groups, some export in internal vert order, adding g labels as needed.

Implications
Some commercial packages use objects as shape keys for morphs and animations. A specific requirement is that vertex order is maintained. This includes faces and group compositions. This has worked correctly up until the new C++ OBJ importer.

Steps
Import an OBJ with the faces of a single group spread over multiple g labels, with the only option being "poly groups" and "keep vertex order". Export with same options. The exported OBJ in 3.1.0 has broken groups. The v order, f structures, and f order are correct. But some of the f's show up in the wrong g's

Note: the attached files contain a commercial figure consisting of proprietary information. Do not use or distribute beyond what is required for this BUG process.

Dawn.obj
2.93.4-Dawn.obj-import.blend
2.93.4-Dawn.obj-import.obj
3.1.0-Dawn.obj-import.blend
3.1.0-Dawn.obj-import.obj

The following script compares the files. The vert count and face count's are equal, but their distribution within groups in the 3.1.0 exported OBJ are incorrect

import re

files = [
	'Dawn.obj',
	'2.93.4-Dawn.obj-import.obj',
	'3.1.0-Dawn.obj-import.obj'
]

raw = {}
obj = {}

print('FILES:', files)
print()

print('{: <30} {: <20} {: <20}'.format(
	'FILENAME',
	'VERT COUNT',
	'FACE COUNT'
))

for filename in files:
	file = open(filename, 'r')
	data = file.readlines()

	raw[filename] = []
	obj[filename] = {'v':[],'g':{}}

	grp = None
	for line in data:
		line = re.sub(r'\n$', r'', line)
		raw[filename].append(line)
		ins = line[:2]
		if ins == 'v ':
			obj[filename]['v'].append(line)
		elif ins == 'g ' :
			grp = line.split(' ')[1]
			obj[filename]['g'].setdefault(grp, [])
		elif ins == 'f ' :
			obj[filename]['g'][grp].append(line)

	print('{: <32} {: <20} {: <20}'.format(
		filename,
		len(obj[filename]['v']),
		sum([len(obj[filename]['g'][x]) for x in obj[filename]['g']])
	))

print()
print('{: <15} {: ^75}'.format('', 'GROUP FACE COUNTS'))
print('{: <15} {: <15} {: <30} {: <30}'.format('', *files))

for grp in obj[files[0]]['g']:
	print('{: <15} {: <15} {: <30} {: <30}'.format(
		grp,
		len(obj[files[0]]['g'][grp]),
		len(obj[files[1]]['g'][grp]),
		len(obj[files[2]]['g'][grp])
	))

output

FILES: ['Dawn.obj', '2.93.4-Dawn.obj-import.obj', '3.1.0-Dawn.obj-import.obj']

FILENAME                       VERT COUNT           FACE COUNT
Dawn.obj                         39096                39326
2.93.4-Dawn.obj-import.obj       39096                39326
3.1.0-Dawn.obj-import.obj        39096                39326

                                             GROUP FACE COUNTS
                Dawn.obj        2.93.4-Dawn.obj-import.obj     3.1.0-Dawn.obj-import.obj
head            10188           10188                          10350
lowerJaw        4582            4582                           4504
lEye            624             624                            624
rEye            624             624                            624
tongue01        72              72                             80
tongue02        72              72                             72
tongue03        72              72                             72
tongue04        312             312                            288
rForeArm        568             568                            645
rCollar         463             463                            536
rShldr          453             453                            380
rHand           691             691                            766
rThumb1         141             141                            133
rThumb2         182             182                            181
rCarpal         318             318                            322
rIndex2         160             160                            200
rIndex3         301             301                            281
rIndex1         120             120                            80
rMid2           160             160                            200
rMid1           141             141                            100
rRing2          160             160                            200
rRing1          120             120                            80
rPinky2         160             160                            200
rPinky1         125             125                            80
rMid3           301             301                            281
rRing3          302             302                            282
rPinky3         298             298                            278
rThumb3         300             300                            280
lForeArm        568             568                            645
lCollar         463             463                            536
lShldr          453             453                            380
lHand           691             691                            766
lThumb1         141             141                            133
lThumb2         182             182                            181
lCarpal         318             318                            322
lIndex2         160             160                            200
lIndex3         301             301                            281
lIndex1         120             120                            80
lMid2           160             160                            200
lMid1           141             141                            100
lRing2          160             160                            200
lRing1          120             120                            80
lPinky2         160             160                            200
lPinky1         125             125                            80
lMid3           301             301                            281
lRing3          302             302                            282
lPinky3         298             298                            278
lThumb3         300             300                            280
rThigh          626             626                            692
rShin           562             562                            565
rFoot           597             597                            605
rToe            689             689                            744
rBigToe         248             248                            228
rIndexToe       308             308                            288
rMidToe         283             283                            263
rRingToe        263             263                            243
rPinkyToe       245             245                            225
lThigh          626             626                            692
lShin           562             562                            565
lFoot           597             597                            605
lToe            689             689                            744
lBigToe         248             248                            228
lIndexToe       308             308                            288
lMidToe         283             283                            263
lRingToe        263             263                            243
lPinkyToe       245             245                            225
neck            338             338                            442
neck2           260             260                            136
chest2          1678            1678                           1614
chest           446             446                            444
abdomen2        256             256                            256
abdomen         320             320                            326
pelvis          618             618                            618
hip             294             294                            160

**System Information** Operating system: Mojave 10.14.6 Graphics card: NVIDIA GeForce GT 750M 2 GB **Blender Version** Broken: Version 3.1.0 (3.1.0 2022-03-09) Worked: Version 2.93.4 **Summary** Blender OBJ import has the option to "keep vertex order". When exporting the Blender Object back out, the v's, g's and f's should be in the same order as they were imported. **Details** The bug is triggered if the input file spreads a single group's faces over multiple g labels. If a group's faces are in a single g label, then the OBJ exports correctly. the OBJ format allows a group's faces to be declared in multiple separate g labels. They are logically connected into a single group. Some applications export OBJs with faces in single groups, some export in internal vert order, adding g labels as needed. **Implications** Some commercial packages use objects as shape keys for morphs and animations. A specific requirement is that vertex order is maintained. This includes faces and group compositions. This has worked correctly up until the new C++ OBJ importer. **Steps** Import an OBJ with the faces of a single group spread over multiple g labels, with the only option being "poly groups" and "keep vertex order". Export with same options. The exported OBJ in 3.1.0 has broken groups. The v order, f structures, and f order are correct. But some of the f's show up in the wrong g's > Note: the attached files contain a commercial figure consisting of proprietary information. Do not use or distribute beyond what is required for this BUG process. [Dawn.obj](https://archive.blender.org/developer/F12951226/Dawn.obj) [2.93.4-Dawn.obj-import.blend](https://archive.blender.org/developer/F12951258/2.93.4-Dawn.obj-import.blend) [2.93.4-Dawn.obj-import.obj](https://archive.blender.org/developer/F12951262/2.93.4-Dawn.obj-import.obj) [3.1.0-Dawn.obj-import.blend](https://archive.blender.org/developer/F12951263/3.1.0-Dawn.obj-import.blend) [3.1.0-Dawn.obj-import.obj](https://archive.blender.org/developer/F12951264/3.1.0-Dawn.obj-import.obj) The following script compares the files. The vert count and face count's are equal, but their distribution within groups in the 3.1.0 exported OBJ are incorrect ``` import re files = [ 'Dawn.obj', '2.93.4-Dawn.obj-import.obj', '3.1.0-Dawn.obj-import.obj' ] raw = {} obj = {} print('FILES:', files) print() print('{: <30} {: <20} {: <20}'.format( 'FILENAME', 'VERT COUNT', 'FACE COUNT' )) for filename in files: file = open(filename, 'r') data = file.readlines() raw[filename] = [] obj[filename] = {'v':[],'g':{}} grp = None for line in data: line = re.sub(r'\n$', r'', line) raw[filename].append(line) ins = line[:2] if ins == 'v ': obj[filename]['v'].append(line) elif ins == 'g ' : grp = line.split(' ')[1] obj[filename]['g'].setdefault(grp, []) elif ins == 'f ' : obj[filename]['g'][grp].append(line) print('{: <32} {: <20} {: <20}'.format( filename, len(obj[filename]['v']), sum([len(obj[filename]['g'][x]) for x in obj[filename]['g']]) )) print() print('{: <15} {: ^75}'.format('', 'GROUP FACE COUNTS')) print('{: <15} {: <15} {: <30} {: <30}'.format('', *files)) for grp in obj[files[0]]['g']: print('{: <15} {: <15} {: <30} {: <30}'.format( grp, len(obj[files[0]]['g'][grp]), len(obj[files[1]]['g'][grp]), len(obj[files[2]]['g'][grp]) )) ``` **output** ``` FILES: ['Dawn.obj', '2.93.4-Dawn.obj-import.obj', '3.1.0-Dawn.obj-import.obj'] FILENAME VERT COUNT FACE COUNT Dawn.obj 39096 39326 2.93.4-Dawn.obj-import.obj 39096 39326 3.1.0-Dawn.obj-import.obj 39096 39326 GROUP FACE COUNTS Dawn.obj 2.93.4-Dawn.obj-import.obj 3.1.0-Dawn.obj-import.obj head 10188 10188 10350 lowerJaw 4582 4582 4504 lEye 624 624 624 rEye 624 624 624 tongue01 72 72 80 tongue02 72 72 72 tongue03 72 72 72 tongue04 312 312 288 rForeArm 568 568 645 rCollar 463 463 536 rShldr 453 453 380 rHand 691 691 766 rThumb1 141 141 133 rThumb2 182 182 181 rCarpal 318 318 322 rIndex2 160 160 200 rIndex3 301 301 281 rIndex1 120 120 80 rMid2 160 160 200 rMid1 141 141 100 rRing2 160 160 200 rRing1 120 120 80 rPinky2 160 160 200 rPinky1 125 125 80 rMid3 301 301 281 rRing3 302 302 282 rPinky3 298 298 278 rThumb3 300 300 280 lForeArm 568 568 645 lCollar 463 463 536 lShldr 453 453 380 lHand 691 691 766 lThumb1 141 141 133 lThumb2 182 182 181 lCarpal 318 318 322 lIndex2 160 160 200 lIndex3 301 301 281 lIndex1 120 120 80 lMid2 160 160 200 lMid1 141 141 100 lRing2 160 160 200 lRing1 120 120 80 lPinky2 160 160 200 lPinky1 125 125 80 lMid3 301 301 281 lRing3 302 302 282 lPinky3 298 298 278 lThumb3 300 300 280 rThigh 626 626 692 rShin 562 562 565 rFoot 597 597 605 rToe 689 689 744 rBigToe 248 248 228 rIndexToe 308 308 288 rMidToe 283 283 263 rRingToe 263 263 243 rPinkyToe 245 245 225 lThigh 626 626 692 lShin 562 562 565 lFoot 597 597 605 lToe 689 689 744 lBigToe 248 248 228 lIndexToe 308 308 288 lMidToe 283 283 263 lRingToe 263 263 243 lPinkyToe 245 245 225 neck 338 338 442 neck2 260 260 136 chest2 1678 1678 1614 chest 446 446 444 abdomen2 256 256 256 abdomen 320 320 326 pelvis 618 618 618 hip 294 294 160 ```

thoennes commented

2022-03-28 04:08:09 +02:00

Added subscriber: @unreality

Aras Pranckevicius commented

2022-03-28 13:24:25 +02:00

Added subscriber: @aras_p

Aras Pranckevicius commented

2022-03-28 13:24:25 +02:00

@unreality does the issue still happen in the latest 3.1.1 nightly builds (https://builder.blender.org/download/daily/)? There were a handful of new OBJ exporter regression fixes there since 3.1.0.

Holger Hinze commented

2022-03-28 19:37:26 +02:00

Added subscriber: @PrefoX

Holger Hinze commented

2022-03-28 19:37:26 +02:00

they removed the "keep vertex order" in blender 3.1 at least for me a my friends. that counts for export.

And even in Version 3.1.1 the option is completly missing!!!

they removed the "keep vertex order" in blender 3.1 at least for me a my friends. that counts for export. And even in Version 3.1.1 the option is completly missing!!!

Aras Pranckevicius commented

2022-03-28 20:28:32 +02:00

@PrefoX in 3.1+ the "keep vertex order" is effectively always on, as far as I can tell.

thoennes commented

2022-03-28 23:58:06 +02:00

In #96824#1330966, @aras_p wrote:
@unreality does the issue still happen in the latest 3.1.1 nightly builds (https://builder.blender.org/download/daily/)? There were a handful of new OBJ exporter regression fixes there since 3.1.0.

Yes. The OBJ output, as far as v's, g's, and f's is the same as 3.1.0: the order is still maintained correctly (vertex and winding), but the group composition is incorrect.

> In #96824#1330966, @aras_p wrote: > @unreality does the issue still happen in the latest 3.1.1 nightly builds (https://builder.blender.org/download/daily/)? There were a handful of new OBJ exporter regression fixes there since 3.1.0. Yes. The OBJ output, as far as v's, g's, and f's is the same as 3.1.0: the order is still maintained correctly (vertex and winding), but the group composition is incorrect.

thoennes commented

2022-03-29 00:06:11 +02:00

In #96824#1331130, @PrefoX wrote:
they removed the "keep vertex order" in blender 3.1 at least for me a my friends. that counts for export.

And even in Version 3.1.1 the option is completely missing!!!

To be clear, it's removed as an option in export. It's still an option in import. It may be intentional in that selecting it in import; a user would almost certainly want it in export (no other reason to use the option). And the only reason to select it in export is BECAUSE it was selected in import. Essentially, one less option for the user to mess up. That's probably not a bad thought :)

It seems (and I hope it's intentional) that so long as the OBJ was imported with that option, it will export with that option (assuming no changes to vert count and vertex groups).

The help and tip on the "keep vert order" import option should mention that, as the way it's presented in the workflow (with the export option missing) could confuse people looking at any reference to the process prior to version 3.x; they would wonder where the heck the export option went

I'm not sure where blender is storing that info. It must be stored with the file since it persists, and it's stored with the obj, since it's copied with copies of the object. Both of those features are useful.

> In #96824#1331130, @PrefoX wrote: > they removed the "keep vertex order" in blender 3.1 at least for me a my friends. that counts for export. > > And even in Version 3.1.1 the option is completely missing!!! To be clear, it's removed as an option in export. It's still an option in import. It may be intentional in that selecting it in import; a user would almost certainly want it in export (no other reason to use the option). And the only reason to select it in export is BECAUSE it was selected in import. Essentially, one less option for the user to mess up. That's probably not a bad thought :) It seems (and I hope it's intentional) that so long as the OBJ was imported with that option, it will export with that option (assuming no changes to vert count and vertex groups). The help and tip on the "keep vert order" import option should mention that, as the way it's presented in the workflow (with the export option missing) could confuse people looking at *any* reference to the process prior to version 3.x; they would wonder where the heck the export option went I'm not sure where blender is storing that info. It must be stored with the file since it persists, and it's stored with the obj, since it's copied with copies of the object. Both of those features are useful.

thoennes commented

2022-03-29 04:50:26 +02:00

Uhm... Trying to understand the source code. Is the importer/exporter C++? It looks like it's python.

Am I looking at the wrong place? https://developer.blender.org/diffusion/BA/browse/master/io_scene_obj/init.py

class OBJ_PT_export_geometry(bpy.types.Panel):
    bl_space_type = 'FILE_BROWSER'
    bl_region_type = 'TOOL_PROPS'
    bl_label = "Geometry"
    bl_parent_id = "FILE_PT_operator"
    bl_options = {'DEFAULT_CLOSED'}

    @classmethod
    def poll(cls, context):
        sfile = context.space_data
        operator = sfile.active_operator

        return operator.bl_idname == "EXPORT_SCENE_OT_obj"

    def draw(self, context):
        layout = self.layout
        layout.use_property_split = True
        layout.use_property_decorate = False  # No animation.

        sfile = context.space_data
        operator = sfile.active_operator

        layout.prop(operator, 'use_mesh_modifiers')
        layout.prop(operator, 'use_smooth_groups')
        layout.prop(operator, 'use_smooth_groups_bitflags')
        layout.prop(operator, 'use_normals')
        layout.prop(operator, 'use_uvs')
        layout.prop(operator, 'use_materials')
        layout.prop(operator, 'use_triangles')
        layout.prop(operator, 'use_nurbs', text="Curves as NURBS")
        layout.prop(operator, 'use_vertex_groups')
        layout.prop(operator, 'keep_vertex_order')

And yet that's not what the export panel looks like.

Uhm... Trying to understand the source code. Is the importer/exporter C++? It looks like it's python. Am I looking at the wrong place? https://developer.blender.org/diffusion/BA/browse/master/io_scene_obj/__init__.py ``` class OBJ_PT_export_geometry(bpy.types.Panel): bl_space_type = 'FILE_BROWSER' bl_region_type = 'TOOL_PROPS' bl_label = "Geometry" bl_parent_id = "FILE_PT_operator" bl_options = {'DEFAULT_CLOSED'} @classmethod def poll(cls, context): sfile = context.space_data operator = sfile.active_operator return operator.bl_idname == "EXPORT_SCENE_OT_obj" def draw(self, context): layout = self.layout layout.use_property_split = True layout.use_property_decorate = False # No animation. sfile = context.space_data operator = sfile.active_operator layout.prop(operator, 'use_mesh_modifiers') layout.prop(operator, 'use_smooth_groups') layout.prop(operator, 'use_smooth_groups_bitflags') layout.prop(operator, 'use_normals') layout.prop(operator, 'use_uvs') layout.prop(operator, 'use_materials') layout.prop(operator, 'use_triangles') layout.prop(operator, 'use_nurbs', text="Curves as NURBS") layout.prop(operator, 'use_vertex_groups') layout.prop(operator, 'keep_vertex_order') ``` And yet that's not what the export panel looks like.

Aras Pranckevicius self-assigned this 2022-03-29 14:32:36 +02:00

Aras Pranckevicius commented

2022-03-29 14:32:36 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Aras Pranckevicius commented

2022-03-29 14:32:36 +02:00

@unreality in 3.1.x the OBJ exporter is in C++ (the old exporter python code still exists but is disabled), but the importer is the same as in previous version, i.e. written in Python.

Anyway, I can definitely see the problem with your repro files, thanks! Something is messed up in the polygon vertex group calculation code, it seems, will try to fix it.

@unreality in 3.1.x the OBJ exporter is in C++ (the old exporter python code still exists but is disabled), but the importer is the same as in previous version, i.e. written in Python. Anyway, I can definitely see the problem with your repro files, thanks! Something is messed up in the polygon vertex group calculation code, it seems, will try to fix it.

Aras Pranckevicius changed title from ~~Incorrect groups in exported .obj from imported .obj with "keeping vertex order" option~~ to New 3.1 OBJ exporter writes incorrect polygon/vertex groups in some cases

2022-03-29 14:33:59 +02:00

blender-admin commented

2022-04-17 20:54:51 +02:00

This issue was referenced by b32cb0266c

This issue was referenced by b32cb0266c61abb333906dc6848d252f6f4dbd20

Aras Pranckevicius commented

2022-04-17 21:01:09 +02:00

Changed status from 'Confirmed' to: 'Resolved'

Aras Pranckevicius closed this issue

2022-04-17 21:01:09 +02:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

New 3.1 OBJ exporter writes incorrect polygon/vertex groups in some cases #96824