Collada Export assigns improper ids to skin controllers when more than one. #22388

New Issue

Tod Liverseed · 2010-05-21T19:25:55+02:00

Tod Liverseed commented

2010-05-21 19:25:55 +02:00

%%%Reproduce

Start with a model that has two mesh objects. Such as body and hair
Import a BVH (or other) animation
Parent both the meshes to the animation.
Export in Collada

The export will contain 2 skin controllers, one for each mesh object. However both controllers have the same ID.

If the controller IDs are renamed manually in the DAE file, then the DAE file will properly import into other software and works as expected.

build: 28841

%%%

%%%Reproduce 1. Start with a model that has two mesh objects. Such as body and hair 2. Import a BVH (or other) animation 3. Parent both the meshes to the animation. 4. Export in Collada The export will contain 2 skin controllers, one for each mesh object. However both controllers have the same ID. If the controller IDs are renamed manually in the DAE file, then the DAE file will properly import into other software and works as expected. build: 28841 %%%

Tod Liverseed commented

2010-05-21 19:25:55 +02:00

Changed status to: 'Open'

Tod Liverseed commented

2010-05-24 04:00:57 +02:00

%%%I've made some changed to the documentexporter file that resolve the issue (patch incuded).

I append the mesh object name to the skinmesh controller's ID. I do not know if it is possible to have multiple meshes per controller, however this solved the issue. What is important is that controller IDs are unique.

I also changed the params for UV exporting, This was done to match the naming conventions of the 1.4 and 1.5 specs. Also was done for somewhat selfish reasons. Torque3D collada import expects properly named UV params S, and T instead of X and Y, I do know that the FBX-DAE importer for Maya assumes X and Y incorrectly. Actually by the spec, I believe that the names are arbitrary and can be anything, but their naming conventions use S and T for Texture Coords.

Other concerns as related to documentimport. I was trying to modify this to account for multiple controllers. A DAE file with multiple controllers seems to always be linked to the same mesh. I do see unique skincontrollers, but each one is linked to the same mesh when they clearly are not in the DAE file. I am by no means an expert, but it appears to be an opencollada issue? If someone could verify this for me, then I'd stop banging my head :-)

Tod %%%

%%%I've made some changed to the documentexporter file that resolve the issue (patch incuded). I append the mesh object name to the skinmesh controller's ID. I do not know if it is possible to have multiple meshes per controller, however this solved the issue. What is important is that controller IDs are unique. I also changed the params for UV exporting, This was done to match the naming conventions of the 1.4 and 1.5 specs. Also was done for somewhat selfish reasons. Torque3D collada import expects properly named UV params S, and T instead of X and Y, I do know that the FBX-DAE importer for Maya assumes X and Y incorrectly. Actually by the spec, I believe that the names are arbitrary and can be anything, but their naming conventions use S and T for Texture Coords. Other concerns as related to documentimport. I was trying to modify this to account for multiple controllers. A DAE file with multiple controllers seems to always be linked to the same mesh. I do see unique skincontrollers, but each one is linked to the same mesh when they clearly are not in the DAE file. I am by no means an expert, but it appears to be an opencollada issue? If someone could verify this for me, then I'd stop banging my head :-) Tod %%%

Arystanbek Dyussenov commented

2010-06-03 19:56:17 +02:00

%%%Hi,

I applied the following changes from your patch into svn:

texture coordinate params named as S and T (spec-compliant) instead of X and Y
exported controller ids now include mesh object name to avoid duplication

Also, importer now detects when multiple controllers share the same armature and doesn't create duplicate armatures.

Thanks for the patch!

Arystan%%%

%%%Hi, I applied the following changes from your patch into svn: * texture coordinate params named as S and T (spec-compliant) instead of X and Y * exported controller ids now include mesh object name to avoid duplication Also, importer now detects when multiple controllers share the same armature and doesn't create duplicate armatures. Thanks for the patch! Arystan%%%

Tod Liverseed commented

2010-06-04 07:41:14 +02:00

%%%Hi,

I've made some more changes. Patch included below.

Fixed the way bone animations are sampled and exported. This was causing some inconsistent time values and some very strange animations when all frames are not keyed. I have detailed notes in the source. I have ran the export on both Keyed and Non keyed bone animations and both come out the exporter great with this change. I do suggest maybe adding an option in the future to choose for sampling/not sampling animations.
Added Extra tags for GoogleEarth and Max3D for double sided meshes. This also helps models import into other 3D apps/game engines. (Torque3D being one I use)
Fixed up Emmissive, Specular, and transparency so now they export properly per the spec.

Thanks for showing me how a better way to handle the controller_id. Still learning blender structures :-)

Thanks!

Tod
%%%

%%%Hi, I've made some more changes. Patch included below. 1. Fixed the way bone animations are sampled and exported. This was causing some inconsistent time values and some very strange animations when all frames are not keyed. I have detailed notes in the source. I have ran the export on both Keyed and Non keyed bone animations and both come out the exporter great with this change. I do suggest maybe adding an option in the future to choose for sampling/not sampling animations. 2. Added Extra tags for GoogleEarth and Max3D for double sided meshes. This also helps models import into other 3D apps/game engines. (Torque3D being one I use) 3. Fixed up Emmissive, Specular, and transparency so now they export properly per the spec. Thanks for showing me how a better way to handle the controller_id. Still learning blender structures :-) Thanks! Tod %%%

Arystanbek Dyussenov commented

2010-06-06 21:24:43 +02:00

%%%Tod,

I applied the changes #2 and #3 in svn. Question about the change #1: why should animations of bones forming a hierarchy have the same sampling keys?

Thanks!%%%

%%%Tod, I applied the changes #2 and #3 in svn. Question about the change #1: why should animations of bones forming a hierarchy have the same sampling keys? Thanks!%%%

Jan Diederich commented

2010-06-06 21:48:35 +02:00

%%%To sampling:
I don't know why they have the same sampling keys...
or what you mean at all, because I have not completely investigated COLLLADA animations yet (some details are missing).
But I want to annotate that Blender 2.49 COLLADA export had exactly that what Tod Liverseed suggested:
An option for animation sampling!
This should really be re-added!%%%

%%%To sampling: I don't know why they have the same sampling keys... or what you mean at all, because I have not completely investigated COLLLADA animations yet (some details are missing). But I want to annotate that Blender 2.49 COLLADA export had exactly that what Tod Liverseed suggested: An option for animation sampling! This should really be re-added!%%%

Tod Liverseed commented

2010-06-07 03:03:00 +02:00

%%%There is no reason it every frame has to be resampled. However, there are a couple good reasons

In its current implementation, the animation export for non keyed animations is flawed and does not export to the collada spec. (I can provide examples), and the resulting animations do not import well into other software. The flaw is in the FindFrames code. The code makes a unique list of frames for a given bone across 3 channels. It uses this one list to export out times for all three channels. The resulting 'times' in the collada file have a strong possibility of not being in ascending order if the keys on the three different channels do not match each other and the total keys does not match the total animation frames. This is not allowed by the Collada spec. All frame times and their corresponding rotation/translation values must be in ascending order. In its current implementation, I can replicate this disorder this at will.
As indicated in the source comments, in theory it should be fine to export only times/values for each bone channel independently (and we should have an option for that). However, problems can occur when there is a discontinuity in the rotational angles (around +/- 180 degrees). This can cause joints to flip/flop around in the importing software incorrectly. Third party software importing collada files can easily butcher the animations, and I have seen it first hand :-)

This is why I suggested that an option be made that can export both ways, only exporting keys for each channel, and resampling the entire animation and exporting keys for every channel/frame. I would be glad to code the independant channel/key export, however I do not understand how to add graphical options to the blender UI.

In the end, I am just suggesting maximum compatibility with other software trying to import the collada files that bender exports.

%%%

%%%There is no reason it every frame has to be resampled. However, there are a couple good reasons 1. In its current implementation, the animation export for non keyed animations is flawed and does not export to the collada spec. (I can provide examples), and the resulting animations do not import well into other software. The flaw is in the FindFrames code. The code makes a unique list of frames for a given bone across 3 channels. It uses this one list to export out times for all three channels. The resulting 'times' in the collada file have a strong possibility of not being in ascending order if the keys on the three different channels do not match each other and the total keys does not match the total animation frames. This is not allowed by the Collada spec. All frame times and their corresponding rotation/translation values must be in ascending order. In its current implementation, I can replicate this disorder this at will. 2. As indicated in the source comments, in theory it should be fine to export only times/values for each bone channel independently (and we should have an option for that). However, problems can occur when there is a discontinuity in the rotational angles (around +/- 180 degrees). This can cause joints to flip/flop around in the importing software incorrectly. Third party software importing collada files can easily butcher the animations, and I have seen it first hand :-) This is why I suggested that an option be made that can export both ways, only exporting keys for each channel, and resampling the entire animation and exporting keys for every channel/frame. I would be glad to code the independant channel/key export, however I do not understand how to add graphical options to the blender UI. In the end, I am just suggesting maximum compatibility with other software trying to import the collada files that bender exports. %%%

Jan Diederich commented

2010-06-07 14:56:34 +02:00

%%%> however I do not understand how to add graphical options to the blender UI.

That's only your way to say that you aren't motivated. ;)

Just look at wm_operators.c, and there at WM_OT_open_mainfile. It's very easy. I give you a patch to give you a further hint and a basic start.
After applying the patch, you must search further for the "load_ui" string, to see how they implemented it, for loading a blenders file stored ui settings.
That should easily be converted to know how to use that for exporting a COLLADA file.
I did such things already for Blender 2.49 COLLADA exporter Python scripts, and that Python scripts are much more confusing, because they feature (a type of) aspect orientated programming, without a really good toolset to support that (it's a mess). It's hard to find the start point, but after you have that, it shouldn't be too hard anymore.

Here the patch-link:
http://www.pasteall.org/13626/c%%%

%%%> however I do not understand how to add graphical options to the blender UI. That's only your way to say that you aren't motivated. ;) Just look at wm_operators.c, and there at WM_OT_open_mainfile. It's very easy. I give you a patch to give you a further hint and a basic start. After applying the patch, you must search further for the "load_ui" string, to see how they implemented it, for _loading_ a blenders file stored ui settings. That should easily be converted to know how to use that for _exporting_ a COLLADA file. I did such things already for Blender 2.49 COLLADA exporter Python scripts, and that Python scripts are _much_ more confusing, because they feature (a type of) aspect orientated programming, without a really good toolset to support that (it's a mess). It's hard to find the start point, but after you have that, it shouldn't be too hard anymore. Here the patch-link: http://www.pasteall.org/13626/c%%%

Arystanbek Dyussenov commented

2010-06-07 22:26:36 +02:00

%%%Tod,

Thanks for explaining the problem in detail.

I should explain now why I chose to collect a union of keys from all three channels and use it for sampling values for each of three channels. The reason is the following: transformation values are converted into different coordinate system before being exported, i.e. each exported key value is completely different from the original key value. This coord. sys. occurs inside the body of the "for" loop in sample_animation(). The conversion is necessary because transformations in COLLADA nodes are in the relative-to-parent coordinate system while transformations in Blender bone channels are relative to bone rest position.

Using a union of keys from all three channels is the solution I found suitable to keep the number of keys as close to the original as possible.

Regarding the order of keys, that's indeed a bug - keys can get in the wrong order. The list should be sorted after being filled.

Regarding the bone flip problem, would you provide an example .blend to help me understand the problem?

Thanks,
Arystan%%%

%%%Tod, Thanks for explaining the problem in detail. I should explain now why I chose to collect a union of keys from all three channels and use it for sampling values for each of three channels. The reason is the following: transformation values are converted into different coordinate system before being exported, i.e. each exported key value is completely different from the original key value. This coord. sys. occurs inside the body of the "for" loop in sample_animation(). The conversion is necessary because transformations in COLLADA nodes are in the relative-to-parent coordinate system while transformations in Blender bone channels are relative to bone rest position. Using a union of keys from all three channels is the solution I found suitable to keep the number of keys as close to the original as possible. Regarding the order of keys, that's indeed a bug - keys can get in the wrong order. The list should be sorted after being filled. Regarding the bone flip problem, would you provide an example .blend to help me understand the problem? Thanks, Arystan%%%

Arystanbek Dyussenov commented

2010-06-07 22:29:30 +02:00

%%%Sorry, a typo: "The coord. sys. occurs" should be "The coord. sys. conversion occurs".%%%

Tod Liverseed commented

2010-06-07 22:57:02 +02:00

%%%Hi Arystan,

The bone flipping problem isn't a Blender issue. I didn't mean to say that it was. :-) Its an issue with many other 3D programs, namely game engines, that use Collada as a means to import assets. Most of these expect animations to be in a 'keyed' format. i.e. Every channel has a key on every frame. They do this because speed is what is important to them.

Some of these game engines (or viewers) will attempt to perform the interpolation of non keyed animations. In doing so, they are very often yield incorrect animations when a joint's channel crosses +/- 180 degrees. What this will do is cause a foot or head or hand joint to flip/flop around occasionally because it handled the interpolation incorrectly.

I had originally modified the code to export each channel of each bone exactly as it was in blender. For instance, the Arm rotation may have had 20 Y rotation keys, 25 X rotation and 14 Z rotation keys (which required a separate '*fra' array for each). Everything exported wonderfully, but when imported into other software, the interpolation problem reared its ugly head. Grumbling then started. :-)

Ultimately, yes, this is not blender's problem. However, it does render blender useless for those people trying to develop models and animations for those other programs. I've experienced these issues in both T3D and UntimateUnwrap. The best way to eliminate that and ensure that the animations are rendered properly in all these different programs is to key every channel on export. Once I coded the collada export to do this, every skeletal animation exported from Blender worked perfectly in these programs. Many headaches disappeared then! :-)

I hope this helps!

Tod

%%%

%%%Hi Arystan, The bone flipping problem isn't a Blender issue. I didn't mean to say that it was. :-) Its an issue with many other 3D programs, namely game engines, that use Collada as a means to import assets. Most of these expect animations to be in a 'keyed' format. i.e. Every channel has a key on every frame. They do this because speed is what is important to them. Some of these game engines (or viewers) will attempt to perform the interpolation of non keyed animations. In doing so, they are very often yield incorrect animations when a joint's channel crosses +/- 180 degrees. What this will do is cause a foot or head or hand joint to flip/flop around occasionally because it handled the interpolation incorrectly. I had originally modified the code to export each channel of each bone exactly as it was in blender. For instance, the Arm rotation may have had 20 Y rotation keys, 25 X rotation and 14 Z rotation keys (which required a separate '*fra' array for each). Everything exported wonderfully, but when imported into other software, the interpolation problem reared its ugly head. Grumbling then started. :-) Ultimately, yes, this is not blender's problem. However, it does render blender useless for those people trying to develop models and animations for those other programs. I've experienced these issues in both T3D and UntimateUnwrap. The best way to eliminate that and ensure that the animations are rendered properly in all these different programs is to key every channel on export. Once I coded the collada export to do this, every skeletal animation exported from Blender worked perfectly in these programs. Many headaches disappeared then! :-) I hope this helps! Tod %%%

Jan Diederich commented

2010-06-08 01:09:20 +02:00

%%%I like this in-depth analysis of the problem. And that seems for me just "screaming" even more for a user decidable option.
The default:

Correct export.
The option:
Incorrect, but compatible export.

Only the end user can know for what he wants to use his COLLADA file, that can't be guessed by the programmer.

Correct me if I misunderstood something.%%%

%%%I like this in-depth analysis of the problem. And that seems for me just "screaming" even more for a user decidable option. The default: - Correct export. The option: - Incorrect, but compatible export. Only the end user can know for what he wants to use his COLLADA file, that can't be guessed by the programmer. Correct me if I misunderstood something.%%%

Nathan Letwory commented

2010-09-29 10:10:32 +02:00

%%%Change resolution to Investigate and assign to myself for further work.%%%

Tod Liverseed commented

2010-09-29 13:44:06 +02:00

%%%The export has been working great since the changes I made, well for at least my purposes :-) Anyway, I was disappointed that IK constraints had no influence on the final frame sampling, so I have added that. I have not yet submitted a patch as I want to improve the way the frame sampling is done. Its a little slow and I have an idea I want to try. Again, this may not be the 'correct' way to handle the IK constraints, but it is the 'useful' way. Game engines most certainly do not incorporate IK solvers in their pipelines. :-)

I will take a peek at how to create export options at the same time. I could see about revisiting this original frame sampling issue at the same time, depending on what you decide.

Tod
%%%

%%%The export has been working great since the changes I made, well for at least my purposes :-) Anyway, I was disappointed that IK constraints had no influence on the final frame sampling, so I have added that. I have not yet submitted a patch as I want to improve the way the frame sampling is done. Its a little slow and I have an idea I want to try. Again, this may not be the 'correct' way to handle the IK constraints, but it is the 'useful' way. Game engines most certainly do not incorporate IK solvers in their pipelines. :-) I will take a peek at how to create export options at the same time. I could see about revisiting this original frame sampling issue at the same time, depending on what you decide. Tod %%%

Nathan Letwory commented

2010-10-18 10:20:51 +02:00

%%%Currently not too much time to work on this, so I'm moving this to our todo list on the wiki: http://wiki.blender.org/index.php/Dev:2.5/Source/Development/Todo/Simple_Todos#COLLADA

Anyone who wants to help out still can provide patches!

Thanks for reporting.%%%

%%%Currently not too much time to work on this, so I'm moving this to our todo list on the wiki: http://wiki.blender.org/index.php/Dev:2.5/Source/Development/Todo/Simple_Todos#COLLADA Anyone who wants to help out still can provide patches! Thanks for reporting.%%%

Nathan Letwory commented

2010-12-21 10:34:04 +01:00

%%%Reopened and assign to Jeroen.%%%

Nathan Letwory commented

2011-03-09 15:44:13 +01:00

%%%Since Jeroen is busy with compositor project, moving back to todo. There is a developer who will try to propose "Full COLLADA animation support" as a GSoC, so pending that...%%%

Nathan Letwory commented

2011-03-09 15:44:13 +01:00

Changed status from 'Open' to: 'Archived'

Nathan Letwory closed this issue

2011-03-09 15:44:13 +01:00

Gaia Clary commented

2012-02-06 22:01:43 +01:00

%%%@Tod Liverseed: I wonder if your patch has ever been made available.
@nathan-45 Letwory: Meanwhile the GSoC 2011 is terminated i believe :)
Is there any improvement regarding this report ?
Could it possibly already be resolved ? %%%

%%%@Tod Liverseed: I wonder if your patch has ever been made available. @nathan-45 Letwory: Meanwhile the GSoC 2011 is terminated i believe :) Is there any improvement regarding this report ? Could it possibly already be resolved ? %%%

Arystanbek Dyussenov commented

2012-02-12 18:35:13 +01:00

%%%This report is a feature request for:

export animation with IK and
user option for switching between sampled and interpolated animation export%%%

%%%This report is a feature request for: - export animation with IK and - user option for switching between sampled and interpolated animation export%%%

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Collada Export assigns improper ids to skin controllers when more than one. #22388