Import Unreal Skeleton Mesh (.psk)/Animation Set (.psa) #48800

Closed
opened 2016-07-07 06:23:51 +02:00 by Tristan Bailey · 50 comments

I found an addon that works better than the existing addon in Blender, for importing psk and psa files with.

GitHub resource

The addon is a revision of the existing "Import Unreal Skeleton Mesh (.psk)/Animation Set (.psa)" addon found within the current version of Blender, and was created approximately 20 months ago. The existing version (2.2.2) does not import PSK files (armatures) properly, appearing to skip over hunks of data. It does not appear to successfully import PSA files (animations) at all.

Commenting from the revision author (Befzz) can be found here:
http://blenderartists.org/forum/archive/index.php/t-328325.html

Mentioned in these websites / web pages:
https://www.epicgames.com/unrealtournament/forums/archive/index.php/t-18198.html
https://www.muvizu.com/forum/topic5765-base-character-reference-models.aspx?p=f
http://www.gildor.org/smf/index.php?topic=718.0;wap2
http://www.gildor.org/smf/index.php/topic,1745.0.html

It must be emphasised, however, that this version of the plugin (v2.6.2) appears to be incapable of importing multiple PSA files at once.

I found an addon that works better than the existing addon in Blender, for importing psk and psa files with. [GitHub resource ](https://github.com/Befzz/blender3d_import_psk_psa) The addon is a revision of the existing "Import Unreal Skeleton Mesh (.psk)/Animation Set (.psa)" addon found within the current version of Blender, and was created approximately 20 months ago. The existing version (2.2.2) does not import PSK files (armatures) properly, appearing to skip over hunks of data. It does not appear to successfully import PSA files (animations) at all. Commenting from the revision author (Befzz) can be found here: http://blenderartists.org/forum/archive/index.php/t-328325.html Mentioned in these websites / web pages: https://www.epicgames.com/unrealtournament/forums/archive/index.php/t-18198.html https://www.muvizu.com/forum/topic5765-base-character-reference-models.aspx?p=f http://www.gildor.org/smf/index.php?topic=718.0;wap2 http://www.gildor.org/smf/index.php/topic,1745.0.html It must be emphasised, however, that this version of the plugin (v2.6.2) appears to be incapable of importing multiple PSA files at once.
Author

Changed status to: 'Open'

Changed status to: 'Open'
Author

Added subscriber: @tristanlbailey

Added subscriber: @tristanlbailey
Author

Actually, I may have jumped the gun on this one. The addon doesn't appear to orient the bones around the correct way. The root of each bone in an imported PSK file (armature) is positioned correctly, but the tips of each bone point outwards from the model. Furthermore, every bone is set to the exact same size, which may very well be undesired when importing the armature.

I have never coded in Python before, but I will have a stab at fixing up the script. Anyone else who excels in armatures is welcome to assist.

Actually, I may have jumped the gun on this one. The addon doesn't appear to orient the bones around the correct way. The root of each bone in an imported PSK file (armature) is positioned correctly, but the tips of each bone point outwards from the model. Furthermore, every bone is set to the exact same size, which may very well be undesired when importing the armature. I have never coded in Python before, but I will have a stab at fixing up the script. Anyone else who excels in armatures is welcome to assist.

Added subscriber: @RhyCynn

Added subscriber: @RhyCynn

In #48800#380685, @tristanlbailey wrote:
Actually, I may have jumped the gun on this one. The addon doesn't appear to orient the bones around the correct way. The root of each bone in an imported PSK file (armature) is positioned correctly, but the tips of each bone point outwards from the model. Furthermore, every bone is set to the exact same size, which may very well be undesired when importing the armature.

I have never coded in Python before, but I will have a stab at fixing up the script. Anyone else who excels in armatures is welcome to assist.

I stumbled on this as my addon has the same issue. My test code detects some reversed bones with cross product calculations. The ‘fix’ is just a hack, multiplying by a negative value in the tail calculation. It breaks my animation import, which needs re-written anyway, but that’s another story. I don't fully understand the math involved with all of this.

I mess with it on and off but have no solution yet. If you haven’t already solved the problem, I’ll post if I manage to come up with anything. One note though, I don't consider my addon to be a viable replacement as I re-wrote it for my own amusement and I'm not a python developer.

> In #48800#380685, @tristanlbailey wrote: > Actually, I may have jumped the gun on this one. The addon doesn't appear to orient the bones around the correct way. The root of each bone in an imported PSK file (armature) is positioned correctly, but the tips of each bone point outwards from the model. Furthermore, every bone is set to the exact same size, which may very well be undesired when importing the armature. > > I have never coded in Python before, but I will have a stab at fixing up the script. Anyone else who excels in armatures is welcome to assist. I stumbled on this as my addon has the same issue. My test code detects some reversed bones with cross product calculations. The ‘fix’ is just a hack, multiplying by a negative value in the tail calculation. It breaks my animation import, which needs re-written anyway, but that’s another story. I don't fully understand the math involved with all of this. I mess with it on and off but have no solution yet. If you haven’t already solved the problem, I’ll post if I manage to come up with anything. One note though, I don't consider my addon to be a viable replacement as I re-wrote it for my own amusement and I'm not a python developer.
Author

Michael, is your addon a revision of the importer featured in the latest version of Blender (v2.2.2, in 2.77a), or a revision of the version that I am proposing they use (v2.6.2)?

I haven't looked much into all the code yet, except to say that it looks pretty daunting. Not to say that I can't fix it, it's just going to take a lot of time to work it all out.

I came across some useful information regarding the format of PSA and PSK files. It appears to be a set of formats (or a specification) written by the guys who developed the Unreal Engine. This information would be best suited to verify if the existing addon is actually importing everything correctly.

https://udn.epicgames.com/Two/BinaryFormatSpecifications.html

I would also need to understand the Blender API, something which I have not looked at yet, either. To top it off, I'm new to armatures and animation, and new to Python. That's like... 4 things that I don't know much about, right there.

Michael, is your addon a revision of the importer featured in the latest version of Blender (v2.2.2, in 2.77a), or a revision of the version that I am proposing they use (v2.6.2)? I haven't looked much into all the code yet, except to say that it looks pretty daunting. Not to say that I can't fix it, it's just going to take a lot of time to work it all out. I came across some useful information regarding the format of PSA and PSK files. It appears to be a set of formats (or a specification) written by the guys who developed the ***Unreal Engine***. This information would be best suited to verify if the existing addon is actually importing everything correctly. https://udn.epicgames.com/Two/BinaryFormatSpecifications.html I would also need to understand the Blender API, something which I have not looked at yet, either. To top it off, I'm new to armatures and animation, and new to Python. That's like... 4 things that I don't know much about, right there.

My addon is a modular re-write of Darknet’s addon and bears little to no resemblance to it. I wrote it to fix the mesh distortion and for my own amusement. It’s more of an experiment. I would submit it as a possible replacement if I thought it was viable, but I don’t for reasons at the end. It is located here: RhyCynn / cynn_import_psa_psk .

I’ve already coded a fix in the Befzz addon after sort of an epiphany. It’s a bit of a hack due to the nature of the animation code. I don’t think it corrects all cases of reversed bones, but the obvious ones, like the finger tips and the spine look correct now. I've assembled models using some semi-automated functions of a newer version of my utilities addon and applied animations to the body, face and hair armatures. They look fine.

I forked the code this morning and will update it later. I’ll add detailed notes on what I’ve done for anyone else to review. I don’t know git / github well but I think I eventually need to do a pull request to see if Befzz wants to integrate my changes if they work out. My capability to heavily test the code is limited but I’ll run it through a sample of models.

My addon builds animations in a similar fashion to Darknet’s and Befzz’s. I don’t think any of them properly use the world / pose / rest state. I would like to re-write the animation code in mine but I don’t quite grasp the concepts. I’ve looked at the BVH and FBX addons but the complexity of the code and the calculations is rough.

FYI, I know how you feel about the API etc. as I've had to try and learn all of it myself.

My addon is a modular re-write of Darknet’s addon and bears little to no resemblance to it. I wrote it to fix the mesh distortion and for my own amusement. It’s more of an experiment. I would submit it as a possible replacement if I thought it was viable, but I don’t for reasons at the end. It is located here: [RhyCynn / cynn_import_psa_psk ](https://github.com/RhyCynn/cynn_import_psa_psk). I’ve already coded a fix in the Befzz addon after sort of an epiphany. It’s a bit of a hack due to the nature of the animation code. I don’t think it corrects all cases of reversed bones, but the obvious ones, like the finger tips and the spine look correct now. I've assembled models using some semi-automated functions of a newer version of my utilities addon and applied animations to the body, face and hair armatures. They look fine. I forked the code this morning and will update it later. I’ll add detailed notes on what I’ve done for anyone else to review. I don’t know git / github well but I think I eventually need to do a pull request to see if Befzz wants to integrate my changes if they work out. My capability to heavily test the code is limited but I’ll run it through a sample of models. My addon builds animations in a similar fashion to Darknet’s and Befzz’s. I don’t think any of them properly use the world / pose / rest state. I would like to re-write the animation code in mine but I don’t quite grasp the concepts. I’ve looked at the BVH and FBX addons but the complexity of the code and the calculations is rough. FYI, I know how you feel about the API etc. as I've had to try and learn all of it myself.
Author

Wow, sounds like you have been busy. I would say you are in a better position than myself to get this done. If you want, I can help test out your addon whenever you want, in the near future. I'll have a look at your current version tomorrow when I get the chance.

Wow, sounds like you have been busy. I would say you are in a better position than myself to get this done. If you want, I can help test out your addon whenever you want, in the near future. I'll have a look at your current version tomorrow when I get the chance.

I pushed two things out. The first is the test code to flip the reversed bones. It has:

  • The new code to detect and flip the bones that are 180 degrees off.
  • A devlog with some info about the addon and why it may be a candidate to replace the current broken one.
  • A changelog saying what I did.

The other thing is a (perpetual) work in progress of my utilities addon. I mention it as is can re-parent and link some multi-file models as explained in the readme. It also reads UE Vewer .mat files and attempts to import and configure the materials and textures.

Sorry about the double notification, It's very late and I smacked the wrong keys.

I pushed two things out. The first is the test code to [flip the reversed bones. ](https://github.com/RhyCynn/blender3d_import_psk_psa/tree/feature/correct_reversed_bones) It has: - The new code to detect and flip the bones that are 180 degrees off. - A devlog with some info about the addon and why it may be a candidate to replace the current broken one. - A changelog saying what I did. The other thing is a (perpetual) work in progress of my [utilities addon. ](https://github.com/RhyCynn/cynn_utilities/tree/feature/mesh_parenting_and_armature_linking) I mention it as is can re-parent and link some multi-file models as explained in the readme. It also reads UE Vewer .mat files and attempts to import and configure the materials and textures. Sorry about the double notification, It's very late and I smacked the wrong keys.
Author

Hmm... I'm still having troubles with the PSK file that I have, using your addon. All of the bones still point straight out from the base model, just like before. No apparent change to be seen.

Also, you might want to update the version number, and add your name to the list of authors at the top of the script.

Hmm... I'm still having troubles with the PSK file that I have, using your addon. All of the bones still point straight out from the base model, just like before. No apparent change to be seen. Also, you might want to update the version number, and add your name to the list of authors at the top of the script.

Thanks for the reminder on the version and author's. I may have to try and expand the source of psk / psa files I'm using to test with since mine are correct. I'm dropping in some utility and logging functions I use to save me some pain. It looks like I'm going to be doing more to this than I first estimated.

At this point I figure I should do what I can to get this considered as a viable replacement for the current add-on, which has been broken for as long as I can remember.

I should have a new one out soon.

Thanks for the reminder on the version and author's. I may have to try and expand the source of psk / psa files I'm using to test with since mine are correct. I'm dropping in some utility and logging functions I use to save me some pain. It looks like I'm going to be doing more to this than I first estimated. At this point I figure I should do what I can to get this considered as a viable replacement for the current add-on, which has been broken for as long as I can remember. I should have a new one out soon.
Author

Thanks, Michael.

I thought I should let you know that I have started on a small PSK file verification program. Basically, it has three purposes:

  • It is teaching me how a PSK file is structured, and what to expect
  • It verifies that a PSK file has its headers, whether or not it has the expected number of bytes in between each header, and thus whether or not it is corrupted in any way

It shows the information that the file contains in a table format

I will let you know when it is finished. I may also be able to extend it to include PSA files without too much trouble.

Thanks, Michael. I thought I should let you know that I have started on a small PSK file verification program. Basically, it has three purposes: - It is teaching me how a PSK file is structured, and what to expect - It verifies that a PSK file has its headers, whether or not it has the expected number of bytes in between each header, and thus whether or not it is corrupted in any way # It shows the information that the file contains in a table format I will let you know when it is finished. I may also be able to extend it to include PSA files without too much trouble.
Author

Err... I just tried downloading and using someone else's PSK file, supposedly generated from ActorX, but it caused your addon to crash Blender.

The error in the Blender console was:
EXCEPTION_ACCESS_VIOLATION

I'm trying to work out why this happened. The file has the necessary headers, but did not appear to have the right "Typeflag" value in at least one place. In any case, the addon should be able to handle PSK files that are not correctly formatted, such as in the case a file may have become corrupted.

I should point out that this may not have been your fault, as it could have easily been code from one of the previous versions of the addon causing Blender to crash.

Err... I just tried downloading and using someone else's PSK file, supposedly generated from ActorX, but it caused your addon to crash Blender. The error in the Blender console was: EXCEPTION_ACCESS_VIOLATION I'm trying to work out why this happened. The file has the necessary headers, but did not appear to have the right "Typeflag" value in at least one place. In any case, the addon should be able to handle PSK files that are not correctly formatted, such as in the case a file may have become corrupted. I should point out that this may not have been your fault, as it could have easily been code from one of the previous versions of the addon causing Blender to crash.

In a few minutes I'm pushing the add-on out again to the same branch correct_reversed_bones , however the structure has changed. If it doesn't work for some reason I'll get on it again tomorrow. It's working on my system but differences in psk / psa files look to be issues from your observations.

It has to be installed correctly into it's own folder. Blender should install it from the zip file from Settings / Add-ons / Install from file. Otherwise the .py files need to be in a folder right under addons, like /addons/io_import_psk_psa for example.

That verification program is a good way to learn the structure and a good idea in general, I don't have a dedicated validation module myself. I've noticed a couple strange things during testing that haven't made a lot of sense. I don't know if it's the psk file or the add-on not getting some data right.

This add-on doesn't do the kind of error checking and trapping that my add-on does. That's not to say that the error you got wouldn't have happened with mine, but this one is pretty loose. I definitely need to tighten up the code that reads and decodes the files. I'm not knocking the original authors, as far as I can tell the add-on was migrated from code written prior to some api changes in Blender. Here's what I have done, much of it for development purposes:

  • Changes the animation import to use the active object if the ui-list isn't used.
  • Broke the add-on up into logical modules: UI, model, animation and utility.
  • Added my logging module that uses python logging instead of print statements.
  • Added a batch import system from an unreleased version of my add-on. It lets me load multiple models and animations with one click using a json file as a script of sorts.

I'm starting to wonder if I would be better merging what makes sense in this add-on and my addon at some point. My parser is almost certainly more sound than this one. Gotta think about it.

In a few minutes I'm pushing the add-on out again to the same branch [correct_reversed_bones ](https://github.com/RhyCynn/blender3d_import_psk_psa/tree/feature/correct_reversed_bones), however the structure has changed. If it doesn't work for some reason I'll get on it again tomorrow. It's working on my system but differences in psk / psa files look to be issues from your observations. It has to be installed correctly into it's own folder. Blender should install it from the zip file from Settings / Add-ons / Install from file. Otherwise the .py files need to be in a folder right under addons, like /addons/io_import_psk_psa for example. That verification program is a good way to learn the structure and a good idea in general, I don't have a dedicated validation module myself. I've noticed a couple strange things during testing that haven't made a lot of sense. I don't know if it's the psk file or the add-on not getting some data right. This add-on doesn't do the kind of error checking and trapping that my add-on does. That's not to say that the error you got wouldn't have happened with mine, but this one is pretty loose. I definitely need to tighten up the code that reads and decodes the files. I'm not knocking the original authors, as far as I can tell the add-on was migrated from code written prior to some api changes in Blender. Here's what I have done, much of it for development purposes: - Changes the animation import to use the active object if the ui-list isn't used. - Broke the add-on up into logical modules: UI, model, animation and utility. - Added my logging module that uses python logging instead of print statements. - Added a batch import system from an unreleased version of my add-on. It lets me load multiple models and animations with one click using a json file as a script of sorts. I'm starting to wonder if I would be better merging what makes sense in this add-on and my addon at some point. My parser is almost certainly more sound than this one. Gotta think about it.
Author

Ah, that's right, you've been writing your own addon from scratch, correct?

I was actually using your addon, so that crash bug is something that you will have to fix. :P

I'll attach the file that caused the addon to crash. It's one I randomly downloaded and tested, from the following website:

http://www.clintons3d.com/vr/ut3_characters_for%20_Lightwave.html

UT3_Male.PSK

Without looking too much into the data itself, the headers of this PSK file are correct, with the exception of the "Typeflag" in the first header (general). According to the official documentation, It should be returning a value of 1999801 or less, but it is returning 2003321 (greater than what it should be). At least, that's what version 1 of the PSK specification says. I haven't found another version of the specification, yet, so this is all I have to work with for the time being.

If you wanted to, you could open up PSK files up a text application. They are mostly gobble-de-gook, as it is all in binary, but the headers are legible, and can be picked out as follows:

  • ACTRHEAD - General Header
  • PNTS0000 - Points Header
    Followed by VPoint data array (x,y,z coords?)
  • VTXW0000 - Wedges Header
    Followed by VVertex wedges array (material positioning information)
  • FACE0000 - Faces Header
    Followed by VTriangle faces array (textured triangles)
  • MATT0000 - Materials Header
    Followed by VMaterials materials array (material names, reference indexes)
  • REFSKELT - Bones Header
    Followed by VBone bones array (bone names, parent indexes, positions)
  • RAWWEIGHTS - Bone influences Header
    Followed by VRawBoneInfluence array (bone weights, indexes, point indexes?)

The "Typeflag" value is the value that comes immediately after the name of every header, where the header name is padded to a total of 20 bytes.
¹ƒ = my original test PSK (correct)
y‘ = in the randomly downloaded PSK (incorrect)

Ah, that's right, you've been writing your own addon from scratch, correct? I was actually using your addon, so that crash bug is something that you will have to fix. :P I'll attach the file that caused the addon to crash. It's one I randomly downloaded and tested, from the following website: http://www.clintons3d.com/vr/ut3_characters_for%20_Lightwave.html [UT3_Male.PSK](https://archive.blender.org/developer/F333027/UT3_Male.PSK) Without looking too much into the data itself, the headers of this PSK file are correct, with the exception of the "Typeflag" in the first header (general). According to the official documentation, It should be returning a value of 1999801 or less, but it is returning 2003321 (greater than what it should be). At least, that's what version 1 of the PSK specification says. I haven't found another version of the specification, yet, so this is all I have to work with for the time being. If you wanted to, you could open up PSK files up a text application. They are mostly gobble-de-gook, as it is all in binary, but the headers are legible, and can be picked out as follows: - ACTRHEAD - General Header - PNTS0000 - Points Header Followed by VPoint data array (x,y,z coords?) - VTXW0000 - Wedges Header Followed by VVertex wedges array (material positioning information) - FACE0000 - Faces Header Followed by VTriangle faces array (textured triangles) - MATT0000 - Materials Header Followed by VMaterials materials array (material names, reference indexes) - REFSKELT - Bones Header Followed by VBone bones array (bone names, parent indexes, positions) - RAWWEIGHTS - Bone influences Header Followed by VRawBoneInfluence array (bone weights, indexes, point indexes?) The "Typeflag" value is the value that comes immediately after the name of every header, where the header name is padded to a total of 20 bytes. ¹ƒ = my original test PSK (correct) y‘ = in the randomly downloaded PSK (incorrect)
Author

sigh

Now I have found more information that suggests there is a different version of PSK specification, one that apparently has another header and section added in, to do with adding extra UV information of some description.

http://me3explorer.wikia.com/wiki/PSK_File_Format

Frustratingly, the page has a link back to the same page that I referenced earlier. I'll continue to hunt down the official documentation for the newer version, something which is proving to be elusive.

*sigh* Now I have found more information that suggests there is a different version of PSK specification, one that apparently has another header and section added in, to do with adding extra UV information of some description. http://me3explorer.wikia.com/wiki/PSK_File_Format Frustratingly, the page has a link back to the same page that I referenced earlier. I'll continue to hunt down the official documentation for the newer version, something which is proving to be elusive.
Author

OK, I'm still not quite sure about the extra UV section, but I have found a newer version of the PSA specification:

https://udn.epicgames.com/Three/BinaryFormatSpecifications.html

The URL was very similar to the other URL, except that the "Two" was changed to a "Three" (a reference to Unreal Engine 3). If you cross-check the PSK versions, there isn't any change; it's exactly the same as it was with the previous version of Unreal Engine (2).

The differences between the older PSA format, and the newer one, is that they added two more headers and sections in: scale keys and curve keys. To make things extra confusing, they also updated the version value given in the General Header TypeFlag: 20090127 (for Maya) 2003321 (for 3dsMax), even though the official version number of the format is still version 1.0.

FYI, it is my understanding that the Unreal Engine developers eventually dumped the PSK/PSA specification, in favour of using FBX files for modelling and animations instead.

Anyway, I'll stop spamming for a while, so you can catch up. ;)

OK, I'm still not quite sure about the extra UV section, but I have found a newer version of the **PSA** specification: https://udn.epicgames.com/Three/BinaryFormatSpecifications.html The URL was very similar to the other URL, except that the "Two" was changed to a "Three" (a reference to Unreal Engine 3). If you cross-check the PSK versions, there isn't any change; it's exactly the same as it was with the previous version of Unreal Engine (2). The differences between the older PSA format, and the newer one, is that they added two more headers and sections in: scale keys and curve keys. To make things extra confusing, they also updated the version value given in the General Header TypeFlag: 20090127 (for Maya) 2003321 (for 3dsMax), even though the official version number of the format is still version 1.0. FYI, it is my understanding that the Unreal Engine developers eventually dumped the PSK/PSA specification, in favour of using FBX files for modelling and animations instead. Anyway, I'll stop spamming for a while, so you can catch up. ;)

I see part of what you are running into. One is that my add-on here is unrelated to the Darknet or Befzz add-ons. It will import the psk files you mentioned without crashing. It does however have some issues I need to work out. Blender did drop to the desktop when I tried to import the UT3_Male.psk file with the Befzz code I'm trying to modify.

I've just about decided that modifying the Darknet / Befzz codebase any further is counterproductive if I'm just going to end up pulling my file parser, data objects and who knows what else into it. Going the other way is making more sense.

And you found the other part before I could reply. The last specs are the correct ones and the final psk / psa version as UE4 seems to be all FBX now. The main difference you will run across are Gildor's additional wedge and face headers for exporting models from UE Viewer that exceed the psk data limitations.

I'm going to regroup. If I shift back to my add-on, which is likely at this point, I'll need to fix the non-manifold geometry bug I recently found first and then start comparing the Befzz matrix math to my own.

I see part of what you are running into. One is that [my add-on here ](https://github.com/RhyCynn/cynn_import_psa_psk) is unrelated to the Darknet or Befzz add-ons. It will import the psk files you mentioned without crashing. It does however have some issues I need to work out. Blender did drop to the desktop when I tried to import the UT3_Male.psk file with the Befzz code I'm trying to modify. I've just about decided that modifying the Darknet / Befzz codebase any further is counterproductive if I'm just going to end up pulling my file parser, data objects and who knows what else into it. Going the other way is making more sense. And you found the other part before I could reply. The last specs are the correct ones and the final psk / psa version as UE4 seems to be all FBX now. The main difference you will run across are Gildor's additional wedge and face headers for exporting models from UE Viewer that exceed the psk data limitations. I'm going to regroup. If I shift back to my add-on, which is likely at this point, I'll need to fix the non-manifold geometry bug I recently found first and then start comparing the Befzz matrix math to my own.
Author

Question: Do you have either Maya or 3DS Max?

If we had access to either one of these programs, and then used the latest ActorX PSK/PSA Exporters from the Epic Games website to export something with, then we would be able to see how the files were structured. It is quite possible that the devs didn't update their PSK/PSA documentation after 2009.

Gildor's website makes reference to the extra UV "sets" being added to ActorX in 2010.

http://www.gildor.org/smf/index.php/topic,228.msg9593.html#msg9593

Also:

http://www.gildor.org/smf/index.php/topic,5.msg9594.html#msg9594

Which points to this official source:

http://udn.epicgames.com/Three/DevelopmentKitBuildUpgradeNotes.html#June%202010:%20UDKInstall-2010-06-BETA

  • Vertex Colors and Multiple UV Sets for Skeletal Meshes
  • Users can now import vertex colors with skeletal meshes and access them in materials.
  • Works with both the FBX importer and the latest ActorX plug-in.
  • Vertex colors only work for GPU-skinned meshes (for now).
  • Up to four UV sets will be taken when importing a skeletal mesh.
  • The skeletal mesh vertex buffer only uses the amount imported so as not to waste memory.

... which still doesn't really explain the updated structure/format of the files.

The main difference you will run across are Gildor's additional wedge and face headers for exporting models from UE Viewer that exceed the psk data limitations.

Speaking of which, do we want to support his file formats (PSKX/PSAX) in the addon?

Question: Do you have either Maya or 3DS Max? If we had access to either one of these programs, and then used the latest ActorX PSK/PSA Exporters from the Epic Games website to export something with, then we would be able to see how the files were structured. It is quite possible that the devs didn't update their PSK/PSA documentation after 2009. Gildor's website makes reference to the extra UV "sets" being added to ActorX in 2010. http://www.gildor.org/smf/index.php/topic,228.msg9593.html#msg9593 Also: http://www.gildor.org/smf/index.php/topic,5.msg9594.html#msg9594 Which points to this official source: http://udn.epicgames.com/Three/DevelopmentKitBuildUpgradeNotes.html#June%202010:%20UDKInstall-2010-06-BETA > - Vertex Colors and Multiple UV Sets for Skeletal Meshes > - Users can now import vertex colors with skeletal meshes and access them in materials. > - Works with both the FBX importer and the latest ActorX plug-in. > - Vertex colors only work for GPU-skinned meshes (for now). > - Up to four UV sets will be taken when importing a skeletal mesh. > - The skeletal mesh vertex buffer only uses the amount imported so as not to waste memory. ... which still doesn't really explain the updated structure/format of the files. > The main difference you will run across are Gildor's additional wedge and face headers for exporting models from UE Viewer that exceed the psk data limitations. Speaking of which, do we want to support his file formats (PSKX/PSAX) in the addon?
Author

Michael, I installed your other addon, but I cannot find the addon in the user preferences menu. Any ideas?

My validation program should be ready for testing PSK files in the next 2-3 days.

Michael, I installed your other addon, but I cannot find the addon in the user preferences menu. Any ideas? My validation program should be ready for testing PSK files in the next 2-3 days.

I don't have access to either of those programs myself. My add-on was written from the beginning to handle pskx extensions, though I've no way to test the extra UV support. It should be under Import-Export / Unreal 3 PSK, PSA Importer. Expanded it should show 'Unreal 3 Model and Animation Importer'. The version is 0.4.0. You can try using the search box in the upper left to search for 'unreal' as well. I'm going to try and get some work done on it tomorrow.

I don't have access to either of those programs myself. My add-on was written from the beginning to handle pskx extensions, though I've no way to test the extra UV support. It should be under Import-Export / Unreal 3 PSK, PSA Importer. Expanded it should show 'Unreal 3 Model and Animation Importer'. The version is 0.4.0. You can try using the search box in the upper left to search for 'unreal' as well. I'm going to try and get some work done on it tomorrow.
Author

Ah, that's OK man. I will look into downloading and installing a trial version of either Maya or 3ds Max. Hopefully I can glean more information from that.

I'm still having troubles getting your addon to work. I can see that it has made a folder called "io_import_psa_psk-0.4.0" in the "addons" directory of Blender, and there's plenty of files in the folder, but I can't get it to come up in Blender's User Preferences window. I could be doing something wrong; I'm not used to using addons in Blender...

Ah, that's OK man. I will look into downloading and installing a trial version of either Maya or 3ds Max. Hopefully I can glean more information from that. I'm still having troubles getting your addon to work. I can see that it has made a folder called "io_import_psa_psk-0.4.0" in the "addons" directory of Blender, and there's plenty of files in the folder, but I can't get it to come up in Blender's User Preferences window. I could be doing something wrong; I'm not used to using addons in Blender...

That's definitely not the right folder as it should start with 'cynn_'. It seems like an older one. I pushed out a new release that has the single mesh builder update / fix in preparation for starting on the armature and animation changes. I tested installing the add-on the normal way and it worked.

Here's what I did so you don't have to look it up or try to remember, it's not that intuitive until you've used Blender a while:

  • Download the .zip file to the downloads folder or the desktop using the 'Clone or download / Download Zip' button near the top right from here cynn_import_psa_psk.
  • Open Blender, go to 'User Preferences' then the 'Add-ons' tab like you did before.
  • Click on the 'Enabled' category on the left just to make sure there isn't a version of mine enabled by checking the 'Author' on the various ones with unreal / psa / psk in the title. If so uncheck it.
  • Click on 'Install from File...' on the bottom left.
  • Navigate to the folder with the zip file, which should be named 'cynn_import_psa_psk-master.zip'.
  • Double-click on it or select it and click on 'Install from file...' on the upper right.
  • It will show up in the list but not enabled as 'Import-Export: Unreal 3 PSK, PSA Importer'. If you expand it the version should be '0.4.1'.
  • Check the box to enable it then click 'Save User Settings' on the lower left.

I hope I got that right.

Imported models still have upside-down bones as I just have test code (which is the same hack). I don't know how it will take to re-write the animation code. Understanding the proper calculations may or may not be trivial at this point, especially if the models are going to be able to be exported. I don't even know it the current exporter works.

"I can't think about that right now. If I do, I'll go crazy. I'll think about that tomorrow." -Scarlett O'Hara

That's definitely not the right folder as it should start with 'cynn_'. It seems like an older one. I pushed out a new release that has the single mesh builder update / fix in preparation for starting on the armature and animation changes. I tested installing the add-on the normal way and it worked. Here's what I did so you don't have to look it up or try to remember, it's not that intuitive until you've used Blender a while: - Download the .zip file to the downloads folder or the desktop using the 'Clone or download / Download Zip' button near the top right from here [cynn_import_psa_psk](https://github.com/RhyCynn/cynn_import_psa_psk). - Open Blender, go to 'User Preferences' then the 'Add-ons' tab like you did before. - Click on the 'Enabled' category on the left just to make sure there isn't a version of mine enabled by checking the 'Author' on the various ones with unreal / psa / psk in the title. If so uncheck it. - Click on 'Install from File...' on the bottom left. - Navigate to the folder with the zip file, which should be named 'cynn_import_psa_psk-master.zip'. - Double-click on it or select it and click on 'Install from file...' on the upper right. - It will show up in the list but not enabled as 'Import-Export: Unreal 3 PSK, PSA Importer'. If you expand it the version should be '0.4.1'. - Check the box to enable it then click 'Save User Settings' on the lower left. I hope I got that right. Imported models still have upside-down bones as I just have test code (which is the same hack). I don't know how it will take to re-write the animation code. Understanding the proper calculations may or may not be trivial at this point, especially if the models are going to be able to be exported. I don't even know it the current exporter works. "I can't think about that right now. If I do, I'll go crazy. I'll think about that tomorrow." -Scarlett O'Hara
Author

I don't remember that Clone button before, so maybe I didn't download it properly the first time around.

OK, so I followed your helpful instructions, and the addon appears in the addons area, except after I tried to enable the addon, it came up with the following error message:

Traceback (most recent call last):
File: "C:\Program Files\Blender\2.77\scripts\modules\addon_utils.py", line 330, in enable
mod = import(module_name)

File: "C:\Users\Tristan Bailey\AppData\Roaming\Blender Foundation\Blender\2.77\scripts\addons\cynn_import_psa_psk-master_init_.py", line 46, in from . ue3_model import import_model, alternate import model
File: "C:\Users\Tristan Bailey\AppData\Roaming\Blender Foundation\Blender\2.77\scripts\addons\cynn_import_psa_psk-master\ue3_model.py", line 39, in import cynn_utilities
ImportError: No module named 'cynn_utilities'

I opened up the zip file, and had a look for 'cynn_utilities', and there is a file by the name 'cynn_utilities.py' there, so not sure why it isn't working...

I don't remember that Clone button before, so maybe I didn't download it properly the first time around. OK, so I followed your helpful instructions, and the addon appears in the addons area, except after I tried to enable the addon, it came up with the following error message: > Traceback (most recent call last): > File: "C:\Program Files\Blender\2.77\scripts\modules\addon_utils.py", line 330, in enable > mod = _import_(module_name) > > File: "C:\Users\Tristan Bailey\AppData\Roaming\Blender Foundation\Blender\2.77\scripts\addons\cynn_import_psa_psk-master\_init_.py", line 46, in <module> from . ue3_model import import_model, alternate import model > File: "C:\Users\Tristan Bailey\AppData\Roaming\Blender Foundation\Blender\2.77\scripts\addons\cynn_import_psa_psk-master\ue3_model.py", line 39, in <module> import cynn_utilities > ImportError: No module named 'cynn_utilities' I opened up the zip file, and had a look for 'cynn_utilities', and there is a file by the name 'cynn_utilities.py' there, so not sure why it isn't working...

I couldn't duplicate the problem at first but finally got it to fail with the same error. It's almost like it was resolving the module to the dev version, but that makes no sense to me. Either way I think I know how to isolate it in the future to make sure the .zip install is working.

I pushed out a new version with a correction to the module import. You can install it the same way and it should overwrite the existing one. I left it in development mode, which means the alternate psk / psa buttons are enabled below the regular ones.

If you use '(Alternate) Import a Model' the bones should point the right way. Then if you use '(Alternate) Import an Animation' they should still point the right way. The alternate import is a test and I saw some distortion on at least one model, so don't be surprised it if fails to work.

I couldn't duplicate the problem at first but finally got it to fail with the same error. It's almost like it was resolving the module to the dev version, but that makes no sense to me. Either way I think I know how to isolate it in the future to make sure the .zip install is working. I pushed out a new version with a correction to the module import. You can install it the same way and it should overwrite the existing one. I left it in development mode, which means the alternate psk / psa buttons are enabled below the regular ones. If you use '(Alternate) Import a Model' the bones should point the right way. Then if you use '(Alternate) Import an Animation' they should still point the right way. The alternate import is a test and I saw some distortion on at least one model, so don't be surprised it if fails to work.
Author

Thanks Michael.

I have now installed your addon successfully. However, I don't see any improvement over the Befzz version. I get the same result regardless of the import button that I use on your addon.

I'll upload a screenshot to show you what I'm seeing.

screenshot.png

My PSK validation program is almost ready for your first inspection. It is by no means complete, and I intend to continue updating it for as long and as much as necessary. I will probably upload it today, sometime.

Thanks Michael. I have now installed your addon successfully. However, I don't see any improvement over the Befzz version. I get the same result regardless of the import button that I use on your addon. I'll upload a screenshot to show you what I'm seeing. ![screenshot.png](https://archive.blender.org/developer/F334914/screenshot.png) My PSK validation program is almost ready for your first inspection. It is by no means complete, and I intend to continue updating it for as long and as much as necessary. I will probably upload it today, sometime.
Author

OK, the validator is ready for your perusal. I have included a ReadMe text file.

Please let me know if there is anything wrong with it. There is a very good chance that I will need to add dependency files to it, which could take me another day to organise.

<file removed, no longer valid>

OK, the validator is ready for your perusal. I have included a ReadMe text file. Please let me know if there is anything wrong with it. There is a very good chance that I will need to add dependency files to it, which could take me another day to organise. <file removed, no longer valid>

There's at least one ocx file missing, it seems to be a tab control. If you used some version of Visual Studio it should list them somewhere.

The screen shot you sent looks like a global orientation problem. I would say the model was created or saved on the wrong axis, which is something you can't really validate.

I can put drop downs on the file browser that let you pick the up and forward axis, it would be trial and error.

I simplified the bone flipping code and I'm trying to make some progress on the animation rewrite but it is going slow.

There's at least one ocx file missing, it seems to be a tab control. If you used some version of Visual Studio it should list them somewhere. The screen shot you sent looks like a global orientation problem. I would say the model was created or saved on the wrong axis, which is something you can't really validate. I can put drop downs on the file browser that let you pick the up and forward axis, it would be trial and error. I simplified the bone flipping code and I'm trying to make some progress on the animation rewrite but it is going slow.
Author

Thanks for trying it, Michael. I was afraid it would do something like that. I guess I was hoping that you would already have at least some, if not all of the dependency files from previous program installations.

We may have to go backwards and forwards a few times to get all of the necessary dependencies included and working. If you could let me know each time what file the program is asking for each time, that would be great. In this instance, I believe the file required is the "tabctl32.ocx" file, so I'll include that one in the code.

The screen shot you sent looks like a global orientation problem. I would say the model was created or saved on the wrong axis, which is something you can't really validate.

The orientation issue makes sense, except I'm pretty sure that you would be able to validate and auto-correct the issue.

Correct me if I'm wrong, but as far as I know, it is standard for all models to be built initially in a T-pose, which means that the mesh and bones would (or should) all have a common axis. The code to correct their orientation may not be trivial, but not impossible.

While not the most ideal solution, giving the user the option to choose the axis is better than what we have at the moment.

Just so you know, I appreciate your assistance with building this addon. Also, don't feel pressured to plough all of your time into it if you don't want to. For me at least, this is much more of a hobby thing, than something that I have to work out for a living.

Thanks for trying it, Michael. I was afraid it would do something like that. I guess I was hoping that you would already have at least some, if not all of the dependency files from previous program installations. We may have to go backwards and forwards a few times to get all of the necessary dependencies included and working. If you could let me know each time what file the program is asking for each time, that would be great. In this instance, I believe the file required is the "tabctl32.ocx" file, so I'll include that one in the code. > The screen shot you sent looks like a global orientation problem. I would say the model was created or saved on the wrong axis, which is something you can't really validate. The orientation issue makes sense, except I'm pretty sure that you would be able to validate and auto-correct the issue. Correct me if I'm wrong, but as far as I know, it is standard for all models to be built initially in a T-pose, which means that the mesh and bones would (or should) all have a common axis. The code to correct their orientation may not be trivial, but not impossible. While not the most ideal solution, giving the user the option to choose the axis is better than what we have at the moment. Just so you know, I appreciate your assistance with building this addon. Also, don't feel pressured to plough all of your time into it if you don't want to. For me at least, this is much more of a hobby thing, than something that I have to work out for a living.
Author

Ooooh Kaaayyy.... so the most obvious dependency files (OCX / Active X control files) are now included in the validator executable. The program is designed to extract these from the program itself, and so won't be visible until you run the program. Again, the program may want more files, but I can't tell exactly what for sure, until you run it on your side.

PSK Validator v0.1.zip

Just noticed this website doesn't show the correct file size when uploading files. I suspect they divided the number of bytes by 1000 in their web script, instead of 1024, displaying more KB than there actually is...

Ooooh Kaaayyy.... so the most obvious dependency files (OCX / Active X control files) are now included in the validator executable. The program is designed to extract these from the program itself, and so won't be visible until you run the program. Again, the program may want more files, but I can't tell exactly what for sure, until you run it on your side. [PSK Validator v0.1.zip](https://archive.blender.org/developer/F335568/PSK_Validator_v0.1.zip) Just noticed this website doesn't show the correct file size when uploading files. I suspect they divided the number of bytes by 1000 in their web script, instead of 1024, displaying more KB than there actually is...

I finally got some time and the validator works good. I did have to register the ocx files, which is normal without an installer. The summary on top is nice. I've needed to browse the data before and this makes it much easier. I've got two suggestions if it could be done. One is output the vector and quaternion decimal values to a fixed precision like 6 or 8 places. That would make them uniform and eliminate exponential output. The other would be using a fixed font for them, such as Consolas or Lucida Console. Both are included with Windows. I'm not sure if the grid can do fonts or fonts on a per-cell basis.

I put in some basic new code for the animations but it's not even close to working yet. In theory axis are standard to any given model type but that doesn't mean the models are always built and / or exported correctly. They can import and even animate right but still have everything oriented wrong.

Blender's axis correction factory generates matrices used to change the orientation, I use it as Blender is Z-up, -Y-forward and (as far as I can tell) PSK is Z-up, X-forward. I plan to get it integrated into the UI correctly so the axis can be overridden by the user as you mentioned.

I wrote this stuff for my own amusement long ago and mess with it for the fun of it, though I do end up needing breaks. I would like to get the whole thing working as well as possible.

I finally got some time and the validator works good. I did have to register the ocx files, which is normal without an installer. The summary on top is nice. I've needed to browse the data before and this makes it much easier. I've got two suggestions if it could be done. One is output the vector and quaternion decimal values to a fixed precision like 6 or 8 places. That would make them uniform and eliminate exponential output. The other would be using a fixed font for them, such as Consolas or Lucida Console. Both are included with Windows. I'm not sure if the grid can do fonts or fonts on a per-cell basis. I put in some basic new code for the animations but it's not even close to working yet. In theory axis are standard to any given model type but that doesn't mean the models are always built and / or exported correctly. They can import and even animate right but still have everything oriented wrong. Blender's axis correction factory generates matrices used to change the orientation, I use it as Blender is Z-up, -Y-forward and (as far as I can tell) PSK is Z-up, X-forward. I plan to get it integrated into the UI correctly so the axis can be overridden by the user as you mentioned. I wrote this stuff for my own amusement long ago and mess with it for the fun of it, though I do end up needing breaks. I would like to get the whole thing working as well as possible.
Author

Hey Michael, thanks for your reply, and thanks for testing my program.

Again, I was hoping that the program would just work without needing you to do anything extra, so I'm going to have to look over my initialisation code and fix that registration issue. Really, a simple application that does not need to be integrated with a user's OS, shouldn't require installation in the most literal sense.

Glad to hear that it would make it easier for you to retrieve data with. ;)
Let's just hope that I'm actually formatting the data correctly...

Remember, if you see any values that are out of place in the first few versions, let me know. To be honest with you, I haven't done much with fractional values before, so I really don't know if the values that we are seeing in the columns at the moment are correct. The strings and integers definitely appear to be correct, but I'm not sure about the fractional values. They may look OK, but you never know... for example, there's a chance that a particular value appears correct to begin with, but may have some digits truncated from the end. With a proper validator (and, by extent, your addon), we want to make sure that we are retrieving the precise values that are actually stored in the files, unaltered.

I was thinking the same thing about fixing the fractional values to a set number of decimal places. It shouldn't be too hard to implement, but I fear that it could significantly add to the amount of processing required to "load" each PSK file. Then again, my obsession of creating high-performance apps may be getting the better of me...

Changing fonts used overall? Piece of cake. Giving individual items / records their own unique font? Not so easy. I can change the field text colours of each individual item / record so each item (or even each individual field) has its own unique color. I can also make them appear bold in much the same way.

An "axis correction factory"? Sounds interesting. If that's something already built into Blender, then utilising it correctly would certainly help to cut your work load down.

Breaks are good. On several occasions I have stared at the computer screen, trying to think of how to solve a problem, and then got up to take a break, and solved the problem (or at least come up with a potential solution) usually within 10 minutes away from the screen. Also, many health reasons that I can personally attest to.

Hey Michael, thanks for your reply, and thanks for testing my program. Again, I was hoping that the program would just work without needing you to do anything extra, so I'm going to have to look over my initialisation code and fix that registration issue. Really, a simple application that does not need to be integrated with a user's OS, shouldn't require installation in the most literal sense. Glad to hear that it would make it easier for you to retrieve data with. ;) Let's just hope that I'm actually formatting the data correctly... Remember, if you see any values that are out of place in the first few versions, let me know. To be honest with you, I haven't done much with fractional values before, so I really don't know if the values that we are seeing in the columns at the moment are correct. The strings and integers definitely appear to be correct, but I'm not sure about the fractional values. They may *look* OK, but you never know... for example, there's a chance that a particular value appears correct to begin with, but may have some digits truncated from the end. With a proper validator (and, by extent, your addon), we want to make sure that we are retrieving the precise values that are actually stored in the files, unaltered. I was thinking the same thing about fixing the fractional values to a set number of decimal places. It shouldn't be too hard to implement, but I fear that it could significantly add to the amount of processing required to "load" each PSK file. Then again, my obsession of creating high-performance apps may be getting the better of me... Changing fonts used overall? Piece of cake. Giving individual items / records their own unique font? Not so easy. I can change the field text colours of each individual item / record so each item (or even each individual field) has its own unique color. I can also make them appear **bold** in much the same way. An "axis correction factory"? Sounds interesting. If that's something already built into Blender, then utilising it correctly would certainly help to cut your work load down. Breaks are good. On several occasions I have stared at the computer screen, trying to think of how to solve a problem, and then got up to take a break, and solved the problem (or at least come up with a potential solution) usually within 10 minutes away from the screen. Also, many health reasons that I can personally attest to.
Author

Validator Update:
In the upcoming version, I have added font selection functionality to the program, so you can now choose your own font on the fly. I have also added in a set of menus at the top, to make it look a bit more tidy, and to allow for keyboard combinations with particular functions. Furthermore, it now has a copy feature included, so you can now select multiple list items / data records at once, and then copy them to the clipboard for pasting into another program. The program's resizing behaviour has been corrected to allow controls to be stretched downwards, as well as sideways.

I have come across several issues that need to be tended to before I can give you an updated version. One is that the program will crash if it tries to open a file that has not been formatted correctly, or is corrupted in some way. Another is the large amount of memory that the program is consuming, which is far greater (~3x) than the amount it should be using. The list controls are the primary cause for this, and are also responsible for the slow loading of PSK files, so I will need to look replacing them with something more efficient.

As I have studies to tend to, I anticipate that the next release will be some time next week. Hopefully, I will have all the issues ironed-out by then.

**Validator Update:** In the upcoming version, I have added font selection functionality to the program, so you can now choose your own font on the fly. I have also added in a set of menus at the top, to make it look a bit more tidy, and to allow for keyboard combinations with particular functions. Furthermore, it now has a copy feature included, so you can now select multiple list items / data records at once, and then copy them to the clipboard for pasting into another program. The program's resizing behaviour has been corrected to allow controls to be stretched downwards, as well as sideways. I have come across several issues that need to be tended to before I can give you an updated version. One is that the program will crash if it tries to open a file that has not been formatted correctly, or is corrupted in some way. Another is the large amount of memory that the program is consuming, which is far greater (~3x) than the amount it should be using. The list controls are the primary cause for this, and are also responsible for the slow loading of PSK files, so I will need to look replacing them with something more efficient. As I have studies to tend to, I anticipate that the next release will be some time next week. Hopefully, I will have all the issues ironed-out by then.
Author

I am making good progress on the Validator. It could still be a few more days before I can release it.

Unfortunately, the official specifications are not clear cut on many things, potentially leading people to assume many things, which may or may not be true. Michael, I need to check your knowledge on a few things, just so that I know I'm going in the right direction, and not simply assuming things as I go along. Two brains are better than one, as they say.

You may confirm that the following information (or my understanding of PSK files) is true or false. If at all false, please let me know where I have gone wrong. If you don't really know, just say whether or not something sounds right, and hopefully that should suffice.

Each "face" which uses the WedgeIndex member of the VTriangle (Faces Data) structure, is presumed to reference three "wedges" that use the VVertex (Wedges Data) structure. The PointIndex member of the VVertex (Wedges Data) structure is presumed to reference each "point" that uses the VPoint stucture.

As the WedgeIndex member of the VTriangle (Faces Data) structure is a _WORD data type (2 bytes), it cannot store values over 65,535. Also, as the PointIndex member of the VVertex (Wedges Data) structure is a _WORD data type, it cannot store values over 65,535. Given this information, it can be assumed that there cannot be more than 65,536 wedges, and no more than 65,536 points, in a PSK file.

I understand that the non-proprietry, third-party PSKX format created by "Gildor", increases this limit.

The ParentIndex of bones appear to simply refer back to other bones in the same data section, effectively building a hierarchy / tree / armature / skeleton of bones. Bone Influence records have a PointIndex and BoneIndex data field; the former appearing to refer back to points in the Points section, and the latter likely referring back to bones in the Bones section. It must be noted here that both PointIndex and BoneIndex fields / members are the INT data type (4 bytes). Although I don't know why the PointIndex is a different data type here, the fact that the BoneIndex field is a 4 byte field means that PSK files could potentially contain more than 65,536 bones (up to 4,294,967,296 bones).

I understand the following to be pretty much correct, but feel free to provide me with any comment as you see fit.

You may have seen the first column in each section list, titled "Index (theoretical)". I named it as such, as I was trying to understand the overall structure of PSK files at the time, and I did not know for sure if it was correct or not. At the time, I understood that each element (face, wedge, point, etc.) did not (and do not) have a literal index marked in the file, and that their index was (and is) based upon the order in which they appear. I now believe that these indexes are zero-based, given the values used in WedgeIndex, PointIndex, and ParentIndex structure members. Thus, the column title will be renamed to "Index (logical)", as the indexes (or indices) are no longer theoretical, and based on order of appearance; and the indexes (or indices) changed to be zero-based. All other columns / fields are literal, as their values actually appear in a PSK file.

When looking at the sizes of the header structures (32 bytes each), and the other data structures that follow, their Highest Common Factor is a multiple of 4. This means that the file size of a PSK file should, at the very least, be a multiple of 4 (in bytes). If not, that means the file has not been formatted correctly, or is corrupt in some way. Note that this would be a very basic check, compared to calculating the expected file size from the DataCount field values. I have to think about this some more, such as how to manage cases where either the DataCount field values are incorrect, and/or the number of data fields that appear in each section is incorrect.

I was thinking at one stage, whether or not PSK files need to be padded out to a certain multiple of bytes, but given my recent understanding of the HCF, I doubt that they would need to be. If they were, however, required to be padded, it would rule out the possibility of using the HCF as a check. A multiple of 4 seems to be a common multiple to use when padding structured files.

I am making good progress on the Validator. It could still be a few more days before I can release it. Unfortunately, the official specifications are not clear cut on many things, potentially leading people to assume many things, which may or may not be true. Michael, I need to check your knowledge on a few things, just so that I know I'm going in the right direction, and not simply assuming things as I go along. Two brains are better than one, as they say. You may confirm that the following information (or my understanding of PSK files) is true or false. If at all false, please let me know where I have gone wrong. If you don't really know, just say whether or not something sounds right, and hopefully that should suffice. Each "face" which uses the WedgeIndex member of the VTriangle (Faces Data) structure, is presumed to reference three "wedges" that use the VVertex (Wedges Data) structure. The PointIndex member of the VVertex (Wedges Data) structure is presumed to reference each "point" that uses the VPoint stucture. As the WedgeIndex member of the VTriangle (Faces Data) structure is a _WORD data type (2 bytes), it cannot store values over 65,535. Also, as the PointIndex member of the VVertex (Wedges Data) structure is a _WORD data type, it cannot store values over 65,535. Given this information, it can be assumed that there cannot be more than 65,536 wedges, and no more than 65,536 points, in a PSK file. I understand that the non-proprietry, third-party PSKX format created by "Gildor", increases this limit. The ParentIndex of bones appear to simply refer back to other bones in the same data section, effectively building a hierarchy / tree / armature / skeleton of bones. Bone Influence records have a PointIndex and BoneIndex data field; the former appearing to refer back to points in the Points section, and the latter likely referring back to bones in the Bones section. It must be noted here that both PointIndex and BoneIndex fields / members are the INT data type (4 bytes). Although I don't know why the PointIndex is a different data type here, the fact that the BoneIndex field is a 4 byte field means that PSK files could potentially contain more than 65,536 bones (up to 4,294,967,296 bones). I understand the following to be pretty much correct, but feel free to provide me with any comment as you see fit. You may have seen the first column in each section list, titled "Index (theoretical)". I named it as such, as I was trying to understand the overall structure of PSK files at the time, and I did not know for sure if it was correct or not. At the time, I understood that each element (face, wedge, point, etc.) did *not* (and do not) have a literal index marked in the file, and that their index was (and is) based upon the order in which they appear. I now believe that these indexes are zero-based, given the values used in WedgeIndex, PointIndex, and ParentIndex structure members. Thus, the column title will be renamed to "Index (logical)", as the indexes (or indices) are no longer theoretical, and based on order of appearance; and the indexes (or indices) changed to be zero-based. All other columns / fields are literal, as their values actually appear in a PSK file. When looking at the sizes of the header structures (32 bytes each), and the other data structures that follow, their Highest Common Factor is a multiple of 4. This means that the file size of a PSK file should, at the very least, be a multiple of 4 (in bytes). If not, that means the file has not been formatted correctly, or is corrupt in some way. Note that this would be a very basic check, compared to calculating the expected file size from the DataCount field values. I have to think about this some more, such as how to manage cases where either the DataCount field values are incorrect, and/or the number of data fields that appear in each section is incorrect. I was thinking at one stage, whether or not PSK files need to be padded out to a certain multiple of bytes, but given my recent understanding of the HCF, I doubt that they would need to be. If they were, however, required to be padded, it would rule out the possibility of using the HCF as a check. A multiple of 4 seems to be a common multiple to use when padding structured files.
Author

Actually, I could be wrong about the BoneIndex field of bones. Given that:

  • root bones don't have a parent
  • the specification header file (UnrealAnimDataStructs.h) states ParentIndex is "0/NULL if this is the root bone"

This leads me to believe that the bones in the Bones Data section are more likely to have one-based indexes / indices. If a single root bone were logically assigned the index value of zero, how would another bone (child bone) reference it, if the spec says that a value of 0 in the ParentIndex field signifies a bone to be a root bone?

Furthermore, I am not sure whether or not a PSK file is allowed to have more than one root bone. The following link I found appears to suggest that only one is allowed:

https://answers.unrealengine.com/questions/109814/multiple-roots-are-found-in-the-bone-hierarchy-we.html

This may suggest that only one root bone is allowed in a PSK file, but what about other applications? I tried looking into the Blender documentation, but it doesn't actually say:

https://www.blender.org/manual/rigging/armatures/structure.html

Actually, I could be wrong about the BoneIndex field of bones. Given that: - root bones don't have a parent - the specification header file (UnrealAnimDataStructs.h) states ParentIndex is "0/NULL if this is the root bone" This leads me to believe that the bones in the Bones Data section are more likely to have one-based indexes / indices. If a single root bone were logically assigned the index value of zero, how would another bone (child bone) reference it, if the spec says that a value of 0 in the ParentIndex field signifies a bone to be a root bone? Furthermore, I am not sure whether or not a PSK file is allowed to have more than one root bone. The following link I found appears to suggest that only one is allowed: https://answers.unrealengine.com/questions/109814/multiple-roots-are-found-in-the-bone-hierarchy-we.html This *may* suggest that only one root bone is allowed in a PSK file, but what about other applications? I tried looking into the Blender documentation, but it doesn't actually say: https://www.blender.org/manual/rigging/armatures/structure.html
Author

After further investigation, I have arrived at the conclusion that PSK files most likely only have a single root bone, as apparently many game engines and 3D modelling programs (perhaps with the exception of Blender), prefer armatures to have only one root bone. If this is the case, then the first bone found in a PSK file must automatically become the root bone (with an index of '0'), with all child bones of the root bone having a ParentIndex value of '0'. Since the root bone cannot refer to another root bone, its ParentIndex value would also likely be zero, and possibly ignored altogether.

Explicitly stated in Ogre3D:

  1. Ogre requires that there be a 'single' root bone.

http://ogre3d.org/tikiwiki/Exporter+3DStudioMax#Exporting_Bones

Hinted at in the CryENGINE documentation:

No bones can be outside of the skeletal hierarchy, the exporter starts at one root bone and steps through all its children.

http://docs.cryengine.com/display/SDKDOC2/Character+Rigging+Guidelines

Supposedly a CryENGINE official talking about it in regards to Blender exporter for CryENGINE:
https://www.cryengine.com/community/viewtopic.php?f=315&t=103136&start=2100#p1273950

and many complaints of people having troubles importing something exported from Blender, relating to having too many root bones (can't be bothered putting sources for this here, just "Google it" ;))

After further investigation, I have arrived at the conclusion that PSK files most likely only have a single root bone, as apparently many game engines and 3D modelling programs (perhaps with the exception of Blender), prefer armatures to have only one root bone. If this is the case, then the first bone found in a PSK file must automatically become the root bone (with an index of '0'), with all child bones of the root bone having a ParentIndex value of '0'. Since the root bone cannot refer to another root bone, its ParentIndex value would also likely be zero, and possibly ignored altogether. Explicitly stated in Ogre3D: >1. Ogre requires that there be a 'single' root bone. http://ogre3d.org/tikiwiki/Exporter+3DStudioMax#Exporting_Bones Hinted at in the CryENGINE documentation: > No bones can be outside of the skeletal hierarchy, the exporter starts at one root bone and steps through all its children. http://docs.cryengine.com/display/SDKDOC2/Character+Rigging+Guidelines Supposedly a CryENGINE official talking about it in regards to Blender exporter for CryENGINE: https://www.cryengine.com/community/viewtopic.php?f=315&t=103136&start=2100#p1273950 and many complaints of people having troubles importing something exported from Blender, relating to having too many root bones (can't be bothered putting sources for this here, just "Google it" ;))
Author

Sorry Michael, this is taking me a lot longer than anticipated. I would rather give you something that is complete, instead of something that is half-finished and/or buggy. I am still making progress, it's just slow.

You may like to know that I have added a bone hierarchy to the program, which somewhat resembles the hierarchy that can been seen in Blender. Each bone is linked to other bones via a dotted line in the list, and can be expanded and collapsed using the + and - symbols next to them, which will show and hide (respectively) the connected child bones underneath, in the hierarchy. When a bone is clicked on, it will give you the same information as is displayed in the initial list on the first tab/screen, just in a vertical format. I could add something more here, but not quite sure what yet. Maybe bone depth?

I'll focus on getting the features that I have put in all fixed up and bug free, lest I get carried away adding more new features. I won't give you a time frame, mainly because I don't think I could stick to it, so I'll just say that I'll get it to you ASAP.

Feel free to send me a message some time. I'm starting to feel like I'm talking to myself again. :P

Sorry Michael, this is taking me a lot longer than anticipated. I would rather give you something that is complete, instead of something that is half-finished and/or buggy. I am still making progress, it's just slow. You may like to know that I have added a bone hierarchy to the program, which somewhat resembles the hierarchy that can been seen in Blender. Each bone is linked to other bones via a dotted line in the list, and can be expanded and collapsed using the + and - symbols next to them, which will show and hide (respectively) the connected child bones underneath, in the hierarchy. When a bone is clicked on, it will give you the same information as is displayed in the initial list on the first tab/screen, just in a vertical format. I could add something more here, but not quite sure what yet. Maybe bone depth? I'll focus on getting the features that I have put in all fixed up and bug free, lest I get carried away adding more new features. I won't give you a time frame, mainly because I don't think I could stick to it, so I'll just say that I'll get it to you ASAP. Feel free to send me a message some time. I'm starting to feel like I'm talking to myself again. :P
Author

Still not entirely finished, but whatever. I resolved the file dependency issue today. If you want, you can test this by un-registering the Active X controls, and then just running the program.

PSK Validator v0.2.zip

Now includes a Word Document, so you don't have to burn your eyes reading a text document in Notepad. :P

Still not entirely finished, but whatever. I resolved the file dependency issue today. If you want, you can test this by un-registering the Active X controls, and then just running the program. [PSK Validator v0.2.zip](https://archive.blender.org/developer/F350316/PSK_Validator_v0.2.zip) Now includes a Word Document, so you don't have to burn your eyes reading a text document in Notepad. :P

I'm grabbing the update and should be able to look at it in a day or so. I've haven't been able to digest or respond to your previous posts due to overriding unexpected events. I'm a little too overloaded to figure out how to send a private message here, I'm not even sure it's possible. If you see a way let me know. If not I've no opposition to explaining my absence via direct email.

I'm grabbing the update and should be able to look at it in a day or so. I've haven't been able to digest or respond to your previous posts due to overriding unexpected events. I'm a little too overloaded to figure out how to send a private message here, I'm not even sure it's possible. If you see a way let me know. If not I've no opposition to explaining my absence via direct email.
Author

Yeah, I had a look around myself, but I don't see a way to PM other users.

You don't have to explain yourself, mate. Just let me know when you are going to be away for a while, then I don't have to freak out. :P

When we (or I) get this PSK stuff ironed out, then we can move on to the PSA stuff. No pressure.

If you would still like my email, I put it here for a few days, or until you say you have it; which ever is sooner.

Yeah, I had a look around myself, but I don't see a way to PM other users. You don't have to explain yourself, mate. Just let me know when you are going to be away for a while, then I don't have to freak out. :P When we (or I) get this PSK stuff ironed out, then we can move on to the PSA stuff. No pressure. If you would still like my email, I put it here for a few days, or until you say you have it; which ever is sooner. <email removed>
Author

Dude, I'm sure it isn't hard to make a quick post or email. I gave you my address, then removed it after almost a week...

Are you with me on this, or not?

Dude, I'm sure it isn't hard to make a quick post or email. I gave you my address, then removed it after almost a week... Are you with me on this, or not?
Author

Secondary project abandoned.

Blender admin, please include addon as per first post (v2.6.2).

Secondary project abandoned. Blender admin, please include addon as per first post (v2.6.2).
Member

Added subscribers: @darknet, @BrendonMurphy

Added subscribers: @darknet, @BrendonMurphy
Member

hi, as we are no longer accepting new addons, I'm currently backlogging this, I've added @darknet as a subscriber.
This is something we could look at in Blender 2.8, the api is likely not to change too much & all i/o addons will need to be reviewed.
What will be needed is: Sample Models & Scenes with information on their creation.
Watertight code, either it works or it reports to the user that a particular element is not supported, no crashes.
Let's make this work & well, then you will need to submit for code review, make sure is pep8 & conforms to Blender standards.
Thanks

hi, as we are no longer accepting new addons, I'm currently backlogging this, I've added @darknet as a subscriber. This is something we could look at in Blender 2.8, the api is likely not to change too much & all i/o addons will need to be reviewed. What will be needed is: Sample Models & Scenes with information on their creation. Watertight code, either it works or it reports to the user that a particular element is not supported, no crashes. Let's make this work & well, then you will need to submit for code review, make sure is pep8 & conforms to Blender standards. Thanks

Mr. Murphy,

Real life issues still prevent me from solving the other animation code changes I want to make in my code base. I don't expect this to change for several more weeks at best. Although I fixed the animation distortions in my code long ago the best I can say is this:

  • The befzz code's advantage is that it also corrects the animation distortions and is directly based on Mr. Phan's code.
  • The reversed bones seem more of a visual annoyance than an actual problem. I can fix them in the bind pose with a cross product calculation but they revert back when the animation is applied.
  • My code is slower than Mr. Phan's code and the befzz code and is larger in size as I wrote it in a similar fashion to how I would write a vb.net or c# program.

While I would like to see my code included in Blender it wouldn't be the 2.8 release as it won't be finished. I'm not sure it can pass pep8 in its current state and was not aware of official Blender standards (I'll have to look for them.)

It seems like the optimum solution in the shortest amount of time would be updating the current code to fix the animation distortions. I've no idea if Mr. Phan has the time or inclination to do so. If he does I will be happy to try and help in whatever way I can given my current status. Otherwise I will still plan to submit my code per the requirements you listed when I am able to finish the changes and integrate several other fixes from my current development version.

Michael

Mr. Murphy, Real life issues still prevent me from solving the other animation code changes I want to make in my code base. I don't expect this to change for several more weeks at best. Although I fixed the animation distortions in my code long ago the best I can say is this: - The befzz code's advantage is that it also corrects the animation distortions and is directly based on Mr. Phan's code. - The reversed bones seem more of a visual annoyance than an actual problem. I can fix them in the bind pose with a cross product calculation but they revert back when the animation is applied. - My code is slower than Mr. Phan's code and the befzz code and is larger in size as I wrote it in a similar fashion to how I would write a vb.net or c# program. While I would like to see my code included in Blender it wouldn't be the 2.8 release as it won't be finished. I'm not sure it can pass pep8 in its current state and was not aware of official Blender standards (I'll have to look for them.) It seems like the optimum solution in the shortest amount of time would be updating the current code to fix the animation distortions. I've no idea if Mr. Phan has the time or inclination to do so. If he does I will be happy to try and help in whatever way I can given my current status. Otherwise I will still plan to submit my code per the requirements you listed when I am able to finish the changes and integrate several other fixes from my current development version. Michael
Member

You can sumbit the code. As long it not messy and clean. Easy to read.

You can sumbit the code. As long it not messy and clean. Easy to read.
Author

Removed subscriber: @tristanlbailey

Removed subscriber: @tristanlbailey
Member

Changed status from 'Open' to: 'Archived'

Changed status from 'Open' to: 'Archived'
Brendon Murphy self-assigned this 2017-04-15 08:20:17 +02:00
Member

Closing as archived as remains unsolved & author unsubscribed

Closing as archived as remains unsolved & author unsubscribed
Member

Added subscriber: @Mets

Added subscriber: @Mets
Sign in to join this conversation.
No Milestone
No project
No Assignees
5 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender-addons#48800
No description provided.