The file doc/blender_file_format/mystery_of_the_blend.html is badly out of date, and just plain wrong in some respects. This set of patches tries to make it more useful.
Hi Lawrence, can you elaborate on what you find Badly out of date and plainly wrong?
I reviewed the text, but expected more difference if you find something badly out of date and plainly wrong.
I do agree with the update of the links and adding more details to the doc, but would not add personal reflections of the file format to the doc as that is not the place to have discussions or opinions.
I also believe that the document by itself should be moved to perhaps wiki.blender.org or code.blender.org
I also believe that currently we have the option to move the rest to the source/tools location where currently the other tools are located.
From the original text:
Blender is known to have excellent downward and upward compatibility
Wrong. “Upward” (forward?) compatibility was never a feature.
The article is written to be programming language independent
Wrong. It uses C-language structs.
and I've setup a web-site for support.
Out of date. The page in question has gone.
the default blend file of Blender 2.48 contains more than 400 of these file-blocks
Out of date. The default blend file of Blender 2.72 contains more than 800 of these file-blocks. Particularly since you do mention post-2.4x versions of Blender elsewhere.
- No mention of (rather important) GLOB block, or of the USER block.
- No mention that ENDB block might be truncated.
- No mention that the version number in the file header governs translation of fields and structures which may look the same as in older versions, but whose meanings have changed.
- No mention that the size of a block must always be a multiple of 4 bytes.
- No mention of the significant difference between principal versus subsidiary blocks, or that the SDNA index can be invalid in subsidiary blocks.
- No mention of the (important!) distinction between type index and structure index.
- No mention of possibly-invalid pointer fields.
Does that give you enough of a start on making sense of it all?
What do you mean by “personal reflections”?
Upwards/Forward compatibility (to extend of loading data) is a demonstrated feature. See first picture of http://zgodzinski.com/blender-prehistory/
Also it is used by production houses to temp switch for some reason to a latest version of blender to do stuff and continue with the version of blender in fixed their pipeline.
Agree the reference to the web-site is gone. this section can be removed as it is replaced with a generator that generates the website from a blend file.
Out of date blend file data. Is not out of date. A blend file from blender 2.48 still has 400 file-blocks. You can adapt it every time blender does a new release, but at the end it is not important that this document reflects the latest version. The intention of the document is to describe the blend file format.
About the rest of your comments
You explain the GLOB and USER data blocks in more depth. The original document only described the bits and bytes. Don't mind adding that.
The meaning of the data is blender version specific. It is mentioned here:
In the source code of blender there is actually logic which can transform and translate every structure used by a Blender release to the one of the release you're using [ref: http://download.blender.org/source/blender-2.48a.tar.gz blender/blenloader/intern/readfile.c lines 4946-7960]. The more difference between releases the more logic is executed
Alignment: As DNA struct must be aligned, the file data are therefore also aligned. It is not mentioned, so we could add that.
The other points have valid value to be added. Would add more wording to it though.
The rest of your points are valid so no problem adding them.
Would remove the second paragraph of your conclusion as that should IMO be part of a different document.
Sorry about the personal reflections. I have re-read the additions and I have mis-read them previously.
Where do you suggest we could add the document including the modifications?
I would recommend to move the code parts to a blender/source/tools repository and the document to the blender developer wiki.
I' not deep into Blender (as you are), but I wrote several parsers in the past, but I fail to parse the .blend files:
The process of flattening a structure is not explained well enough, especially when the structure contains another structure. All documents I found root in one document that doesn't explain the process, to nobody else does. For example the only comment I received for How to interpret Blender's SDNA structure field type index? was "Not really sure, but for what I remember (and doc/blender_file_format/ directory) you never refer to structure index, only to type index. Structure itself is linked with some type though, so you get that via reverse lookup."
Note "Not really sure, "... the public seems to do not know. So the document can be improved clearly .
Structures inside structures work the same way as in C—they have to. My rework explains exactly when the structure index is used, and when the type index is used. Which is just one reason why I think it’s a big improvement on the doc that’s already there!