Page MenuHome
Create Task
Maniphest T81226

OldNewMap handling of pointers in blenloader does not work correctly with FD_FLAGS_SWITCH_ENDIAN
Closed, ResolvedPublic
Actions

Description

System Information
Operating system: Linux/ppc64
Graphics card: llvmpipe

Blender Version
Broken: 2.83.4, master
Worked: ?

Short description of error
The functions switch_endian_bh4 and switch_endian_bh8 in readfile.c do not byteswap the "old" pointer field. In theory this is fine, as it is just used as an opaque handle, which is mapped to an actual pointer via the OldNewMap. However, in practice this does not work because it seems that the reference pointers to the object are byteswapped, and so if the pointer used as key to create the entry in the OldNewMap is not byteswapped as well, the values will not match and the lookup will fail.

This can easily fixed by just swapping the "old" pointer in the bh swapping functions. That also removes the need for the workaround in bh4_from_bh8. Please see the following patch:

blender.patch1 KBDownload

Exact steps for others to reproduce the error

  1. Start blender on a big endian 64-bit system
  2. Startup crashes as it fails to load the builtin WM blendfile, getting a NULL back from oldnewmap_lookup_and_inc

Revisions and Commits

rB Blender
rBffb220590ea7 Fix T81226: Crash opening 64bit files with endian switching
rB724a4f467658 Fix T81226: Crash opening 64bit files with endian switching

Related Objects

Event Timeline

Marcus Comstedt (zeldin) created this task.Sep 27 2020, 12:53 PM
Germano Cavalcante (mano-wii) added subscribers: Jacques Lucke (JacquesLucke), Germano Cavalcante (mano-wii).Oct 5 2020, 4:58 PM

Thanks for the report, unfortunately I don't have a big endian 64-bit system to test.

@Jacques Lucke (JacquesLucke), if I'm not mistaken, you worked in this area recently.
Do you know what's going on here?

Jacques Lucke (JacquesLucke) added a comment.EditedOct 5 2020, 5:07 PM

Just last week I found this issue as well. My fix (which is essentially the same) is part of D9089.

@Marcus Comstedt (zeldin), can you provide a .blend file that has been saved on a big endian 64 bit system? Can you also check if D9089 fixes it for you?

Marcus Comstedt (zeldin) added a comment.Oct 5 2020, 10:28 PM

@Jacques Lucke (JacquesLucke) Sure. Here you go:

untitled.blend712 KBDownload

I'll try your patch later, need to make a new git checkout since the patch does not apply cleanly to 2.83.4. :-)

Jacques Lucke (JacquesLucke) mentioned this in D9089: DNA: Cleanup endian switching when loading file..Oct 6 2020, 1:51 PM
Jacques Lucke (JacquesLucke) added a comment.Oct 6 2020, 1:55 PM

Thanks!

Did any older version of Blender work on your hardware?

Marcus Comstedt (zeldin) added a comment.Oct 6 2020, 6:26 PM

Yes. The "?" in "Worked" just means that I can't remember what the version number was. :-)

Well, I can't say for sure that it was possible to load little endian .blend files from disk, but running it in BE worked fine.

Campbell Barton (campbellbarton) closed this task as Resolved by committing rB724a4f467658: Fix T81226: Crash opening 64bit files with endian switching.Oct 7 2020, 1:00 PM
Campbell Barton (campbellbarton) claimed this task.
Campbell Barton (campbellbarton) added a commit: rB724a4f467658: Fix T81226: Crash opening 64bit files with endian switching.
Jeroen Bakker (jbakker) mentioned this in T77348: Blender LTS: Maintenance Task.Oct 7 2020, 1:03 PM
Jeroen Bakker (jbakker) added a commit: rBffb220590ea7: Fix T81226: Crash opening 64bit files with endian switching.Oct 28 2020, 9:59 AM