OldNewMap handling of pointers in blenloader does not work correctly with FD_FLAGS_SWITCH_ENDIAN #81226

New Issue

Marcus Comstedt · 2020-09-27T12:53:40+02:00

Marcus Comstedt commented

2020-09-27 12:53:40 +02:00

System Information
Operating system: Linux/ppc64
Graphics card: llvmpipe

Blender Version
Broken: 2.83.4, master
Worked: ?

Short description of error
The functions switch_endian_bh4 and switch_endian_bh8 in readfile.c do not byteswap the "old" pointer field. In theory this is fine, as it is just used as an opaque handle, which is mapped to an actual pointer via the OldNewMap. However, in practice this does not work because it seems that the reference pointers to the object are byteswapped, and so if the pointer used as key to create the entry in the OldNewMap is not byteswapped as well, the values will not match and the lookup will fail.

This can easily fixed by just swapping the "old" pointer in the bh swapping functions. That also removes the need for the workaround in bh4_from_bh8. Please see the following patch:
blender.patch

Exact steps for others to reproduce the error

Start blender on a big endian 64-bit system
Startup crashes as it fails to load the builtin WM blendfile, getting a NULL back from oldnewmap_lookup_and_inc

**System Information** Operating system: Linux/ppc64 Graphics card: llvmpipe **Blender Version** Broken: 2.83.4, master Worked: ? **Short description of error** The functions switch_endian_bh4 and switch_endian_bh8 in readfile.c do not byteswap the "old" pointer field. In theory this is fine, as it is just used as an opaque handle, which is mapped to an actual pointer via the OldNewMap. However, in practice this does not work because it seems that the reference pointers to the object *are* byteswapped, and so if the pointer used as key to create the entry in the OldNewMap is not byteswapped as well, the values will not match and the lookup will fail. This can easily fixed by just swapping the "old" pointer in the bh swapping functions. That also removes the need for the workaround in bh4_from_bh8. Please see the following patch: [blender.patch](https://archive.blender.org/developer/F8928134/blender.patch) **Exact steps for others to reproduce the error** 1) Start blender on a big endian 64-bit system 2) Startup crashes as it fails to load the builtin WM blendfile, getting a NULL back from oldnewmap_lookup_and_inc

Marcus Comstedt commented

2020-09-27 12:53:40 +02:00

Added subscriber: @zeldin

Germano Cavalcante commented

2020-10-05 16:58:28 +02:00

Added subscribers: @JacquesLucke, @mano-wii

Germano Cavalcante commented

2020-10-05 16:58:28 +02:00

Thanks for the report, unfortunately I don't have a big endian 64-bit system to test.

@JacquesLucke, if I'm not mistaken, you worked in this area recently.
Do you know what's going on here?

Thanks for the report, unfortunately I don't have a big endian 64-bit system to test. @JacquesLucke, if I'm not mistaken, you worked in this area recently. Do you know what's going on here?

Jacques Lucke commented

2020-10-05 17:07:22 +02:00

Just last week I found this issue as well. My fix (which is essentially the same) is part of D9089.

@zeldin, can you provide a .blend file that has been saved on a big endian 64 bit system? Can you also check if D9089 fixes it for you?

Just last week I found this issue as well. My fix (which is essentially the same) is part of [D9089](https://archive.blender.org/developer/D9089). @zeldin, can you provide a .blend file that has been saved on a big endian 64 bit system? Can you also check if [D9089](https://archive.blender.org/developer/D9089) fixes it for you?

Marcus Comstedt commented

2020-10-05 22:28:05 +02:00

@JacquesLucke Sure. Here you go:
untitled.blend

I'll try your patch later, need to make a new git checkout since the patch does not apply cleanly to 2.83.4. :-)

@JacquesLucke Sure. Here you go: [untitled.blend](https://archive.blender.org/developer/F8958873/untitled.blend) I'll try your patch later, need to make a new git checkout since the patch does not apply cleanly to 2.83.4. :-)

Jacques Lucke commented

2020-10-06 13:55:05 +02:00

Thanks!

Did any older version of Blender work on your hardware?

Thanks! Did any older version of Blender work on your hardware?

Marcus Comstedt commented

2020-10-06 18:26:07 +02:00

Yes. The "?" in "Worked" just means that I can't remember what the version number was. :-)

Well, I can't say for sure that it was possible to load little endian .blend files from disk, but running it in BE worked fine.

Yes. The "?" in "Worked" just means that I can't remember what the version number was. :-) Well, I can't say for sure that it was possible to load little endian .blend files from disk, but running it in BE worked fine.

blender-admin commented

2020-10-07 12:48:59 +02:00

This issue was referenced by ffb220590e

This issue was referenced by ffb220590ea76bc279eeb3498776a360467125c1

blender-admin commented

2020-10-07 12:48:59 +02:00

This issue was referenced by 724a4f4676

This issue was referenced by 724a4f4676586d65b718deab7f04f207e0ad6488

Campbell Barton commented

2020-10-07 13:00:44 +02:00

Changed status from 'Needs Triage' to: 'Resolved'

Campbell Barton closed this issue

2020-10-07 13:00:44 +02:00

Campbell Barton self-assigned this 2020-10-07 13:00:44 +02:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

OldNewMap handling of pointers in blenloader does not work correctly with FD_FLAGS_SWITCH_ENDIAN #81226