Unify blend file thumbnail extraction #63736

New Issue

Campbell Barton · 2019-04-19T15:21:09+02:00

Campbell Barton commented

2019-04-19 15:21:09 +02:00

Currently there are two utilities for extracting thumbnails from Blend files.

./source/blender/blendthumb/src/BlenderThumb.cpp
./release/bin/blender-thumbnailer.py

This is causing a problem for adding a new compression method D4402: LZ4 Compression for blend file IO because we need to support a new kind of decompression for both.

Besides this, Python doesn't support LZ4, making it likely we would need to write a new utility in C/C++ anyway.

This task proposes to share use a single thumbnail extraction utility (probably based on BlenderThumb.cpp) which all platforms can share.

Currently there are two utilities for extracting thumbnails from Blend files. - `./source/blender/blendthumb/src/BlenderThumb.cpp` - `./release/bin/blender-thumbnailer.py` This is causing a problem for adding a new compression method [D4402: LZ4 Compression for blend file IO](https://archive.blender.org/developer/D4402) because we need to support a new kind of decompression for both. Besides this, Python doesn't support LZ4, making it likely we would need to write a new utility in C/C++ anyway. ---- This task proposes to share use a single thumbnail extraction utility (probably based on `BlenderThumb.cpp`) which all platforms can share.

Campbell Barton commented

2019-04-19 15:21:09 +02:00

Added subscriber: @ideasman42

blender-admin commented

2019-04-19 15:21:09 +02:00

#91841 was marked as duplicate of this issue

Ante Lausic commented

2019-10-03 14:38:20 +02:00

Added subscriber: @alausic

Ante Lausic commented

2019-10-03 14:38:20 +02:00

I am starting to work on this. I'll post here my progress.

Ante Lausic commented

2019-10-11 20:08:52 +02:00

Hi guys,

Last couple of days I've been working on this task, mainly investigating how thumbnailer handles are working on different operating systems and what the BlendThumb.cpp actually does.

Now I've come up with the idea how to approach with the solution and would like to hear your comments. So:

each OS (win, linux, macOS) handles thumbnails differently so I presume that we would need to keep three different thumbnail handlers (or whatever they're called), for each OS
I would then create a separate ThumbExtract utility that would actually extract thumbnail from .blend file. This utility could then be called from different thumbnail handlers to provide thumbnail image from .blend file

Would that be acceptable solution? What do you think?

Kind regard,
Ante

Hi guys, Last couple of days I've been working on this task, mainly investigating how thumbnailer handles are working on different operating systems and what the BlendThumb.cpp actually does. Now I've come up with the idea how to approach with the solution and would like to hear your comments. So: - each OS (win, linux, macOS) handles thumbnails differently so I presume that we would need to keep three different thumbnail handlers (or whatever they're called), for each OS - I would then create a separate ThumbExtract utility that would actually extract thumbnail from .blend file. This utility could then be called from different thumbnail handlers to provide thumbnail image from .blend file Would that be acceptable solution? What do you think? Kind regard, Ante

Campbell Barton commented

2019-10-14 05:16:44 +02:00

Hi, this seems reasonable.

I'd suggest to first split out thumbnail extraction into a separate, source file which exposes a single function.

Then this can be called from the current windows DLL.

After that, the Python script can be made into a single command line utility (in C/C++) that also calls this function.

We could drop the GFileWrapper functionality of the Python script initially, although later on it might be nice to support it again.

Hi, this seems reasonable. I'd suggest to first split out thumbnail extraction into a separate, source file which exposes a single function. Then this can be called from the current windows DLL. After that, the Python script can be made into a single command line utility (in C/C++) that also calls this function. We could drop the `GFileWrapper` functionality of the Python script initially, although later on it might be nice to support it again.

Ante Lausic commented

2019-10-14 14:27:23 +02:00

Hi,

would creating static lib be ok for this case? I should probably use only standard C/C++ functions for file manipulation in order to write portable code, right?

Hi, would creating static lib be ok for this case? I should probably use only standard C/C++ functions for file manipulation in order to write portable code, right?

Campbell Barton commented

2019-10-15 06:51:37 +02:00

Yes, this can be a function in a single source file which can be included in the DLL and command-line utility.

Ante Lausic commented

2019-10-21 14:46:24 +02:00

Hi,

after further investigation I’ve come to the point where I need guidance/decision.

Windows implements Thumbnail handlers as a COM servers. In order to work correctly we need to implement GetThumbnail method of the IThumbnailProvider interface. Also, we need to implement Initialize method of one of the initialization interfaces. And here is a problem: Microsoft recommends using IInitializeWithStream interface because it is more secure and reliable. But that when working with that interface Window Shell calls it with IStream object which means that we get stream and we don’t know path and/or filename. We could use IIntializeWithFile interface which would expose full path of the file, but that approach is not recommended. Also, it is said to be very uncommon.

So, what should we do? If we stick with IIntializeWithStream implementation we need to create at least two different Thumbnail extraction functions (one for Windows which works with IStream, and another for other platforms which works with file handler). If we implement IIntializeWithFile we could have just one function but I don’t know whether that would cause any problems since this wouldn’t be recommended approach (although I haven’t find any references to problems with such approach).

Hi, after further investigation I’ve come to the point where I need guidance/decision. Windows implements Thumbnail handlers as a COM servers. In order to work correctly we need to implement GetThumbnail method of the IThumbnailProvider interface. Also, we need to implement Initialize method of one of the initialization interfaces. And here is a problem: Microsoft recommends using IInitializeWithStream interface because it is more secure and reliable. But that when working with that interface Window Shell calls it with IStream object which means that we get stream and we don’t know path and/or filename. We could use IIntializeWithFile interface which would expose full path of the file, but that approach is not recommended. Also, it is said to be very uncommon. So, what should we do? If we stick with IIntializeWithStream implementation we need to create at least two different Thumbnail extraction functions (one for Windows which works with IStream, and another for other platforms which works with file handler). If we implement IIntializeWithFile we could have just one function but I don’t know whether that would cause any problems since this wouldn’t be recommended approach (although I haven’t find any references to problems with such approach).

Ante Lausic commented

2019-11-07 12:14:01 +01:00

Just to document my findings.

It seems that although documentation states differently only IInitializeWithStream is working on Windows (please check https://stackoverflow.com/questions/57168578/can-i-initialize-a-ithumbnailprovider-object-with-a-file).

This actually means that in our Thumbnail Handler for Windows we don't know the file and path because we just get IStream object with data in it. Further that means that for Windows we need to implement extraction function that uses native Windows API and IStream object, and for other systems we need to implement extraction function using standard C/C++.

If you agree I'll proceed with that approach.

Regards,
Ante

Just to document my findings. It seems that although documentation states differently only IInitializeWithStream is working on Windows (please check https://stackoverflow.com/questions/57168578/can-i-initialize-a-ithumbnailprovider-object-with-a-file). This actually means that in our Thumbnail Handler for Windows we don't know the file and path because we just get IStream object with data in it. Further that means that for Windows we need to implement extraction function that uses native Windows API and IStream object, and for other systems we need to implement extraction function using standard C/C++. If you agree I'll proceed with that approach. Regards, Ante

Campbell Barton commented

2019-11-07 20:31:21 +01:00

In this case we could abstract away the file object entirely, we only need to pass in a struct with a read callback.

See uses of FileDataReadFn & FileDataSeekFn which do this in in readfile.c.

In this case we could abstract away the file object entirely, we only need to pass in a struct with a read callback. See uses of `FileDataReadFn` & `FileDataSeekFn` which do this in in `readfile.c`.

Ante Lausic commented

2019-12-13 14:29:22 +01:00

Diff was uploaded for review:

https://developer.blender.org/D6408

Diff was uploaded for review: https://developer.blender.org/D6408

Campbell Barton commented

2021-10-01 07:05:37 +02:00

Added subscriber: @dfelinto