Issues with cache files and file sequences #55007

New Issue

Isaac Weaver · 2018-05-08T19:06:37+02:00

Isaac Weaver commented

2018-05-08 19:06:37 +02:00

It looks like BAT doesn't handle fluid/smoke simulation caches (looking at modifier_walkers.py there aren't any walkers for those two types).

I've also had issues packing particles caches. I think the issue is in pack/__init__.py around line 219:

            if not asset_path.exists():
                log.warning('Missing file: %s', asset_path)
                self.missing_files.add(asset_path)
                self._progress_cb.missing_file(asset_path)
                continue

If the asset_path is actually a glob path (as it will be in particle caches), then it wouldn't exist so it would add it to the missing paths without continuing.

In the attached zip I have a .blend file with some backed out cache files, if you try to pack it, none of the caches will get packed.
cachetest.zip

It looks like BAT doesn't handle fluid/smoke simulation caches (looking at `modifier_walkers.py` there aren't any walkers for those two types). I've also had issues packing particles caches. I think the issue is in `pack/__init__.py` around line 219: ``` if not asset_path.exists(): log.warning('Missing file: %s', asset_path) self.missing_files.add(asset_path) self._progress_cb.missing_file(asset_path) continue ``` If the `asset_path` is actually a glob path (as it will be in particle caches), then it wouldn't exist so it would add it to the missing paths without continuing. In the attached zip I have a .blend file with some backed out cache files, if you try to pack it, none of the caches will get packed. [cachetest.zip](https://archive.blender.org/developer/F3588095/cachetest.zip)

Isaac Weaver commented

2018-05-08 19:06:37 +02:00

Added subscriber: @IsaacWeaver

Philipp Oeser commented

2018-05-09 11:54:41 +02:00

Added subscribers: @dr.sybren, @lichtwerk

Sybren A. Stüvel was assigned by Philipp Oeser

2018-05-09 11:54:41 +02:00

Philipp Oeser commented

2018-05-09 11:54:41 +02:00

Can confirm both issues here, @dr.sybren: mind having a look?

Having a quick look at the particle/glob case, it seems we could do something like [not working fully though -- usage of 'file_sequence' prevents this]:

            if not usage.is_sequence:
                if not asset_path.exists():
                    log.warning('Missing file: %s', asset_path)
                    self.missing_files.add(asset_path)
                    self._progress_cb.missing_file(asset_path)
                    continue
            else:
                try:
                    yield from file_sequence.expand_sequence(asset_path)
                except file_sequence.DoesNotExist:
                    log.warning('Missing sequence: %s', asset_path)
                    self.missing_files.add(asset_path)
                    self._progress_cb.missing_file(asset_path)
                    continue

I'm also not sure if a Usage path [in case of a sequence] should always be represented by a glob?
This seems like it could possibly lead to a lot of false positives? [e.g. 1-100 used in the file, but 1-1000 lying around on disk]?

Can confirm both issues here, @dr.sybren: mind having a look? Having a quick look at the particle/glob case, it seems we could do something like [not working fully though -- usage of 'file_sequence' prevents this]: ``` if not usage.is_sequence: if not asset_path.exists(): log.warning('Missing file: %s', asset_path) self.missing_files.add(asset_path) self._progress_cb.missing_file(asset_path) continue else: try: yield from file_sequence.expand_sequence(asset_path) except file_sequence.DoesNotExist: log.warning('Missing sequence: %s', asset_path) self.missing_files.add(asset_path) self._progress_cb.missing_file(asset_path) continue ``` I'm also not sure if a `Usage` path [in case of a sequence] should always be represented by a glob? This seems like it could possibly lead to a lot of false positives? [e.g. 1-100 used in the file, but 1-1000 lying around on disk]?

Sybren A. Stüvel commented

2018-05-29 12:29:46 +02:00

@IsaacWeaver can you update your bug report with one or more example files that demonstrate this issue? Just having to unzip a file and opening/bat-packing a blend file makes it much faster for me to fix, and also ensures that I'm looking at the same issue as you are.