Details
- Tag
- v2.4.1
Shaman is a file storage server. It accepts uploaded files via HTTP, and stores them based on their SHA256-sum and their file length. It can recreate directory structures by symlinking those files. Shaman is intended to complement Blender Asset Tracer (BAT) and Flamenco, but can be used as a standalone component.
The overall use looks like this:
After this process, the checkout directory contains symlinks to all the files in the Checkout Definition File. The user only had to upload new and changed files.
The Shaman file store is structured as follows:
shaman-store/ .. uploading/ .. /{checksum[0:2]}/{checksum[2:]}/{filesize}-{unique-suffix}.tmp .. stored/ .. /{checksum[0:2]}/{checksum[2:]}/{filesize}.blob
When a file is uploaded, it goes through several stages:
To prevent infinite growth of the File Store, the Shaman will periodically perform a garbage collection sweep. Garbage Collection can be configured by setting the following settings in shaman.yaml:
Every time a file is symlinked into a checkout directory, it is 'touched' (that is, its modification time is set to 'now').
Files that are not referenced in any checkout, and that have a modification time that is older than garbageCollectMaxAge will be deleted.
To perform a dry run of the garbage collector, use shaman -gc.
SHAman uses JWT with ES256 signatures. The public keys of the JWT-signing authority need to be known, and stored in jwtkeys/*-public*.pem. For more info, see jwtkeys/README.md
_py_client: An example client in Python. Just hacked together as a proof of
concept and by no means of any official status.