Job archival
Closed, ResolvedPublic


Job archival:

  • Sets job status "archiving"
  • For each task, de-chunks the task logs and gz-compresses them.
  • Creates a ZIP file with the job+task definitions in JSON and compressed logs.
  • Uploads the ZIP to the project's file storage.
  • Records the link of the ZIP in the job document.
  • Deletes the tasks and task logs in MongoDB.
  • Sets the job status to "archived".


To Do
Sybren A. Stüvel (sybren) triaged this task as Normal priority.Mar 21 2017, 3:34 PM
Sybren A. Stüvel (sybren) raised the priority of this task from Normal to High.May 24 2017, 11:05 AM
Sybren A. Stüvel (sybren) closed this task as Resolved.Jun 7 2017, 1:27 PM

Last bit was implemented in fdbdc0191507ffc826e71d2c4ad03d0adada05be, preceeding commits also relate to this.