Diffusion Flamenco Manager (master) History

flamenco-manager/flamenco
mastereb59c020ddf2783e5b001aa…

Feb 21 2019

Worker cleanup: Requeue active tasks before deleting worker
eb59c020ddf2
·Sybren A. Stüvel (sybren) on Feb 21 2019, 1:45 PM.
Worker cleanup: configurable set of statuses to auto-remove
c52c65d2b48b
·Sybren A. Stüvel (sybren) on Feb 21 2019, 12:11 PM.
Automatically delete offline workers
7bbbe3c0c5ca
·Sybren A. Stüvel (sybren) on Feb 21 2019, 12:11 PM.

Feb 20 2019

Log server: human-readable sizes in 'Skipped ... bytes' message
7db66cb69d18
·Sybren A. Stüvel (sybren) on Feb 20 2019, 9:47 AM.

Feb 19 2019

When serving log file, conditionally only show head + tail of the log
ea917be44c80
·Sybren A. Stüvel (sybren) on Feb 19 2019, 6:44 PM.
Limit number of workers that can retry a task after it failed
5f9bc2e2d452
·Sybren A. Stüvel (sybren) on Feb 19 2019, 4:14 PM.
Allow soft-failed tasks to be run by other workers
e47780a63378
·Sybren A. Stüvel (sybren) on Feb 19 2019, 3:38 PM.
TaskUpdateQueue::QueueTaskUpdateWithExtra now expects outer update dict
8f827a2eb22a
·Sybren A. Stüvel (sybren) on Feb 19 2019, 3:38 PM.
Typo fix
9639e1cb476d
·Sybren A. Stüvel (sybren) on Feb 19 2019, 3:38 PM.

Feb 15 2019

When a worker times out, its active task is now re-queued
2dc054b23fe3
·Sybren A. Stüvel (sybren) on Feb 15 2019, 5:26 PM.

Feb 14 2019

Soft-fail tasks when there are workers left to retry it
e9c67553a372
·Sybren A. Stüvel (sybren) on Feb 14 2019, 2:32 PM.
More detailed testing of task updates when blacklisting
d62e23d6f89c
·Sybren A. Stüvel (sybren) on Feb 14 2019, 11:52 AM.

Feb 12 2019

Fix T59491: Manager should detect starvation due to blacklisting
72c46706ea80
·Sybren A. Stüvel (sybren) on Feb 12 2019, 3:05 PM.

Jan 11 2019

Send "this log file does not exist" as log file when it doesn't exist.
0f9fb203b4f2
·Sybren A. Stüvel (sybren) on Jan 11 2019, 6:20 PM.
Added missing return statement
cfb5cc825dfa
·Sybren A. Stüvel (sybren) on Jan 11 2019, 5:48 PM.

Jan 9 2019

Upload task log files when requested from the Flamenco Server
5735bbee2e16
·Sybren A. Stüvel (sybren) on Jan 9 2019, 5:00 PM.
Less strict timeout checks
65c74bc3035d
·Sybren A. Stüvel (sybren) on Jan 9 2019, 5:00 PM.
Don't pass pointer to array
135f195d9ce9
·Sybren A. Stüvel (sybren) on Jan 9 2019, 2:20 PM.

Dec 18 2018

Include ffmpeg variable in default settings
e85a902fb7f5
·Sybren A. Stüvel (sybren) on Dec 18 2018, 2:35 PM.
Fix T58779: allow lazy status change requests
cfe561c79e75
·Sybren A. Stüvel (sybren) on Dec 18 2018, 10:51 AM.

Dec 17 2018

Compatibility with older MongoDB
e64ffe098dbe
·Sybren A. Stüvel (sybren) on Dec 17 2018, 5:18 PM.
Dashboard: make worker blacklist visible
f15e445baa65
·Sybren A. Stüvel (sybren) on Dec 17 2018, 5:08 PM.
Fix T50981 Worker deallocation from job if fails n tasks
a6e5900f0982
·Sybren A. Stüvel (sybren) on Dec 17 2018, 2:28 PM.
Fixed bug in sleep scheduler
1cf3c1971f49
·Sybren A. Stüvel (sybren) on Dec 17 2018, 2:26 PM.

Dec 14 2018

Sleep scheduler: only log at debug level when there is nothing to do
e9d6cb017fc6
·Sybren A. Stüvel (sybren) on Dec 14 2018, 4:39 PM.
Schedule is now event-based and can be temporarily overridden
dce92f90d3db
·Sybren A. Stüvel (sybren) on Dec 14 2018, 11:32 AM.
Log Server URL at info level at startup
cb933e473aba
·Sybren A. Stüvel (sybren) on Dec 14 2018, 11:21 AM.
Less verbose logging while testing
b7b8cab39074
·Sybren A. Stüvel (sybren) on Dec 14 2018, 11:21 AM.

Dec 13 2018

Dashboard: added link to task on Flamenco Server
702a773b395d
·Sybren A. Stüvel (sybren) on Dec 13 2018, 6:14 PM.
Deactivate sleep schedule when explicitly changing the worker state
24d108b78ebd
·Sybren A. Stüvel (sybren) on Dec 13 2018, 5:04 PM.
Sleep Schedule implementation
7680c65daf35
·Sybren A. Stüvel (sybren) on Dec 13 2018, 4:49 PM.
Fixed compiler warnings
5fab85b3a765
·Sybren A. Stüvel (sybren) on Dec 13 2018, 4:38 PM.
Renamed 'reporter' to 'dashboard'
409675ed38a1
·Sybren A. Stüvel (sybren) on Dec 13 2018, 4:38 PM.
Added missing comment
fb376b1657ba
·Sybren A. Stüvel (sybren) on Dec 13 2018, 10:48 AM.

Dec 12 2018

Big modernisation steps (bootstrap 4 + Vue.js):
98a4e2ecf12e
·Sybren A. Stüvel (sybren) on Dec 12 2018, 12:10 PM.
Added config for run mode (develop/production)
704e0a08c8c4
·Sybren A. Stüvel (sybren) on Dec 12 2018, 11:39 AM.

Dec 4 2018

Use "active" → statusActive
ce56ce4b242b
·Sybren A. Stüvel (sybren) on Dec 4 2018, 4:09 PM.
Refuse to let the Worker set a task status to 'active'
980b609a79d2
·Sybren A. Stüvel (sybren) on Dec 4 2018, 4:09 PM.
Worker sign-off: only queue 'asleep' status when no other status requested
6a58e461fe75
·Sybren A. Stüvel (sybren) on Dec 4 2018, 4:09 PM.

Nov 30 2018

Allow task log download from the dashboard
0b001fc31245
·Sybren A. Stüvel (sybren) on Nov 30 2018, 5:29 PM.
Only send the last 10 lines of log in `log_tail`, not in `log`
eb6cfe311d8b
·Sybren A. Stüvel (sybren) on Nov 30 2018, 5:16 PM.
HTTP endpoint for getting task logs at /logfile/{job-id}/{task-id}
2b22fe3744c9
·Sybren A. Stüvel (sybren) on Nov 30 2018, 5:09 PM.
Rotate log files when tasks become active
61fc041b043d
·Sybren A. Stüvel (sybren) on Nov 30 2018, 4:15 PM.
Added unit test for log rotation (no implementation yet)
dba87d034458
·Sybren A. Stüvel (sybren) on Nov 30 2018, 12:42 PM.
Deduplicated some test code into a single function
001bc481a1fc
·Sybren A. Stüvel (sybren) on Nov 30 2018, 12:42 PM.
"statuses" → "task statuses"
3ad8692a11c0
·Sybren A. Stüvel (sybren) on Nov 30 2018, 11:27 AM.
Added some tests for newline handling
f8aa55a9b898
·Sybren A. Stüvel (sybren) on Nov 30 2018, 10:48 AM.

Nov 29 2018

Write task logs to local files instead of sending them all to the Server
f809c994ec92
·Sybren A. Stüvel (sybren) on Nov 29 2018, 6:37 PM.
Moved task queue functionality into a TaskUpdateQueue struct
e9824f5469a9
·Sybren A. Stüvel (sybren) on Nov 29 2018, 6:37 PM.
Some extra documentation
e676b0eaa123
·Sybren A. Stüvel (sybren) on Nov 29 2018, 6:26 PM.
Allow Workers to return tasks to the queue
2c38437a4bcd
·Sybren A. Stüvel (sybren) on Nov 29 2018, 11:21 AM.
Add HTTP Content-Type header + better code coverage
317373e25774
·Sybren A. Stüvel (sybren) on Nov 29 2018, 10:30 AM.

Nov 28 2018

Moved task-requeueing to TaskScheduler
6a1434b7b36d
·Sybren A. Stüvel (sybren) on Nov 28 2018, 5:16 PM.
Don't queue address-of-pointer, just use the pointer
448cc9287080
·Sybren A. Stüvel (sybren) on Nov 28 2018, 5:15 PM.

Nov 23 2018

Include example configuration for the ffmpeg variable + mention in README
2325613821b5
·Sybren A. Stüvel (sybren) on Nov 23 2018, 3:45 PM.

Nov 22 2018

Send the set of task types supported by our workers to Flamenco Server
b6ca4cfb3837
·Sybren A. Stüvel (sybren) on Nov 22 2018, 1:38 PM.
startup_notification.go → upstream_notification.go
961b3f3ec467
·Sybren A. Stüvel (sybren) on Nov 22 2018, 1:38 PM.
Rename StartupNotifier → UpstreamNotifier
c6a27a4a5d5a
·Sybren A. Stüvel (sybren) on Nov 22 2018, 1:38 PM.
Reduced log level of timer shutdown message
fba4cedf2e40
·Sybren A. Stüvel (sybren) on Nov 22 2018, 1:38 PM.
Fixed warning about superfluous type declaration
ca9e0045a89d
·Sybren A. Stüvel (sybren) on Nov 22 2018, 1:38 PM.

Nov 16 2018

Remember old status when timing out
6a9b0cc03fd2
·Sybren A. Stüvel (sybren) on Nov 16 2018, 2:32 PM.

Nov 12 2018

Support worker status 'error'
e751154b0aac
·Sybren A. Stüvel (sybren) on Nov 12 2018, 12:39 PM.

Nov 9 2018

Moved WorkerMayRunTask function to task scheduler
51fe199abbb0
·Sybren A. Stüvel (sybren) on Nov 9 2018, 11:09 AM.
Reduce hashed password complexity
332517429e10
·Sybren A. Stüvel (sybren) on Nov 9 2018, 11:09 AM.
Removed TimeoutAfter utility function
edcf5726dcea
·Sybren A. Stüvel (sybren) on Nov 9 2018, 11:08 AM.

Sep 11 2018

Include upstream queue size on dashboard
9b9d397dd9ad
·Sybren A. Stüvel (sybren) on Sep 11 2018, 7:26 PM.
Log body as string instead of bytes
2c5413a29a98
·Sybren A. Stüvel (sybren) on Sep 11 2018, 7:11 PM.
Fixed: used wrong logger
9ad005a8b09a
·Sybren A. Stüvel (sybren) on Sep 11 2018, 6:41 PM.

Jun 14 2018

Accept log entries for tasks that are no longer runnable
80e639128099
·Sybren A. Stüvel (sybren) on Jun 14 2018, 10:48 AM.
Added test for WorkerPingedTask
ef246eceb4f9
·Sybren A. Stüvel (sybren) on Jun 14 2018, 10:21 AM.
Fixed nil pointer dereference panic
869866883908
·Sybren A. Stüvel (sybren) on Jun 14 2018, 10:17 AM.

Mar 26 2018

Log hex of ObjectID, not the ID object itself
bc69c9a22124
·Sybren A. Stüvel (sybren) on Mar 26 2018, 3:02 PM.

Feb 23 2018

Replaced Debugf() with Debug() when no formatting used
7df9e8820685
·Sybren A. Stüvel (sybren) on Feb 23 2018, 10:17 AM.

Feb 20 2018

Allow erasing idle workers from the dashboard
2ce108a8916e
·Sybren A. Stüvel (sybren) on Feb 20 2018, 2:54 PM.
Log body as string instead of byte[]
b7e832126504
·Sybren A. Stüvel (sybren) on Feb 20 2018, 12:00 PM.
Log a warning when backslashes are used in path replacement variables
f974d2422ab0
·Sybren A. Stüvel (sybren) on Feb 20 2018, 12:00 PM.
Added unit test for duplicate variables in config file
30f2244cd9eb
·Sybren A. Stüvel (sybren) on Feb 20 2018, 11:32 AM.
Add `job_storage` path replacement variable to defaults
127d62fb28f6
·Sybren A. Stüvel (sybren) on Feb 20 2018, 11:28 AM.

Feb 9 2018

Some logging improvements
da681e61bb88
·Sybren A. Stüvel (sybren) on Feb 9 2018, 11:37 AM.
Send immediate notification when browser connects to SSE events.
f09148640c1c
·Sybren A. Stüvel (sybren) on Feb 9 2018, 11:37 AM.
More logging tweaks
594c18ce1361
·Sybren A. Stüvel (sybren) on Feb 9 2018, 11:06 AM.
Using logging fields instead of formatting
e633047d9aa4
·Sybren A. Stüvel (sybren) on Feb 9 2018, 10:46 AM.

Dec 15 2017

Properly escape quotes in test task stamp note
72d466ccde25
·Sybren A. Stüvel (sybren) on Dec 15 2017, 4:54 PM.
Not finding template file is no longer fatal
94602abb15c5
·Sybren A. Stüvel (sybren) on Dec 15 2017, 4:54 PM.
Prevent sending updates on Manager-local tasks to upstream Server
4c0b73b13fa3
·Sybren A. Stüvel (sybren) on Dec 15 2017, 4:32 PM.
Test task: add Blender Render command
ff7bc8f5343b
·Sybren A. Stüvel (sybren) on Dec 15 2017, 3:37 PM.
Improved logging style of http.go
2d4237f9deda
·Sybren A. Stüvel (sybren) on Dec 15 2017, 3:36 PM.
Added creation of test tasks.
e1dbc7b3bfa3
·Sybren A. Stüvel (sybren) on Dec 15 2017, 3:24 PM.

Dec 14 2017

Added support for worker status 'testing'
e7193b6a54f0
·Sybren A. Stüvel (sybren) on Dec 14 2017, 2:57 PM.
Added missing error check
e7e9edd4c6b5
·Sybren A. Stüvel (sybren) on Dec 14 2017, 2:56 PM.

Oct 20 2017

Prevent endless shutdown loops.
d254c1c60439
·Sybren A. Stüvel (sybren) on Oct 20 2017, 3:12 PM.
Don't assume workers are awake when 'Seen' is called.
3c64a65ecbb3
·Sybren A. Stüvel (sybren) on Oct 20 2017, 2:51 PM.
Queue 'asleep' status change when an asleep worker signs off.
cc8a9289b572
·Sybren A. Stüvel (sybren) on Oct 20 2017, 2:41 PM.
Added ability to shut down worker and acknowledge timeouts
d23d8adfd1a6
·Sybren A. Stüvel (sybren) on Oct 20 2017, 1:01 PM.
Fixed logging mistakes
7049df178831
·Sybren A. Stüvel (sybren) on Oct 20 2017, 12:13 PM.

Oct 19 2017

Added ability to send workers to sleep (and wake them up again)
01860e221e46
·Sybren A. Stüvel (sybren) on Oct 19 2017, 6:55 PM.
Renamed worker status "down" to "offline"
e0882c937f83
·Sybren A. Stüvel (sybren) on Oct 19 2017, 2:41 PM.
Using constants for Worker status strings.
f2c0cd60fa99
·Sybren A. Stüvel (sybren) on Oct 19 2017, 2:39 PM.
Modernised logging in scheduler.go
a46c7e5a2fa7
·Sybren A. Stüvel (sybren) on Oct 19 2017, 1:25 PM.
Scale latest images down to max 2 megapixels (maintains aspect ratio).Scale…
4aa2d6b0b0f9
·Sybren A. Stüvel (sybren) on Oct 19 2017, 11:44 AM.

flamenco-manager/flamencomastereb59c020ddf2783e5b001aa…

Feb 21 2019

Feb 20 2019

Feb 19 2019

Feb 15 2019

Feb 14 2019

Feb 12 2019

Jan 11 2019

Jan 9 2019

Dec 18 2018

Dec 17 2018

Dec 14 2018

Dec 13 2018

Dec 12 2018

Dec 4 2018

Nov 30 2018

Nov 29 2018

Nov 28 2018

Nov 23 2018

Nov 22 2018

Nov 16 2018

Nov 12 2018

Nov 9 2018

Sep 11 2018

Jun 14 2018

Mar 26 2018

Feb 23 2018

Feb 20 2018

Feb 9 2018

Dec 15 2017

Dec 14 2017

Oct 20 2017

Oct 19 2017

flamenco-manager/flamenco
mastereb59c020ddf2783e5b001aa…