Hi @Richard Antalik (ISS)

What are the requirements of such a library. I would expect that headless rendering is one of those.
Did you have a look at Halide? https://halide-lang.org/
They promote them as a language, but actually is more a framework.

BTW I am totally not aware what the future target of the VSE is, but this seems to be a large undertaking when you want to select something that is also for other areas (like compositor etc)

@Jeroen Bakker (jbakker)
Thanks for reminding me of halide - forgot about that one.

What are the requirements of such a library.

From VSE perspective it could be quite simple. Just basic image operations on raw data would be OK. Compositor nodes doesn't look that much different from what I would expect from such library. Question would be more about how to handle Blender specific controls, or splitting image to multiple components for example.
Advantage using existing universal library is, that it is already done, tested and maintained.

Using frameworks/CL we get more freedom to do case-specific optimizations, but we have to develop and maintain processing algorithms ourselves, but still we don't have to think too much about OS/HW.

I don't think it is really apropriate writing low level code such as CPU dispatchers in this case.

I am totally not aware what the future target of the VSE

Me too :)
My goal is to review current features, and make sure, that they work correctly and efficiently.

but this seems to be a large undertaking when you want to select something that is also for other areas (like compositor etc)

I am fully aware. That's why it's "long term project" :)
This task is rather idea placeholder, then design document. I don't expect to start working on this for next 2 releases at least.