Starting with CY2020, the VFX Reference Platform uses Python 3.7. This gives us an opportunity to align with it. We should decide if this is something we want to do, and what that means exactly.
The goal would be to align all releases in 2020 to the CY2020 platform, which in our cases would be for Blender 2.82. If we upgrade to a too new version of a library in 2.81 we can't downgrade easily though.
The VFX reference platform is currently for Linux only. It is planned to be extended to Windows and macOS in CY2021.
Being compatible with the reference platform does not mean we have to use all the exact compiler or library versions listed. What matters is effective API/ABI compatibility, and file format compatibility. There are a few different considerations.
GCC and glibc
We are already compatible with the required glibc version 2.17.
We can use newer GCC or Clang versions that produce more optimized code, as long as they are ABI compatible with the stated GCC version.
We currently do not have a public C/C++ API, and are also not exposing any of the external libraries as an API.
A change we could make is to hide symbols by default. We currently hide some symbols that are likely to conflict, but not all. This is already automatically done on Windows, but not on Linux or macOS. Plug-ins can generally handle this on their side too, but we may as well make it easier to not conflict.
To guarantee full compatibility, we'd have to use Python 3.7 and Numpy 1.17. Python scripts written for 3.7 should generally work fine in 3.8, but not the other way around. For extensions there is a stable ABI, but it's not clear this is widely used and so extensions would be incompatible.
The downside is that we'd have to wait a year use Python 3.8 after its release. As the Python ecosystem moves to 3.8, we then deviate from the rest of the Python community that are not following the VFX reference platform.
For example on Windows the default version to download from the Python website in 2020 would be 3.8. But this means it would not be compatible with Blender or give PYTHONPATH problems.
File Format Compatibility
Libraries like OpenEXR, OpenVDB and OpenColorIO read and write files. We want those versions to be new enough that we can read files written by other software. Further, using a newer version can give files that other applications can't read.
Note for example that the OpenVDB file format remained compatible across some major versions. In such cases we can use a newer library version, if it contains functionality that we need.
On the other hand OpenColorIO 2 is breaking compatibility. It contains improvements that are quite useful to us, but is only planned for CY2021.