Library upgrades for 2.90 #76184

Closed
opened 2020-04-28 14:10:11 +02:00 by Brecht Van Lommel · 32 comments

Still up for discussion

dep Current Latest Notes suggestion decision
MESA 18.3.1 20.0.6 Keep Current Version

Marked for update

dep Current Latest DIFF/TASK
PYTHON 3.7.4 3.7.7 #77846
IDNA 2.8 2.9 #77846
URLLIB3 1.25.3 1.25.9 #77846
CERTIFI 2019.6.16 2020.4.5.1 #77846
REQUESTS 2.22.0 2.23.0 #77846
NUMPY v1.17.0 1.17.5 #77846
OIDN 1.0.0 1.2.0 D7641
OPENAL 1.18.2 1.20.1 #77005
PNG 1.6.35 1.6.37 D7727
JPEG 1.5.3 2.0.4 D7727
OPENCOLORIO 1.1.0 1.1.1 D7727
OPENIMAGEIO 2.1.15.0 2.1.15.0 D7727
TIFF 4.0.9 4.1.0 D7727
OSL 1.10.9 1.10.10 D7727
OPENJPEG 2.3.0 2.3.1 D7727
PUGIXML 1.9 1.10 D7727
EMBREE 3.8.0 3.10.0 D7623
FREETYPE 2.10.1 2.10.2 #77006
OPENSUBDIV v3_4_0_RC2 3.4.3 #77007
SDL 2.0.8 2.0.12 #77010
OGG 1.3.3 1.3.4 #77035
FLAC 1.3.2 1.3.3 #77035
VPX 1.7.0 1.8.2 #77035
XVIDCORE 1.3.5 1.3.7 #77035
FFMPEG 4.0.2 4.2.3 #77035
X264 some githash most recent githash #77035
ICONV 1.15 1.16 #77233
JEMALLOC 5.0.1 5.2.1 #78252
XML2 2.9.4 2.9.10 #78252
BZIP2 1.0.6 1.0.8 #78252
FFI 3.2.1 3.3 #78252
LZMA 5.2.4 5.2.5 #78252
SSL 1.1.0i 1.1.1g #78252
SQLITE 3.24.0 3.31.1 #78252
USD 19.11 20.05 #77806
XR_OPENXR_SDK 1.0.6 1.0.8 #77011
OpenImageDenoise 1.0.0 1.2.1 D7641

Marked for removal

dep Current Latest reason DIFF
HDF5 1.8.17 1.12.0 not been tested in ages 0c38436227 and 0102b9d47e
LAPACK 3.6.0 3.9.0 not used anymore d69bb06
FAAD 2-2.8.8 2-2.8.8 Only used by ffmpeg, which dropped support ages ago 5db2d9c
HIDAPI 0.8.0-rc1 ? was only used by the openhmd branch d69bb06

Deps not updated

dep Current Latest reason
ZLIB_VERSION 1.2.11 1.2.11 no new version
BOOST 1.70.0 1.72.0 vfx refrerence prevents update
BLOSC 1.5.0 1.81.1 1.5.0 is mandated by openvdb
PTHREADS 3.0.0 3.0.0 no new version
OPENEXR 2.4.0 2.5.1 vfx refrerence prevents update
FREEGLUT 3.0.0 3.2.1 used for cycles standalone only
ALEMBIC 1.7.12 1.7.12 no new version
OPENCOLLADA v1.6.68 1.6.88 no new version
TBB 2019_U9 2020_U2 vfx refrerence prevents update
OPENVDB 7.0.0 7.0.0 no new version
SNDFILE 1.0.28 1.0.28 no new version
FLEXBISON 2.5.5 2.5.18 build dep for osl only,works fine as is
LAME 3.1 3.1 no new version
VORBIS 1.3.6 1.3.6 no new version
THEORA 1.1.1 1.1.1 no new version
OPUS 1.3.1 1.3.1 no new version
FFTW 3.3.8 3.3.8 no new version
SPNAV 0.2.3 0.2.3 no new version
LCMS 2.9 2.9 no new version
TINYXML 2_6_2 2.6.2 no new version
YAMLCPP 0.6.3 0.6.3 no new version
GLEW 1.13.0 2.1.0 only used for cycles standalone
CHARDET 3.0.4 3.0.4 no new version
LIBGLU 9.0.1 9.0.1 no new version
LLVM 9.0.1 10.0.0 large library, no clear reason to upgrade
WEBP 0.6.1 1.0.3 still optional
**Still up for discussion** |dep|Current|Latest|Notes|suggestion|decision| | -- | -- | -- | -- | -- | -- | |MESA|18.3.1|20.0.6||Keep Current Version| | **Marked for update** |dep|Current|Latest|DIFF/TASK | -- | -- | -- | -- | |PYTHON|3.7.4|3.7.7|#77846| |IDNA|2.8|2.9|#77846| |URLLIB3|1.25.3|1.25.9|#77846| |CERTIFI|2019.6.16|2020.4.5.1|#77846| |REQUESTS|2.22.0|2.23.0|#77846| |NUMPY|v1.17.0|1.17.5|#77846| |OIDN|1.0.0|1.2.0|[D7641](https://archive.blender.org/developer/D7641) |OPENAL|1.18.2|1.20.1| #77005| |PNG|1.6.35|1.6.37|[D7727](https://archive.blender.org/developer/D7727) |JPEG|1.5.3|2.0.4|[D7727](https://archive.blender.org/developer/D7727) |OPENCOLORIO|1.1.0|1.1.1|[D7727](https://archive.blender.org/developer/D7727) |OPENIMAGEIO|2.1.15.0|2.1.15.0|[D7727](https://archive.blender.org/developer/D7727) |TIFF|4.0.9|4.1.0|[D7727](https://archive.blender.org/developer/D7727) |OSL|1.10.9|1.10.10|[D7727](https://archive.blender.org/developer/D7727) |OPENJPEG|2.3.0|2.3.1|[D7727](https://archive.blender.org/developer/D7727) |PUGIXML|1.9|1.10|[D7727](https://archive.blender.org/developer/D7727) |EMBREE|3.8.0|3.10.0|[D7623](https://archive.blender.org/developer/D7623) |FREETYPE|2.10.1|2.10.2|#77006 |OPENSUBDIV|v3_4_0_RC2|3.4.3|#77007 |SDL|2.0.8|2.0.12|#77010 |OGG|1.3.3|1.3.4|#77035 |FLAC|1.3.2|1.3.3|#77035 |VPX|1.7.0|1.8.2|#77035 |XVIDCORE|1.3.5|1.3.7|#77035 |FFMPEG|4.0.2|4.2.3|#77035 |X264|some githash|most recent githash|#77035 |ICONV|1.15|1.16|#77233| |JEMALLOC|5.0.1|5.2.1|#78252| |XML2|2.9.4|2.9.10|#78252| |BZIP2|1.0.6|1.0.8|#78252| |FFI|3.2.1|3.3|#78252| |LZMA|5.2.4|5.2.5|#78252| |SSL|1.1.0i|1.1.1g|#78252| |SQLITE|3.24.0|3.31.1|#78252| |USD|19.11|20.05| #77806| |XR_OPENXR_SDK|1.0.6|1.0.8|#77011| |OpenImageDenoise|1.0.0|1.2.1|[D7641](https://archive.blender.org/developer/D7641)| **Marked for removal** |dep|Current|Latest|reason|DIFF| | -- | -- | -- | -- | -- | |HDF5|1.8.17|1.12.0|not been tested in ages|0c38436227 and 0102b9d47e | |LAPACK|3.6.0|3.9.0|not used anymore|d69bb06| |FAAD|2-2.8.8|2-2.8.8|Only used by ffmpeg, which dropped support ages ago|5db2d9c| |HIDAPI|0.8.0-rc1|?|was only used by the openhmd branch|d69bb06| **Deps not updated** |dep|Current|Latest|reason | -- | -- | -- | -- | |ZLIB_VERSION|1.2.11|1.2.11|no new version |BOOST|1.70.0|1.72.0|vfx refrerence prevents update |BLOSC|1.5.0|1.81.1|1.5.0 is mandated by openvdb |PTHREADS|3.0.0|3.0.0|no new version |OPENEXR|2.4.0|2.5.1|vfx refrerence prevents update |FREEGLUT|3.0.0|3.2.1|used for cycles standalone only |ALEMBIC|1.7.12|1.7.12|no new version |OPENCOLLADA|v1.6.68|1.6.88|no new version |TBB|2019_U9|2020_U2|vfx refrerence prevents update |OPENVDB|7.0.0|7.0.0|no new version |SNDFILE|1.0.28|1.0.28|no new version |FLEXBISON|2.5.5|2.5.18|build dep for osl only,works fine as is |LAME|3.1|3.1|no new version |VORBIS|1.3.6|1.3.6|no new version |THEORA|1.1.1|1.1.1|no new version |OPUS|1.3.1|1.3.1|no new version |FFTW|3.3.8|3.3.8|no new version |SPNAV|0.2.3|0.2.3|no new version |LCMS|2.9|2.9|no new version |TINYXML|2_6_2|2.6.2|no new version |YAMLCPP|0.6.3|0.6.3|no new version |GLEW|1.13.0|2.1.0|only used for cycles standalone |CHARDET|3.0.4|3.0.4|no new version | |LIBGLU|9.0.1|9.0.1|no new version| |LLVM|9.0.1|10.0.0|large library, no clear reason to upgrade| |WEBP|0.6.1|1.0.3|still optional|
Author
Owner

Changed status from 'Needs Triage' to: 'Confirmed'

Changed status from 'Needs Triage' to: 'Confirmed'
Author
Owner

Added subscribers: @brecht, @LazyDodo, @Sergey

Added subscribers: @brecht, @LazyDodo, @Sergey
Author
Owner

There's no urgency to this, just creating the task now for the 2.90 workboard.

There's no urgency to this, just creating the task now for the 2.90 workboard.
Member

OIIO is somewhat of a problematic one, it detects a bunch of stuff on its own and decides from that what to build with, some of the cmake variables have changed in recent versions so things may gone missing, we should make a unit test to validate that the formats we expect are actually in there similar to the ffmpeg unit test.

OIIO is somewhat of a problematic one, it detects a bunch of stuff on its own and decides from that what to build with, some of the cmake variables have changed in recent versions so things may gone missing, we should make a unit test to validate that the formats we expect are actually in there similar to the ffmpeg unit test.

@brecht . are the corresponding changes done to versions.cmake? (as in, is it enough to simply git pull, make deps and call it done)?

I'm also confused with "Maybe". Do or do not I do it?

@brecht . are the corresponding changes done to `versions.cmake`? (as in, is it enough to simply `git pull`, `make deps` and call it done)? I'm also confused with "Maybe". Do or do not I do it?
Author
Owner

Nothing is done yet.

Nothing is done yet.

Added subscriber: @zebus3dream

Added subscriber: @zebus3dream
Member

Added subscriber: @Stefan_Werner

Added subscriber: @Stefan_Werner
Member

I'd like to upgrade to Embree 3.9.0 as well, since it supports thick line segments for hair. I can provide a patch. If we're now settled on TBB being a requirement, it might also be worth it to turn on TBB support in Embree.

I'd like to upgrade to Embree 3.9.0 as well, since it supports thick line segments for hair. I can provide a patch. If we're now settled on TBB being a requirement, it might also be worth it to turn on TBB support in Embree.
Author
Owner

@Stefan_Werner upgrading Embree and enabling TBB seems fine.

@Stefan_Werner upgrading Embree and enabling TBB seems fine.
Member

This comment was removed by @LazyDodo

*This comment was removed by @LazyDodo*
Author
Owner

For upgrading OpenImageIO and other libraries, here's a patch from a few months ago that works towards this: P1368.

For upgrading OpenImageIO and other libraries, here's a patch from a few months ago that works towards this: [P1368](https://archive.blender.org/developer/P1368.txt).
Member

Just so we don't duplicate work, i'm currently working on oidn 1.2

Just so we don't duplicate work, i'm currently working on oidn 1.2

Added subscriber: @AndreaMonzini

Added subscriber: @AndreaMonzini
Member

Working on oiio

Working on oiio
Member

Added subscribers: @JulianEisel, @dr.sybren

Added subscribers: @JulianEisel, @dr.sybren
Member

i took a quick survey of all our deps, made some recommendations , the deps without notes i really have no opinion on, there's some module specific deps where the owners will have to decide,

@dr.sybren for USD
@JulianEisel for openXR
@Sergey for opensubdiv
@Stefan_Werner for embree, you want to stick to 3.9? 3.10 is available

i took a quick survey of all our deps, made some recommendations , the deps without notes i really have no opinion on, there's some module specific deps where the owners will have to decide, @dr.sybren for USD @JulianEisel for openXR @Sergey for opensubdiv @Stefan_Werner for embree, you want to stick to 3.9? 3.10 is available
Member

I checked the OpenXR SDK releases. There's nothing important or interesting for us really, so no need to update that.

Also checked the FreeType change-log for 2.10.2, but there's nothing interesting for us there either. WOFF2 support got added, I don't see a need for that though.

I checked the OpenXR SDK releases. There's nothing important or interesting for us really, so no need to update that. Also checked the [FreeType change-log ](https://sourceforge.net/projects/freetype/files/freetype2/2.10.2/) for 2.10.2, but there's nothing interesting for us there either. WOFF2 support got added, I don't see a need for that though.

While OpenSubdiv does not really bring any user-measurable difference I still suggest updating it. Reasoning:

  1. Is bringing us from RC to actual release
  2. There are fixes for MSVC 2019
While OpenSubdiv does not really bring any user-measurable difference I still suggest updating it. Reasoning: 1. Is bringing us from RC to actual release 2. There are fixes for MSVC 2019
Author
Owner

As for overall coordination here. I think it would be most convenient to review and commit the code for the library upgrades in parts, and then when it's all in place let the platform maintainers update the precompiled libraries all at once. At least as macOS maintainer I'd like to avoid doing this multiple times.

Regarding the proposed library upgrades. I'm ok with updated all the ones that are proposed to be updated, and removing all the ones that are proposed to be removed.

I wouldn't mind updating more libraries just for the sake of not missing out on any potential security fixes or other bug fixes. If they are easy and just a matter of bumping the version and hash in versions.cmake. If it's more complicated, I don't think it's important.

As for overall coordination here. I think it would be most convenient to review and commit the code for the library upgrades in parts, and then when it's all in place let the platform maintainers update the precompiled libraries all at once. At least as macOS maintainer I'd like to avoid doing this multiple times. Regarding the proposed library upgrades. I'm ok with updated all the ones that are proposed to be updated, and removing all the ones that are proposed to be removed. I wouldn't mind updating more libraries just for the sake of not missing out on any potential security fixes or other bug fixes. If they are easy and just a matter of bumping the version and hash in `versions.cmake`. If it's more complicated, I don't think it's important.

+1 on the USD upgrade; I've updated the task description for this.

+1 on the USD upgrade; I've updated the task description for this.

I wouldn't mind having WebP support upgraded and integrated with Blender as supported image format. It's a nice image format, and rather widely used, and a patch has been waiting for 5 years already (D1598).

I wouldn't mind having WebP support upgraded and integrated with Blender as supported image format. It's a nice image format, and rather widely used, and a patch has been waiting for 5 years already ([D1598](https://archive.blender.org/developer/D1598)).
Member

I've pretty much picked off the easy bits, the ones still open are linux only, and USD which needs changes to the integration code, @dr.sybren mind taking this one on linux? once the build issues in blender are taken care of i'll do the windows libs.

I've pretty much picked off the easy bits, the ones still open are linux only, and USD which needs changes to the integration code, @dr.sybren mind taking this one on linux? once the build issues in blender are taken care of i'll do the windows libs.

I would like to see Python upgraded to 3.7.7, as there have been a bunch of security fixes.

As far as the other Python libraries go, they seem to be Requests + its dependencies. Keeping those updated is also a good idea, especially because that also impacts which TLS certificates are considered trustworthy.

I would like to see Python upgraded to 3.7.7, as there have been a bunch of security fixes. As far as the other Python libraries go, they seem to be Requests + its dependencies. Keeping those updated is also a good idea, especially because that also impacts which TLS certificates are considered trustworthy.

This issue was referenced by d69bb06db8

This issue was referenced by d69bb06db857553dc2bf1cad58aa3c742acfe1e8
Author
Owner

I believe we are now ready with the CMake builder updates for 2.90, and that we can do the precompiled library updates now.

On Windows most precompiled libraries are already upgraded. I believe still remaining is embree (with TBB), openimagedenoise, and removing hidapi.

I believe we are now ready with the CMake builder updates for 2.90, and that we can do the precompiled library updates now. On Windows most precompiled libraries are already upgraded. I believe still remaining is embree (with TBB), openimagedenoise, and removing hidapi.
Member

embree is done, tbb got added when i took out the erroneous exports, oidn still needs to be done indeed.

embree is done, tbb got added when i took out the erroneous exports, oidn still needs to be done indeed.
Member

Added subscriber: @EAW

Added subscriber: @EAW

This issue was referenced by ff444da7c4

This issue was referenced by ff444da7c45403ac0d112042c9d1d0a1992168d4
Author
Owner

As far as the official release is concerned this is now done. There are still two sets of libraries that have not been updated yet for install deps:

As far as the official release is concerned this is now done. There are still two sets of libraries that have not been updated yet for install deps: * [D7727: Deps: Update OIIO + ImageLibs + OSL + Helpers](https://archive.blender.org/developer/D7727) * [D7641: Update OIDN to 1.2.1](https://archive.blender.org/developer/D7641)
Author
Owner

Changed status from 'Confirmed' to: 'Resolved'

Changed status from 'Confirmed' to: 'Resolved'
Brecht Van Lommel self-assigned this 2020-08-18 18:29:52 +02:00
Author
Owner

This should be all done now.

This should be all done now.
Thomas Dinges added this to the 2.90 milestone 2023-02-08 16:27:21 +01:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
10 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#76184
No description provided.