Proposal to amend the default build configuration #42569

New Issue

Campbell Barton · 2014-11-11T17:34:12+01:00

Campbell Barton commented

2014-11-11 17:34:12 +01:00

Building on Linux has become fairly complicated, with specific dependencies.

In some cases this just means installing the 'right' version of a library (using install_deps.sh), but even then, there are complications.

Examples:

ffmpeg: http://blenderartists.org/forum/archive/index.php/t-229623.html
openimageio: http://blenderartists.org/forum/showthread.php?353402-Updating-and-building-blende-Fedora-Linux
openimageio: (reported on IRC) http://www.pasteall.org/55131
opencollada: https://developer.blender.org/T25772

Ideally we could detect which Libraries will work & find the correct linking flags, but some of the dependencies are just very error prone & we can't easily support many different systems.

To be clear, these will stay supported & included in official releases, its just that anyone who wants to build Blender from source for the first time, would likely not be able to solve these kinds of problems listed above (without a lot of work & even then... some library linking errors can become quite involved).

Proposal

On Linux/Unix use a minimal set of features.
Make some compromise, a better experience for a entry level developer, at the expense of having something which isn't including all features.
Add a make release configuration (similar to what we have for make lite now - see: https://developer.blender.org/diffusion/B/browse/master/build_files/cmake/config/blender_lite.cmake)

Defaults (proposed defaults to disable, may change)

WITH_OPENCOLLADA - uses quite a few indirect dependencies & is a large library which isn't always well maintained by each distribution, to use only to support one format.
WITH_OPENCOLORIO - only used for color management.
WITH_CYCLES_OSL (open-shading-language) - uses LLVM which is a huge dependancy.
WITH_CODEC_FFMPEG - also many indirect dependencies, linking flags can be different on each system.
WITH_LIBMV_SCHUR_SPECIALIZATION - not needed for functional libmv.
WITH_SDL - only used for joystick/BGE & audio.
WITH_NDOF - spnav isn't such a common library. quite specialized.
WITH_BUILDINFO - slows down rebuilds, isn't so useful for personal.

Others

WITH_GAMEENGINE - not a very active area of Blender at the moment (takes a while to build)
WITH_JACK - specialized.
WITH_IMAGE_OPENEXR
WITH_IMAGE_REDCODE
WITH_IMAGE_PSD

Building on Linux has become fairly complicated, with specific dependencies. In some cases this just means installing the 'right' version of a library (using `install_deps.sh`), but even then, there are complications. Examples: - ffmpeg: http://blenderartists.org/forum/archive/index.php/t-229623.html - openimageio: http://blenderartists.org/forum/showthread.php?353402-Updating-and-building-blende-Fedora-Linux - openimageio: (reported on IRC) http://www.pasteall.org/55131 - opencollada: https://developer.blender.org/T25772 Ideally we could detect which Libraries will work & find the correct linking flags, but some of the dependencies are just very error prone & we can't easily support many different systems. To be clear, these will stay supported & included in official releases, its just that anyone who wants to build Blender from source for the first time, would likely not be able to solve these kinds of problems listed above (without a lot of work & even then... some library linking errors can become quite involved). Proposal ---- - On Linux/Unix use a minimal set of features.<br>Make some compromise, a better experience for a entry level developer, at the expense of having something which isn't including all features. - Add a `make release` configuration (similar to what we have for `make lite` now - see: https://developer.blender.org/diffusion/B/browse/master/build_files/cmake/config/blender_lite.cmake) Defaults (proposed defaults to disable, may change) ---- - `WITH_OPENCOLLADA` - uses quite a few indirect dependencies & is a large library which isn't always well maintained by each distribution, to use only to support one format. - `WITH_OPENCOLORIO` - only used for color management. - `WITH_CYCLES_OSL` (open-shading-language) - uses LLVM which is a huge dependancy. - `WITH_CODEC_FFMPEG` - also many indirect dependencies, linking flags can be different on each system. - `WITH_LIBMV_SCHUR_SPECIALIZATION` - not needed for functional libmv. - `WITH_SDL` - only used for joystick/BGE & audio. - `WITH_NDOF` - spnav isn't such a common library. quite specialized. - `WITH_BUILDINFO` - slows down rebuilds, isn't so useful for personal. Others - `WITH_GAMEENGINE` - not a very active area of Blender at the moment (takes a while to build) - `WITH_JACK` - specialized. - `WITH_IMAGE_OPENEXR` - `WITH_IMAGE_REDCODE` - `WITH_IMAGE_PSD`

Campbell Barton commented

2014-11-11 17:34:12 +01:00

Changed status to: 'Open'

Sergey Sharybin was assigned by Campbell Barton

2014-11-11 17:34:12 +01:00

Campbell Barton commented

2014-11-11 17:34:12 +01:00

Added subscriber: @ideasman42

Sergey Sharybin commented

2014-11-11 17:57:38 +01:00

Generally agree, with some remarks and ideas:

Would think we should try preserving behavior of make invoked from the sources directory, to minimize possible frustration.
Think default behavior should be to enable everything possible which doesn't require extra dependencies. Hence LIBMV could be enabled. Schur specializations better be disabled.
PSD we might keep, at least for until we'll teach Cycles to compile without OIIO. For image loading it's not that difficult to not use OIIO in Cycles, but some other stuff like ustring (which is a part of OIIO) used in Cycles might take a bit of effort to solve.

Now i'll just throw an idea, could we have something like make auto which would scan for dependencies and enable features which dependencies are met? This way, i.e. if FFmpeg dev packages is installed, it'll be enabled in Blender?

Generally agree, with some remarks and ideas: - Would think we should try preserving behavior of `make` invoked from the sources directory, to minimize possible frustration. - Think default behavior should be to enable everything possible which doesn't require extra dependencies. Hence LIBMV could be enabled. Schur specializations better be disabled. - PSD we might keep, at least for until we'll teach Cycles to compile without OIIO. For image loading it's not that difficult to not use OIIO in Cycles, but some other stuff like `ustring` (which is a part of OIIO) used in Cycles might take a bit of effort to solve. Now i'll just throw an idea, could we have something like `make auto` which would scan for dependencies and enable features which dependencies are met? This way, i.e. if FFmpeg dev packages is installed, it'll be enabled in Blender?

Campbell Barton commented

2014-11-13 12:16:40 +01:00

Added subscriber: @Sergey

Campbell Barton commented

2014-11-13 12:16:40 +01:00

@Sergey,

I think make from the sources, especially should be changed. for a full release compatible build make release. Existing builds wont change configuration.
re: make auto, cmake already detects and disables libraries that don't exist in many cases, it could do better auto-detection of course, and in some cases its not as simple as detecting if libs exist or not, (incompatible versions, direct vs indirect linking... etc). So I'd not bother with make auto and just improve auto-detection.

@Sergey, - I think `make` from the sources, *especially* should be changed. for a full release compatible build `make release`. Existing builds wont change configuration. - re: `make auto`, cmake already detects and disables libraries that don't exist in many cases, it could do better auto-detection of course, and in some cases its not as simple as detecting if libs exist or not, (incompatible versions, direct vs indirect linking... etc). So I'd not bother with `make auto` and just improve auto-detection.

Sergey Sharybin commented

2014-11-13 12:28:34 +01:00

Well, if the change in behavior of make is explicitly mentioned in the ML and so it should be all fine i guess.

As for make auto -- well, improvements for auto-detection are also nice :) Just would be nice if i've got, say, OSD installed (well, in the future) it would be used by blender by default. But this things are not something what we'd better keep for further changes.

Well, if the change in behavior of `make` is explicitly mentioned in the ML and so it should be all fine i guess. As for make auto -- well, improvements for auto-detection are also nice :) Just would be nice if i've got, say, OSD installed (well, in the future) it would be used by blender by default. But this things are not something what we'd better keep for further changes.

Bastien Montagne commented

2014-11-13 13:15:20 +01:00

Added subscriber: @mont29

Bastien Montagne commented

2014-11-13 13:15:20 +01:00

I do not mind having a minimal feature-set in default build, what bothers me is that it could have the side effect of even fewer devs building often full-featured Blender.

We of course have the recurrent example of the BGE and infamous missing 'stubs', but the lighter Blender you build, the more you risk breaking some 'optional' part without noticing. So even though it can be (really) difficult sometimes, I think new devs should make the effort to build a reasonably complete Blender.

So if we decide to go this way, we should communicate well it’s just a first step, if you want to be a (blender) dev, you need to be able to build most of it.

I do not mind having a minimal feature-set in default build, what bothers me is that it could have the side effect of even fewer devs building often full-featured Blender. We of course have the recurrent example of the BGE and infamous missing 'stubs', but the lighter Blender you build, the more you risk breaking some 'optional' part without noticing. So even though it can be (really) difficult sometimes, I think new devs should make the effort to build a reasonably complete Blender. So if we decide to go this way, we should communicate well it’s just a first step, if you want to be a (blender) dev, you need to be able to build most of it.

Diego Gangl commented

2014-11-13 14:20:32 +01:00

Added subscriber: @Januz

Diego Gangl commented

2014-11-13 14:20:32 +01:00

Would requiring ranges of versions (instead of a specific one) add too much maintenance?

Most package managers let you install a specific version (alongside the latest they have). Maybe there could be a wiki page with some links to docs of several package managers to get people started.

Also, like @mont29 said, maybe print a message at the start or end of building to communicate that it's a light build.

Would requiring ranges of versions (instead of a specific one) add too much maintenance? Most package managers let you install a specific version (alongside the latest they have). Maybe there could be a wiki page with some links to docs of several package managers to get people started. Also, like @mont29 said, maybe print a message at the start or end of building to communicate that it's a light build.

Sergey Sharybin commented

2014-11-13 14:42:34 +01:00

@mont29, if someone want to help hacking into motion tracking that'd be weird to require him to deal with boost and co.

@Januz, i'm not sure how it's gonna to help.

@ideasman42, we should consider disabling WITH_LIBMV_SCHUR_SPECIALIZATIONS by default btw.

@mont29, if someone want to help hacking into motion tracking that'd be weird to require him to deal with boost and co. @Januz, i'm not sure how it's gonna to help. @ideasman42, we should consider disabling WITH_LIBMV_SCHUR_SPECIALIZATIONS by default btw.

Diego Gangl commented

2014-11-13 23:42:26 +01:00

@Sergey The fact that these settings are disabled by default doesn't mean users won't enable them at some point and get the same strange library errors. Paste some of those errors in a wiki page, with some likely causes, and some links to package manager docs and you have a nice link to send people to when they ask about these errors in IRC/ML/etc. And they could find it by googling, before they have to ask.
(I assume you meant the wiki idea btw)

@Sergey The fact that these settings are disabled by default doesn't mean users won't enable them at some point and get the same strange library errors. Paste some of those errors in a wiki page, with some likely causes, and some links to package manager docs and you have a nice link to send people to when they ask about these errors in IRC/ML/etc. And they could find it by googling, before they have to ask. (I assume you meant the wiki idea btw)

Campbell Barton commented

2014-11-14 09:52:06 +01:00

Changed status from 'Open' to: 'Resolved'

Campbell Barton closed this issue

2014-11-14 09:52:06 +01:00

Campbell Barton commented

2014-11-14 09:52:06 +01:00

Committed bf63e348a2

@Januz, making sure builds work is an on-going effort, what you suggest can help.
Also, I didnt see general docs online about how to resolve linking issues (these aren't blender specific). If they exist could help to link to them... or write our own.

Committed bf63e348a2 @Januz, making sure builds work is an on-going effort, what you suggest can help. Also, I didnt see general docs online about how to resolve linking issues (these aren't blender specific). If they exist could help to link to them... or write our own.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Proposal to amend the default build configuration #42569

Proposal

Defaults (proposed defaults to disable, may change)