Build: add scripts to build dependencies for Windows and macOS.
ClosedPublic

Authored by Brecht Van Lommel (brecht) on Jul 27 2017, 6:11 PM.

Details

Summary

Note these are intended for platform maintainers, we do not intend to
support users making their own builds with these. For that precompiled
libraries from lib/ should be used.

Implemented by Martijn Berger, Ray Molenkamp and Brecht Van Lommel.

This is based on the following branch of blender-dependencies, which
adds the latest macOS / Linux changes and tweaks to try to match the
Blender directory layout and naming better:
https://github.com/brechtvl/blender_dependencies/commits/merge

Then moving this into the Blender repo there were a few extra changes:

  • Move blendthumb code to release/windows/blendthumb.
  • Integrate into GNUmakefile, so "make deps" works.
  • Move install_deps_patches/ contents into patches/ directory.
  • Move windows build folder out of source, to fix issue with long paths.

For Windows I left in the batch scripts instead of integrating into
make.bat (quick test to do that: P517). I leave it to @LazyDodo (LazyDodo) to
decide if we just leave it as is or want to integrate it more.

Diff Detail

Repository
rB Blender
LazyDodo (LazyDodo) added a comment.EditedJul 27 2017, 6:20 PM

Left a comment in the patch already, i don't like integrating it into make.bat at all, would make it way to accessible, people will use it, and it'll have to support it, given the script has some rather specific requirements, needs admin priv for mklink for python for instance , also i ran into path length issues with llvm in the past. Also when not installed it'll download all the bits of mingw it needs onto the users system with very little signature validation. so yeah..

beyond that...

The patch folder is kind of a mess (mostly my fault i have been rather sloppy) it's hard to tell which patches apply to which platforms, so we might have to come up with a better naming scheme or directory structure.

Also i would really like to get rid of the harvest step on windows, but we'll have to change the lib folders structure (and blenders build scripts) to do this.

Ok, let's leave it out of make.bat then.

Cleaning up the patches would be good. I guess we can just consistently add some postfixes to the patches, like _darwin, _windows, _vs2013, .. . Maybe after this is committed.

I think part of harvest is leaving out files to avoid making the lib/ directory too big? Not sure how that would be handled then, we could remove files after install but that code might not be much simpler.

Harvest is the way it is due to the scripts having to match whatever was in svn when i started, i have no idea how some of these things came to be honestly...

Right, the specific directory layout is mostly due to historical reasons and there's no good reason to keep that. But at least on macOS the harvest step reduces the libraries from 880MB to 379MB, by leaving out files not needed by the Blender build, so it's not entirely useless.

If there's no further objections I'd like to commit this, and we can then continue improving it in master.

No objections to move further development of this in master. But here are some notes:

  • The patch needs some tweaks to be rebased on top of latest master
  • If we expose this scripts to GNUmakefile, think we'd also need to do same for make.bat
  • it is a bit weird to have source files in release/ folder. But also not sure of a better place for this without making it confusing why some of the library is not covered by Blender's CMake if we move those files to somewhere else,
  • CMake files are lacking copyiright/license header
  • CMake files are mixing some code styles (mixed space/tabs indentation)

Not saying those are stoppers, just something to be looked at. Don't mind if that's done after initial commit to master.

This revision was automatically updated to reflect the committed changes.
  • If we expose this scripts to GNUmakefile, think we'd also need to do same for make.bat

I'll leave that up to @LazyDodo (LazyDodo) to decide, if we intend these scripts to be for platform maintainers it doesn't really matter either way I think.

  • it is a bit weird to have source files in release/ folder. But also not sure of a better place for this without making it confusing why some of the library is not covered by Blender's CMake if we move those files to somewhere else,

Agreed. This should really become part of the regular build instead of precompiled libraries.

  • CMake files are lacking copyiright/license header
  • CMake files are mixing some code styles (mixed space/tabs indentation)

Committed now with license headers and style cleanups.

  • If we expose this scripts to GNUmakefile, think we'd also need to do same for make.bat

I'll leave that up to @LazyDodo (LazyDodo) to decide, if we intend these scripts to be for platform maintainers it doesn't really matter either way I think.

I think i have been pretty clear on my feelings on this subject, i have no interest in adding this to make.bat *EVER* , this is not something regular users should ever run, if we add it to some common batch file, they will run it, they will have issues with it and i'll have to support it 'cause it's there, so it should work' . I'll happily spend a couple of hours getting a fellow platform dev to get things going, doing this for every user with a compiler installed will be an unreasonable drain on my available time.

  • it is a bit weird to have source files in release/ folder. But also not sure of a better place for this without making it confusing why some of the library is not covered by Blender's CMake if we move those files to somewhere else,

Agreed. This should really become part of the regular build instead of precompiled libraries.

it can't. this dll gets loaded into the explorer process , and given you can run x86 blender on x64 windows it'll need to build a 64 bit copy of this dll as well as a 32 bit , and for that to work, it'll need 64 bit zlib. so unless we start adding 64 bit libs to our 32 bit lib folder, this dll cannot be build from blenders cmake.

it can't. this dll gets loaded into the explorer process , and given you can run x86 blender on x64 windows it'll need to build a 64 bit copy of this dll as well as a 32 bit , and for that to work, it'll need 64 bit zlib. so unless we start adding 64 bit libs to our 32 bit lib folder, this dll cannot be build from blenders cmake.

Ah, that's interesting, seems like we'll have to leave it as is then.