Page MenuHome

Include Python executable with Blender.
Closed, ResolvedPublic

Description

This tasks proposed to include python in Blender (python.exe on windows, so every Blender installation can be ensured to run the same version of Python that's included with Blender).

Motivation

Short Version: subprocesses.

Blender currently uses Python for many tasks, but anything that takes longer then a few seconds will lock Blender up, Currently some addons load a separate Blender process (SketchFab for example), However if this is only to transfer data, the overhead of running an entire Blender instance, just for its Python interpreter isn't very efficient.

Uses for Python subprocesses:

  • General online integration (file upload/download ... asset manager, cloud... etc).
  • Background processing (avoid locking up Blender).

Blender could just run Python, but this means you have to install Python and have it available in the systems PATH which isn't so common on Windows.

Including the python binary means we can ensure that there will be a python3.4 for certain.

Size considerations

On my system (64bit Linux) (python+libpython3.4) compress to ~1mb. It will vary between platforms but should stay under 2-3mb.

Platform maintainers

Since we already include a full python for each platform. Including the python binary in fact isn't much extra work (will be included in svn lib/(platform)/, will have to be copied across from the system).
Note that this will have to be a static build (since we aren't LD_PRELOADing libpython)

Access from Blender

Currently blender includes: 2.73/python/lib/python3.4

The python binary can be located at: 2.73/python/bin/python

We can have an attribute to access this to avoid each script having to find the path.
eg bpy.app.python_path. (as we have for bpy.app.binary_path to access Blender's binary)

For Linux distributions which don't bundle Python, this can simply point to $(which python)

Details

Type
Design

Event Timeline

Wouldn't mind that, but I’m not a maintainer so… ;)

Anyway, my main question is: is it really worth it? Do we have use cases where it would really benefit to use lighter py process instead of full blender?

Here, empty py process is about 3MB, and empty blender process (in background) is about 37MB. This is much much bigger, but is it really an issue still? I would not expect more than a few of those processes running at once (usually no more than one actually), and since blender is already in mem, loading it again is really fast too. So unless you intend to start hundreds of processes in a very short time…

+1
As a user I can see the benefit of this as well. Do you think the Blenderplayer should have this too? Or should it be optional?

@Bastien Montagne (mont29), yep, we can keep using blender just for py access, its not horrible but not really nice either, Since we are considering having asset managers that can download data in the background, I would rather this not be depending on an entire blender process.

@Dalai Felinto (dfelinto), blenderplayer using same python as Blender, so some way for it to find the Python binary is trivial.

+10000
I've recently implemented some code that works in the background for an addon, and had to use the Blender binary just for Python. A simple py script would be lighter, faster and it'd allow me to decouple it completely from Blender so users can call it from the cli (or other scripts).

You mention it has to be a static build. Is bpy.app.python_path always going to point to a python bin, or do we have to try it (and have a fallback)?

@Diego Gangl (januz) bpy.app.python_path Would always point to a valid python binary, weather on portable blender.org setup's or on linux distro's which include python.
Of course we cant help if someone manually deletes python from blender, but this isn't some case that needs to be supported - if someone manually removes files in Blender, it can be considered a broken installation.

A Python script writer should be able to call python in a single line of code, without searching for it.

@Campbell Barton (campbellbarton) you want to have linux pointing to the system installed python? why? this could lead to mismatching python versions.

@Inês Almeida (brita_) re: system python.

I'm referring to packaged versions of Blender (as you would install on debian/redhad/fedora.... etc), these dont bundle Python.
We can assume python3.4 is in the PATH in these cases (We can use include defines to construct the path dynamically so Py3.5+ works too, whichever version of Python Blender is compiled against).

@Marcin Nijaki (marcino15)

Initially no, it wouldnt support pip, at the moment we remove distutils from the bundled Python installation.

Including it isn't especially difficult, it was just removed because there was no reason to have it and to save a bit of space.

Im not against this, but if we include distutils it may open us up to some maintenance issues. (users upgrading Blender and wanting to keep existing pip modules for eg).
So I would first like to evaluate the use-case for having pip with Blender to see if its really worth bothering with.

Thats easy doing and i see it's benefits.
I will provide a bin to our libs and adapt scons to place it in VERSION/python/bin

So we can test this and decide pro/contra much better in the end.

Jens

Is there any patch to test this? I believe this would fix issues like this: T43869

Update. added rBbc2f77e1da8f4a4e9074223824c9d3d29fc87a7b

Now we only have to copy the executable over on install, and include in lib/

@Campbell Barton (campbellbarton) but if we are distributing the Python binary with Blender we can(/should?) also link Blender against it (which would take care of the forementioned issue).

That is basically what we do in OSX with -DWITH_PYTHON_FRAMEWORK.

@ Dalai: the python binary i added to darwin libs is fully static against python and has nothing todo with python_framework.
Place it in the bundle in an apropiate way ( ....2.47/python/bin ) and you are done.
Now you must just call it instead of new blender py instances for subprocesses ....

Jens

Added for linux/osx/windows closing.