Force GPU rendering python script #54099

New Issue

Dennis Teusink · 2018-02-19T08:49:22+01:00

Dennis Teusink commented

2018-02-19 08:49:22 +01:00

In our quest to get our GPU workers to use all available GPU's we have found a script.

This script doesn't work when added to the script>startup directory. We don't know why.

But, when added to the commandline in the yaml file of the manager it works perfectly.

Add this to the commandline. --enable-autoexec --python "S:/-location script-/gpurender.py"

gpurender.py is below script.

- ##### BEGIN GPL LICENSE BLOCK #####
#
- This program is free software; you can redistribute it and/or
- modify it under the terms of the GNU General Public License
- as published by the Free Software Foundation; either version 2
- of the License, or (at your option) any later version.
#
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- GNU General Public License for more details.
#
- You should have received a copy of the GNU General Public License
- along with this program; if not, write to the Free Software Foundation,
- Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
#
# ##### END GPL LICENSE BLOCK #####

# <pep8 compliant>
import bpy
bpy.context.user_preferences.addons['cycles'].preferences.compute_device_type = 'CUDA'
bpy.context.user_preferences.addons['cycles'].preferences.compute_device = 'CUDA_MULTI_2'

Our request is to have this be added in the flamenco worker code or something. And have a checkbox somewhere to enable this feature..

We hope it helps!

With kind regards,
Dennis Teusink

In our quest to get our GPU workers to use all available GPU's we have found a script. This script doesn't work when added to the script>startup directory. We don't know why. But, when added to the commandline in the yaml file of the manager it works perfectly. Add this to the commandline. **--enable-autoexec --python "S:/-location script-/gpurender.py"** gpurender.py is below script. ``` - ##### BEGIN GPL LICENSE BLOCK ##### # - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License - as published by the Free Software Foundation; either version 2 - of the License, or (at your option) any later version. # - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. # - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software Foundation, - Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. # # ##### END GPL LICENSE BLOCK ##### # <pep8 compliant> import bpy bpy.context.user_preferences.addons['cycles'].preferences.compute_device_type = 'CUDA' bpy.context.user_preferences.addons['cycles'].preferences.compute_device = 'CUDA_MULTI_2' ``` Our request is to have this be added in the flamenco worker code or something. And have a checkbox somewhere to enable this feature.. We hope it helps! With kind regards, Dennis Teusink

Sybren A. Stüvel was assigned by Dennis Teusink

2018-02-19 08:49:22 +01:00

Dennis Teusink commented

2018-02-19 08:49:22 +01:00

Added subscriber: @Alientrick

Jack Frost commented

2018-02-19 12:03:54 +01:00

Added subscriber: @nguyenbs

Jack Frost commented

2018-02-19 12:03:54 +01:00

We are urgently need this feature

Brecht Van Lommel commented

2018-02-19 13:43:35 +01:00

Added subscriber: @brecht

Brecht Van Lommel commented

2018-02-19 13:43:35 +01:00

Something like this should work:

import bpy

scene = bpy.context.scene
scene.cycles.device = 'GPU'

prefs = bpy.context.user_preferences
cprefs = prefs.addons['cycles'].preferences

# Attempt to set GPU device types if available
for compute_device_type in ('CUDA', 'OPENCL', 'NONE'):
    try:
        cprefs.compute_device_type = compute_device_type
        break
    except TypeError:
        pass

# Enable all CPU and GPU devices
for device in cprefs.devices:
    device.use = True

Something like this should work: ``` import bpy scene = bpy.context.scene scene.cycles.device = 'GPU' prefs = bpy.context.user_preferences cprefs = prefs.addons['cycles'].preferences # Attempt to set GPU device types if available for compute_device_type in ('CUDA', 'OPENCL', 'NONE'): try: cprefs.compute_device_type = compute_device_type break except TypeError: pass # Enable all CPU and GPU devices for device in cprefs.devices: device.use = True ```

joel heethaar commented

2018-02-19 16:30:27 +01:00

Added subscriber: @joel_nl

joel heethaar commented

2018-02-19 16:30:27 +01:00

This comment was removed by @joel_nl

*This comment was removed by @joel_nl*

Sybren A. Stüvel commented

2018-02-23 10:13:07 +01:00

Thanks @brecht!

@joel_nl I've added Brecht's script to Flamenco Worker, to the blender/blender#53099-gpu-rendering branch, but I haven't created anything around that yet. Can you test that the script works by adding --enable-autoexec --python "/path/to/flamenco-worker/flamenco_worker/enable_all_gpus.py" to the blender variable in your flamenco-manager.yaml?

Thanks @brecht! @joel_nl I've added Brecht's script to Flamenco Worker, to the [blender/blender#53099-gpu-rendering](https://developer.blender.org/source/flamenco-worker/history/T53099-gpu-rendering/) branch, but I haven't created anything around that yet. Can you test that the script works by adding ` --enable-autoexec --python "/path/to/flamenco-worker/flamenco_worker/enable_all_gpus.py"` to the `blender` variable in your `flamenco-manager.yaml`?

A K commented

2018-10-22 23:10:34 +02:00

Added subscriber: @campino

A K commented

2018-10-22 23:10:34 +02:00

Adding the script to my blender variable in flamenco-manager.yaml works for me.

Dennis Teusink commented

2019-01-04 09:46:47 +01:00

@dr.sybren I previously haven't had a chance to test this script properly. For now we are working with Backburner as our main render farm tool. We would like to use Flamenco though. But it is not very compatible with our Windows network. Everytime we try to work with Flamenco we run into strange issues.

I've tested this script though and it works fine. Now we need a checkbox or something in the Flamenco panel to activate it or something. Thanks...

@dr.sybren I previously haven't had a chance to test this script properly. For now we are working with Backburner as our main render farm tool. We would like to use Flamenco though. But it is not very compatible with our Windows network. Everytime we try to work with Flamenco we run into strange issues. I've tested this script though and it works fine. Now we need a checkbox or something in the Flamenco panel to activate it or something. Thanks...

Sybren A. Stüvel commented

2019-01-04 11:18:15 +01:00

In #54099#593120, @Alientrick wrote:
@dr.sybren I previously haven't had a chance to test this script properly. For now we are working with Backburner as our main render farm tool. We would like to use Flamenco though. But it is not very compatible with our Windows network. Everytime we try to work with Flamenco we run into strange issues.

Please make sure that you have the latest Blender Cloud add-on 1.11.0 (just released) as it contains Windows and Blender 2.8 fixes. If you still see issues, please provide us with a proper bug report.

I've tested this script though and it works fine.

That is good to know.

Now we need a checkbox or something in the Flamenco panel to activate it or something. Thanks...

Why? As @campino said you can just add it to the Flamenco Manager config.

> In #54099#593120, @Alientrick wrote: > @dr.sybren I previously haven't had a chance to test this script properly. For now we are working with Backburner as our main render farm tool. We would like to use Flamenco though. But it is not very compatible with our Windows network. Everytime we try to work with Flamenco we run into strange issues. Please make sure that you have the latest Blender Cloud add-on 1.11.0 (just released) as it contains Windows and Blender 2.8 fixes. If you still see issues, please provide us with a proper bug report. > I've tested this script though and it works fine. That is good to know. > Now we need a checkbox or something in the Flamenco panel to activate it or something. Thanks... Why? As @campino said you can just add it to the Flamenco Manager config.

Dennis Teusink commented

2019-01-08 15:07:49 +01:00

Please make sure that you have the latest Blender Cloud add-on 1.11.0 (just released) as it contains Windows and Blender 2.8 fixes. If you still see issues, please provide us with a proper bug report.

We will.

I think the same problem is occurring as that path resolve thing here Projects on mapped network drives fail .

When i change the script location to a network drive and set the variable in the yaml file
windows: S:/!Apz/Blender/Flamenco Blender 2.79/Blender/Blender.exe --python //rackstation/software/!Apz/Blender/enable_all_gpus.py

it returns this.

OSError: Python file "C:\Users\Administrator\Desktop\flamenco-worker-2.1.0-windows//RACKSTATION/software/!Apz/Blender/enable_all_gpus.py" could not be opened: No such file or directory

It doesn't matter if i use a drive letter, backslashes or forward slashes. If i put it between quotation marks, like in the documentation, the manager won't even run.

> Please make sure that you have the latest Blender Cloud add-on 1.11.0 (just released) as it contains Windows and Blender 2.8 fixes. If you still see issues, please provide us with a proper bug report. We will. I think the same problem is occurring as that path resolve thing here [Projects on mapped network drives fail ](https://developer.blender.org/T60131). When i change the script location to a network drive and set the variable in the yaml file windows: S:/!Apz/Blender/Flamenco Blender 2.79/Blender/Blender.exe --python //rackstation/software/!Apz/Blender/enable_all_gpus.py it returns this. OSError: Python file "C:\Users\Administrator\Desktop\flamenco-worker-2.1.0-windows\//RACKSTATION/software/!Apz/Blender/enable_all_gpus.py" could not be opened: No such file or directory It doesn't matter if i use a drive letter, backslashes or forward slashes. If i put it between quotation marks, like in the documentation, the manager won't even run.

Sybren A. Stüvel commented

2019-01-09 10:17:38 +01:00

I recommend not using any exclamation marks in your path. Also only use mapped network drives (so S:/ instead of *rackstation/software), as * means something else to Blender. That, and of course blender/blender-asset-tracer#60131 should be fixed.

I recommend not using any exclamation marks in your path. Also only use mapped network drives (so `S:/` instead of `*rackstation/software`), as `*` means something else to Blender. That, and of course blender/blender-asset-tracer#60131 should be fixed.

Dennis Teusink commented

2019-01-10 08:47:05 +01:00

Also only use mapped network drives (so S:/ instead of rackstation/software), as means something else to Blender. That, and of course blender/blender-asset-tracer#60131 should be fixed.

That why i tried every which version i know of url. Nothing worked. This version was with the slashes...

I recommend not using any exclamation marks in your path.

Ok... That worked. Which is weird because the path to Blender itself also contains an exclamation mark. Don't know why it works but it does. so i'm happy. We are changing the structure of our Software/Install server anyway. It is a mess. This folder for instance is like 20 years old. :)

Thanks for all the help!

>Also only use mapped network drives (so S:/ instead of *rackstation/software), as* means something else to Blender. That, and of course blender/blender-asset-tracer#60131 should be fixed. That why i tried every which version i know of url. Nothing worked. This version was with the slashes... >I recommend not using any exclamation marks in your path. Ok... That worked. Which is weird because the path to Blender itself also contains an exclamation mark. Don't know why it works but it does. so i'm happy. We are changing the structure of our Software/Install server anyway. It is a mess. This folder for instance is like 20 years old. :) Thanks for all the help!

Dalai Felinto commented

2019-12-23 18:41:18 +01:00

Added subscriber: @dfelinto

Dalai Felinto commented

2019-12-23 18:41:18 +01:00

Changed status from 'Confirmed' to: 'Archived'

Dalai Felinto closed this issue

2019-12-23 18:41:18 +01:00

Dalai Felinto commented

2019-12-23 18:41:18 +01:00

Hi, thanks for your patch.

We are undergoing a Tracker Curfew where we are automatically closing old patches.

If you think the patch is still relevant please update and re-submit it. For new features make sure there is a clear design from the user level perspective.

Hi, thanks for your patch. We are undergoing a [Tracker Curfew ](https://code.blender.org/?p=3861) where we are automatically closing old patches. If you think the patch is still relevant please update and re-submit it. For new features make sure there is a clear design from the user level perspective.

Chris commented

2020-04-28 21:30:02 +02:00

Added subscriber: @itschrisho

Chris commented

2020-04-28 21:30:02 +02:00

Hi,

I currently have the script that @brecht wrote. Then using the command line on windows im running:

blender.exe blender_scenes\bmw27\bmw27_gpu.blend -b -f 0 -P GPURender.py

I see the python script gets executed as I put print statements to make sure it actually runs. However, looking at my task manager, the cpu usage is still 100%.

Can someone help as to why this is not making my GPU do the rendering?

Thanks for the help!

Hi, I currently have the script that @brecht wrote. Then using the command line on windows im running: blender.exe blender_scenes\bmw27\bmw27_gpu.blend -b -f 0 -P GPURender.py I see the python script gets executed as I put print statements to make sure it actually runs. However, looking at my task manager, the cpu usage is still 100%. Can someone help as to why this is not making my GPU do the rendering? Thanks for the help!

Simon commented

2020-10-21 12:29:49 +02:00

Added subscriber: @simon-28

Simon commented

2020-10-21 12:29:49 +02:00

Hi, having the same problem...

On a headless Virtual Machine in the cloud running Ubuntu 18.04 installation with CUDA and NVIDIA drivers all installed and showing up correctl.

Using Blender 2.8 but file continues to render on CPU,

Here's the command line

./blender -b /shots/xxxx/GPU_RenderTest.blend -f 1 -P use_gpu.py

this is the use_gpu.py python file

import bpy

# force rendering to GPU
bpy.context.scene.cycles.device = 'GPU'
cpref = bpy.context.preferences.addons['cycles'].preferences
cpref.compute_device_type = 'CUDA'
# Use GPU devices only
cpref.get_devices()
for device in cpref.devices:
    device.use = True if device.type == 'CUDA' else False

Any help would be greatly appreciated.

Simon

Hi, having the same problem... On a headless Virtual Machine in the cloud running Ubuntu 18.04 installation with CUDA and NVIDIA drivers all installed and showing up correctl. Using Blender 2.8 but file continues to render on CPU, Here's the command line ``` ./blender -b /shots/xxxx/GPU_RenderTest.blend -f 1 -P use_gpu.py ``` this is the use_gpu.py python file ``` import bpy # force rendering to GPU bpy.context.scene.cycles.device = 'GPU' cpref = bpy.context.preferences.addons['cycles'].preferences cpref.compute_device_type = 'CUDA' # Use GPU devices only cpref.get_devices() for device in cpref.devices: device.use = True if device.type == 'CUDA' else False ``` Any help would be greatly appreciated. Simon

Brecht Van Lommel commented

2020-10-21 19:11:21 +02:00

You can run Blender with --debug-cycles to see what it is doing, and also check that there is no Python error in executing that script.

The example script I gave was a simple example, for use in production it needs error handling and verification there is actually a CUDA device, and if not abort rendering.

You can run Blender with `--debug-cycles` to see what it is doing, and also check that there is no Python error in executing that script. The example script I gave was a simple example, for use in production it needs error handling and verification there is actually a CUDA device, and if not abort rendering.

Seal La-Tunje commented

2020-10-22 10:50:03 +02:00

Added subscriber: @seal

horatio_engelmann commented

2023-10-01 22:52:38 +02:00

looking at my flamenco task logs, it seems like the gpy python script runs AFTER all frames have been rendered:

2023-10-01T22:36:57+02:00 Task assigned to worker hostname (841bc2cf-48ca-42db-95eb-ea4c61c08871)
2023-10-01T22:36:57+02:00 task changed status queued -> active
going to run: /Applications/blender.app/Contents/MacOS/blender ["-b" "-y" "-P" "../gpu.py" "/path/to/flamenco/projects/asdf.flamenco.blend" "--render-output" "/path/to/flamenco/renders/asdf/2023-10-01_223640/######" "--render-format" "JPEG" "--render-frame" "10..19"]
pid=74897 > Blender 3.6.4 (hash 21bfc5e7fe3f built 2023-09-25 23:33:06)
pid=74897 > Read prefs: "/Users/xyz/Library/Application Support/Blender/3.6/config/userpref.blend"
pid=74897 > Read blend: "/path/to/flamenco/projects/asdf.flamenco.blend"
pid=74897 > Fra:10 Mem:13.71M (Peak 13.71M) | Time:00:00.00 | Mem:0.00M, Peak:0.00M | Scene, ViewLayer | Synchronizing object | glass
[...]
pid=74897 > Fra:19 Mem:99.83M (Peak 382.31M) | Time:00:01.24 | Compositing | Tile 12-12
pid=74897 > Fra:19 Mem:99.77M (Peak 382.31M) | Time:00:01.24 | Compositing | De-initializing execution
pid=74897 > Saved: '/path/to/flamenco/renders/asdf/2023-10-01_223640/000019.jpg'
pid=74897 >  Time: 00:01.24 (Saving: 00:00.00)
pid=74897 > 
pid=74897 > ############ hostname
pid=74897 > ## METAL
pid=74897 > ## - Apple M2 Pro
pid=74897 > ## - Apple M2 Pro (GPU - 16 cores)
pid=74897 > ############
pid=74897 > 
pid=74897 > Blender quit
2023-10-01T22:37:10+02:00 task changed status active -> completed

can that be?
🤨

looking at my flamenco task logs, it seems like the gpy python script runs AFTER all frames have been rendered: ``` 2023-10-01T22:36:57+02:00 Task assigned to worker hostname (841bc2cf-48ca-42db-95eb-ea4c61c08871) 2023-10-01T22:36:57+02:00 task changed status queued -> active going to run: /Applications/blender.app/Contents/MacOS/blender ["-b" "-y" "-P" "../gpu.py" "/path/to/flamenco/projects/asdf.flamenco.blend" "--render-output" "/path/to/flamenco/renders/asdf/2023-10-01_223640/######" "--render-format" "JPEG" "--render-frame" "10..19"] pid=74897 > Blender 3.6.4 (hash 21bfc5e7fe3f built 2023-09-25 23:33:06) pid=74897 > Read prefs: "/Users/xyz/Library/Application Support/Blender/3.6/config/userpref.blend" pid=74897 > Read blend: "/path/to/flamenco/projects/asdf.flamenco.blend" pid=74897 > Fra:10 Mem:13.71M (Peak 13.71M) | Time:00:00.00 | Mem:0.00M, Peak:0.00M | Scene, ViewLayer | Synchronizing object | glass [...] pid=74897 > Fra:19 Mem:99.83M (Peak 382.31M) | Time:00:01.24 | Compositing | Tile 12-12 pid=74897 > Fra:19 Mem:99.77M (Peak 382.31M) | Time:00:01.24 | Compositing | De-initializing execution pid=74897 > Saved: '/path/to/flamenco/renders/asdf/2023-10-01_223640/000019.jpg' pid=74897 > Time: 00:01.24 (Saving: 00:00.00) pid=74897 > pid=74897 > ############ hostname pid=74897 > ## METAL pid=74897 > ## - Apple M2 Pro pid=74897 > ## - Apple M2 Pro (GPU - 16 cores) pid=74897 > ############ pid=74897 > pid=74897 > Blender quit 2023-10-01T22:37:10+02:00 task changed status active -> completed ``` can that be? 🤨

Brecht Van Lommel commented

2023-10-02 11:02:50 +02:00

@horatio_engelmann if you think you found a bug please open a new report, this 5 year old report is not going to be reopened.

Sign in to join this conversation.