Page MenuHome

Blender BenchmarkProject
ActivePublic

Members (6)

Details

Description

This is a project about Blender's benchmark bundle and website. Both of this projects are highly under development, more information will be available once things are stabilized and ready for public tests.

Recent Activity

Tue, Jul 26

ExLibris (ExLibris68) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

Tue, Jul 26, 7:24 AM · Blender Open Data, Blender Benchmark
ExLibris (ExLibris68) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

On juli 12th 2022 two another unrealistic values appear. This time it is a RTX 3080 Laptop GPU.

Tue, Jul 26, 7:21 AM · Blender Open Data, Blender Benchmark

Jun 27 2022

ExLibris (ExLibris68) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

Since 24-6 the Apple M2 appears in the open data, with unrealistic values for the Junkshop scene

Jun 27 2022, 7:59 AM · Blender Open Data, Blender Benchmark

Jun 24 2022

Richard Antalik (ISS) closed T96635: Launcher hanging as Archived.

No activity for more than a week. As per the tracker policy we assume the issue is gone and can be closed.

Jun 24 2022, 7:42 PM · Blender Benchmark
Richard Antalik (ISS) closed T97763: Blender Benchmark 2.0.4 Error while communicating with the server. as Archived.

No activity for more than a week. As per the tracker policy we assume the issue is gone and can be closed.

Jun 24 2022, 7:39 PM · Blender Open Data, Blender Benchmark
Brecht Van Lommel (brecht) added a revision to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data: D15287: Cycles: stop Metal rendering on command buffer error.
Jun 24 2022, 3:39 PM · Blender Open Data, Blender Benchmark
Brecht Van Lommel (brecht) merged T99134: Junkshop scene samples per minute reporting bug on Apple M2 chips into T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.
Jun 24 2022, 3:33 PM · Blender Open Data, Blender Benchmark
Brecht Van Lommel (brecht) merged task T99134: Junkshop scene samples per minute reporting bug on Apple M2 chips into T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.
Jun 24 2022, 3:33 PM · Blender Benchmark
Jeroen Bakker (jbakker) renamed T99136: Benchmark: Add benchmarking for Eevee viewport from Add benchmarking for Eevee/workbench viewport playback performance to Benchmark: Add benchmarking for Eevee/workbench viewport.
Jun 24 2022, 3:23 PM · BF Blender (3.3), Restricted Project, EEVEE & Viewport
ExLibris (ExLibris68) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

A simular problem appears with (some) M2 processors as described in Track T99134

Jun 24 2022, 2:43 PM · Blender Open Data, Blender Benchmark
ExLibris (ExLibris68) added a comment to T99134: Junkshop scene samples per minute reporting bug on Apple M2 chips.

This is probably related to track T96519.

Jun 24 2022, 2:40 PM · Blender Benchmark
Omar Emara (OmarSquircleArt) edited projects for T99134: Junkshop scene samples per minute reporting bug on Apple M2 chips, added: Blender Benchmark; removed BF Blender.
Jun 24 2022, 2:35 PM · Blender Benchmark

Jun 10 2022

Richard Antalik (ISS) changed the status of T96635: Launcher hanging from Needs Triage to Needs Information from User.

@David Butterworth (Frappe) can you check if this happens with new benchmark?

Jun 10 2022, 7:17 PM · Blender Benchmark

Jun 8 2022

Richard Antalik (ISS) changed the status of T97763: Blender Benchmark 2.0.4 Error while communicating with the server. from Needs Triage to Needs Information from User.

Is this still issue with new benchmark?

Jun 8 2022, 6:21 PM · Blender Open Data, Blender Benchmark

May 30 2022

Anna Sirota (railla) added a comment to T98358: benchmark-launcher-cli fails to submit results.

I'm not sure where the "bad handshake" error comes from, if that is caused by the websocket connection upgrade failing (HTTPS -> WSS)

May 30 2022, 4:29 PM · Blender Benchmark, Blender Open Data
Anna Sirota (railla) closed T98358: benchmark-launcher-cli fails to submit results as Resolved by committing rBOD0ae0bca34df6: Fix T98358: set HTTP Upgrade headers for benchmark-launcher-cli WS connection.
May 30 2022, 4:03 PM · Blender Benchmark, Blender Open Data
Francesco Siddi (fsiddi) assigned T98358: benchmark-launcher-cli fails to submit results to Anna Sirota (railla).

Looping in Anna, as this might be related to changes recently done on the infrastructure.

May 30 2022, 1:03 PM · Blender Benchmark, Blender Open Data

May 29 2022

Richard Antalik (ISS) added a comment to T98358: benchmark-launcher-cli fails to submit results.

Can you check if there are any rules for outgoing connections? Not sure what port is in question here though or if this could be debugged somehow. I would have to check sources, to see if it is possible to have some exchange perhaps with python script to see if connection was made at least or not.

May 29 2022, 12:52 AM · Blender Benchmark, Blender Open Data

May 27 2022

Paul Melis (paulmelis) added a comment to T98358: benchmark-launcher-cli fails to submit results.

Yes, I just tried it again and got the same handshake error. Note that I'm running the benchmark on a GPU node of a supercomputer, but I've never had issues with outgoing network connections there. The weird thing is that apparently the connection to the submit server can easily be made, but something goes wrong after that. I'm not sure where the "bad handshake" error comes from, if that is caused by the websocket connection upgrade failing (HTTPS -> WSS), or if is related to the missing token (which might be part of the submission handshake?).

May 27 2022, 8:35 AM · Blender Benchmark, Blender Open Data
Richard Antalik (ISS) changed the status of T98357: benchmark-launcher-cli segfaults for unknown options or missing option values from Needs Triage to Confirmed.
May 27 2022, 2:50 AM · Blender Benchmark, Blender Open Data
Richard Antalik (ISS) changed the status of T98358: benchmark-launcher-cli fails to submit results from Needs Triage to Needs Information from User.

@Paul Melis (paulmelis) is this still an issue? I wasn't able to reproduce error

May 27 2022, 2:49 AM · Blender Benchmark, Blender Open Data

May 25 2022

Pratik Borhade (PratikPB2123) edited projects for T98358: benchmark-launcher-cli fails to submit results, added: Blender Open Data, Blender Benchmark; removed BF Blender.
May 25 2022, 5:49 AM · Blender Benchmark, Blender Open Data
Pratik Borhade (PratikPB2123) edited projects for T98357: benchmark-launcher-cli segfaults for unknown options or missing option values, added: Blender Open Data, Blender Benchmark; removed BF Blender.
May 25 2022, 4:54 AM · Blender Benchmark, Blender Open Data

May 24 2022

Omar Emara (OmarSquircleArt) closed T80156: Blender Benchmark segfaults with Optix initialization error as Archived.

No activity for more than a week. As per the tracker policy we assume the issue is gone and can be closed.

May 24 2022, 4:38 PM · Blender Benchmark, Cycles, BF Blender

May 2 2022

ExLibris (ExLibris68) added a comment to T97763: Blender Benchmark 2.0.4 Error while communicating with the server..

As far I can see in the data, this is not a specific Quadro k2200 issue, because there were several successful submissions with this card before.

May 2 2022, 8:08 AM · Blender Open Data, Blender Benchmark
Pratik Borhade (PratikPB2123) edited projects for T97763: Blender Benchmark 2.0.4 Error while communicating with the server., added: Blender Benchmark, Blender Open Data; removed BF Blender.
May 2 2022, 4:47 AM · Blender Open Data, Blender Benchmark

Apr 20 2022

Matthieu (MHEonM1) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

Classroom may be older but is still a 2.8.x Blender demo file.

Apr 20 2022, 8:32 PM · Blender Open Data, Blender Benchmark
Brecht Van Lommel (brecht) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

The source code is here, I don't know if there is documentation outside of that.
https://developer.blender.org/source/blender-open-data/

Apr 20 2022, 8:09 PM · Blender Open Data, Blender Benchmark
Matthieu (MHEonM1) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

Also, where can I find documentation on how the Open Data Benchmark performs its Benchmark ? What are the settings for the files you are computing ? Why and how those settings were chosen ? How can I correlate the Open Data Benchmark results with what I can expect to see in Blender ?

Apr 20 2022, 7:41 PM · Blender Open Data, Blender Benchmark
Matthieu (MHEonM1) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

@Brecht Van Lommel (brecht) What is the CPU score of Junkshop on your 16G machine ?

Apr 20 2022, 7:24 PM · Blender Open Data, Blender Benchmark
Brecht Van Lommel (brecht) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

Ok, perhaps the junkshop scene is running out of memory and causing issues on both CPU and GPU. Swapping and running very slowly on the CPU, and just failing on the GPU.

Apr 20 2022, 7:07 PM · Blender Open Data, Blender Benchmark
Matthieu (MHEonM1) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

@Brecht Van Lommel (brecht) I'm looking at the number of samples per second at the very end of the log.

Apr 20 2022, 5:53 PM · Blender Open Data, Blender Benchmark
Brecht Van Lommel (brecht) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

@Matthieu (MHEonM1) regarding CPU rendering time, which numbers are you looking at exactly?

Apr 20 2022, 5:36 PM · Blender Open Data, Blender Benchmark
Brecht Van Lommel (brecht) updated subscribers of T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

Thanks, there is indeed an error that doesn't seem to stop the render:

BLENDER: CommandBuffer Failed: cycles_metal_integrator_reset

@Michael Jones (michael_jones) I guess there are two issues to fix here. One is to stop the render on this error, and the other would be to figure out why the error happens in the first place. Would you have time to look into this?

Apr 20 2022, 5:33 PM · Blender Open Data, Blender Benchmark
Matthieu (MHEonM1) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

Here we go.

Apr 20 2022, 5:11 PM · Blender Open Data, Blender Benchmark
Brecht Van Lommel (brecht) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

Likely there is an error while rendering, which causes the render to stop quickly, but this goes undetected and the render time is considered correct.

Apr 20 2022, 2:54 PM · Blender Open Data, Blender Benchmark
Matthieu (MHEonM1) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

I am still having nonsensical numbers (13inch MBP M1 8 GPU cores, 8Go RAM), both on CPU and GPU

Apr 20 2022, 2:33 PM · Blender Open Data, Blender Benchmark
ExLibris (ExLibris68) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

Since April 10th I see no new unrealistic scores. I don't know if that is a coincidence.

Apr 20 2022, 11:25 AM · Blender Open Data, Blender Benchmark
Patrick Mours (pmoursnv) closed T73657: Benchmark GUI launcher does not run on Quadro cards as Resolved.

This turned out to be a driver issue that has been fixed in 512.15+.

Apr 20 2022, 11:13 AM · Blender Benchmark, Blender Open Data

Apr 6 2022

Omar Emara (OmarSquircleArt) merged task T83898: Bug in the benchmark into T85513: Blender benchmark issue.
Apr 6 2022, 12:54 PM · Blender Open Data (1.0.0-beta.2), Blender Benchmark

Mar 28 2022

ExLibris (ExLibris68) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

That is usefull info for the Metal/Blender developers I think if they are going to look into this bug.

Mar 28 2022, 4:24 PM · Blender Open Data, Blender Benchmark
Matthieu (MHEonM1) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

Concerning the CPU score of Junkshop:

Mar 28 2022, 3:56 PM · Blender Open Data, Blender Benchmark
ExLibris (ExLibris68) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

No problem :)

Mar 28 2022, 2:54 PM · Blender Open Data, Blender Benchmark
Matthieu (MHEonM1) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

Sorry for the late response. Here are some screenshots of what I get in OpenData :

Mar 28 2022, 1:57 PM · Blender Open Data, Blender Benchmark

Mar 25 2022

ExLibris (ExLibris68) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

I cannot see the low overal scores for Apple M1 CPU scores for junkshop (see screenshot), but I notice quite a large difference between the lowest and highest scores. Is your score specific for your benchmark?

Mar 25 2022, 2:27 PM · Blender Open Data, Blender Benchmark
Matthieu (MHEonM1) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

Just to add a precision

Mar 25 2022, 1:24 PM · Blender Open Data, Blender Benchmark
ExLibris (ExLibris68) added a comment to T96519: Some benchmarks considering Metal device type show unrealistic values in Blender Open Data.

The unrealistic values are still being submitted. Because the very high values the overall scores of Apple benchmarks are not really usable:

Mar 25 2022, 8:32 AM · Blender Open Data, Blender Benchmark

Mar 22 2022

David Butterworth (Frappe) added a comment to T96635: Launcher hanging.

Hi Sergy,

Mar 22 2022, 3:36 PM · Blender Benchmark

Mar 21 2022

Sergey Sharybin (sergey) added a comment to T96635: Launcher hanging.

Need to know system information (windows version, GPU, usual things like that).

Mar 21 2022, 4:39 PM · Blender Benchmark
Francesco Siddi (fsiddi) reassigned T96635: Launcher hanging from Francesco Siddi (fsiddi) to Sergey Sharybin (sergey).

Thanks for the report. Sergey, is there anything that we can do to investigate this issue further?
Something similar has been reported by @Pablo Vazquez (pablovazquez) as well.

Mar 21 2022, 3:56 PM · Blender Benchmark