This is a project about Blender's benchmark bundle and website. Both of this projects are highly under development, more information will be available once things are stabilized and ready for public tests.
Details
Tue, Jul 26
On juli 12th 2022 two another unrealistic values appear. This time it is a RTX 3080 Laptop GPU.
Jun 27 2022
Since 24-6 the Apple M2 appears in the open data, with unrealistic values for the Junkshop scene
Jun 24 2022
No activity for more than a week. As per the tracker policy we assume the issue is gone and can be closed.
No activity for more than a week. As per the tracker policy we assume the issue is gone and can be closed.
This is probably related to track T96519.
Jun 10 2022
@David Butterworth (Frappe) can you check if this happens with new benchmark?
Jun 8 2022
Is this still issue with new benchmark?
May 30 2022
I'm not sure where the "bad handshake" error comes from, if that is caused by the websocket connection upgrade failing (HTTPS -> WSS)
Looping in Anna, as this might be related to changes recently done on the infrastructure.
May 29 2022
Can you check if there are any rules for outgoing connections? Not sure what port is in question here though or if this could be debugged somehow. I would have to check sources, to see if it is possible to have some exchange perhaps with python script to see if connection was made at least or not.
May 27 2022
Yes, I just tried it again and got the same handshake error. Note that I'm running the benchmark on a GPU node of a supercomputer, but I've never had issues with outgoing network connections there. The weird thing is that apparently the connection to the submit server can easily be made, but something goes wrong after that. I'm not sure where the "bad handshake" error comes from, if that is caused by the websocket connection upgrade failing (HTTPS -> WSS), or if is related to the missing token (which might be part of the submission handshake?).
@Paul Melis (paulmelis) is this still an issue? I wasn't able to reproduce error
May 25 2022
May 24 2022
No activity for more than a week. As per the tracker policy we assume the issue is gone and can be closed.
May 2 2022
As far I can see in the data, this is not a specific Quadro k2200 issue, because there were several successful submissions with this card before.
Apr 20 2022
Classroom may be older but is still a 2.8.x Blender demo file.
The source code is here, I don't know if there is documentation outside of that.
https://developer.blender.org/source/blender-open-data/
Also, where can I find documentation on how the Open Data Benchmark performs its Benchmark ? What are the settings for the files you are computing ? Why and how those settings were chosen ? How can I correlate the Open Data Benchmark results with what I can expect to see in Blender ?
@Brecht Van Lommel (brecht) What is the CPU score of Junkshop on your 16G machine ?
Ok, perhaps the junkshop scene is running out of memory and causing issues on both CPU and GPU. Swapping and running very slowly on the CPU, and just failing on the GPU.
@Brecht Van Lommel (brecht) I'm looking at the number of samples per second at the very end of the log.
@Matthieu (MHEonM1) regarding CPU rendering time, which numbers are you looking at exactly?
Thanks, there is indeed an error that doesn't seem to stop the render:
BLENDER: CommandBuffer Failed: cycles_metal_integrator_reset
@Michael Jones (michael_jones) I guess there are two issues to fix here. One is to stop the render on this error, and the other would be to figure out why the error happens in the first place. Would you have time to look into this?
Here we go.
Likely there is an error while rendering, which causes the render to stop quickly, but this goes undetected and the render time is considered correct.
I am still having nonsensical numbers (13inch MBP M1 8 GPU cores, 8Go RAM), both on CPU and GPU
Since April 10th I see no new unrealistic scores. I don't know if that is a coincidence.
This turned out to be a driver issue that has been fixed in 512.15+.
Apr 6 2022
Mar 28 2022
That is usefull info for the Metal/Blender developers I think if they are going to look into this bug.
Concerning the CPU score of Junkshop:
No problem :)
Sorry for the late response. Here are some screenshots of what I get in OpenData :
Mar 25 2022
I cannot see the low overal scores for Apple M1 CPU scores for junkshop (see screenshot), but I notice quite a large difference between the lowest and highest scores. Is your score specific for your benchmark?
Just to add a precision
The unrealistic values are still being submitted. Because the very high values the overall scores of Apple benchmarks are not really usable:
Mar 22 2022
Hi Sergy,
Mar 21 2022
Need to know system information (windows version, GPU, usual things like that).
Thanks for the report. Sergey, is there anything that we can do to investigate this issue further?
Something similar has been reported by @Pablo Vazquez (pablovazquez) as well.