Attribute Statistic Node Outputting Weird Mean. #97685

New Issue

Gerstmann Bradley · 2022-04-28T18:49:30+02:00

Gerstmann Bradley commented

2022-04-28 18:49:30 +02:00

System Information
Operating system: Windows-10-10.0.17134-SP0 64 Bits
Graphics card: NVIDIA GeForce GTX 970M/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 466.11

Blender Version
Broken: version:
3.2.0 Alpha, branch: master, commit date: 2022-04-27 22:46, hash: 52a5f68562
3.1.2, branch: master, commit date: 2022-03-31 17:40, hash: cc66d1020c

Short description of error

Statistics.blend

**System Information** Operating system: Windows-10-10.0.17134-SP0 64 Bits Graphics card: NVIDIA GeForce GTX 970M/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 466.11 **Blender Version** Broken: version: 3.2.0 Alpha, branch: master, commit date: 2022-04-27 22:46, hash: `52a5f68562` 3.1.2, branch: master, commit date: 2022-03-31 17:40, hash: `cc66d1020c` **Short description of error** ![image.png](https://archive.blender.org/developer/F13035434/image.png) [Statistics.blend](https://archive.blender.org/developer/F13035441/Statistics.blend)

Gerstmann Bradley commented

2022-04-28 18:49:30 +02:00

Added subscriber: @Bradley_G

Hans Goudey commented

2022-04-28 18:58:36 +02:00

Added subscriber: @HooglyBoogly

Hans Goudey commented

2022-04-28 18:58:36 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Hans Goudey commented

2022-04-28 18:58:36 +02:00

This is probably because it adds up all of the values and then divides by the total. It's quite easy to get overflow in that case. I'm not sure if there's a great solution, but there are probably ways to improve this case.

Lukas Tönne commented

2022-06-11 18:23:46 +02:00

Added subscriber: @LukasTonne

Lukas Tönne commented

2022-06-11 18:23:46 +02:00

Hans is correct: storing a value of 0.2 as floating point has an error of about 2.980232e-9, which adds up quite a bit over a 1000000 loop. Here's a python snippet that shows the effect (using numpy's float32 because python floats are usually double precision while Blender uses single precision):

import numpy as np

print((np.float32(0.2) * 1000000) / 1000000)

x = np.float32(0.2)
for i in range(1000000):
    x = x + np.float32(0.2)
x = x / 1000000
print(x)

The effect is quite noticeable:

multiply and divide: 0.20000000298023224
loop sum and divide: 0.201916890625

Hans is correct: storing a value of 0.2 as floating point has an error of about 2.980232e-9, which adds up quite a bit over a 1000000 loop. Here's a python snippet that shows the effect (using numpy's float32 because python floats are usually double precision while Blender uses single precision): ``` import numpy as np print((np.float32(0.2) * 1000000) / 1000000) x = np.float32(0.2) for i in range(1000000): x = x + np.float32(0.2) x = x / 1000000 print(x) ``` The effect is quite noticeable: ``` multiply and divide: 0.20000000298023224 loop sum and divide: 0.201916890625 ```

Evan Wilson commented

2022-06-11 18:35:48 +02:00

Added subscriber: @EAW

Evan Wilson commented

2022-06-11 18:35:48 +02:00

Would Kahan Summation help?

Lukas Tönne commented

2022-06-11 19:02:25 +02:00

Oh this is exciting, i didn't know about Kahan sum:

import numpy as np

print("multiply and divide: ", (np.float32(0.2) * 1000000) / 1000000)

s = np.float32(0.0)
for i in range(1000000):
    s = s + np.float32(0.2)
s = s / 1000000
print("simple sum: ", s)

s = np.float32(0.0)
c = np.float32(0.0)
for i in range(1000000):
    y = np.float32(0.2) - c
    t = s + y
    c = (t - s) - y
    s = t
s = s / 1000000
print("Kahan sum: ", s)

This is an ideal case of course, all the terms being exactly equal, but still nice to see it works out. Thanks @EAW, i learned something useful today.

multiply and divide:  0.20000000298023224
simple sum:  0.2019166875
Kahan sum:  0.2

Oh this is exciting, i didn't know about Kahan sum: ``` import numpy as np print("multiply and divide: ", (np.float32(0.2) * 1000000) / 1000000) s = np.float32(0.0) for i in range(1000000): s = s + np.float32(0.2) s = s / 1000000 print("simple sum: ", s) s = np.float32(0.0) c = np.float32(0.0) for i in range(1000000): y = np.float32(0.2) - c t = s + y c = (t - s) - y s = t s = s / 1000000 print("Kahan sum: ", s) ``` This is an ideal case of course, all the terms being exactly equal, but still nice to see it works out. Thanks @EAW, i learned something useful today. ``` multiply and divide: 0.20000000298023224 simple sum: 0.2019166875 Kahan sum: 0.2 ```

Evan Wilson commented

2022-06-12 11:55:46 +02:00

In #97685#1372874, @LukasTonne wrote:
Oh this is exciting, i didn't know about Kahan sum:
Thanks @EAW, i learned something useful today.

Glad to help!

> In #97685#1372874, @LukasTonne wrote: > Oh this is exciting, i didn't know about Kahan sum: > Thanks @EAW, i learned something useful today. Glad to help!

Gerstmann Bradley commented

2023-01-08 02:30:36 +01:00

I've found another bug about statistical mean and recalled this report. perhaps due to same reason or?
Epsilon is as small as 1e-08.

Compare False.blend

I've found another bug about statistical mean and recalled this report. perhaps due to same reason or? Epsilon is as small as 1e-08. ![image.png](https://archive.blender.org/developer/F14132297/image.png) [Compare False.blend](https://archive.blender.org/developer/F14132300/Compare_False.blend)

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Attribute Statistic Node Outputting Weird Mean. #97685