Blender 3.0 consistently crashes when RSS nears 5GiB on Linux (easy to trigger with video playback) #94615

New Issue

George Hawkins · 2022-01-03T20:15:00+01:00

George Hawkins commented

2022-01-03 20:15:00 +01:00

System Information
Operating system: Ubuntu 20.04.3 LTS
Graphics card: GeForce RTX 2060

Blender Version
Broken: 3.0.0, f1cca30557, master, 2021-12-02
Worked: 2.93.7, a5b7f9dc90, master, 2021-12-14

Note: I also downloaded the December 29th 3.0.0 daily build from https://builder.blender.org/download/daily/ but it has the same hash as the official release, the only difference seems to be the build date:

$ ./blender --version
Blender 3.0.0
	build date: 2021-12-29
	build time: 00:34:14
	build commit date: 2021-12-02
	build commit time: 18:35
	build hash: f1cca3055776
	build platform: Linux
	build type: release

Short description of error

Any action that pushes Blender's RSS near the 4.7GiB mark consistently causes Blender to crash on my very generic Ubuntu 20.04.4 LTS system (with all apt updates applied and the latest 470.86 proprietary Nvidia driver version).

E.g. if I play a video file, I can watch the RSS gradually grow (in top or htop) as the frame count rises and then as it reaches around 4.7GiB it crashes.

No such issue is seen if I switch back to a 2.93 version of Blender and do exactly the same thing.

Exact steps for others to reproduce the error

For a given video and a freshly opened copy of Blender, the frame at which it crashes, on playback, is entirely consistent.

E.g. for the 4K Footage.mp4 file here (this is just footage that accompanied this Blender Daily video on YouTube), I can consistently play from frame 1 to frame 129 (using left-arrow to step frame-by-frame as I approach 129) and then Blender crashes as I step onto frame 130.

The frame I can reach varies depending on the file used, e.g. I can get much further in a typical 1080p file. But if I watch what's happening with htop then the failure always seems to occur as the RSS goes just a little above 4700MiB.

E.g. here you can see that Blender's RSS is 580MiB before I started playback:
{F12787266, size=full}

By frame 129, RSS has grown to 4703MiB:
{F12787272, size=full}

Stepping onto the next frame caused it to crash (and it consistently crashes on just that frame). As you can see from the screenshots, the system has plenty of free memory and is not heavily loaded.

If I run Blender with strace then the last few lines I see are:

read(28, "\301i2\325\251\263xu\215\267\206Y\357\207\324\301\4.\347\207M\333\332(RR\207\353\331NT\242"..., 32768) = 32768
futex(0x7f578aed7a04, FUTEX_WAKE_PRIVATE, 1) = 1
madvise(0x7f5766c0a000, 33558528, MADV_DONTNEED) = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x10} ---
write(1, "Writing: /tmp/blender.crash.txt\n", 32Writing: /tmp/blender.crash.txt
) = 32

The full strace output is here: strace-output.txt

The blender.crash.txt is here: blender.crash.txt

Exact steps:

Start Blender and select VFX.
In the middle Movie Clip editor (the one with mode Tracking and view Clip), click Open and select a downloaded copy of the Footage.mp4 file linked to above.
Press Space to start playback and then, as the frame count nears 129, pause and start to step forward one frame at a time with cursor left-arrow.
Everything should go fine up to including frame 129 but on stepping onto frame 130 Blender crashes.

I can reproduce this with any reasonably long video clip - the frame at which Blender crashes depends on the video (4K videos blow up much earlier than 1080p videos) but the frame at which it crashes is consistent for any given video, i.e. I can get it to crash repeatedly at exactly the same point.

**System Information** Operating system: Ubuntu 20.04.3 LTS Graphics card: GeForce RTX 2060 **Blender Version** Broken: 3.0.0, f1cca3055776, master, 2021-12-02 Worked: 2.93.7, a5b7f9dc9011, master, 2021-12-14 Note: I also downloaded the December 29th 3.0.0 *daily* build from <https://builder.blender.org/download/daily/> but it has the same hash as the official release, the only difference seems to be the build date: ``` $ ./blender --version Blender 3.0.0 build date: 2021-12-29 build time: 00:34:14 build commit date: 2021-12-02 build commit time: 18:35 build hash: f1cca3055776 build platform: Linux build type: release ``` **Short description of error** Any action that pushes Blender's [RSS](https://en.wikipedia.org/wiki/Resident_set_size) near the 4.7GiB mark consistently causes Blender to crash on my very generic Ubuntu 20.04.4 LTS system (with all `apt` updates applied and the latest 470.86 proprietary Nvidia driver version). E.g. if I play a video file, I can watch the RSS gradually grow (in `top` or `htop`) as the frame count rises and then as it reaches around 4.7GiB it crashes. No such issue is seen if I switch back to a 2.93 version of Blender and do exactly the same thing. **Exact steps for others to reproduce the error** For a given video and a freshly opened copy of Blender, the frame at which it crashes, on playback, is entirely consistent. E.g. for the 4K `Footage.mp4` file [here](https:*drive.google.com/drive/folders/1WHNNXcz74CZb4kF51ZqYNloraMeQHtJd) (this is just footage that accompanied this Blender Daily [video](https:*www.youtube.com/watch?v=88S6gBFLkdo) on YouTube), I can consistently play from frame 1 to frame 129 (using left-arrow to step frame-by-frame as I approach 129) and then Blender crashes as I step onto frame 130. The frame I can reach varies depending on the file used, e.g. I can get much further in a typical 1080p file. But if I watch what's happening with `htop` then the failure always seems to occur as the RSS goes just a little above 4700MiB. E.g. here you can see that Blender's RSS is 580MiB before I started playback: {[F12787266](https://archive.blender.org/developer/F12787266/Screenshot_from_2022-01-03_19-31-23.png), size=full} By frame 129, RSS has grown to 4703MiB: {[F12787272](https://archive.blender.org/developer/F12787272/Screenshot_from_2022-01-03_19-32-22.png), size=full} Stepping onto the next frame caused it to crash (and it consistently crashes on just that frame). As you can see from the screenshots, the system has plenty of free memory and is not heavily loaded. If I run Blender with `strace` then the last few lines I see are: ``` read(28, "\301i2\325\251\263xu\215\267\206Y\357\207\324\301\4.\347\207M\333\332(RR\207\353\331NT\242"..., 32768) = 32768 futex(0x7f578aed7a04, FUTEX_WAKE_PRIVATE, 1) = 1 madvise(0x7f5766c0a000, 33558528, MADV_DONTNEED) = 0 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x10} --- write(1, "Writing: /tmp/blender.crash.txt\n", 32Writing: /tmp/blender.crash.txt ) = 32 ``` The full `strace` output is here: [strace-output.txt](https://archive.blender.org/developer/F12787281/strace-output.txt) The `blender.crash.txt` is here: [blender.crash.txt](https://archive.blender.org/developer/F12787287/blender.crash.txt) Exact steps: * Start Blender and select VFX. * In the middle *Movie Clip* editor (the one with mode *Tracking* and view *Clip*), click *Open* and select a downloaded copy of the `Footage.mp4` file linked to above. * Press *Space* to start playback and then, as the frame count nears 129, pause and start to step forward one frame at a time with cursor left-arrow. * Everything should go fine up to including frame 129 but on stepping onto frame 130 Blender crashes. I can reproduce this with any reasonably long video clip - the frame at which Blender crashes depends on the video (4K videos blow up much earlier than 1080p videos) but the frame at which it crashes is consistent for any given video, i.e. I can get it to crash repeatedly at exactly the same point.

George Hawkins commented

2022-01-03 20:15:00 +01:00

Added subscriber: @GeorgeHawkins

George Hawkins commented

2022-01-03 20:25:58 +01:00

I just tried the latest 3.1 alpha for Linux - blender-3.1.0-alpha+master.ea8d749587dd-linux.x86_64-release - and I no longer see this issue. So, I guess it has probably already been covered by a previous bug report.

Hopefully, whatever the underlying fix is, it will soon appear in the next stable version, i.e. 3.0.1. As noted in my original bug report it is not fixed in the latest 3.0.0 daily build (however, this daily seems to have exactly the same hash as the Dec 2nd 3.0.0 stable release, i.e. it's got no additional commits over the official 3.0.0 release).

Update: I spoke too soon, see next comment.

I just tried the latest 3.1 alpha for Linux - blender-3.1.0-alpha+master.ea8d749587dd-linux.x86_64-release - and I no longer see this issue. So, I guess it has probably already been covered by a previous bug report. Hopefully, whatever the underlying fix is, it will soon appear in the next stable version, i.e. 3.0.1. As noted in my original bug report it is not fixed in the latest 3.0.0 daily build (however, this daily seems to have exactly the same hash as the Dec 2nd 3.0.0 stable release, i.e. it's got no additional commits over the official 3.0.0 release). **Update:** I spoke too soon, see next comment.

George Hawkins commented

2022-01-03 20:41:32 +01:00

Interesting - neither 2.93.7 and 3.1.0-alpha+master.ea8d749587dd crash on playing back Footage.mp4 but they both hang on the frame where 3.0.0 crashes if I try to track - ctrl-T - to that frame:

{F12787346, size=full}

Here's the .blend file (you'll also need the Footage.mp4 file linked to in the original bug report): blender-daily-2.93.7.blend

Interesting - neither 2.93.7 and 3.1.0-alpha+master.ea8d749587dd crash on playing back `Footage.mp4` but they both hang on the frame where 3.0.0 crashes if I try to track - `ctrl-T` - to that frame: {[F12787346](https://archive.blender.org/developer/F12787346/Screenshot_from_2022-01-03_20-32-50.png), size=full} Here's the `.blend` file (you'll also need the `Footage.mp4` file linked to in the original bug report): [blender-daily-2.93.7.blend](https://archive.blender.org/developer/F12787350/blender-daily-2.93.7.blend)

Jesse Yurkovich commented

2022-01-03 23:02:03 +01:00

Added subscriber: @deadpin

Jesse Yurkovich commented

2022-01-03 23:02:03 +01:00

The MEM_CacheLimiter_unmanage crash as seen in your crash file was fixed by 7061d1e39f and is already on the list of patches to include in 3.0.1 if/when that happens (#93479)

Beyond that, I'm unable to reproduce the hang here locally on 3.1. I was able to complete the track for all frames. If it's reliable for you it would be best to create a new report, just for the hang, so things are clear for other folks looking through the bugs.

The `MEM_CacheLimiter_unmanage` crash as seen in your crash file was fixed by 7061d1e39f and is already on the list of patches to include in 3.0.1 if/when that happens (#93479) Beyond that, I'm unable to reproduce the hang here locally on 3.1. I was able to complete the track for all frames. If it's reliable for you it would be best to create a new report, just for the hang, so things are clear for other folks looking through the bugs.

George Hawkins commented

2022-01-04 09:16:51 +01:00

Please consider this bug closed from my perspective (I don't see any way for me to close it directly).

Thanks for the quick response and apologies for not being better at analyzing blender.crash.txt to see that it was already covered by another bug.

I agree that it makes sense to split out the tracking issue - will look into it more and see if there really is an issue and if I can clearly summarize it in a new bug.

Please consider this bug closed from my perspective (I don't see any way for me to close it directly). Thanks for the quick response and apologies for not being better at analyzing `blender.crash.txt` to see that it was already covered by another bug. I agree that it makes sense to split out the tracking issue - will look into it more and see if there really is an issue and if I can clearly summarize it in a new bug.

Jesse Yurkovich commented

2022-01-04 11:06:46 +01:00

Changed status from 'Needs Triage' to: 'Resolved'

Jesse Yurkovich closed this issue

2022-01-04 11:06:46 +01:00

Jesse Yurkovich self-assigned this 2022-01-04 11:06:46 +01:00

Jesse Yurkovich commented

2022-01-04 11:06:46 +01:00

No worries, it would have been a bit of a pain to find the already-fixed issue really. I'll resolve this task but if you can reliably reproduce the hang, please do file a new report.

George Hawkins commented

2022-01-05 20:29:53 +01:00

Thanks, Jesse. The "hang" was actually the result of hitting the video-sequencer cache's default limit of 4GiB. When I upped it to 16GiB the track ran through to the finish.

I'm not quite sure why tracking through a video file (or simply playing it) should cause Blender to fill up and then grind to a halt once you've read thru e.g. 129 frames of 4K video in my case.

Surely the point of a cache is that you start throwing away the older data (i.e. the earlier frames) once full and that, ideally, ejection from the cache shouldn't be really expensive?

But at the moment, once the cache is full that seems to be Blender pretty much done for - it will grind on from that point but extremely slowly (but with htop showing very low CPU usage - so it's not maxing out there).

Thanks, Jesse. The "hang" was actually the result of hitting the video-sequencer cache's default limit of 4GiB. When I upped it to 16GiB the track ran through to the finish. I'm not quite sure why tracking through a video file (or simply playing it) should cause Blender to fill up and then grind to a halt once you've read thru e.g. 129 frames of 4K video in my case. Surely the point of a cache is that you start throwing away the older data (i.e. the earlier frames) once full and that, ideally, ejection from the cache shouldn't be really expensive? But at the moment, once the cache is full that seems to be Blender pretty much done for - it will grind on from that point but extremely slowly (but with `htop` showing very low CPU usage - so it's not maxing out there).

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Blender 3.0 consistently crashes when RSS nears 5GiB on Linux (easy to trigger with video playback) #94615