Page MenuHome

cuda out of memory error after exactly 669 frames
Closed, ResolvedPublic

Description

System Information
Operating system: Windows-10-10.0.18362 64 Bits
Graphics card: 2x GeForce GTX 1080/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 441.20 + 1 x GeForce GTX 1070/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 441.20

Blender Version
Broken: version: 2.81 (sub 16), branch: unknown, commit date: 1970-01-01 00:00, hash: rBunknown
Worked: This is E-Cycles. But the same problem occurs with the normal 2.81

Short description of error
after exactly 669 frames I get a error cuda out of memory

Exact steps for others to reproduce the error
This is a huge file with a lot of geometry that is bought from other firms.

So please don't spread.

Event Timeline

It sounds like you have run out of GPU memory.

yes of course but why only after 669 frames? Moreover, I have 8 gigbyte graphic cards and the peak memory is only 3084 m!

Jacques Lucke (JacquesLucke) changed the task status from Needs Triage to Needs Information from User.Dec 28 2019, 3:10 PM

What happens when you start rendering at a later frame?
What happens when you only render the frame that was crashing?

What happens when you start rendering at a later frame? The same if I for example start randering at frame 100 it stops at frame 769.
What happens when you only render the frame that was crashing? First when Blender stops rendering I have to restart Blender otherwise it won't render. So
after restarting it renders normally.
When I do render in eevee wich I normally don't it renders the whole sequence.

Just got some more questions that might help to track down the issue.

  1. Does it work fine on CPU? Do you see a continuous increase in RAM usage over all frames?
  2. When you simplify the scene (e.g. remove a texture), does it render for more frames?
  3. Does the same happen when you only render with one sample and a small resolution?
  1. It takes to long to render on CPU. I render on 3 x 1080gtx cards there is no increase in ram usage for as far I can tell.
  2. No.
  3. I did not tested that. But will do so. Then I will report when I did that.
  1. I did not tested that. But will do so. Then I will report when I did that.

Did that give any insights?

I did a lot of testing the last days. Different scenes and different amounts of polygons and textures. It always happens when the amount of polygons and textures are large but not so big as it stops immediately. Cycles stops rendering almost every time around the 650 frames. Maybe it is the biggest problem in the world but it is strange!

Paul Kolbrink (XYZ-Animation) changed the visibility from "Public (No Login Required)" to "Paul Kolbrink (XYZ-Animation)".Jan 12 2020, 10:14 AM
Policy changed the edit policy from "Paul Kolbrink (XYZ-Animation)" to "Sergey Sharybin (sergey)".
Sergey Sharybin (sergey) changed the visibility from "Sergey Sharybin (sergey)" to "Public (No Login Required)".Thu, Feb 6, 5:05 PM

We can only fix issues we can reproduce. Waiting 100s of frames to be rendered is not something we can afford as a bug fixing team.

Please:

  • Simplify file as much as possible
  • Render with as few samples as possible
  • Lower resolution to lowest possible value

Odds are that this is something up to how driver manages VRAM, which is not in control of Blender. With simple enough file you can easily see if it's indeed something driver related.

P.S. Please do not modify task policy.

Aaron Carlisle (Blendify) claimed this task.

No activity for more than a week. As per the tracker policy we assume the issue is gone and can be closed.

Thanks again for the report. If the problem persists please open a new report with the required information.