VSE: Block artefacts in WEBM video on preview or export #91405

New Issue

forceengine · 2021-09-14T14:57:30+02:00

forceengine commented

2021-09-14 14:57:30 +02:00

System Information
Operating system: Windows-10-10.0.19041-SP0 64 Bits
Graphics card: Radeon RX 5500M ATI Technologies Inc. 4.5.13587 Core Profile Context 19.40.30.02 26.20.14030.2002

Blender Version
Broken: version: 2.93.2
Worked: blender-2.93.1

Short description of error
When I import a WEBM video (Codec: Google/On2's VP9 Video (VP90) according to VLC) I downloaded from YouTube in the WEBM container I get block artefacts in the video (in the render as well as in the preview window of the sequence editor)

This is how rendered image and video files look:

Exact steps for others to reproduce the error
Add following video to VSE
F10551482

**System Information** Operating system: Windows-10-10.0.19041-SP0 64 Bits Graphics card: Radeon RX 5500M ATI Technologies Inc. 4.5.13587 Core Profile Context 19.40.30.02 26.20.14030.2002 **Blender Version** Broken: version: 2.93.2 Worked: blender-2.93.1 **Short description of error** When I import a WEBM video (Codec: Google/On2's VP9 Video (VP90) according to VLC) I downloaded from YouTube in the WEBM container I get block artefacts in the video (in the render as well as in the preview window of the sequence editor) This is how rendered image and video files look: ![blender 2.93.4 bug webm block artefacts 1.png](https://archive.blender.org/developer/F10412013/blender_2.93.4_bug_webm_block_artefacts_1.png) ![blender 2.93.4 bug webm block artefacts 2.png](https://archive.blender.org/developer/F10412014/blender_2.93.4_bug_webm_block_artefacts_2.png) **Exact steps for others to reproduce the error** Add following video to VSE [F10551482](https://archive.blender.org/developer/F10551482/About_that_Discord_server....webm)

forceengine commented

2021-09-14 14:57:30 +02:00

Added subscriber: @forceengine

blender-admin commented

2021-09-14 14:57:30 +02:00

#92349 was marked as duplicate of this issue

forceengine commented

2021-09-14 15:47:47 +02:00

I tested this on a live Zorin OS 16 (64bit, Windowing System X11) from a USB stick on 2.93.4 and 3.0.0-alpha and I got the same results.

System Information
Operating system: Linux-5.11.0-25-generic-x86_64-with-glibc2.31 64 Bits
Graphics card: AMD RENOIR (DRM 3.40.0, 5.11.0-25-generic, LLVM 12.0.0) AMD 4.6 (Core Profile) Mesa 21.0.3

Blender Versions
I tested this on the following versions:

2.93.4, branch: master, commit date: 2021-08-31 09:23, hash: b7205031ce
3.0.0 Alpha, branch: master, commit date: 2021-09-13 23:08, hash: 9b0b78d58f

Additional notes
I thought that maybe Windows or JDownloader may have messed up the video so I used "Video Downloader" from the Zorin OS store (https://github.com/Unrud/video-downloader) to download the file. I got the same results.

I tested this on a live Zorin OS 16 (64bit, Windowing System X11) from a USB stick on 2.93.4 and 3.0.0-alpha and I got the same results. **System Information** Operating system: Linux-5.11.0-25-generic-x86_64-with-glibc2.31 64 Bits Graphics card: AMD RENOIR (DRM 3.40.0, 5.11.0-25-generic, LLVM 12.0.0) AMD 4.6 (Core Profile) Mesa 21.0.3 **Blender Versions** I tested this on the following versions: - 2.93.4, branch: master, commit date: 2021-08-31 09:23, hash: `b7205031ce` - 3.0.0 Alpha, branch: master, commit date: 2021-09-13 23:08, hash: `9b0b78d58f` **Additional notes** I thought that maybe Windows or JDownloader may have messed up the video so I used "Video Downloader" from the Zorin OS store (https://github.com/Unrud/video-downloader) to download the file. I got the same results.

forceengine commented

2021-09-15 18:56:16 +02:00

Some more testing:

I converted the MP4 version of the original video file (displayed in the original screenshots) to WEBM/VP9 using Handbrake 1.4.1 (2021081500). There was no problem
I tried to use a different WEBM file with VP9 from Wikimedia Commons (I just used the media of the day: https://commons.wikimedia.org/wiki/File:When_the_Clouds_Roll_By_(1919).webm). There was no problem
In a project file I had a problem with the WEBM container with VP9 where the VSE didn't even start to render the scene (but it worked fine with MP4 and h.264) I tested if a default scene with WEBM + VP9 causes any problems. There was no problem
I tested if the WEBM + VP9 file from one line above causes any problems. There was no problem

(I will update this list when I had more ideas for further testing)

Some more testing: - I converted the MP4 version of the original video file (displayed in the original screenshots) to WEBM/VP9 using Handbrake 1.4.1 (2021081500). There was **no problem** - I tried to use a different WEBM file with VP9 from Wikimedia Commons (I just used the media of the day: https://commons.wikimedia.org/wiki/File:When_the_Clouds_Roll_By_(1919).webm). There was **no problem** - In a project file I had a problem with the WEBM container with VP9 where the VSE didn't even start to render the scene (but it worked fine with MP4 and h.264) I tested if a default scene with WEBM + VP9 causes any problems. There was **no problem** - I tested if the WEBM + VP9 file from one line above causes any problems. There was **no problem** (I will update this list when I had more ideas for further testing)

Richard Antalik commented

2021-09-21 21:48:32 +02:00

Added subscriber: @iss

Richard Antalik commented

2021-09-21 21:48:32 +02:00

Changed status from 'Needs Triage' to: 'Needs User Info'

Richard Antalik commented

2021-09-21 21:48:32 +02:00

Thanks for report, can you upload broken webm file directly to this site? Though file should be ideally free to distribute like this one https://www.youtube.com/watch?v=fxNlpQYRz7s

Mp4 file you uploaded here causes crash when added to VSE and in VLC it causes same artifacts as shown in pictures. Not sure what this file is exactly.

Thanks for report, can you upload broken webm file directly to this site? Though file should be ideally free to distribute like this one https://www.youtube.com/watch?v=fxNlpQYRz7s Mp4 file you uploaded here causes crash when added to VSE and in VLC it causes same artifacts as shown in pictures. Not sure what this file is exactly.

forceengine commented

2021-09-22 22:33:06 +02:00

In #91405#1223283, @iss wrote:
Thanks for report, can you upload broken webm file directly to this site? Though file should be ideally free to distribute like this one https://www.youtube.com/watch?v=fxNlpQYRz7s

I can upload the source file here that causes the problem, but since I'm not the original author nor is it freely licenced, I would only upload the WEBM-file to this webpage if you say so. Keep in mind though that it's just the WEBM-file I download directly from YouTube.
Nevertheless it shouldn't be hard to get the permission by the original author since he is on twitter: https://twitter.com/thelinuxEXP
Maybe he could be asked via the official blender twitter account

Mp4 file you uploaded here causes crash when added to VSE and in VLC it causes same artifacts as shown in pictures. Not sure what this file is exactly.

The video file I uploaded is the rendered end result of the WEBM file from YouTube.
The process was: Download video from YouTube -> Import video to blender -> render result
This video is the very last step in bold. The original WEBM video was added to the VSE and rendered out.

Judging from the preview, the artifact problems usually get worse towards the end as far as I have seen (but this also might be because of the proxy that clears the artifact problem for the viewport).

I took the "Blender 2.93 LTS - Features Reel Showcase" video, downloaded it as WEBM file and rendered it out in blender with the same settings as the innitial video (so MP4 container, with H.264 with default settings and no sound). I get the same atrifact results, but less often . I only see artifacts when the scene content changes (eg. 00:18, 00:29 - see examples below). The video described in the original report has much more of these artifacts, but is also 60fps The demo reel is only 24fps. Since the artifacts appear in this video when the content changes it may is related to bandwidth/throughput/kbps.

Blender 2.93 Features Reel: Downloaded as WEBM from YouTube and rendered to MP4 in blender with no changes. Also note the strange end credits at the very end.
F10549431

Screenshots of said file from above at 00:18

and 00:29

Right now I'm downloading the video "Walking in LINDAU / Germany 🇩🇪- 4K 60fps (UHD)" as WEBM from YouTube (https://www.youtube.com/watch?v=Ootn6VUuhtw) in 4K as well as 1080p. It's a 60fps video licenced under CC-BY. The idea is to test another 60fps file and compare 1080p to 4K. If the bug is related to bandwidth/throughput, there could be a visible difference.

But since both files (1080p and 4K) are about 8.5GB in size it will take a bit of time. I will update the edit once I have the files.

EDIT 1: Thinking and rewatching the video I thought maybe it could also be related to keyframes. I have no idea when, why and how WEBM (with VP90) uses keyframes. But drastic changes in the image could be a good place for one. Maybe it's the keyframe itself? Or an unexpected one that does not follow the set rules?

> In #91405#1223283, @iss wrote: > Thanks for report, can you upload broken webm file directly to this site? Though file should be ideally free to distribute like this one https://www.youtube.com/watch?v=fxNlpQYRz7s I can upload the source file here that causes the problem, but since I'm not the original author nor is it freely licenced, I would only upload the WEBM-file to this webpage if you say so. Keep in mind though that it's just the WEBM-file I download directly from YouTube. Nevertheless it shouldn't be hard to get the permission by the original author since he is on twitter: https://twitter.com/thelinuxEXP Maybe he could be asked via the official blender twitter account > Mp4 file you uploaded here causes crash when added to VSE and in VLC it causes same artifacts as shown in pictures. Not sure what this file is exactly. The video file I uploaded is the rendered end result of the WEBM file from YouTube. The process was: Download video from YouTube -> Import video to blender -> **render result** This video is the very last step in bold. The original WEBM video was added to the VSE and rendered out. Judging from the preview, the artifact problems usually get worse towards the end as far as I have seen (but this also might be because of the proxy that clears the artifact problem for the viewport). ---- I took the "Blender 2.93 LTS - Features Reel Showcase" video, downloaded it as WEBM file and rendered it out in blender with the same settings as the innitial video (so MP4 container, with H.264 with default settings and no sound). I get the same atrifact results, but *less often* . I only see artifacts when the scene content changes (eg. 00:18, 00:29 - see examples below). The video described in the original report has much more of these artifacts, but is also 60fps The demo reel is only 24fps. Since the artifacts appear in this video when the content changes it may is related to bandwidth/throughput/kbps. Blender 2.93 Features Reel: Downloaded as WEBM from YouTube and rendered to MP4 in blender with no changes. Also note the strange end credits at the very end. [F10549431](https://archive.blender.org/developer/F10549431/Blender_2.93_Features_Reel-WEBM_to_MP4.mp4) Screenshots of said file from above at 00:18 ![Blender 2.93 Features Reel-WEBM to MP4-02.png](https://archive.blender.org/developer/F10549651/Blender_2.93_Features_Reel-WEBM_to_MP4-02.png) and 00:29 ![Blender 2.93 Features Reel-WEBM to MP4-01.png](https://archive.blender.org/developer/F10549657/Blender_2.93_Features_Reel-WEBM_to_MP4-01.png) Right now I'm downloading the video "Walking in LINDAU / Germany 🇩🇪- 4K 60fps (UHD)" as WEBM from YouTube (https://www.youtube.com/watch?v=Ootn6VUuhtw) in 4K as well as 1080p. It's a 60fps video licenced under CC-BY. The idea is to test another 60fps file and compare 1080p to 4K. If the bug is related to bandwidth/throughput, there could be a visible difference. But since both files (1080p and 4K) are about 8.5GB in size it will take a bit of time. I will update the edit once I have the files. EDIT 1: Thinking and rewatching the video I thought maybe it could also be related to keyframes. I have no idea when, why and how WEBM (with VP90) uses keyframes. But drastic changes in the image could be a good place for one. Maybe it's the keyframe itself? Or an unexpected one that does not follow the set rules?

Richard Antalik commented

2021-09-22 23:25:29 +02:00

I asked you to upload webm, because I don't really want to install random software to do this step myself. In this case I would be debugging the issue, but in other cases any number of devs may want to check the issue ant they all would have to install something. That is unreasonable in my opinion.

forceengine commented

2021-09-23 00:55:56 +02:00

In #91405#1224124, @iss wrote:
I asked you to upload webm, because I don't really want to install random software to do this step myself. In this case I would be debugging the issue, but in other cases any number of devs may want to check the issue ant they all would have to install something. That is unreasonable in my opinion.

Well then, here is the WEBM file I downloaded from YouTube:
About that Discord server....webm

> In #91405#1224124, @iss wrote: > I asked you to upload webm, because I don't really want to install random software to do this step myself. In this case I would be debugging the issue, but in other cases any number of devs may want to check the issue ant they all would have to install something. That is unreasonable in my opinion. Well then, here is the WEBM file I downloaded from YouTube: [About that Discord server....webm](https://archive.blender.org/developer/F10551482/About_that_Discord_server....webm)

Richard Antalik commented

2021-09-23 01:26:14 +02:00

Changed status from 'Needs User Info' to: 'Confirmed'

Richard Antalik commented

2021-09-23 01:26:14 +02:00

Thanks, can confirm the issue.

Evan Wilson commented

2021-09-23 04:27:20 +02:00

Added subscriber: @EAW

Evan Wilson commented

2021-09-23 04:27:20 +02:00

I can confirm that 2.93.1 is ok, but 2.93.2 has the artifacts.

Richard Antalik commented

2021-10-08 08:36:33 +02:00

Added subscriber: @ZedDB

Richard Antalik commented

2021-10-08 08:36:33 +02:00

Checked this in detail, and this issue doesn't happen when blender is forced to use 2 or less threads. I don't know why exactly this happens though.

@ZedDB can you reproduce this on Linux? Can you test this with newer libs than we ship? I am not able to make my set of libs for some reason.

Checked this in detail, and this issue doesn't happen when blender is forced to use 2 or less threads. I don't know why exactly this happens though. @ZedDB can you reproduce this on Linux? Can you test this with newer libs than we ship? I am not able to make my set of libs for some reason.

Richard Antalik commented

2021-10-11 10:38:45 +02:00

Just for the record, the issue occurs in Linux builds too. I wasn't able to reproduce with ffplay, so likely cause is, that we fail to initialize decoder to it's default values.

Richard Antalik commented

2021-11-02 06:16:27 +01:00

Did more in depth digging, and found, that issue started with 8d6264ea12. This commit did not change thread count, it really is just API change.

Then while looking at log, ffmpeg reports File is broken, keyframes not correctly marked!. This is error from demuxer, it doesn't really make sense why that would be affected by thread count used by decoder. Possibly decoder could return frame with incorrect data we use to seek? Don't think this file is broken though.

Finally similar to limiting thread count, using FF_THREAD_SLICE threading method does also fix this issue. I assume that only 1 thread is used then.

Transcoding to proxy works also well and it uses maximum thread count and FF_THREAD_FRAME method (exactly same settings as playback). This issue happens when playing video frame by frame so this is not really seek bug. So as an experiment I have created loop to decode all frames, which works correctly with either settings. This must mean, that issue is in ffmpeg_decode_video_frame. Will slowly iterate from code that works to one that doesn't work and see if I can isolate a problem.

Did more in depth digging, and found, that issue started with 8d6264ea12. This commit did not change thread count, it really is just API change. Then while looking at log, ffmpeg reports `File is broken, keyframes not correctly marked!`. This is error from demuxer, it doesn't really make sense why that would be affected by thread count used by decoder. Possibly decoder could return frame with incorrect data we use to seek? Don't think this file is broken though. Finally similar to limiting thread count, using `FF_THREAD_SLICE` threading method does also fix this issue. I assume that only 1 thread is used then. Transcoding to proxy works also well and it uses maximum thread count and `FF_THREAD_FRAME` method (exactly same settings as playback). This issue happens when playing video frame by frame so this is not really seek bug. So as an experiment I have created loop to decode all frames, which works correctly with either settings. This must mean, that issue is in `ffmpeg_decode_video_frame`. Will slowly iterate from code that works to one that doesn't work and see if I can isolate a problem.

Richard Antalik commented

2021-11-02 07:13:49 +01:00

I found the problem: Issue is, that we are using new FFmpeg API incorrectly, however this was kinda concious decision, see https://developer.blender.org/D10338#inline-86120.

I think this would be good to resolve for 3.0 still so not sure if I should lower the priority. Since these commits are backported to 2.93 that maybe too, but it may be mistake that these were backported in first place.

I found the problem: Issue is, that we are using new FFmpeg API incorrectly, however this was kinda concious decision, see https://developer.blender.org/D10338#inline-86120. I think this would be good to resolve for 3.0 still so not sure if I should lower the priority. Since these commits are backported to 2.93 that maybe too, but it may be mistake that these were backported in first place.

blender-admin commented

2021-11-15 20:20:33 +01:00

This issue was referenced by d3c45e1c39

This issue was referenced by d3c45e1c391eaafd9c54dba5b574b1a7ee23c82a

Richard Antalik commented

2021-11-15 20:33:19 +01:00

Changed status from 'Confirmed' to: 'Resolved'

Richard Antalik closed this issue

2021-11-15 20:33:19 +01:00

Richard Antalik self-assigned this 2021-11-15 20:33:19 +01:00

Richard Antalik commented

2021-11-17 01:46:15 +01:00

Added subscribers: @gudvinr, @mano-wii

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

VSE: Block artefacts in WEBM video on preview or export #91405