Page MenuHome

Write metadata to video output file
Needs ReviewPublic

Authored by Sybren A. Stüvel (sybren) on Oct 5 2016, 4:34 PM.

Details

Summary

For production work (like we do now for Agent 327) it is very useful to have metadata in preview videos. This allows us to:

  • for a shot, compute the frame range(s) *in the original blend file* that will be visible in the final edit.
  • in the edit, view the shot name (as the blendfile name that rendered the video is the most authoritative source of this information).
  • directly from the edit (i.e. sequence editor), open a shot blendfile in a new Blender.
  • possibly in the future: given a frame of a previs video, open the file in Blender at that frame (which can be taken further, and, for example, provide a list of characters currently visible).

To accomplish this. this patch writes the following metadata to a video output file to make this possible:

  • base filename of blendfile (so no directory)
  • scene name
  • start frame
  • end frame

Diff Detail

Repository
rB Blender
Branch
temp-sybren-ffmpeg-metadata
Build Status
Buildable 1390
Build 1390: arc lint + arc unit

Event Timeline

Sybren A. Stüvel (sybren) retitled this revision from to Write metadata to video output file.Oct 5 2016, 4:34 PM
Sybren A. Stüvel (sybren) updated this object.
Sybren A. Stüvel (sybren) set the repository for this revision to rB Blender.
Sybren A. Stüvel (sybren) updated this object.
Sybren A. Stüvel (sybren) updated this revision to Diff 7549.

Why not couple this with the 'stamp' feature? Those are pretty much same info, just two different ways to store them in render, could be nice to have a way to combine them imho (not totally sure how though, UI wise).

The "stamp metadata" mostly contains time-changing information, like the current camera, frame, render time for the frame, etc. The only thing that probably doesn't change is the scene being rendered (although this is also debatable for VSE renders).

Of course we could add a checkbox to enable this type of metadata to the Metadata panel. However, it would be highly dependent on the chosen output format (currently it only works with FFmpeg video, and as far as I've seen only with MKV files). I'm not a big fan of having such inter-panel dependencies in the UI.

Sybren A. Stüvel (sybren) updated this object.
Sybren A. Stüvel (sybren) updated this revision to Diff 7550.
  • Added loading of metadata from movie file into strip ID property 'metadata'.

Metadata keys are written in all-caps and with more explicit names.

The metadata keys are now in all-caps to be compliant with the Matroska spec for tags.

One thing that I would like to discuss is the storage location of the metadata. At the moment I'm simply adding an ID property "metadata" to the sequence strip. This allows a user to inspect the data in the custom properties panel, and possibly even delete it. Alternatively I could add a struct IDProperty *metadata property to the Sequence class and add this metadata to the UI in a different spot.

Note that interpretation of the metadata is left to add-ons; everything is left as a string. We could even go as far as to read all metadata, and not just the tags starting with "BLENDER:".

Sergey Sharybin (sergey) requested changes to this revision.

Some quick feedback.

First of all, it is fully weird to have read of metadata only in sequencer. This should be more generic on anim_movie.c level, so other areas can also access and show this data. Such behavior was actually requested during Gooseberry days to make movies behaves same way as images to display metadata.

Second of all, it is like adding yet-another-implementation of metadata specifically for video files. This is going to be quite annoying to maintain and keep it all in sync with existing Metadata panel. Decoupling video metadata from existing interface settings also leads to fully non-intuitive behavior.

Third of all, doing everything hardcoded is not really a great way to go. Majority of the existing options can be re-used here. Others can be re-grouped in a way that it's clear whether they have affect on video file output or not.

It is also worrying that it's build around matroska. What about other containers? If they are not supported it should be clear in the interface what;s going on. Or at least show a warning when starting rendering.

This revision now requires changes to proceed.Oct 13 2016, 10:44 AM

First of all, it is fully weird to have read of metadata only in sequencer. This should be more generic on anim_movie.c level, so other areas can also access and show this data. Such behavior was actually requested during Gooseberry days to make movies behaves same way as images to display metadata.

I can see about merging metadata code and sharing it between video & images, and moving it to anim_movie.c

Second of all, it is like adding yet-another-implementation of metadata specifically for video files. This is going to be quite annoying to maintain and keep it all in sync with existing Metadata panel. Decoupling video metadata from existing interface settings also leads to fully non-intuitive behavior.

Of course I can add some options to the metadata panel. However, also see my notes about that above. In short, if possible I'd rather not add any GUI options at all, and just have Blender write the metadata to the video files.

Third of all, doing everything hardcoded is not really a great way to go. Majority of the existing options can be re-used here. Others can be re-grouped in a way that it's clear whether they have affect on video file output or not.

Which existing options are you thinking about?

It is also worrying that it's build around matroska. What about other containers? If they are not supported it should be clear in the interface what;s going on. Or at least show a warning when starting rendering.

It's a side-effect of using the metadata API offered by FFmpeg. It only seems to work with free-form tags for Matroska files. FFmpeg also has support for writing XMP metadata to MOV files, but that then doesn't properly work with neither Matroska nor MP4, and is more difficult to write & parse.

Sybren A. Stüvel (sybren) edited edge metadata.EditedApr 16 2017, 5:47 PM
Sybren A. Stüvel (sybren) edited the summary of this revision. (Show Details)
Sybren A. Stüvel (sybren) updated this revision to Diff 8651.

Major update in which I've removed the custom metadata writing stuff, and expanded the existing metadata stamping. This made it possible to reuse the metadata stamp UI and metadata formatting code.

To discuss: are the files in render/ the correct place to call BKE_static_stamp_info(scene) and create the StampData struct? This is consistent with the calls to BKE_render_result_stamp_info(...), which is why I made it in this way. However, since only the scene pointer is necessary, we could also drop that consistency, which in turn allows us to avoid the extra parameter to start_movie(), and call BKE_static_stamp_info(scene) only where it is needed.

Sybren A. Stüvel (sybren) edited the summary of this revision. (Show Details)EditedApr 5 2018, 11:47 AM
Sybren A. Stüvel (sybren) updated this revision to Diff 10334.

Reading & writing updated:

  • Metadata handling is now separate from ImBuf *, allowing it to be used with a generic IDProperty *.
  • Merged IMB_metadata_add_field() and IMB_metadata_change_field() into a more robust IMB_metadata_set_field(). This new function doesn't return any status (it now always succeeds, and the previously existing return value was never checked anyway).
  • Removed IMB_metadata_del_field() as it was never actually used anywhere.
  • Use IMB_metadata_ensure() instead of having IMB_metadata_set_field() create the containing IDProperty for you.
  • Deduplicated function declarations, moved intern/IMB_metadata.h out of intern/. Note that this does mean that we have some extra #include "IMB_metadata.h" lines now, as the metadata functions are no longer declared in IMB_imbuf.h.
  • Moved BKE_static_stamp_info() into BKE_ffmpeg_start(), as this is the only writer that seems to support metadata (contrary to AVI RAW, AVI JPEG, and framebuffer).
  • Loading metadata into metadata RNA property for dict-like behaviour in Python. This has been implemented for both MovieSequence and MovieClip types.
  • Flipped a boolean parameter to not be negative.

Implemented Movie{Clip,Sequence}.metadata() as RNA function call.

For this I also implemented a little wrapper code to allow RNA
functions to return an IDProperty pointer as PointerRNA.

  • FIXUP reverted unnecessary whitespace edits
Campbell Barton (campbellbarton) added inline comments.
source/blender/makesrna/intern/rna_movieclip.c
114

Should be NULL checked too.

source/blender/makesrna/intern/rna_sequencer.c
599

This should probably be checked for NULL and return PointerRNA_NULL too.

Sybren A. Stüvel (sybren) marked 2 inline comments as done.Apr 5 2018, 3:51 PM