Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement support for encoding Dolby Vision from RPU file #3250

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

quietvoid
Copy link

@quietvoid quietvoid commented Aug 7, 2023

The changes allow to encode AV1 files with Dolby Vision metadata for playback on supported devices.

As there is no official specification, the minimum I've tested is that the metadata is recognized and used correctly on playback.
The metadata OBU placement is based on #3000 (the AV1 HDR10+ specification), and I've tested it to be working fine with 2 GOPs.

From CLI, the metadata is expected to be passed as a RPU binary file, which follows the same format as encoders like x265.
The metadata parsing/encoding is done through the dolby_vision crate.
The CLI opt can be either --dovi-rpu or --dolby-vision-rpu (used by x265)

From Rust, the metadata must be encoded into the final T.35 and provided for the frames that require it.

For muxing, it's possible to use the GPAC utilities but it currently requires patching for AV1: gpac/gpac#2549
mkvmerge will also have support for raw OBU. IVF has to be added.

@quietvoid quietvoid force-pushed the t35_dovi branch 2 times, most recently from 8cc3d6b to e6d6adb Compare August 8, 2023 00:19
@quietvoid
Copy link
Author

quietvoid commented Aug 8, 2023

There are some issues with the current T35 code..

  • With --keyint 1, some frames are missing metadata because the input frame number is incorrect when fetching the metadata from t35_q.
  • With inter frames, the lookahead causes some frame invariants to be created twice for the same frame number, with the second time having t35_metadata overwritten since the metadata has been removed from t35_q on the first time.

I don't know if the changes I made are any correct but they seem to fix the issues for me.
But the metadata is still incorrectly ordered.

@codecov
Copy link

codecov bot commented Aug 8, 2023

Codecov Report

Patch coverage is 54.54% of modified lines.

Files Changed Coverage
src/bin/common.rs 12.00%
src/bin/rav1e.rs 59.09%
src/encoder.rs 76.92%
src/api/util.rs 90.00%
src/api/internal.rs 100.00%

📢 Thoughts on this report? Let us know!.

@quietvoid
Copy link
Author

There doesn't seem to be an easy way of fixing the T35 metadata without cloning from the queue, and removing later.
The alternative would be breaking the API and fixing the FrameInvariants by setting t35_metadata prior to actually encoding the packet instead of in build_frame_properties.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant