The IMF UG held a hybrid plugfest on December 13 and 14, 2023, with physical locations in Los Angeles (USA) and Cologne (Germany), and remote participation through shared cloud storage, virtual breakout sessions and instant messaging. The plugfest, the first since February 2020, gathered more than 50 participants from the content creator, user, and implementer communities.

Over 2 days, participants created, exchanged, and consumed IMF content. Cross-checking IMF content between implementations ensures the standard is supported and helps reveal potential improvements to both implementations and the standard.

This report summarizes the use cases studied during the plugfest.

Lossless conversion from MOV to IMF

IMF Application ProRes (SMPTE RDD 45) enables lossless conversion of existing ProRes QuickTime files to IMF. For this use case, the ProRes video contained in MOV files was rewrapped into IMF Compositions without any decoding and re-encoding, demonstrating lossless conversion and eliminating generation losses.

Use case and source content provided by BBC.

Deduplication

Deduplication refers to the reuse of identical essence in multi-version titles, also known as “versioning”. A typical ProRes workflow produces one flat MOV file for every version of a title, even if only a few frames differ between those versions. For this use case, five MOV files were provided: one was identified as the source version and the others as derived versions with different inserts, cuts and pre-rolls. From these five flat files, five IMF composition playlists, each corresponding to one of the versions, and a collection of video components were created. One of the components contained all the frames of the source version while the others contained only inserts and pre-rolls. By allowing portions of the source version components to be reused across composition playlists, IMF achieved a 75% reduction in storage compared to the five flat files.

Use case and source content provided by RTL.

Immersive Audio Bitstream (IAB)

IAB is a mature format for encoding immersive sound fields as a collection of audio channels and audio objects. This use case involved successful QC and playback of an IMF composition whose audio essence consisted solely of IAB essence and did not contain multichannel audio essence.

Use case and source content provided by Sony Pictures Entertainment.

Audio Definition Model (ADM)

S-ADM (Serialized Audio Definition Model) and ADM (Audio Definition Model) are recent ITU-R Recommendations for interactive and immersive audio applications associating metadata with audio essence. The use case involved creating IMF compositions starting from audio essence with S-ADM and ADM metadata wrapped in MXF files.

Use case and source content provided by Fraunhofer IIS.

High Throughput JPEG 2000 (HTJ2K)

HTJ2K significantly improves the encoding and decoding speed of JPEG 2000, reducing processing costs and allowing real time playback on consumer desktops and laptops. This use case involved QC and playback of compositions that used HTJ2K-encoded video.

Use case and source content provided by Warner Bros Discovery, Sony Pictures Entertainment, NBC Universal, and Walt Disney Studios.

ISXD

Carrying metadata as a separate component allows its addition, modification, and removal without having to modify associated essence components. This use case involved QC and playback of a composition that contained Dolby Vision metadata stored in a separate track file conforming to the Isochronous Stream of XML Documents (ISXD) plug-in.

Use case and source content provided by Walt Disney Studios.

Stereoscopic 3D

Stereoscopic 3D video has been a core, but infrequently exercised, capability of IMF since its inception. This use case involved QC and playback of a Stereoscopic 3D composition.

Use case and source content provided by Walt Disney Studios.

Lossless To Lossy Transcode

IMF Application 2E supports both lossless and lossy JPEG 2000 video essence. This use case involved the transcoding of lossless JPEG 2000 (Main level 7 Sublevel 0) compositions (HDR and SDR) to lossy JPEG 2000 (Main level 6 Sublevel 4 and Main level 6 Sublevel 3) compositions, which were subsequently subjected to QC and playback.

Use case and source content provided by Sony Pictures Entertainment.

Observations

As IMF matures, the focus has increasingly turned to operational improvements and incremental upgrades.

• The plugfest highlighted the importance of a common validation strategy: different versions of the widely used Photon OSS validator performed differently when faced with newer use cases.
• The plugfest was an opportunity to experiment with new capabilities such as S-ADM and ADM, which are expected to be leading topics of future plugfests.
• Support for IMF across proprietary and open-source tools can always be improved. There is, for example, an opportunity to expand support for IMF beyond multichannel audio and monoscopic video in the popular open-source FFMPEG toolkit.

Future plugfests

We are already working on future plugfests. In the meantime, testing should never stop, and the content used during the plugfest remains available to all IMF UG members.

Join the UG today to get access to past plugfest content and help the community plan future plugfests!

Acknowledgments

Many thanks to David Gageos, Mike Krause, Wolfgang Ruppel, Brian Holter and Dave Deelo for their contributions to the program committee; to Walt Disney Studios and RTL, who hosted the event; and to Colorfront who sponsored lunch at the Los Angeles location.

Do you have a content library or an implementation that speaks IMF, then join us at the December 2023 IMF plugfest, which will be held concurrently in Los Angeles, USA and Cologne, Germany.

https://www.imfug.com/imf-plugfest-2023/

Only a few days left to register!

Test content from BBC, NBC Universal, RTL Technology, Fraunhofer DMT and The Walt Disney Studios is now available, with more to come.

This is a unique opportunity to gain implementation experience with IMF and network.

The HPA IMF UG hosted a workshop at IBC 2023 where 28 stakeholders from the media supply chain, representing both users and suppliers, explored using quality control (QC) with the Interoperable Master Format (IMF). This report summarizes their discussions.

What is QC?

QC ensures consistent consumer experiences, prevents technical failures, and avoids regulatory penalties. QC involves testing a media asset against a list of requirements, ranging from objective, e.g., ensuring that the audio loudness level remains below a specified level, to subjective, e.g., detecting incorrect color conversions.

Performing QC is costly. QC involves complex operations on large media files and expert human review. It is therefore critical to avoid performing QC multiple times on the same media assets.

Failing QC is costly. Failing QC nearly always means human intervention, which does not scale, and redelivery, which takes time and incurs expenses. It is therefore critical to minimize QC failures.

Minimizing QC by sharing assets across program versions

IMFallows the re-use of media assets across versions of a program. As such, IMF makes it possible to avoid repeating QC on media assets that are shared across versions of a program. This results in significant cost savings when, for example, a small part of a program is modified to correct an error or when multiple language versions have largely identical video content.

This, however, comes with challenges. Tools must provide context is needed when performing QC on a new part of a program: while the new part might pass QC on its own, it might not when considered with the rest of the program. For example, a video frame might have been duplicated where the new part meets the original program. It is also necessary to design workflows such that QC results on the original part of the program can be trusted when performing QC on new parts.

Avoiding rejections from expected failures

Consider, for example, a program where a long silence is part of the storyline. Such a program will likely repeatedly fail QC tests designed to detect missing audio, potentially resulting in multiple rejections as the program moves across the supply chain. QC tests that a program is expected to fail should therefore be captured as machine-readable data and as early as possible in the supply chain. It is also critical that such expected-failure metadata remain synchronized with the program across the supply chain.

IMF is uniquely positioned to be the source of expected-failure metadata: such metadata is generated at mastering, where IMF is typically used, and IMF allows rich metadata to be synchronized with a program. Additional specification work however remains to fully define the syntax of expected-failure metadata.

Automating QC using machine readable reports

Automation is impossible without standardization. Today, QC reports are routinely shared in a non-machine-readable form, e.g., as a PDF file, and using vendor-specific terminology. This makes it difficult to automate QC, keep QC data synchronized with the program, and trust QC data across the supply chain. Standardizing QC data is a non-trivial problem given the historical disparities in QC practices worldwide. Efforts, such as EBU QC (https://tech.ebu.ch/qc), are underway to standardize QC tests.

Next steps

As the number of programs explodes worldwide, IMF offers the opportunity to reduce QC costs by reusing assets across versions and being the source of QC data for the supply chain. Challenges remain in standardizing QC data and designing workflows and tools that make use of it. Get involved today with the HPA IMF UG to contribute to these efforts.

Acknowledgments

Many thanks to Tedial, who kindly sponsored the location and catering, and to our program committee: Fereidoon Khosravi (chair), Andrew Dunne, Andy Quested, Brian Holter, Harvey Landy, and Bruce Devlin.

Attendees

Pixelogic Media Partners, Venera Technologies, BBC, EBU, Amazon Studios, Tedial, RTL, Marquise Technologies, Andy Quested, Mr. MXF, Lum::Invent, Netflix, MediaArea, Colorfront, Motion Picture Solutions, Deluxe, Sky UK, nomalab, BFI, Bitmax, Walt Disney Studios, Sky UK, Ateliere Creative Technologies, Dolby Laboratories, DI Factory.

Subscribe

Sign-up for our mailing list and stay up-to-date with the latest on IMF and component-based workflows!

Email address:

Leave this field empty if you’re human:

The HPA IMF UG hosted a workshop at IBC 2022 where 34 stakeholders from the broadcasting community, representing both users and suppliers, explored using the Interoperable Master Format (IMF) as the source content format in broadcast workflows. This report summarizes their discussions.

UHD Transition

The broadcast community is facing a transition from HD to UHD programming. This transition requires a new program exchange format but also offers the opportunity to upgrade workflows so that they are more efficient today and more resilient to future requirements. For example, the time and cost involved with re-delivering complete programs when only a small part is defective is amplified by the larger size of UHD programs.

The Interoperable Master Format (IMF) is a worldwide standard for exchanging and archiving TV programs, movies, and ads. It is component-based: audio, video and access services files are stored separately and synchronized on a common timeline using a single playlist file.

Strengths

IMF is deployed today, both in internal workflows and as a delivery format by large content providers and broadcasters: Netflix, BBC, Disney, Deluxe… Several attendees indicated that they are actively evaluating it.

IMF supports UHD today and is implemented by a wide range of commercial and open-source tools, including mastering, QC and transcoding tools. For example, the recent addition of an IMF demuxer to the FFmpeg, Avisynth and VapourSynth open-source toolkits has significantly reduced the barrier to adopting IMF.

IMF allows the re-use of components across deliveries and versions, reducing QC, making deliveries faster and requiring less storage for archives. One participant reported that they had saved 40,000₤ in QC costs alone by avoiding having to QC multiple times components shared by multiple deliveries. Component-based delivery and storage is also particularly attractive for cloud-based applications where bandwidth is limited.

Challenges

While the IMF technology is mature, there is still little operational familiarity with component-based media processing and IMF is often used as a flat linear mezzanine, creating an obstacle to realizing the full benefits of IMF. For example, while IMF carries rich metadata that allows the location of color bars or candidate commercial insertion points to be identified, the automated use of such metadata during playout is not common practice.

Participants highlighted that it is preferable to modify one’s workflow to take advantage of component-based media, and thus of IMF, instead of shoehorning IMF into existing linear workflows. Such a transition should be planned over 2 to 3 years and account for education and onboarding.

It was also noted that several traditional playout and media server vendors do not support IMF, despite IMF using many of the technologies (playlist and MXF files) that they are familiar with.

Opportunities

It is unlikely that IMF will displace current formats and practices for HD. It was however emphasized that the future is not flat, and that IMF allows a broadcaster to both transition to UHD today and be prepared for future technologies. The future might include, for example, object-based media as identified by Ofcom, rights metadata or access services such as sign language.

Now is therefore a good opportunity for the broadcasting community to collaborate around developing best practices for component-based workflows and develop delivery specifications around IMF.

Links

IMF Product and services	https://www.imfug.com/suppliers/
IMF Open-source	https://www.imfug.com/open-source/
IMF Explainer	https://www.imfug.com/explainer/

Attendees

BBC, Colorfront, BFI, Ateliere, DI-Factory, Mr. MXF, NRJ, Netflix, Sky UK, Deluxe, Venera Technologies, Sky Italy, CST, Vidispine, Arte, Amazon, filter Media, Tedial, Black Photon, RTL, Marquise Technologies, Lum Invent, Aveco, ITV, Via Play, RAI, Fraunhofer IIS, Dolby, Rohde & Schwarz, MediA Digital Nutty, BBC Studios.

Acknowledgments

Many thanks to Ateliere and Deluxe, who kindly sponsored the location and catering, and to our program committee: Andrew Dunne (chair), Julián Fernández-Campón, Kirk Bradford, Laurence Stoll, Mark Pascoe and Simon Thompson.

Subscribe

Sign-up for our mailing list and stay up-to-date with the latest on IMF and component-based workflows!

Email address:

Leave this field empty if you’re human: