4K HDR framerate lag on latest official build

[davilla]

Is the next build eminent, I.e. within few days, say next week?
BTW: selecting Video Toolbox delivers washed-out colors for HDR contents; this is not the case for AVFoundation, however, so what is the reason for this behavior?

The reason is videotoolbox uses GLES for rendering and there is no tone mapping of HDR -> SDR.
Since AVFoundation codec can handle HDR -> SDR, adding tone mapping to videotoolbox is low priority.

[Everbrave]

Is the next build eminent, I.e. within few days, say next week?
BTW: selecting Video Toolbox delivers washed-out colors for HDR contents; this is not the case for AVFoundation, however, so what is the reason for this behavior?

The reason is videotoolbox uses GLES for rendering and there is no tone mapping of HDR -> SDR.
Since AVFoundation codec can handle HDR -> SDR, adding tone mapping to videotoolbox is low priority.

So, what is Video Toolbox doing that AVFoundation doesn’t do? Low-level HW Access? Isn’t AVFoundation using Video Toolbox as well?

[davilla]

VideoToolBox is a video codec. It takes video demux packets and decodes them into video frames of yuv (8 or 10 bit, depending on source content). It's the caller's job to render those video frames. So MrMC takes those video frames and renders them into onto a NSView with GLES, which is then displayed on iOS/tvOS.

When VideoToolBox is selected as the codec, MrMC uses GLES to do the rendering. GLES can only do 8 bit, so HDR -> SDR but since tone mapping is not done, you get washed out colors. MrMC would have to render in Metal to get 10 bit HDR rendered with correct colors.

AVFoundation codec actually uses an API called AVSampleBufferDisplayLayer. It's the same thing that AVPlayer (apple's player) uses. It uses VideoToolBox for decode and Metal internally to decode AND render to a "layer" which we associate with a NSView. This NSView is under the GLES layer (which is transparent). So what you see is video with GLES over top.

Calling AVFoundation codec CDVDVideoCodecAVFoundation is actually over general. But calling it CDVDVideoCodecAVSampleBufferDisplayLayer looked funny so we named it CDVDVideoCodecAVFoundation

[Everbrave]

VideoToolBox is a video codec. It takes video demux packets and decodes them into video frames of yuv (8 or 10 bit, depending on source content). It's the caller's job to render those video frames. So MrMC takes those video frames and renders them into onto a NSView with GLES, which is then displayed on iOS/tvOS.

When VideoToolBox is selected as the codec, MrMC uses GLES to do the rendering. GLES can only do 8 bit, so HDR -> SDR but since tone mapping is not done, you get washed out colors. MrMC would have to render in Metal to get 10 bit HDR rendered with correct colors.

AVFoundation codec actually uses an API called AVSampleBufferDisplayLayer. It's the same thing that AVPlayer (apple's player) uses. It uses VideoToolBox for decode and Metal internally to decode AND render to a "layer" which we associate with a NSView. This NSView is under the GLES layer (which is transparent). So what you see is video with GLES over top.

Calling AVFoundation codec CDVDVideoCodecAVFoundation is actually over general. But calling it CDVDVideoCodecAVSampleBufferDisplayLayer looked funny so we named it CDVDVideoCodecAVFoundation

Now I understand, thanks a lot

[davilla]

@LeoNatan, you are clueless about how video is decoded and rendered on iOS/tvOS

The issue is with AVSampleBufferDisplayLayer, Apple changed its default behavior and you have to add magic sauce to restore the original behavior. AVPlayer works because they (dh'u) have added the magic sauce to AVPlayer. But they neglected to tell 3rd party developers that use AVSampleBufferDisplayLayer. Typical

[Samekas]

Same problem here... Any way to downgrade version?

[amet]

Same problem here... Any way to downgrade version?

MrMC ? No

but... debuglog might have answers why you have issues

[davilla]

Same problem here... Any way to downgrade version?

Fixed in MrMC 3.9.2 testflight. Will be released sometime this week.

[Samekas]

Same problem here... Any way to downgrade version?

Fixed in MrMC 3.9.2 testflight. Will be released sometime this week.

Awesome, thank you!

[MrSpidey]

Same problem here... Any way to downgrade version?

Fixed in MrMC 3.9.2 testflight. Will be released sometime this week.

Thank you very much.

As I am not on testflight (and assuming this works correctly), what do you think would be a reasonable time we would see it as an update to everyone else?