• Thanks: 7
  • Thanks for the post: 1

Topic: Request support for YUV444P8/P10/P16 format

Request support for YUV444P8/P10/P16 format

AviSynth Filter recently supports YUV444P10/P16 format, and I hope SVP will follow suit.
https://github.com/CrendKing/avisynth_f … 2189374110

If SVP can support YUV444P10 format, when playing 4K UHD HDR video, it can convert from 4K YUV420P10 to 1080P YUV444P10 and keep the resolution of UV plane.
The RIFE filter can also convert from RGBS to YUV444 instead of YUV420, keeping the resolution of the UV plane. (SmoothFps_RIFE)

2 Reply by UHD (edited by UHD 22-04-2022 16:07:02)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: Request support for YUV444P8/P10/P16 format

blackmickey1007, could you explain to me how exactly this works with RIFE?

I found this: https://github.com/n00mkrad/flowframes/issues/123 and I think this is the same thing you write about.

I'll try to describe how I understand it, and you can correct me if I'm wrong somewhere:


Real-time interpolation:

Step 1: RIFE converts all original frames from YUV420 to RGBS - lossless conversion for colours.

Step 2: RIFE adds the interpolated frames - no effect on colours.

Step 3... and here I have doubts. Encoder for example FFmpeg converts all original and these interpolated frames from RGBS to YUV420 and here is an obvious loss of information about colours.

...but how is it with real-time interpolation? After all, it is standard for a PC to send an image in RGBS format to the monitor from a word processor, for example, so that we have clear black text on a white background without colour artefacts.

Can't RIFE, or rather VapourSynth (ncnn Vulkan) send frames in RGBS format directly to the monitor? I'm not familiar with coding, so I don't understand everything about the RIFE filter for VapourSynth (ncnn Vulkan) code: https://github.com/HomeOfVapourSynthEvo … cnn-Vulkan

I know that later information from this filter is captured by SVP and passed to mpv. Is it there that some other conversion happens that affects the colour change?

My desire to use RIFE for real-time interpolation rather than encoding comes from the fact that I would like to retain as much information from the original video as possible, including colour information.

I see that you have a lot of knowledge about colour conversion and maybe you can explain to me what is needed and at what stage to enjoy the interpolation quality of the RIFE filter for VapourSynth (ncnn Vulkan) in real time without losing the original colour information?

3 Reply by blackmickey1007 (edited by blackmickey1007 23-04-2022 11:16:30)

  • Thanks: 7
  • Thanks for the post: 1

Re: Request support for YUV444P8/P10/P16 format

That looks like the difference is caused by using the wrong function for color space conversion.
For example:
YUV --(BT.709 conversion) --> RGB --(BT.601 conversion) --> YUV

This can be checked using the following script.

import vapoursynth as vs
import statistics
core = vs.core
core.num_threads = 4

input_clip = VpsFilterSource
input_YUV = input_clip.resize.Point(format=vs.YUV420P16)

clip_RGB = input_YUV.resize.Point(format=vs.RGBS, matrix_in_s="709")
clip_YUV = clip_RGB.resize.Point(format=vs.YUV420P16, matrix_s="709")


# expr: x - y
# Y Plane
expr = 'x y - '.format(neutral=1 << (input_YUV.format.bits_per_sample - 1))
Diff = core.std.Expr([input_YUV, clip_YUV], [0])
Diff = core.std.PlaneStats(Diff)

# Show the difference value (PlaneStatus - Y Plane)
# If the value is 0, it means it is exactly the same, non-zero means there is a difference.
clip_YUV = core.std.CopyFrameProps(clip_YUV, Diff)
clip_YUV = core.text.FrameProps(clip_YUV)
clip_YUV.set_output()

https://i.imgur.com/QF3Es1Z.png


Incorrect conversion

clip_RGB = input_YUV.resize.Point(format=vs.RGBS, matrix_in_s="709")
clip_YUV = clip_RGB.resize.Point(format=vs.YUV420P16, matrix_s="470bg")

https://i.imgur.com/6z3oLGF.png


Accuracy error (from floating to integer conversion process)

clip_RGB = input_YUV.resize.Point(format=vs.RGB24, matrix_in_s="709")
clip_YUV = clip_RGB.resize.Point(format=vs.YUV420P16, matrix_s="709")

https://i.imgur.com/tazoNcb.png


My request is for the video resolution, not the color space.
The common video format is YUV420, which has three planes, Y-plane, U-plane, and V-plane, where the length and width of UV-plane is only half of Y-plane.
In 2160P YUV420 video, the resolution of Y-plane is 2160P, U-plane is 1080P, and V-plane is 1080P.

YUV444 format has the same resolution for Y-plane, U-plane, and V-plane.
RGB format, there are three planes, R plane, G plane and B plane, all three planes have the same resolution.

RIFE only supports RGB format, but VapourSynth Filter does not support RGB format output, so you must convert RGB format to YUV format.
If you convert from RGB to YUV420 format, the information of UV plane will be lost.
If you want to keep the full resolution, you should convert to YUV444 format.


NOW
             4K HDR Video ---> scale(power saving) ---> RGBS Convert(RIFE) ---> YUV444P10 Convert(Output)
Format:        YUV420                YUV444                   RGBS                  YUV420
Y or R:         2160p                 1080p                  1080p                   1080p
U or G:         1080p                 1080p                  1080p                    540p    
V or B:         1080p                 1080p                  1080p                    540p
Depth:          10bit                 10bit                  32bit                   10bit


Wanted
             4K HDR Video ---> scale(power saving) ---> RGBS Convert(RIFE) ---> YUV444P10 Convert(Output)
Format:        YUV420                YUV444                   RGBS                  YUV444
Y or R:         2160p                 1080p                  1080p                   1080p
U or G:         1080p                 1080p                  1080p                   1080p    
V or B:         1080p                 1080p                  1080p                   1080p
Depth:          10bit                 10bit                  32bit                   10bit
  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: Request support for YUV444P8/P10/P16 format

Thank you for your comprehensive reply. I did not understand everything, but for me personally this passage was the most important:

blackmickey1007 wrote:

RIFE only supports RGB format, but VapourSynth Filter does not support RGB format output, so you must convert RGB format to YUV format.

Did I understand correctly that you want to watch 4K UHD HDR video (10bit) interpolated with RIFE?

As I understood it correctly, such a conversion, would give better quality and resolution than interpolating the same material from a 1080p YUV420 source, part of whose information is encoded in 540p anyway.

By converting from 4K UHD HDR video (10bit) you want to preserve 1080p throughout the chain?

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: Request support for YUV444P8/P10/P16 format

By the way, can RIFE interpolate 10 bit video? 32 bit  RGB means 8 bits each of red, green, blue, and alpha...

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: Request support for YUV444P8/P10/P16 format

Filmscans (RGB) → Blu-ray (YUV420) → RIFE (RGB) → VapourSynth (YUV420 now) → Monitor/TV (RGB) roll

... and any conversion to YUV420 is a loss of half the resolution in terms of colour... sad

7 Reply by blackmickey1007 (edited by blackmickey1007 23-04-2022 11:17:29)

  • Thanks: 7
  • Thanks for the post: 1

Re: Request support for YUV444P8/P10/P16 format

UHD wrote:

Thank you for your comprehensive reply. I did not understand everything, but for me personally this passage was the most important:
Did I understand correctly that you want to watch 4K UHD HDR video (10bit) interpolated with RIFE?
As I understood it correctly, such a conversion, would give better quality and resolution than interpolating the same material from a 1080p YUV420 source, part of whose information is encoded in 540p anyway.

By converting from 4K UHD HDR video (10bit) you want to preserve 1080p throughout the chain?

I have some videos in YUV444P10 format and I want to keep the UV plane resolution, so I want SVP to support YUV444 format.
On the other hand, it is hard to realistically use RIFE for 4K video in a few years. Scaling 4K video to 1080P is a trade-off.



UHD wrote:

By the way, can RIFE interpolate 10 bit video? 32 bit  RGB means 8 bits each of red, green, blue, and alpha...

The RGBS format in VapourSynth is 32bits floating point per channel. It is 32bits(R) + 32bits(G) + 32bits(B), without alpha channel.
The 8bits per channel integer format in VapourSynth is called RGB24. 8bits(R) + 8bits(G) + 8bits(B)

List of formats supported by VapourSynth:
http://www.vapoursynth.com/doc/pythonre … -constants
List of formats supported by AviSynth:
http://avisynth.nl/index.php/Avisynthplus_color_formats



UHD wrote:

Filmscans (RGB) → Blu-ray (YUV420) → RIFE (RGB) → VapourSynth (YUV420 now) → Monitor/TV (RGB)
... and any conversion to YUV420 is a loss of half the resolution in terms of colour...

Master Tape (YUV422) → Blu-ray (YUV420) → RIFE (RGBS) → SmoothFps_RIFE (YUV420 now) → Monitor/TV (RGB)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: Request support for YUV444P8/P10/P16 format

There is hope for a better output format:

https://www.svp-team.com/forum/viewtopi … 1&p=12

smile