Search options (Page 4 of 12)

Chainik, I think it would also be useful in SVP to have a similar function in the GUI to test RIFE interpolation speed without the need for encoding:

https://github.com/styler00dollar/VSGAN … benchmarks

As you can see from the test results there, this makes a big difference when it comes to performance on fast graphics cards:

for example:

Rife 4.6 ensemble True - the highest quality!!! but slower
NVIDIA GeForce RTX 4090, TensorRT8.5

320 / 401.6* fps (720p)
160 / 207* fps (1080p)

...and such a result would be more revealing of a configuration's potential for real-time video interpolation with RIFE.

and just as a reminder:

Rife 4.6 ensemble False - the fastest
NVIDIA GeForce RTX 4090, TensorRT8.5

541* fps (720p)
288* fps (1080p)

Lots of new options, lots of tweaking.

Most importantly for the performance boost is the use of FP16 mode on GPUs with Tensor Cores. To set FP16 mode it is now essential to convert the colour format to RGBH.

https://github.com/HolyWu/vs-rife/releases/tag/v3.0.0

Apart from that, we'll probably need lots of VRAM and an ultra-fast SSD or RAM Disk:

:param trt_max_workspace_size: Maximum workspace size for TensorRT engine.

https://github.com/HolyWu/vs-rife/blob/ … t__.py#L48

 trt_max_workspace_size: int = 1 << 30,

https://github.com/HolyWu/vs-rife/blob/ … t__.py#L26

:param trt_cache_path:          Path for TensorRT engine and timing cache files.
                                    Engine will be cached when it's built for the first time.
                                    Note each engine is created for specific settings.

https://github.com/HolyWu/vs-rife/blob/ … t__.py#L49

 trt_cache_path: str = dir_name,

https://github.com/HolyWu/vs-rife/blob/ … t__.py#L27

We will probably also need to test this:

 :param num_streams: Number of CUDA streams to enqueue the kernels.

https://github.com/HolyWu/vs-rife/blob/ … t__.py#L42

In other words, there can be many bottlenecks and performance gains can vary not only depending on the graphics card, but also on the speed of the SSD, RAM and CPU.

+40~50% comes from the developer vs-rife quote. This is a new filter published last night, so no independent tests yet.

However, we have tests of the TensorRT-optimised version of RIFE in VSGAN-tensorrt-docker, which seem to confirm this: 288fps/164fps-1=+75%

288 new fps (1080p) for 4090+13900k (TensorRT8.5+vs_threads=4+fp16) (rife46) (num_streams=10) (benchmark was done with vspipe file.py -p . instead of piping into ffmpeg and rendering to avoid cpu bottleneck)

164 new fps (1080p) for 4090+5950x (ncnn+2 threads+4 vs threads+ffmpeg (ultrafast) (rife4.6)

Source: https://github.com/styler00dollar/VSGAN-tensorrt-docker

It's good to know that all is well, as I too intend to use Windows 11 and model 4.6 in the future.

Please try 3 variations of the test for the above 4K HDR file:

1. Normal playback without interpolation on a monitor or TV set up with HDR. Only pay attention to the colours in this test.

2A. Real-time interpolation with RIFE. Even if footage is very choppy with frameskipping it is interesting to see if colours have changed compared to test number 1. Try using Triple Buffering
in the graphics card settings to improve smoothness: https://tweakguides.pcgamingwiki.com/Graphics_10.html

2B. Same as 2A only if it continues to stutter reduce the size before interpolation e.g. "Decrease to HD" https://www.svp-team.com/wiki/Manual:Resizing for testing purposes whether HD colours will be retained when interpolating in real time using RIFE.

3. Reencoding with RIFE. I'm interested in the fps during transcoding 4K file, and whether later playing back normally this file, which will already have a fixed 50 fps will retain the colours from test 1 or 2.

Very strange...

Could you please check if it is possible to interpolate x2 in real time with RIFE:

LG 4K HDR Demo - New York.ts
File size: 448 MiB
Duration: 1 min 13 sec
Overall bit rate: 51.4 Mbps
HDR format: SMPTE ST 2086, HDR10 compatible
Width: 3 840 pixels
Height: 2 160 pixels
Frame rate: 25.000 FPS
Color space: YUV
Chroma subsampling: 4:2:0
Bit depth: 10 bits
Direct link: https://drive.google.com/file/d/1dfR5TT … _bGfEXUvJ/
Source: http://hdr4k.blogspot.com/

"performance did not change" and "is the fastest" how is this possible at the same time?

Now a question for everyone: is it possible to set the number of GPU threads higher than 4?

Thanks a lot!

Make sure you are using the latest version of SVP: 4.6.0.220 2022-10-10: https://www.svp-team.com/news/

It only has the NCNN RIFE version: https://www.svp-team.com/forum/viewtopi … 052#p81052

I think most people will use 4 models on a daily basis:

https://github.com/megvii-research/ECCV … 1225279362

2 models automatically profile-switched by SVP for animation:

for real-time HD animation interpolation - the fastest version of the v4.4 model:
rife-v4.4 (ensemble=False / fast=True)

for real-time SD animation interpolation and HD animation transcoding - highest quality v4.4 model:
rife-v4.4 (ensemble=True / fast=False)

2 models automatically profile-switched by SVP for live action:

for real-time HD live action interpolation - fastest version of the model v4.5 or v4.6:
rife-v4.5 or v4.6 (ensemble=False)

for real-time SD live action interpolation and HD live action transcoding - highest quality version of the model v4.5 or v4.6:
rife-v4.5 or v4.6 (ensemble=True)

I don't know which of the v4.5 or v4.6 models would be better. However, there must have been some reason that they don't use contextnet and unet anymore: https://github.com/styler00dollar/Vapou … 1271982617 I also suspect that perhaps one of the v4.5 or v4.6 models is the successor to v4.4 for animation.

We would have to ask hzwer for more details. I plan to do this and ask him to evaluate the models using one of the perceptual quality metrics:
FloLPIPS https://arxiv.org/abs/2207.08119 https://github.com/danielism97/flolpips
VFIPS https://arxiv.org/abs/2210.01879 https://github.com/hqqxyy/vfips
LPIPS https://arxiv.org/abs/1801.03924 https://github.com/richzhang/PerceptualSimilarity

Before I do that I would like to complete my "Rankings of Video Frame Interpolation Models Based on PSNR, SSIM, LPIPS, FloLPIPS Metrics" beforehand: https://github.com/AIVFI/Rankings-of-Vi … PS-Metrics with rankings based on LPIPS. I will later ask some questions of RIFE models based on it.

If anyone reading this post and would like to test the RIFE models themselves with FloLPIPS or VFIPS, which this year have just been developed as perceptual quality metrics specifically for video footage as opposed to LPIPS, which is more suitable for single images then of course this would be appreciated

In any case, we are only at the beginning of the development of AI interpolation methods and the best is yet to come.

It would be good if the SVP had the possibility to manually select through the GUI any RIFE model and the possibility to set profiles for automatic model selection depending on video resolution - faster for HD and UHD and higher quality for SD. Maybe one day it will also be possible to automatically recognise whether we are dealing with animation or with live action and automatically select the RIFE model to match - that would be the real deal.

I think this is what should distinguish SVP with RIFE compared to using only VapourSynth-RIFE-ncnn-Vulkan directly with mpv: ease and speed of use thanks to the GUI, automation of model selections via profiles and the wide selection of different settings and features that SVP is famous for.

This is great news!

Until now, I thought there would be some problems based on statements from n00mkrad, who uses RIFE in his Flowframes:

https://github.com/n00mkrad/flowframes/ … -803571016