New RIFE filter - 3x faster AI interpolation possible in SVP!!! (Page 8 of 85)

From discord

Many thanks to Chainik for the tests! Even more so, for doing more tests and capturing the differences between the different versions.

Amazing progress!

Chainik, the results you presented are very promising in terms of using a lighter version of RIFE together with SVP for real-time motion interpolation.

Now I am intrigued by an anomaly that has come to light through testing on this thread:

Flowframes:

https://www.svp-team.com/forum/viewtopi … 531#p79531

SVP:

https://www.svp-team.com/forum/viewtopi … 526#p79526

Flowframes probably uses RIFE ncnn Vulkan directly: https://github.com/nihui/rife-ncnn-vulkan

SVP on the other hand uses RIFE filter for VapourSynth, based on RIFE ncnn Vulkan: https://github.com/HomeOfVapourSynthEvo … cnn-Vulkan

I don't know if this is the reason for the difference of about 50% more performance of Flowframes vs SVP or maybe just some settings are chosen differently. Anyway I have a big request to you to check how it looks like for you. We already have the results of the test you did for the base RIFE ncnn Vulkan tool:

https://www.svp-team.com/forum/viewtopi … 140#p80140

If we had comparison data for the following 3 variants, it would be easier to predict where the difference comes from:

4:30 RTX 2060 model 3.1 RIFE ncnn Vulkan 20210520
?:?? RTX 2060 model 3.1 SVP+RIFE filter for VapourSynth, based on RIFE ncnn Vulkan
?:?? RTX 2060 model 3.1 Flowframes

The free version of Flowframes still probably uses the 3.1 RIFE model.

I know, in light of new testing this old model is now obsolete, but it is still the basis of the RIFE filter for VapourSynth (ncnn Vulkan). If it turns out that there are actually differences between the results of these 3 tests, I think you could figure out from the code where the difference arises in the software. If it's the RIFE filter for VapourSynth, then we could ask HolyWu to include possible changes in a new update: https://github.com/HomeOfVapourSynthEvo … /issues/14

Thanks!!! Great news!!!

Chainik, could you check how much closer with SVP and the new VapourSynth-RIFE-ncnn-Vulkan filter?

You gave us the results for the base RIFE ncnn Vulkan 20220330:

Which gives us the following fps result:

original demo video from the creator of RIFE at: https://github.com/hzwer/arXiv2020-RIFE
720p (1280x720), 25FPS, 53 s 680 ms

25FPS*53.68=1342frames

1342frames*2/54s=49.7fps

Chainik, what results do you get out on your hardware in case of:

1. SVP & RIFE filter for VapourSynth (PyTorch CUDA) re-encoding with x2 interpolation; RIFE model: 4.0; scale=1.0; FP16 or FP32 (whichever is faster)

2. SVP & RIFE filter for VapourSynth (ncnn Vulkan) re-encoding with x2 interpolation; RIFE model: 4.0

I am very much asking you for these tests, please. It's important that we have a comparison of these two tests along with your earlier result, 54 seconds (49.7fps) on the same hardware.

Is there any difference when you change:

gpu_thread

What are the impressions with:

multiplier

set to 3 for the 4.0 model?

70fps SVP & RIFE filter for VapourSynth (PyTorch CUDA) or
SVP & RIFE filter for VapourSynth (ncnn Vulkan)?

Thanks for the tests and update

Thanks, this 70fps is not only hugely impressive compared to the previous 8fps, but also to the max results given here earlier:

720p - SVP & RIFE filter for VapourSynth (PyTorch CUDA) Model 4.0:

82,4fps NVIDIA GeForce RTX 3070 Ti - dlr5668
https://www.svp-team.com/forum/viewtopi … 691#p79691
91.3fps NVIDIA GeForce RTX 3080 Ti - Quaternions
https://www.svp-team.com/forum/viewtopi … 727#p79727

1080p - SVP & RIFE filter for VapourSynth (PyTorch CUDA) Model 4.0:

42.5fps NVIDIA GeForce RTX 3070 Ti - dlr5668
https://www.svp-team.com/forum/viewtopi … 699#p79699
45.0fps NVIDIA GeForce RTX 3080 Ti - Quaternions
https://www.svp-team.com/forum/viewtopi … 719#p79719

Can we enable rife profile as default ? I want to use it for <1080p content. Regular rules dont work

Thanks for the test! Amazing performance boost!

Interpolation x2 and 188.2fps means 94.1 original frames and 94.1 interpolated frames per second. This means that we can have real-time x4 interpolation without any problem - 25 original frames and 75 interpolated frames!!!!!

Am I counting correctly?

Dlr5668, have you tried x3 and x4 realtime interpolation with this new RIFE filter?
Does the "gpu_thread" setting affect fps, %GPU, VRAM?

.

Nice! Is it possible to now run RIFE at real time on a RTX 2070 Super & 5800X CPU?

I remember it was impossible in the past, but maybe with these versions it is?

PS: If I do, most content I watch is Bluray Remux 4K HDR & SDR content. So it will be very demanding, but has v4 of RIFE improved so much that it can now work real time with a 2070 Super?

any progress on making rife profile use generic rules ?

function gen_vs()
{
    // RIFE override for low resolution
    // Make sure default profile for such content is set to Automatic and interpolation disabled
    // https://imgur.com/a/cx6cKo5
    if(media.src_fps < 51 && media.dst_h_par * media.src_fps <= 1.2 * 1080 * 30)
    {
        profile.rife = true;
        profile.rife_model = 9;
        profile.rife_threads = 2;
        profile.rife_gpu = 0;
    }

https://imgur.com/a/cx6cKo5