New RIFE filter - 3x faster AI interpolation possible in SVP!!! (Page 7 of 85)

Thanks for the new information dlr5668. I just don't know where the drop in fps to 30-35 comes from? Is the earlier result some kind of GPU boost?

I also have a question, would you have the ability to do the above test I ask for Quaternions in post above? I am keen to know the performance information in fps combined with GPU load (CUDA)

I don't think you need to install the CUDA toolkit. It is probably enough to disable hardware scheduling, as lwk7454 wrote about: https://www.svp-team.com/forum/viewtopi … 497#p79497

You could probably find that the 3D load will equal the CUDA load. That's what I think, after what lwk7454 described, but I'm not sure. Unfortunately, it's hard to write something when I don't have a proper graphics card myself yet.

I don't know if lwk7454 is reading this thread and will be able to do some more testing.That's why it would be good if you could do a similar test that I'm writing about for both the 3.8 and 4.0 models. The idea is to be consistent in the data and see how the computing power utilization as a percentage changed between these models on the same graphics card.  The load on the 3090 and 3070 Ti cards can be quite different.

I think it's worth it...

and this...

https://github.com/HolyWu/vs-rife/blob/ … t__.py#L20

I was able to get the CUDA graph to show up by disabling GPU hardware scheduling, the 3D reporting is very different from when GPU hardware scheduling was turned on but here is my results:
1080p half

1080p full

4k full (thanks for pointing out that it fixes the artifacts)

demo clip full

OK, thanks, now I understand.

Here are the results showing 66% maximum CUDA load for the 3090 and the 3.8 model:
https://www.svp-team.com/forum/viewtopi … 526#p79526

If you could do similar tests on your card for model 3.8 and 4.0, we would have a confirmation how the CUDA load has changed compared to the old model on the same graphics card. If in fact in addition to the performance increase the CUDA usage has decreased, it means that the potential for further optimization of the 4.0 model is even greater than for the 3.8 model and that the bottleneck is not CUDA, but something else entirely, as I wrote about here:  https://github.com/hzwer/arXiv2020-RIFE/issues/217

Then we could add the results of such tests to the above post and maybe the RIFE developer will find a way to optimize. I think we should all care about that.

And here are the details of the test:

Fixed test parameters:

SVP & RIFE filter for VapourSynth (PyTorch)
re-encoding with x2 interpolation
scale=1.0

Variable test parameters:

Math precision: FP16 and FP32
RIFE model: 4.0 and 3.8

Test results:

re-encoding speed [FPS]
CUDA utilisation [%]

Video file:

original demo video from the creator of RIFE at: https://github.com/hzwer/arXiv2020-RIFE
720p (1280x720), 25FPS, 53 s 680 ms, 4:2:0 YUV, 8 bits
direct link: https://drive.google.com/file/d/1i3xlKb … sp=sharing

Instructions simplified (https://www.svp-team.com/forum/viewtopi … 695#p79695) cause Python 3.9 and Vapoursynth R57 are now installed/updated with SVP.

if you think that asking the same question ten times will change anything - you're wrong

Of course, if you perceived my posts as time pressure then I sincerely apologise.

I am keen to provide feedback to the RIFE developer as soon as possible on the new 4.0 model in terms of performance and CUDA core loading on the latest graphics cards. I posted my request to the RIFE developer when the 3.8 model was available: https://github.com/hzwer/arXiv2020-RIFE/issues/217 However, we now have the 4.0 model which, as the tests posted here show, is a colossal leap in performance.

I am afraid that if there is no feedback, the RIFE developer will close the thread, concluding that since he added a new model and there are no comments, the issue is closed. So far we have tests done by Quaternions, which shows that paradoxically the 4.0 model is not only faster, but also probably less demanding on CUDA cores than the 3.8 model: https://www.svp-team.com/forum/viewtopi … 727#p79727

I write probably, because we have the test results of the 720p RIFE demo file  showing the following load on the CUDA cores:

Cuda: 58%, FPS: 69.8, FP32, Model 3.8, NVIDIA GeForce RTX 3090
https://www.svp-team.com/forum/viewtopi … 723#p79723

Cuda: 46%, FPS: 91.3, FP32, Model 4.0, NVIDIA GeForce RTX 3080 Ti
https://www.svp-team.com/forum/viewtopi … 727#p79727

Because the tests were done on two different computers and two different graphics cards, we can't be entirely sure of a similar conclusion when testing different models on the same hardware. This is why I asked 3 people on this thread to test both models or completing the test of the missing model.

That said, the 3080 Ti and 3090 cards are pretty close in performance and the 12% difference in CUDA load between the two different RIFE models is likely to be confirmed on tests on the same hardware.  This would mean that despite the optimisation of the 4.0 model, the potential for further optimisation has paradoxically increased even more as the CUDA load has decreased, which as you can see is not the bottleneck here.

Now the most important thing: 91.3 fps indicates that we are very close to x3 interpolation of 720p files in real time! Perhaps this is already possible if x3 interpolation uses some synergy effect whether better parallel processing, lower VRAM bandwidth needs, or better use of CUDA processing power. Without testing we don't know that. And for testing we need your help Chainik and adding this parameter https://github.com/HolyWu/vs-rife/blob/ … t__.py#L20 to work with SVP.

With full x2 and x3 test results we will give a full feedbeck to the RIFE developer and maybe he can find a way to make it even more effective by making better use of CUDA cores and maybe even Tensor cores.

Hi, first post. I've been using SVP for a while and thought I'd look into RIFE after seeing it in the installer.

I'm eager to test it out, but to clarify first, I notice the wiki and SVP itself say it can only be used for 2X interpolation, but some of the posts in this thread (I think) seem to suggest this limitation is no longer in effect, SVP just needs to be updated to account for it. Is this correct?

I have a project I've been planning for a while now and if there's a major new feature like this on the horizon I'll put it off longer. Does anyone know when this feature might be added to SVP? Or might I be better off trying to install standalone RIFE and using that for now?

Thanks

4.0 works the best for me. I watched at least 10 hours with it

A few hours ago:

RIFE ncnn Vulkan - Release 20220330:
https://github.com/nihui/rife-ncnn-vulk … g/20220330

"update ncnn with nvidia tensorcore optimization"

I'm extremely curious, how much of a performance boost can this give?

If it could manage to get performance close to vs-rife: https://github.com/HolyWu/vs-rife that would be something great

Dont think it will be faster than cuda version

Try the 4.1 model, and if the result is worse than 3.8 then report this particular case to the RIFE developer, as here: https://github.com/hzwer/Practical-RIFE/issues/10

For millions of pictures it is best to use a RAM Disk, such as OSFMount:
.

4K and FP16 is a known issue: https://github.com/hzwer/arXiv2021-RIFE/issues/188

Yes, it is normal. The RIFE ncnn/Vulkan version takes up much less space, but is much slower. At least it was. There is hope for a speedup, but it's not clear how big. See next post...