New RIFE filter - 3x faster AI interpolation possible in SVP!!! (Page 1 of 86)

Yes. These will be the main disadvantages of this filter. I am, however, at the moment preparing a huge list of benefits of the new filter over the previous one. It is going to be really long posts. I hope that I will be able to convince you, but also at the same time encourage other users to the new filter. Please give me some time though, English is not my native language, so it will take me until late evening or maybe even tomorrow to write all the posts. I really care - I am a big fan of real-time motion interpolation.

But now I will briefly address the drawbacks. A LOT of dependencies and a huge loading time can probably be offset by using a fast NVMe SSD based on PCIe 4.0 x4 interface, for example: Seagate FireCuda 530 with 7300 MB/s sequential read. In a year's time we may have 2x faster drives with the widespread adoption of the PCIe 5.0 x4 interface, so maybe even 14600 MB/s!

But what is most important and what I value most about SVP is that it gives users choice. SVP gives countless configurations and choices for each user and thus leaves other programs like DmitriRender behind.

Please give users the choice to use the RIFE filter for VapourSynth based on ncnn or the RIFE filter for VapourSynth based on PyTorch in SVP.

And I'm back to preparing more posts....

And now I will present below the main advantages of the new RIFE filter for VapourSynth over its predecessor:

1. The RIFE filter for VapourSynth based on PyTorch: https://github.com/HolyWu/vs-rife is at least 3 times faster in performance than the RIFE filter for VapourSynth based on ncnn: https://github.com/HomeOfVapourSynthEvo … cnn-Vulkan

Developer confirmation of the above filters:
https://github.com/HomeOfVapourSynthEvo … n/issues/5

SVP developer confirmation:
https://github.com/nihui/rife-ncnn-vulkan/issues/22

Confirmation by one more person:
https://github.com/nihui/rife-ncnn-vulkan/issues/26

This is obviously a major advantage, especially if we think about real-time RIFE AI interpolation. This new PyTorch-based filter is especially useful for SVP users, as SVP is the only program on the market that allows real-time motion interpolation using AI with a GUI, that is friendly and easy to use for any user.

If someone wants to transcode video using RIFE AI motion interpolation then of course they have a wider choice of tools, because in addition to SVP they can choose from Flowframes, DVDFab or Rife-App among others.  When transcoding, the speed does not matter so much, because whether the procedure takes 3 hours or 9 hours, you still have to wait patiently.

However, when we want to perform interpolation in real time the situation looks completely different. Here, either the interpolation works in time or it does not. Either it is possible to watch or it is not.

The situation is worse than when playing computer games, where for example we either have 120fps or 40fps. In the latter case it is still possible to play, although with much less comfort. However, gamers can spend a lot of money, time and overclocking to gain at least those extra 10% fps!

And we have the chance to increase the performance by 3 times!!!

Thank you very much!

I was just writing the 2nd point of my essay, but it looks like it's not necessary. OK, maybe I'll still finish it and paste it in here sometime. It is not important now!

Chainik, please write if you were able to interpolate in real time a higher resolution file using the 3.8 model than last time?

https://www.svp-team.com/forum/viewtopi … 96&p=2

3.8 model is the fastest!

https://github.com/hzwer/arXiv2020-RIFE/issues/176

I hope that a solution can be found. It is the real-time interpolation that I care most about. Of course it could be tomorrow, the day after tomorrow next week, whenever you have some time to find the cause.

Thank you once again!

> I guess "scale"

yep

> but then where do you set the calculation precision

there's no "precision" param in the "vsrife" filter
https://github.com/HolyWu/vs-rife/blob/ … t__.py#L16

Chainik, am I perhaps too impatient with asking about the problem of mpv hanging after switching to RIFE profile in a real time? Do you need a little more time to look for solutions?

At the moment we have 4 cases:

1. SVP+RIFE filter for VapourSynth (ncnn) - transcode very slowly
2. SVP+RIFE filter for VapourSynth (ncnn) - run very slowly in real time
3. SVP+RIFE filter for VapourSynth (PyTorch) - transcode 2-3 times faster
4. SVP+RIFE filter for VapourSynth (PyTorch) - mpv hangs

.https://www.svp-team.com/wiki/RIFE_AI_interpolation

Have you tried using VLC or any DirectShow player?
Have you tried using some very, very low resolution file?
Have you tried waiting a few minutes?

I'd love to help, but I'm neither a programmer nor have a proper graphics card.

Maybe someone else besides Chainik could test the latest RIFE filter for VapourSynth based on PyTorch and share their impressions on this thread?

If anyone hasn't tried it yet, here are the advantages of RIFE over MVTools2:
http://forum.doom9.net/showpost.php?p=1 … stcount=17
http://forum.doom9.net/showpost.php?p=1 … stcount=24

This is an improvement on an aspect that has been raised many times on this forum:
https://www.svp-team.com/forum/viewtopic.php?id=5500
https://www.svp-team.com/forum/viewtopic.php?id=6139
https://www.svp-team.com/forum/viewtopic.php?id=4382

Now it is!!!!!!!!!!!!!!

HolyWu just released a new version yesterday:
https://github.com/HolyWu/vs-rife/releases/tag/v1.1.0

I guess he reads this thread;-) If so, thanks a lot HolyWu!

Selecting half precision (FP16) increases the performance by up to 4x:

https://github.com/hzwer/arXiv2020-RIFE/issues/188 

Chainik, this is yet another reason to take a second look at running this filter in SVP in real time. The speed should increase dramatically!

HolyWu is actively supporting the RIFE filter for VapourSynth based on PyTorch, maybe he will be able to help you over the implementation of this filter for real-time interpolation?

Chainik, you will know best what to ask. I can of course write a request myself, but apart from the fact that it doesn't work I won't be able to add anything else, because I simply don't know anything about it :-(

Thank you very much for the upgrade and sharing the test results.

Strange...
cheesywheesy claims x4 on 3090.
n00mkrad, creator of Flowframes, claims that fp16 does not work with newer models:
https://github.com/hzwer/arXiv2020-RIFE/issues/188

Maybe it's a matter of the interpolation model used, or the GPU? I unfortunately can't check this as I don't have a compatible graphics card yet. Maybe someone else could check the speed in SVP and in Flowframes?

Does this mean that you are using a default model in the Pytorch version? If so, this is version 3.5:
https://github.com/HolyWu/vs-rife/blob/ … t__.py#L10

If so, it is neither the latest model nor, unfortunately, the fastest.
3.8 model is the fastest!

https://github.com/hzwer/arXiv2020-RIFE/issues/176

https://grisk.itch.io/rife-app/devlog/2 … 207-update

Chainik, could you take a second look at running this filter in SVP in real time.

Could you please try to use VLC or any other DirectShow player?
Could you please try to use other some very, very low resolution file?
Could you please try to wait a few more minutes?
Have you checked, maybe hardware is the bottleneck? VRAM? RAM?

I'd love to help, but I'm neither a programmer nor have a proper graphics card currently.

HolyWu is actively supporting the RIFE filter for VapourSynth based on PyTorch, maybe he will be able to help you over the implementation of this filter for real-time interpolation?

Chainik, you will know best what to ask. I can of course write a request myself, but apart from the fact that it doesn't work I won't be able to add anything else, because I simply don't know anything about it :-(

RIFE filter for VapourSynth (PyTorch) is already completely capable of producing enough frames per second in HD files today to think about real-time interpolation. Below are examples of using RIFE in FlowFrames:

RTX 3080 - 1280x720p - 55.5 FPS
RTX 3090 - 1280x720p - 60 FPS

More examples here:
https://github.com/n00mkrad/flowframes/ … chmarks.md

It should be mentioned that these are examples from 8 months ago, and the latest 3.8 algorithm is much faster, as I wrote above.

Are there any people reading this thread who have already tried the RIFE filter for VapourSynth (PyTorch) implemented in SVP?

I would be very grateful if you could test SVP with RIFE filter for VapourSynth (PyTorch) on 3070ti.

I would be particularly interested in how the following 2 issues compare on the new Ampere microarchitecture:

1. Do you have the same problem as Chainik on the Turing microarchitecture when attempting real-time interpolation?

2. What is the performance when choosing half precision (FP16)? Here quite different results appear:

https://github.com/hzwer/arXiv2020-RIFE/issues/188

ok, so 3070 Ti is the absolute minimum for 1080p real-time playback

what is GPU temperature after 1 hour of encoding?

> I was changing this line and restarting svp, right ?

yep