Search options (Page 8 of 12)

Full list as per details in previous post:

x2    1h41m
x3    3h21m
x4    5h2m
x5    6h42m
x6    8h22m
x7    10h3m
x8    11h43m
x9    13h23m
x10  15h4m

Of course, it may be that 2 new interpolated frames at the x3 setting will not necessarily require 2x the time to interpolate 1 frame. It is possible that some synergy effect will take place and x1.9 time or 1.8 will be enough. This needs to be verified in a real test.

If I understand what is written here correctly: https://arxiv.org/pdf/2011.06294.pdf

Position 0 is the first real frame.
Position 1 is the second real frame.

Before RIFEm and RIFE 4.0, only position 0.5 (x2) was possible.
To get positions 0.25 and 0.75 we had to repeat the process again.

Now it is possible to get any position between 0 and 1 the first time on the principle:
0, 0.5, 1 (x2)
0, 0.333, 0.667, 1 (x3)
0, 0.25, 0.5, 0.75, 1 (x4)
0, 0.2, 0.4, 0.6, 0.8, 1 (x5)
... and so on

I, for quality reasons would never use a method that omits real frame 1.
It is better to use: Custom Resolution Utility (CRU): https://www.monitortests.com/forum/Thre … tility-CRU

In anyway, trying to use x2.5 should generate an error report:
https://github.com/HolyWu/vs-rife/blob/ … t__.py#L40

Check out my post here: https://www.svp-team.com/forum/viewtopi … 493#p79493 and especially lwk7454's reply here: https://www.svp-team.com/forum/viewtopi … 497#p79497

Am I seeing correctly? Even 42.5fps manages to be achieved?

Many thanks for the tests! A huge improvement in the RIFE algorithm can be seen!

These are for comparison the results on a GeForce RTX 3090 graphics card done by: lwk7454:

https://www.svp-team.com/forum/viewtopi … 526#p79526

We can see how similar results the 3.8 model gave for GeForce RTX 3090 and GeForce RTX 3070 Ti graphics cards:

28.1 FPS vs. 24 FPS

and how huge the difference is with the 4.0 model: 40.9 FPS!!!!!

dlr5668, could you please specify the CUDA load for these tests. This is very important data to determine if the new model makes better use of the graphics cards capabilities or if there is still room for optimization, as I described here: https://github.com/hzwer/arXiv2020-RIFE/issues/217

By the way, thanks for checking that FP32 performance is close to FP16. However, if you would provide precise data for which precision this is, that would be additional information that I could paste in the link provided above.

It would be good if you could present data in the same range as lwk7454, it would be consistent with what I presented here https://github.com/hzwer/arXiv2020-RIFE/issues/217 and maybe the RIFE developer will come up with something extra to achieve 50FPS for 1080p, which would be satisfactory for real time interpolation!

Chainik, please if you find a moment of time add a new parameter to the RIFE configuration in SVP:

https://github.com/HolyWu/vs-rife/blob/ … t__.py#L20

Yes, finally we can with vs-rife not only interpolate x2 but also x3, x4, x5, x6, x7, x8, x9, x10...

If anyone is surprised by x3, x5 - this is the first and most important change compared to previous models.

The second change is the correction of the bugs that lowered the quality in the previous models:
https://github.com/hzwer/Practical-RIFE/issues/3
https://github.com/hzwer/Practical-RIFE/issues/5

The third change is an even better performance of the 4.0 model, which is based on the 3.9 model with the elimination of a bug: https://github.com/hzwer/arXiv2020-RIFE/issues/219

.
https://github.com/n00mkrad/flowframes/ … g.full.txt

Возможно, теперь в SVP можно использовать больше алгоритмов, а не только RIFE: https://www.svp-team.com/forum/viewtopi … 553#p79553

This seems to be the CAIN model, but it can be changed to anything else:

# currenlty has a hardcoded JIT cain model, you can replace that with anything you want

https://github.com/styler00dollar/vs-vf … ference.py

I don't need CAIN, but there is another model that is interesting.

Nothing more?

Until now only RIFE models could be used with vs-rife.
Now we can probably use any algorithm based on machine learning (artificial intelligence)!

Isn't it beautiful? Everyone could choose something for himself and in one software tool.

For a programmer like you this is probably nothing new and you could probably write something like this yourself. However, for a person like me all this code is a big black magic. What is important for me is that the greater the possibility of choice the faster the development of better interpolation methods and this also means better quality. And this is what all of us enthusiasts of smooth video should care about most.

Many thanks for the tests and for commenting on them.

What can I add? It is a great pity that no one has done such tests before and shared the results publicly. I am very curious what the developer of RIFE will say about this. After all, he knows the RIFE code best and how it works. If anyone would like to perform such tests on another graphics card I highly encourage you to do it. In about a week I will post the results on the RIFE project website: https://github.com/hzwer/arXiv2020-RIFE/issues

Once again all results together:

FP32

FPS 63.28 - Cuda ~45% - v3.8
FPS 59.43 - Cuda ~50% - v3.1
FPS 58.22 - Cuda ~70% - v2.4
FPS 58.39 - Cuda ~70% - v2.3
FPS 55.98 - Cuda 87% - v1.8

FP16

FPS 61.32 - Cuda ~45% - v3.8
FPS 60.03 - Cuda ~40% - v3.1
FPS 57.06 - Cuda ~50% - v2.4
FPS 57.20 - Cuda ~55% - v2.3
FPS 58.87 - Cuda ~70% - v1.8

lwk7454, when it comes to re-encoding tests I think we already have the full picture regarding the potential of today's most efficient consumer graphics card with relation to all major RIFE models and both computational precisions. I hope you'll continue to follow this thread and the place where I intend to share your results with a wider group of people interested in RIFE along with its creator: https://github.com/hzwer/arXiv2020-RIFE/issues  Maybe someone will come up with some ideas on how to increase the utilization of the graphics card power? I'm especially counting on the creator of RIFE, who is very helpful, as you can see in this example: https://github.com/hzwer/arXiv2020-RIFE/issues/207  If you would like to post results and questions yourself, let us know. After all, it is thanks to your work that we know there is still a lot of potential in the NVIDIA GeForce RTX 3090 graphics card to use.

Very interesting results and again surprising. It turns out that theory is theory and still practice shows best what is possible. Many thanks for the test results I requested as well as for doing the additional tests, which are always welcome. In my requests I ask for some minimum that answers the questions I have. I know that each additional test is additional time required to complete it and I respect your time and am hugely grateful that you are here and sharing invaluable information.

However, the results show that each additional test adds important data to the discussion.

In fact, the test results using Vulkan show the biggest differences and the most surprising. It seemed that Compute_1: 100% and FPS: 25.1 is already the maximum, and here such a surprise. Especially since it seems that Flowframes with Vulkan does not load CUDA at all. Maybe it's a matter of optimization and use of a certain number of GPU threads, which this version allows as opposed to the version dedicated to CUDA? In any case, this is just a curiosity, because for the Vulkan version to be suitable for real-time interpolation, more radical performance improvements are needed, for example by filling in the missing one layer (Interp) in this project: https://github.com/atanmarko/ncnn-with-cuda

The comparison of 720p and 1080p tests (PyTorch) also brings something important.

The FPS for the 1080p file can be assumed to be identical for both Flowframes and SVP+vsrife. However, for a 720p file the advantage of more than 10% in FPS in favour of SVP+vsrife is already significant. It is difficult to deduce what this might be due to. We can see that even lower CUDA load in Flowframes does not translate into performance increase. Maybe VRAM bandwidth is the bottleneck here? Maybe data I/O process?

What matters most to me is that using SVP+vsrife doesn't degrade performance, and that matters firstly because of real-time interpolation, and secondly because it shows us where to look for optimization.

Here I want to emphasize that the tests made are not only to satisfy my curiosity, or to provide a benchmark for testing other graphics cards, but above all to serve the development of the use of RIFE for real-time interpolation. The data we already have can be shared with the developer of RIFE and perhaps some optimizations can still be made to improve performance.

There is one more thing that I would like to share with the developer of RIFE, but before that it still needs to be tested:

Thank you! This is what it was all about! Very interesting results and a bit surprising.

The first thing that comes to mind after reviewing the results is how powerful the NVIDIA GeForce RTX 3090 is. Nearly 70FPS for HD video using an interpolation algorithm based on machine learning (artificial intelligence) and without downscaling (scale=1.0) is pretty impressive.

The second thing that comes to mind after reviewing the results is how much of the NVIDIA GeForce RTX 3090 graphics card's potential is still unfulfilled: 56-66% CUDA utilization and worse FP16 results compared to FP32.

I think a lot of people don't realize what potential there is in RIFE and don't even look at this thread. Yes I know that not everyone has a top of the line graphics card (I know because I don't have one yet either), but it is up to all of us now to make progress and practical use of algorithms based on machine learning for real time interpolation. Up to us, because firstly we can show that we need them. Secondly, because we can signal where there is still room for improvement in terms of efficiency and quality.

The third thing that comes to mind after reviewing the results is how big the differences are when it comes to real-time interpolation during playback versus during re-encoding: the percentage usage of CUDA power and the possibility of using FP32.

In upcoming posts later today and over the next few days, I will outline the areas that I think would be worth testing based on the results that lwk7454 presented and explain why.

Thanks lwk7454 for another test and showing how hardware scheduling in Windows 11 with RIFE works in practice.

The result surprised me a bit. I counted that it will be 100% and everything will be clear. That's why such tests are worth doing. It looks like for some reason 50% CUDA utilization is the ceiling. Or maybe it's not CUDA that is the bottleneck? Maybe it's not the amount of VRAM either, because there's plenty of VRAM in the 3090, maybe it's the VRAM bandwidth, something similar to when RAM bandwidth is the bottleneck in the native SVP algorithm: https://www.svp-team.com/forum/viewtopic.php?id=6349
Or maybe as you wrote earlier there is still potential for optimization? After all, 40% or even 46% GPU utilization for 720p gives hope for the future for 1080p.

And here's another surprise. Looks like there is a problem somewhere in the software.

Here is a confirmation that indeed the RIFE filter for VapourSynth (PyTorch) uses CUDA, while the RIFE filter for VapourSynth (ncnn) does not: https://www.svp-team.com/forum/viewtopi … 96&p=2 Although there is hope that the latter filter will also use CUDA, as Chainik mentioned https://github.com/nihui/rife-ncnn-vulkan/issues/22 pointing to this unfinished project: https://github.com/atanmarko/ncnn-with-cuda Maybe some talented interpolation enthusiast will fill the missing one layer (Interp) of the above project?

Until then, however, it is worth focusing on what currently gives us the most potential for real-time interpolation, namely the RIFE filter for VapourSynth (PyTorch).

I would be very interested to see the results of a few more simple tests, which maybe will show where the cause of the FP32 problems lies and what the potential is for interpolating 1080p files.  However, I will be away from the forum for 2 days and will try to describe it all on 3 November. I wouldn't want to ask for something when I won't have the opportunity to thank and comment on the results of my request.  I will try to get it all worded properly somehow by November 3.

Thanks again for the tests and I encourage others to check how the real-time RIFE intertpolation results look on their graphics cards, other media players or other Windows.