51 Reply by UHD (edited by UHD 26-09-2021 18:38:36)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

buy 3090!

Thanks Chainik! If you know someone who owns this unicorn and could share his or her results, that would obviously be very helpful.

At the moment we still don't have any confirmation on what card allows smooth real-time interpolation at what resolution at scale=1.

It would be great to have such a list on this thread, so that not only me, but also others can decide if it is worth it and what graphics card to buy for real-time RIFE interpolation.

The main SVP algorithm relies mainly on CPU power, so if someone doesn't play computer games and the main load on the computer is SVP then he or she probably isn't using a high-end graphics card. If he/she wants to buy a GPU to have interpolation with less artifacts, then such a list would be very helpful.

Something along the lines of what you provided for the filter for Vulcan:


Chainik - NVIDIA GeForce RTX 2060 Mobile
https://www.techpowerup.com/gpu-specs/g … bile.c3348
SVP+RIFE filter for VapourSynth (ncnn)

Chainik wrote:

GPU; VRAM; CPU utilization; SVP index

424*240 - 23% (65% compute_1); 0.9 GB; zero; 1.0
640*360 - 9% (100% compute_1); 0.9 GB; zero; 0.76

https://www.svp-team.com/forum/viewtopi … 96&p=2

Above is an example of data that would satisfy me.

Of course now we need some kind of confirmation for the 3x faster filter:
SVP+RIFE filter for VapourSynth (PyTorch)

and the fastest RIFE model 3.8
with scale=1

Chainik, were you able to do real-time interpolation with this newer vs-rife filter?

  • Thanks: 230
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

realtime broke for me after some experiments and now hungs up big_smile

3.5 and 3.8 models takes ~5 GB VRAM under pytorch

53 Reply by UHD

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dlr5668 wrote:

realtime broke for me after some experiments and now hungs up big_smile

3.5 and 3.8 models takes ~5 GB VRAM under pytorch


Thanks.

Could you try some smaller: 240p 360p or 480p file?

  • Thanks: 230
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD wrote:
dlr5668 wrote:

realtime broke for me after some experiments and now hungs up big_smile

3.5 and 3.8 models takes ~5 GB VRAM under pytorch


Thanks.

Could you try some smaller: 240p 360p or 480p file?

source https://www.youtube.com/watch?v=4Ps6nV4wiCE
model 3.5 scale 1.0
1280x720  50 fps 3.2 GB VRAM
854x480 92 fps 1.7GB VRAM

No luck with playing. Doesnt matter what res or windowed / fullscreen

55 Reply by UHD (edited by UHD 26-09-2021 21:27:03)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Thank you very much for checking.

Some time ago you wrote:

dlr5668 wrote:

At least my mpv is not crashing big_smile rewind takes 3-4 sec though

Does this mean that with scale=0.5 you were able to successfully interpolate in real time playback?

  • Thanks: 230
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD wrote:

Thank you very much for checking.

Some time ago you wrote:

dlr5668 wrote:

At least my mpv is not crashing big_smile rewind takes 3-4 sec though

Does this mean that with scale=0.5 you were able to successfully interpolate in real time playback?

yep https://www.playbook.com/vadash/NPbJXaD … ABj6bV6hwQ

57 Reply by UHD

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Then we know that:

1. vs-rife can interpolate in real time, but at scale=0.5
2. vs-rife has a problem with realtime interpolation at scale=1, even when fps with reencoding and VRAM is sufficient.

https://www.playbook.com/vadash/NPbJXaD … ABj6bV6hwQ

requires a google account, which I do not have

  • Thanks: 230
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD wrote:

Then we know that:

1. vs-rife can interpolate in real time, but at scale=0.5
2. vs-rife has a problem with realtime interpolation at scale=1, even when fps with reencoding and VRAM is sufficient.

https://www.playbook.com/vadash/NPbJXaD … ABj6bV6hwQ

requires a google account, which I do not have

https://www.playbook.com/s/vadash/wcVXb … ABj6bV6hwQ

59 Reply by UHD (edited by UHD 27-09-2021 13:06:04)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

del

60 Reply by UHD (edited by UHD 27-09-2021 13:06:49)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

del

  • Thanks: 230
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I solved freezing. You will need to disable black bar detection

C:\Program Files (x86)\SVP 4\mpv64\mpv conf

input-ipc-server=mpvpipe

# hardware video decoder
hwdec=auto-copy
hwdec-codecs=all

vo=gpu

# high quality video output, require rather fast video card
profile=gpu-hq

# D3D11 renderer (default) is required for the HDR playback
#gpu-api=d3d11

# use VULKAN API if you have problems with the default d3d11 renderer
gpu-api=vulkan

# miscellaneous options

# comment this one if you want mpv to remember the playback position
save-position-on-quit=yes

# fixes audio desync
hr-seek-framedrop=no

# "ReClock" replacement
video-sync=display-resample
video-sync-max-video-change=25

C:\Program Files (x86)\SVP 4\script\base.py

def interpolate(clip):
# input_um - original frame in 4:2:0
# input_m  - cropped and resized (if needed) frame
# input_m8 - input_m converted to 8-bit
#RESIZE-CODE
    input_m = input_m.misc.SCDetect()
    input_m = input_m.resize.Bicubic(format=vs.RGBS, matrix_in_s="709")
    smooth = RIFE(input_m,model_ver=3.8,fp16=False,scale=1.0,device_index=rife_gpu)
    smooth = smooth.resize.Point(format=input_um.format.id, matrix_s="709")
    smooth = core.svp2.SmoothFps_RIFE(smooth,smoothfps_params,src=input_um,fps=src_fps*2)
    return smooth
#FORCE-CFR
smooth =  interpolate(clip)

switch profile when mpv in windowed mode. 70ti is enough to watch 720p with few drops (about 10 per 2 minutes). I ll need to increase buffer

62 Reply by UHD

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Thank you so much. You are a genius!
I see you sat on the problem a little longer and found a solution!

Thanks also for uploading the detailed configuration files. This will be very useful for me and others who would like to use RIFE for real-time interpolation.

Thanks to Chainik for implementing RIFE AI interpolation in SVP!
Thanks to dlr5668 for testing and finding a working configuration for real-time HD file interpolation!

63 Reply by UHD (edited by UHD 27-09-2021 14:49:30)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Finally, it can be stated that vs-rife allows real-time interpolation of HD files without the need for image scaling!

NVIDIA GeForce RTX 3070 Ti
SVP with mpv with vs-rife https://github.com/HolyWu/vs-rife
real time playback with x2 interpolation:
720p - yes, with few drops (about 10 per 2 minutes)
RIFE model: 3.8
scale=1.0
FP32

  • Thanks: 1622
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dlr5668
I solved freezing. You will need to disable black bar detection

nope, this would be too easy...
it just hangs at random times. with black bars detection the chances are higher just because two script initializations instead of one.

  • Thanks: 230
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

dlr5668
I solved freezing. You will need to disable black bar detection

nope, this would be too easy...
it just hangs at random times. with black bars detection the chances are higher just because two script initializations instead of one.

every time I click https://i.imgur.com/FGJ2AbW.png it will 100% freeze. I watched 2h realtime and so far it never hungs. Well unless I decide to rewind big_smile its 100% freeze as well

  • Thanks: 1622
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

> Well unless I decide to rewind  big_smile its 100% freeze as well

this is what I'm saying
every re-initialization will hangs
and for me the first-time initialization hangs too in 99% of times

  • Thanks: 230
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

> Well unless I decide to rewind  big_smile its 100% freeze as well

this is what I'm saying
every re-initialization will hangs
and for me the first-time initialization hangs too in 99% of times

did u try vulkan render ?

68 Reply by UHD (edited by UHD 27-09-2021 22:51:07)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dlr5668 wrote:

I watched 2h realtime and so far it never hungs.


You read my mind and answered a question that has not yet been asked smile



dlr5668 wrote:

Well unless I decide to rewind big_smile its 100% freeze as well


Does it also freeze on pause and when resuming playback?
What about fast forward?

Have you checked the VRAM and GPU consumption when freezing, not general but in detail:
compute_0
compute_1
CUDA
?

Is the freezing temporary or permanent (over 1 minute)?

Does it freeze the same way on light files like 180p (scale=1.0)?

69 Reply by UHD (edited by UHD 27-09-2021 22:52:06)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I also continue to encourage other SVP users with vs-rife to test real-time playback interpolation.

Please, share your results and experiences.

70 Reply by UHD

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dlr5668 wrote:
Chainik wrote:

> Well unless I decide to rewind  big_smile its 100% freeze as well

this is what I'm saying
every re-initialization will hangs
and for me the first-time initialization hangs too in 99% of times

did u try vulkan render ?


Did it help?

71 Reply by UHD

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dlr5668 wrote:

Well unless I decide to rewind big_smile its 100% freeze as well

Does it also freeze on pause and when resuming playback?

For me this is a very important question. I can live without rewind, but if pause would cause permanent freezing, that would be a serious problem.

  • Thanks: 6
  • Thanks for the post: 2

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Hello,

My computer is a laptop AMD 5800H with NVIDIA RTX 3070 Mobile (115W).

I have installed SVP Pro with the RIFE AI Engine plugin.

My tests results:

Video 512x384
SVP index 1.0
GPU load average 80%
VRAM usage 0.7GB
Good result, smooth and no artifacts smile

Video 720p
SVP index 0,42
GPU load average 100%
VRAM usage 1,3GB

Video 1080p
SVP index 0,20
GPU load average 100%
VRAM usage 2.3GB

Video 4k
SVP index 0,19
GPU load average 100%
VRAM usage 2.6GB

All tests were made with FP16 and MPV.

73 Reply by UHD

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Thank you very much XKCorp for testing and sharing your results smile

Could you please further clarify the parameters with which you performed the test with 512x384 resolution?
I am interested in the following data:

1. Which filter did you use?

Chainik wrote:

in the RIFE video profile set
- TTA = disabled -> use RFIE/ncnn/Vulkan
- TTA = enabled -> use RIFE/torch/CUDA.

https://www.svp-team.com/forum/viewtopi … 023#p79023

2. Which scale? 1.0 or 0.5?

3. Which RIFE model? 3.8 or other?

4. Did you try FP32 with 512x384 resolution? Or only FP16 was able to interpolate in real time with SVP index 1.0?

5. Did it freeze on pause and when resuming playback? If yes: temporary (how long) or permanent (over 1 minute)?

6. Did it freeze on rewind? If yes: temporary (how long) or permanent (over 1 minute)?

7. Did you freeze on fast forward? If yes: temporary (how long) or permanent (over 1 minute)?

  • Thanks: 6
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Hi,

1. I tested with the default RIFE filter ncnn/Vulkan.

2. I din't change the scale, should be 1.

3. AI model "generic" is selected, don't know the version.

4. FP16 (GPU@80%) and FP32 (GPU@95%) are able to interpolate in real time with SVP Index 1.

5. No freeze on pause and resume.

6/7. Didn't try rewind or fast forward.

I followed the instructions to install torch/CUDA but when selecting "TTA = enabled" as others reported this filter is not working in real-time.

75 Reply by UHD (edited by UHD 13-10-2021 22:49:54)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Thanks a lot for answering all my points and clarifying the test parameters.

I am most interested in real-time interpolation with the RIFE PyTorch/CUDA filter especially because of the 3x better performance than with the default RIFE ncnn/Vulkan filter.

Have you tried the Chainik recommendations from these two posts:

https://www.svp-team.com/forum/viewtopi … 023#p79023
https://www.svp-team.com/forum/viewtopi … 183#p79183

and especially the settings and recommendations of dlr5668, who is the only one so far who has managed to run real-time interpolation with the RIFE PyTorch/CUDA filter:

https://www.svp-team.com/forum/viewtopi … 315#p79315
https://www.svp-team.com/forum/viewtopi … 321#p79321

?