• Thanks: 47

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Criptaike wrote:

Hello, so I've wanted to try the rife engine, the vulkan version works fine, but I sure want the tensorRT version, so when selecting rife after opening a movie, a cmd window pops up and mpc-hc crashes, it just goes to shit and it doesn't really say that there's an error or something, try to search the problem on google and this forum and no luck. Here is a gif of all that happening -> https://imgur.com/a/VHVCvBZ

Any ideas on what's happening? The vulkan one works.

Thanks

Need to wait for 3-4 minutes every time that cmd window pops up.

  • Thanks: 13

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

It usually does it for all the different frame sizes. Once its done it once per frame size though it doesn't happen again.

  • Thanks: 230
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

zerosoul9901 wrote:
dlr5668 wrote:
Chainik wrote:

> while v2 models significantly reduces the playback seek time, improve performance

dunno, I can't see any improvements
not "significantly", not even slightly

----
unpack "rife_v2" into SVP 4\rife\models (i.e. there must be SVP 4\rife\models\rife_v2\rife_v4.6.onnx)
replace SVP 4\rife\vsmlrt.py

switch back to "v1" models - rename "rife_v2" folder to any other name, like "_rife_v2"

v2 takes 7.5/8 and crashes at seek, v1 stays at 3.5/8 as it should

Don't use dynamic shapes, it doesn't seem to work well with v2 models and requires more VRAM usage compared to v1, use static shapes instead, and use flag force_fp16 instead of fp16 during inference, reducing memory usage and engine build time during engine build, you can also add _implementation=2 flag to easily switch v2 models.

Thanks works great. Same VRAM requirements and much faster almost instant rewind

https://i.imgur.com/NM5g5Zk.png

704 Reply by dlr5668 (edited by dlr5668 11-03-2023 08:58:46)

  • Thanks: 230
  • Thanks for the post: 2

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I thinks its worth including in SVP. Performance boost = false should call V1 model (for now) and true V2 model

https://i.imgur.com/5yQiUhR.mp4

Post's attachments

vsmlrt.py 53.63 kb, 179 downloads since 2023-03-11 

  • Thanks: 1622
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

> Performance boost = false should call V1 model (for now) and true V2 model

...or just wait for a few days and they'll fix v2 for dynamic shapes and probably other issues (if any) wink

  • Thanks: 1
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

And please include also a switch for using "Ensemble" on or off as well...

707 Reply by dawkinscm (edited by dawkinscm 12-03-2023 18:39:04)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I've added the v2 models and seeking peformance is massively improved. There is no longer any difference in seeking whether I'm using RIFE or not. But I'm a little confused because it works with my TensorRT engine as well even though there isn't a direct option to choose this AI model like there is for Vulkan.

Just as an aside, it's a pity that when using the Tensor engine, it has very noticeable artefacts when watching the movie Gravity with the stars in space. They shimmer like a black coat is being run across them. This doesn't happen on Vulkan or Automatic and it's annoying.

708 Reply by svr

  • svr
  • Novice
  • Offline
  • Thanks: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Tried this new feature. I really liked the quality. Even on my not-so-new PC with RTX3060, it works in realtime with 1080 video. But I have a problem. After 30-60 minutes, the video driver freezes with the following events:

The description for Event ID 0 from source nvlddmkm cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
Faulting application name: mpv.exe, version: 2.0.0.0, time stamp: 0x00000000
Faulting module name: nvinfer.dll, version: 0.0.0.0, time stamp: 0x638eab04
Exception code: 0xc0000409
Fault offset: 0x00000000046172f5
Faulting process id: 0x54
Faulting application start time: 0x01d954a7b9c628bd
Faulting application path: C:\Program Files (x86)\SVP 4\mpv64\mpv.exe
Faulting module path: C:\Program Files (x86)\SVP 4\rife\vsmlrt-cuda\nvinfer.dll

I tried different versions of video drivers, tried different players (MPV, MPC-HC, MPC-BE), the result is the same everywhere - it works well, but after a while it crashes.

709 Reply by dawkinscm (edited by dawkinscm 12-03-2023 22:01:15)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

svr wrote:

Tried this new feature. I really liked the quality. Even on my not-so-new PC with RTX3060, it works in realtime with 1080 video. But I have a problem. After 30-60 minutes, the video driver freezes with the following events:

The description for Event ID 0 from source nvlddmkm cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
Faulting application name: mpv.exe, version: 2.0.0.0, time stamp: 0x00000000
Faulting module name: nvinfer.dll, version: 0.0.0.0, time stamp: 0x638eab04
Exception code: 0xc0000409
Fault offset: 0x00000000046172f5
Faulting process id: 0x54
Faulting application start time: 0x01d954a7b9c628bd
Faulting application path: C:\Program Files (x86)\SVP 4\mpv64\mpv.exe
Faulting module path: C:\Program Files (x86)\SVP 4\rife\vsmlrt-cuda\nvinfer.dll

I tried different versions of video drivers, tried different players (MPV, MPC-HC, MPC-BE), the result is the same everywhere - it works well, but after a while it crashes.

I thought you might be running out of memory, but I'm running on a 3080 with 4GB less memory than the 3060. How many GPU threads are you using? With 8GB I've found that any more than 1 GPU thread takes up too much processing and memory. I assume that you are using the TensorRT engine? Also using rife v2 took my GPU processing below 90% while also removing the seek delays I had with rife.

710 Reply by svr

  • svr
  • Novice
  • Offline
  • Thanks: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dawkinscm wrote:

I thought you might be running out of memory, but I'm running on a 3080 with 4GB less memory than the 3060. How many GPU threads are you using? With 8GB I've found that any more than 1 GPU thread takes up too much processing and memory. I assume that you are using the TensorRT engine? Also using rife v2 took my GPU processing below 90% while also removing the seek delays I had with rife.

I use 4 GPU threads. If I use only one, then my system starts to slow down. With 4 everything works smoothly. At the same time, I carefully monitor the GPU memory and the load on the GPU chip. No more than 1.5GB VRAM is used at 11-14% GPU load for 1080p video. The model version is 4.6 (v1). "rife_trt_boost" is enabled. According to monitoring data, GPU memory usage and load on it is not a problem. Of course I use TensorRT, but for example, when the MPV player is running, this warning is displayed:

vstrt: TensorRT version mismatch, built with 2135 but loaded with 2136; continue but fingers crossed...

I used v1 version models that come as part of SVP. I also tried the 4.4 model, tried to turn off the boost, but still after a while the video driver crashes.

Display driver nvlddmkm stopped responding and has successfully recovered.

711 Reply by aloola (edited by aloola 13-03-2023 04:28:16)

  • Thanks: 86

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Rife V2
https://media.discordapp.net/attachments/290709370600423424/1084693213794795610/image.png
old 58.6FPS >61.3FPS

now I can watch 4k@30 x2 in realtime with 3070ti

Rife V1
https://cdn.discordapp.com/attachments/ … /image.png

712 Reply by dawkinscm (edited by dawkinscm 13-03-2023 08:34:51)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

svr wrote:

Of course I use TensorRT, but for example, when the MPV player is running, this warning is displayed:

vstrt: TensorRT version mismatch, built with 2135 but loaded with 2136; continue but fingers crossed...

I used v1 version models that come as part of SVP. I also tried the 4.4 model, tried to turn off the boost, but still after a while the video driver crashes.

Display driver nvlddmkm stopped responding and has successfully recovered.

It looks like you have been tinkering with some back end files? Anyway, I hope the Devs can come up with a solution but if it was me I would uninstall the rife/Tensor RT stuff first. If that doesn't work then I would uninstall SVP completely before reinstalling it.

svr wrote:

I use 4 GPU threads. If I use only one, then my system starts to slow down.

Once you get it working I suggest you install rife v2. You might be able to reduce the number of GPU threads required.

  • Thanks: 2

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Someone got it working with MadVR?

I'm using MPC-BE with MadVR and SVP Rife @ RTX 4080, but I cant get more than 1080p x2. 1080p 60fps would be perfect (120hz TV), but I get Framedrops with this.

Even while using TensorRT 4.6 with 2 GPU Cores it doesn't seems like the GPU works at full load. Taskmanager CUDA is always around 0% and the GPU itself won't go higher than 60-70%.

Anything i could do to get more performance? In the Guide it says to use Vapoursynth Filter. Will this even work with MadVR?

714 Reply by dawkinscm (edited by dawkinscm 13-03-2023 19:07:12)

  • Thanks: 200
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Grinchy wrote:

Someone got it working with MadVR?

I'm using MPC-BE with MadVR and SVP Rife @ RTX 4080, but I cant get more than 1080p x2. 1080p 60fps would be perfect (120hz TV), but I get Framedrops with this.

Even while using TensorRT 4.6 with 2 GPU Cores it doesn't seems like the GPU works at full load. Taskmanager CUDA is always around 0% and the GPU itself won't go higher than 60-70%.

Anything i could do to get more performance? In the Guide it says to use Vapoursynth Filter. Will this even work with MadVR?

I got MPC-BE and MadVR working before with Vapoursynth but could not get a smooth 1080p/60 on my 3080 (although I think that might change with Rife v2). You want more performance, then switch to MPV where I was able to get a smooth 2160p/60 running at about 90% GPU. As I said in a previous post, the key for me was using 1 GPU thread. Any more than that then I started to run into issues but this has also improved with Rife v2.

I recently tried to get it working again with MPC-BE and Vapoursynth but it kept on crashing. I think it's to do with my Python install but I'm settled with the much better performance and picture quality of MPV so I haven't tried to hard to get it working again with MPC-BE and MadVR.

With Rife 2 I now get 4k/60 running at well below 90% GPU.

715 Reply by svr (edited by svr 13-03-2023 22:32:45)

  • svr
  • Novice
  • Offline
  • Thanks: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Just in case, I found the reason for the driver hanging. My GPU undervolting settings were not suitable for such a load. There were no problems at all in games, but when using RIFE, the driver started to fail.

716 Reply by dawkinscm (edited by dawkinscm 14-03-2023 17:35:57)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Just updated to the latest pre-release TensorRT library. Start up time is now about 5 seconds and the seek is just shy of real time but very choppy so not worth the upgrade to the pre-release yet.

717 Reply by Drakko01 (edited by Drakko01 14-03-2023 20:13:35)

  • Thanks: 49

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dawkinscm wrote:

Just updated to the latest pre-release TensorRT library. Start up time is now about 5 seconds and the seek is just shy of real time but very choppy so not worth the upgrade to the pre-release yet.

HI, I I like to try it, can you explain what I need to do for updating to the tensorrt 8.5.3.1 latest version. Thanks.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Drakko01 wrote:
dawkinscm wrote:

Just updated to the latest pre-release TensorRT library. Start up time is now about 5 seconds and the seek is just shy of real time but very choppy so not worth the upgrade to the pre-release yet.

HI, I I like to try it, can you explain what I need to do for updating to the latest version. Thanks.

As I said, I don't recommend it because it caused some issues and I've ended up reinstalling SVP lol. Although thinking back on it, maybe some of that was my doing. Either way it works well enough with Rife v2. But if you insist then Google "v14 latest TensorRT library" smile

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dawkinscm wrote:

Just updated to the latest pre-release TensorRT library. Start up time is now about 5 seconds and the seek is just shy of real time but very choppy so not worth the upgrade to the pre-release yet.

What do you mean start time? Once cached its below 2s for me

  • Thanks: 49

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

ohh You mean this https://docs.nvidia.com/deeplearning/te … index.html , for this dont you have to update all this to
cuDNN 8.8.0
TensorFlow 1.15.5
PyTorch 1.13.1
ONNX 1.12.0
Im no expert i think i have to wait, when Chainik send us via svp4 update.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dlr5668 wrote:
dawkinscm wrote:

Just updated to the latest pre-release TensorRT library. Start up time is now about 5 seconds and the seek is just shy of real time but very choppy so not worth the upgrade to the pre-release yet.

What do you mean start time? Once cached its below 2s for me

You may be right about caching but it seems to be a little variable for me sometimes.

  • Thanks: 2

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Someone else with Short Micro Lags?

It's perfectly smooth, but sometimes there are some short lags. Using an 5900x with RTX4080 and TensorRT

  • Thanks: 16

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Grinchy wrote:

Someone else with Short Micro Lags?

It's perfectly smooth, but sometimes there are some short lags. Using an 5900x with RTX4080 and TensorRT

I was getting something similar when testing Dolby vision videos. Regular HDR or regular SDR videos I have no issues. Same CPU, but a 4090 instead

  • Thanks: 49

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Has anyone noticed a visual change or improvement with the latest 2 nvidia drivers ie. 531.29/531.41, didn't change the settings but the video looks better using rife/mpc-hc/madvr

725 Reply by Blackfyre (edited by Blackfyre 05-04-2023 21:13:07)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Do the versions from the study correspond to the ones we use? I assume not.

The scientific study is up to version 12:

https://arxiv.org/abs/2011.06294