326 Reply by UHD (edited by UHD 30-10-2022 14:36:28)

  • UHD
  • Member
  • Offline
  • Thanks: 39
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Pezede wrote:

(Transcoding speed is around 45fps for 1920*1080@23.976 --> 1920@1080@119.88)

Pezede wrote:

https://i.imgur.com/1QMZ4fL.png

45/5*4=36
75,8/2*1=37,9

The results are repeatable, but clearly something is wrong!

Even NVIDIA GeForce RTX 3070 Ti graphics card performs better - 45 new interpolated frames per second:
https://www.svp-team.com/forum/viewtopi … 588#p80588

On the other hand, NVIDIA GeForce RTX 4090 in combination with RIFE is a real tornado!

Below the benchmark tests for RIFE interpolation (fastmode True, ensemble False):

38 new fps (1080p) for 3090+5950x (CUDA+ffmpeg+FrameEval+vs_threads=20) (rife4.0)

63 new fps (1080p) for 3090+5950x (C++ NCNN+vs_threads=20+ncnn_threads=8) (rife4.0)

164 new fps (1080p) for 4090+5950x (ncnn+2 threads+4 vs threads+ffmpeg (ultrafast)) (rife4.6)

Source: https://github.com/styler00dollar/VSGAN-tensorrt-docker

Pezede, I hope you will stay with us for some time on this forum. I have a lot of questions for you! However, the most important thing now is to find the reason for such a large disparity in test results.

Hence the first and most important question:

Are you using the NCNN version of RIFE or the CUDA version?

327 Reply by Pezede (edited by Pezede 30-10-2022 15:06:41)

  • Thanks: 19
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD wrote:
Pezede wrote:

(Transcoding speed is around 45fps for 1920*1080@23.976 --> 1920@1080@119.88)

Pezede wrote:

https://i.imgur.com/1QMZ4fL.png

45/5*4=36
75,8/2*1=37,9

The results are repeatable, but clearly something is wrong!

Even NVIDIA GeForce RTX 3070 Ti graphics card performs better - 45 new interpolated frames per second:
https://www.svp-team.com/forum/viewtopi … 588#p80588

On the other hand, NVIDIA GeForce RTX 4090 in combination with RIFE is a real tornado!

Below the benchmark tests for RIFE interpolation (fastmode True, ensemble False):

38 new fps (1080p) for 3090+5950x (CUDA+ffmpeg+FrameEval+vs_threads=20) (rife4.0)

63 new fps (1080p) for 3090+5950x (C++ NCNN+vs_threads=20+ncnn_threads=8) (rife4.0)

164 new fps (1080p) for 4090+5950x (ncnn+2 threads+4 vs threads+ffmpeg (ultrafast)) (rife4.6)

Source: https://github.com/styler00dollar/VSGAN-tensorrt-docker

Pezede, I hope you will stay with us for some time on this forum. I have a lot of questions for you! However, the most important thing now is to find the reason for such a large disparity in test results.

Hence the first and most important question:

Are you using the NCNN version of RIFE or the CUDA version?


Hey, I will stay around and provide as much feedback as necessary, as I'd be quite happy to have as much performance as possible :-)

I'm using the rife version (ncnn unless I'm wrong) that comes embedded with SVP, I'm currently trying to set up other models and see if that changes anything.

328 Reply by UHD (edited by UHD 30-10-2022 15:21:49)

  • UHD
  • Member
  • Offline
  • Thanks: 39
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Pezede wrote:

Hey, I will stay around and provide as much feedback as necessary, as I'd be quite happy to have as much performance as possible :-)

Thanks a lot!

Pezede wrote:

I'm using the rife version (ncnn unless I'm wrong) that comes embedded with SVP, I'm currently trying to set up other models and see if that changes anything.

Make sure you are using the latest version of SVP: 4.6.0.220 2022-10-10: https://www.svp-team.com/news/

It only has the NCNN RIFE version: https://www.svp-team.com/forum/viewtopi … 052#p81052

329 Reply by dlr5668 (edited by dlr5668 30-10-2022 15:47:19)

  • Thanks: 230
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Pezede wrote:
Chainik wrote:

I mean, switch the video profile to movie x2 and start transcoding. Will it only give 72 fps?

Yes that's the case:

https://i.imgur.com/1QMZ4fL.png

Something wrong here. 4090 should be twice as fast (at least). Note that a have few vms in background + game

https://i.imgur.com/EIiZM1L.png

330 Reply by Pezede (edited by Pezede 30-10-2022 16:16:51)

  • Thanks: 19
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I have reinstalled SVP (4.6.0.220 2022-10-10) and added the RIFE AI engine component again (Can confirm that this is the ncnn 4.4), performance is as follows now with the same file and encoder settings:

With 1 gpu thread:

https://i.imgur.com/iX3LJka.png

With 2 gpu threads:

https://i.imgur.com/5ofWuhj.png

With 4 gpu threads:

https://i.imgur.com/e7RmuWD.png


It seems that reinstalling did the trick, as this is significantly better than before :-)

331 Reply by UHD (edited by UHD 30-10-2022 16:31:51)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

GREAT!!!

Try using the fastest H264/AVC encoding preset and model 4.6


Pezede wrote:

transcoding doesn't take too long with a lossless NVENC H265 preset

Lossless encoding is best for enjoying videos later, but for testing it's best to use the fastest encoding that doesn't overtax the CPU and SSD. See the RIFE test results at the bottom of the page: https://github.com/styler00dollar/VSGAN-tensorrt-docker The way you encode has an impact on fps.

332 Reply by UHD

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Now a question for everyone: is it possible to set the number of GPU threads higher than 4?

333 Reply by Pezede (edited by Pezede 30-10-2022 16:59:19)

  • Thanks: 19
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD wrote:

Now a question for everyone: is it possible to set the number of GPU threads higher than 4?

I did try setting it higher than 4 in here:

https://i.imgur.com/Od6Onlx.png

https://i.imgur.com/OmIXqC4.png

Performance is unchanged from 4 to 8 threads, and it is not possible to go higher than 8.

I have tried the ultrafast encoding preset, performance did not change but I will be using it from now on as it is the fastest.

Rife 4.6 seems a tiny bit slower on my end, averaging 105fps with it but it has occasional drops to 95-100.

334 Reply by UHD (edited by UHD 30-10-2022 17:38:23)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Have you checked threads 5, 6, 7 and 3?

Try 4.6 ensemble False, there should be more fps.

335 Reply by UHD

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Pezede wrote:

I have tried the ultrafast encoding preset, performance did not change but I will be using it from now on as it is the fastest.

"performance did not change" and "is the fastest" how is this possible at the same time?

336 Reply by Pezede (edited by Pezede 30-10-2022 18:03:49)

  • Thanks: 19
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD wrote:
Pezede wrote:

I have tried the ultrafast encoding preset, performance did not change but I will be using it from now on as it is the fastest.

"performance did not change" and "is the fastest" how is this possible at the same time?

What I mean is that the amount of fps I was getting didn't change between the lossless and ultrafast encoding settings because I probably wasn't being bottlenecked by the encoding capabilities of my system, despite the encoder ultrafast setting probably being capable of outputting more frames outside of interpolation scenarios.

For example when I transcode with ultrafast (non Rife interpolation) I get 400+ fps.

I'm not entirely sure how all the elements of the chain work though so that's just me theorizing.

UHD wrote:

Have you checked threads 5, 6, 7 and 3?

Try 4.6 ensemble False, there should be more fps.

Performance seems to max out at 3 threads and stay constant beyond that (I have tried all the settings).


On 4.6 EnsembleTrue gives me around half the performance (55fps) while EnsembleFalse gives me the results I previously reported (slightly lower than 4.4 performance)

337 Reply by UHD (edited by UHD 30-10-2022 18:29:12)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Have you tried to do a real-time RIFE interpolation test for 1080p:
x3
x4
x5
?

  • Thanks: 19
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD wrote:

Have you tried to do a real-time RIFE interpolation test for 1080p:
x3
x4
x5
?

Realtime doesn't seem to work well beyond x2 on that same anime source, there are desyncs and choppy frames on x2.5, x3, x4,...

Using the madVR renderer with no fancy settings applied.

339 Reply by UHD

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Very strange...

Could you please check if it is possible to interpolate x2 in real time with RIFE:

LG 4K HDR Demo - New York.ts
File size: 448 MiB
Duration: 1 min 13 sec
Overall bit rate: 51.4 Mbps
HDR format: SMPTE ST 2086, HDR10 compatible
Width: 3 840 pixels
Height: 2 160 pixels
Frame rate: 25.000 FPS
Color space: YUV
Chroma subsampling: 4:2:0
Bit depth: 10 bits
Direct link: https://drive.google.com/file/d/1dfR5TT … _bGfEXUvJ/
Source: http://hdr4k.blogspot.com/

340 Reply by UHD (edited by UHD 30-10-2022 19:11:05)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Try not to use madVR, as it actually takes up a lot of GPU resources sometimes. If madVR and RIFE are fighting for GPU resources this could be a bottleneck.

  • Thanks: 19
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD wrote:

Very strange...

Could you please check if it is possible to interpolate x2 in real time with RIFE:

LG 4K HDR Demo - New York.ts
File size: 448 MiB
Duration: 1 min 13 sec
Overall bit rate: 51.4 Mbps
HDR format: SMPTE ST 2086, HDR10 compatible
Width: 3 840 pixels
Height: 2 160 pixels
Frame rate: 25.000 FPS
Color space: YUV
Chroma subsampling: 4:2:0
Bit depth: 10 bits
Direct link: https://drive.google.com/file/d/1dfR5TT … _bGfEXUvJ/
Source: http://hdr4k.blogspot.com/

Just tried it with the MPC-HC default video renderer, footage is very choppy with frameskipping in *2.

342 Reply by Pezede (edited by Pezede 30-10-2022 19:27:42)

  • Thanks: 19
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Actually I just noticed that compute use is basically non-existent in realtime:

https://i.imgur.com/7l7JD1N.png

It goes to and stays at 100% while transcoding, but maxes out at 40-50% and often goes to 0-18% in real-time.

343 Reply by UHD (edited by UHD 31-10-2022 00:35:22)

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Pezede wrote:

Just tried it with the MPC-HC default video renderer, footage is very choppy with frameskipping in *2.

Please try 3 variations of the test for the above 4K HDR file:

1. Normal playback without interpolation on a monitor or TV set up with HDR. Only pay attention to the colours in this test.

2A. Real-time interpolation with RIFE. Even if footage is very choppy with frameskipping it is interesting to see if colours have changed compared to test number 1. Try using Triple Bufferinghttps://tweakguides.pcgamingwiki.com/images/GGDSG_20.jpg
in the graphics card settings to improve smoothness: https://tweakguides.pcgamingwiki.com/Graphics_10.html

2B. Same as 2A only if it continues to stutter reduce the size before interpolation e.g. "Decrease to HD" https://www.svp-team.com/wiki/Manual:Resizing for testing purposes whether HD colours will be retained when interpolating in real time using RIFE.

3. Reencoding with RIFE. I'm interested in the fps during transcoding 4K file, and whether later playing back normally this file, which will already have a fixed 50 fps will retain the colours from test 1 or 2.

344 Reply by UHD

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Most important: for real-time testing, it is best to use mpv player and for 4K HDR files only mpv player: https://www.svp-team.com/wiki/SVP:4K_an … DR_display

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Yep rife works properly with mpv only

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

can't see any problems with MPC-HC + Vapoursynth + MPC VR

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

How do you guys get model 4.6 on svp? The only model I see is Generic (4.4).

  • Thanks: 2

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

> Although, anywhere above 48 fps my GPU usage sky rockets from 46% to 90%

because x2.5 interpolation (i.e. 24->60 fps) need generating TWO times more intermediate frames than x2 (24->48)

> I added another model from github and when selected I get a flickering sensation on any video

then add another model big_smile and post a bug report on the Github regarding the flickering one

The flickering is due to the incompatibility between current rife_vs.dll and model 4.6. I can reproduce this flickering problem on Windows 11. Since most other people are ok, maybe the problem is with Win 11 but not with Win 10.

  • Thanks: 230
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

weiw26 wrote:
Chainik wrote:

> Although, anywhere above 48 fps my GPU usage sky rockets from 46% to 90%

because x2.5 interpolation (i.e. 24->60 fps) need generating TWO times more intermediate frames than x2 (24->48)

> I added another model from github and when selected I get a flickering sensation on any video

then add another model big_smile and post a bug report on the Github regarding the flickering one

The flickering is due to the incompatibility between current rife_vs.dll and model 4.6. I can reproduce this flickering problem on Windows 11. Since most other people are ok, maybe the problem is with Win 11 but not with Win 10.

Shouldnt be a problem. I run win 11 as well

350 Reply by UHD

  • UHD
  • Member
  • Offline
  • Thanks: 39

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

It's good to know that all is well, as I too intend to use Windows 11 and model 4.6 in the future.