Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

aloola
> I just want to report that I got 30fps on 4k transcoding

and you're capped by the encoder speed
try h264 instead
or set preset=fast in 'advanced' mode, cause it's set to "slow" by default

427 (edited by aloola 08-01-2023 19:15:56)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

aloola
> I just want to report that I got 30fps on 4k transcoding

and you're capped by the encoder speed
try h264 instead

it did 10% better smile

https://cdn.discordapp.com/attachments/290709370600423424/1061722152560246834/image.png
https://cdn.discordapp.com/attachments/290709370600423424/1061724906573803663/image.png

428

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

aloola wrote:

it did 10% better smile

Why not try even faster settings?

ultrafast
superfast
veryfast
faster

429 (edited by grobalt 08-01-2023 23:05:12)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

aloola wrote:
Chainik wrote:

aloola
> I just want to report that I got 30fps on 4k transcoding

and you're capped by the encoder speed
try h264 instead

it did 10% better smile

https://cdn.discordapp.com/attachments/290709370600423424/1061722152560246834/image.png
https://cdn.discordapp.com/attachments/290709370600423424/1061724906573803663/image.png

do you have discord ? please show me your profile setup .. you said 3 gpu threads, SVP allowes 1,2 and 4

i tried to replicate what you did .. advanced encoding with preset fast:
23:53:34.822: ===== Starting mpv ======
23:53:34.822: Command line: C:\Program Files (x86)\SVP 4\mpv64\mpv.exe D:/The Northman (2022).mkv --o=D://The Northman (2022).SVP.temporary.mkv --no-audio --no-sub --no-sub-auto --input-ipc-server=mpvencodepipe --input-media-keys=no --no-msg-color --vf=vapoursynth:[C:\Users\patri\AppData\Roaming\SVP4\scripts\ffff.py]:4:12 --of=matroska --ovc=h264_nvenc --ovcopts=b=10000000,preset=slow,profile=high,rc=vbr,maxrate=13000000,bufsize=20000000,time_base=1000/47952,threads=12
23:53:34.879: (+) Video --vid=1 (hevc 3840x2160 23.976fps)
23:53:34.879: Audio --aid=1 --alang=deu (*) 'Surround 7.1' (truehd 8ch 48000Hz)
23:53:34.879: Audio --aid=2 --alang=eng 'Surround 7.1' (truehd 8ch 48000Hz)
23:53:34.879: Audio --aid=3 --alang=eng 'Stereo' (ac3 2ch 48000Hz)
23:53:34.879: Subs --sid=1 --slang=deu (hdmv_pgs_subtitle)
23:53:34.879: Subs --sid=2 --slang=deu (hdmv_pgs_subtitle)
23:53:34.879: Subs --sid=3 --slang=deu (hdmv_pgs_subtitle)
23:53:34.879: Subs --sid=4 --slang=eng (hdmv_pgs_subtitle)
23:53:34.879: Subs --sid=5 --slang=eng (hdmv_pgs_subtitle)
23:53:34.879: Subs --sid=6 --slang=eng (hdmv_pgs_subtitle)
23:53:34.879: File tags:
23:53:34.879: Title: The Northman (2022)
23:53:35.954: vstrt: TensorRT version mismatch, built with 2135 but loaded with 2136; continue but fingers crossed...
23:53:36.859: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-pro … l#env-vars
23:53:36.941: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-pro … l#env-vars
23:53:39.511: [autoconvert] Converting yuv420p10 -> p010
23:53:39.521: VO: [lavc] 3840x2160 p010
23:53:39.521: [vo/lavc] Opening encoder: NVIDIA NVENC H.264 encoder [h264_nvenc]
23:53:39.725: [ffmpeg] h264_nvenc: 10 bit encode not supported
23:53:39.749: [ffmpeg] h264_nvenc: No capable devices found
23:53:39.749: [vo/lavc] Could not initialize encoder.
23:53:39.749: Could not initialize video chain.
23:53:39.947: Video: no video
23:53:39.947: [encode] no data written to target file
23:53:39.947: Exiting... (Interrupted by error)
23:53:40.128: (!!!) Intermediate file may be broken: D:\\The Northman (2022).SVP.temporary.mkv
23:53:40.128: ===== mpv exited with code 2 =====

*edit* error found .. movies was 10bit and ... this does not work. so i have to use convert to 8bit or .. use h265 instead, see below

430

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

grobalt wrote:

Thanks, will invest later tonight how to apply this smile do you read the github thread ?

I hope a solution can be found. Looking at what the 3070Ti card can do, I'm very curious to see what will be achieved with the 4090. Bottlenecks will probably appear somewhere and if they can be identified then the potential for performance gains is huge. You are blazing a new trail, the next ones after you will find it easier wink

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Transcoding is completely different to MPV playback ... Transcoding H265 with profile fast is utilizing my CPU 100% .. playback not.

Post's attachments

transcoding.jpg 198.88 kb, 64 downloads since 2023-01-08 

432

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

DragonicPrime wrote:

Tested this out with an RTX 4090 and seem to be getting around 115fps on a 1080p video. So much better than the default implementation. Used to only get around 80fps with the default

It is good that there are more 4090 card owners on this forum. It will be easier to compare results smile

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

80 to 115 is same result for me .. i would be happy to have a discord evening smile even curious with the rtx3070ti results shown above ... and 3 gpu threads as i cannot select odd numbers big_smile

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

grobalt wrote:

80 to 115 is same result for me .. i would be happy to have a discord evening smile even curious with the rtx3070ti results shown above ... and 3 gpu threads as i cannot select odd numbers big_smile

you can edit to 3 threads here
https://cdn.discordapp.com/attachments/290709370600423424/1061784700680679434/image.png
encoder log
https://cdn.discordapp.com/attachments/290709370600423424/1061784958395502632/image.png




@dlr5668 he also has 3070ti, we may need him to test to compare the results.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Did you rename the profile to the name of the Ride default profile Name ? Generic 4.4 profile is odd as the 4.6 is used in vs-mlrt

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

grobalt wrote:

Did you rename the profile to the name of the Ride default profile Name ? Generic 4.4 profile is odd as the 4.6 is used in vs-mlrt

like I said before, that model does nothing with TRT mode on.trt model Default is 4.6.

you can try to switch to model 4.4, edit base.py file
https://cdn.discordapp.com/attachments/290709370600423424/1061787859276808233/image.png

437 (edited by grobalt 08-01-2023 23:38:55)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

It seems you run the SVP basic RIFE and not vs-mlrt as introduced here: https://www.svp-team.com/forum/viewtopi … 483#p81483

correct ?

anyhow, i tried the basic Rife implementation and nothing changed with 3 threads ..

which nvidia driver version are you using ?

i run windows 11 with the latest driver from january 5th, 528.02

438 (edited by aloola 09-01-2023 00:01:59)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

mine also 528.02

It seems you run the SVP basic RIFE and not vs-mlrt as introduced here: https://www.svp-team.com/forum/viewtopi … 483#p81483

I follow that instruction. so vs-mlrt
proof I run trt ver. %AppData%\Roaming\SVP4\cache\Program Files (x86)\SVP 4\rife\models\rife
https://cdn.discordapp.com/attachments/290709370600423424/1061794561069940756/image.png


here is benchmark with TRT off, it's way slower
https://cdn.discordapp.com/attachments/290709370600423424/1061795235472093224/image.png





also make sure these files in the right place

2. replace generate.js, base.py in SVP 4\script; restart SVP to be sure

maybe you copy these files to SVP 4\MPV64

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Maybe an indication that TRT is not used in my setup .. are you in discord or TeamViewer etc tomorrow? Going to bed now

440 (edited by DragonicPrime 09-01-2023 00:09:36)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

aloola wrote:

mine also 528.02

It seems you run the SVP basic RIFE and not vs-mlrt as introduced here: https://www.svp-team.com/forum/viewtopi … 483#p81483

I follow that instruction. so vs-mlrt
proof I run trt ver. %AppData%\Roaming\SVP4\cache\Program Files (x86)\SVP 4\rife\models\rife
https://cdn.discordapp.com/attachments/290709370600423424/1061794561069940756/image.png


here is benchmark with TRT off, it's way slower
https://cdn.discordapp.com/attachments/290709370600423424/1061795235472093224/image.png



also make sure these files in the right place

2. replace generate.js, base.py in SVP 4\script; restart SVP to be sure

maybe you copy these files to SVP 4\MPV64

I followed all of that as well. I just did everything again to make sure and I get the same result. up to around 115fps. Changing to 4.4 doesn't change anything either. All of my other specs are the same as yours I think. 5900X, 32Gb 3600mhzCL14

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Everything I've seen suggested so far, hasn't changed the speed. It always gets capped at around 60-75% usage like this for the entire duration

Post's attachments

Screenshot 2023-01-08 191450.png 123.33 kb, 68 downloads since 2023-01-08 

442

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Transcoding with TRT ON or OFF seems to have close performance when we look to FPS, but with TRT ON the power consumption is about 25% less  than TRT OFF. So, I think there is a margin for optimizations.

TRT ON
https://i.imgur.com/DzLOFPI.png

TRT OFF
https://i.imgur.com/3msSD67.png

443 (edited by Pezede 09-01-2023 05:27:44)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

That one - https://github.com/AmusementClub/vs-mlrt - is MUUUUCH better
-------------
Let's try this:

0. nothing to do with Python big_smile
1. download --> https://www.svp-team.com/files/temp/rife-trt-0701-1.7z <--, unpack into SVP 4\rife (you should already have this folder), so vstrt.dll must be in the root, i.e. SVP 4\rife\vstrt.dll
2. replace generate.js, base.py in SVP 4\script; restart SVP to be sure
3. menu -> Applications settings -> Additional options -> All settings, go to 'User defined options'
Title: TensortRT; Script name: rife_trt; Other values by default - "FRC profile", "ON or OFF"
Click 'Add option'
Go to the RIFE video profile, see the added TensortRT on/off swtich.
Make a copy of the RIFE video profile, one with TensorRT ON and another with OFF.

Pros:
- no Pytorch!
- works in a real-time
- even faster (?) than vsrife [could be just because of some TensorRT options]
Cons:
- even slower (?) first-time initialization [could be just because of some TensorRT options]
- a nasty command-line window will pop-up for every new video resolution


I'm getting 150fps transcoding with 1080p content with this (4090), this is really good! I'm using 4 gpu threads.

Real time is still slow and choppy but I haven't taken the time to check that everything's properly set-up, I will do it tomorrow.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I see no performance change with the TensorRT setting - so i am still thinking it is not running with Tensorcores

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

if it creates cache files - then it's working

446 (edited by grobalt 09-01-2023 08:26:39)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

i posted some benchmark data with a sample script: https://github.com/AmusementClub/vs-mlr … nt-4630465
numbers should be valid as someone else with RTX4080 is half performance

the parameter peak_performance = True increases performance by ~60%

thanks for the parameter feedback@github smile

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

grobalt wrote:

i posted some benchmark data with a sample script: https://github.com/AmusementClub/vs-mlr … nt-4630465
numbers should be valid as someone else with RTX4080 is half performance

the parameter peak_performance = True increases performance by ~60%

thanks for the parameter feedback@github smile

peak_performance is just for the bench, right ?

448 (edited by Xenocyde 09-01-2023 10:17:09)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Is it safe to install the new RIFE stuff for SVP now? Chainik was saying that real-time was not really working, but I see you guys have been troubleshooting over the weekend. Would like to know if real-time interpolation works with the latest updates.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Depends on your resolution .. 1080p runs without any problem .. 2k as well ... 4k is still not fast enough

450 (edited by Xenocyde 09-01-2023 10:21:36)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

grobalt wrote:

Depends on your resolution .. 1080p runs without any problem .. 2k as well ... 4k is still not fast enough

I have an RTX 3070 145 W and only have a 1080p TV.

Chainik wrote:

=== RIFE / PyTorch+TensorRT installation ===

!!! UPDATED 07-01-2023 !!!
- replaced base.py : fp16 isntead of fp32
+ added fix for vsrife
============

!!! Advanced users only, if you don't know what this is all about - you don't need it !!!

0. Update SVP via online updater (this will update Python to ver. 3.10.9)

1. Dependencies installation:
--------------
Download TensorRT libs: https://github.com/HolyWu/vs-rife/relea … 2_win64.7z
unpack CUDA-blah-blah.7z to SVP 4\mpv64

run cmd as Administrator

cd SVP 4\mpv64

python get-pip.py

python -m pip install torch torchvision --extra-index-url https://download.pytorch.org/whl/cu117
python -m pip install tensorrt-8.5.2.2-cp310-none-win_amd64.whl
python -m pip install torch-tensorrt-fx-only tqdm
python -m pip install -U --no-deps vsrife
python -m vsrife

A quick fix for vsrife's "slowmo" bug: replace __init__.py in SVP 4\mpv64\Lib\site-packages\vsrife\


2. SVP modification
---------------
Replace SVP 4\script\generate.js and base.py with the attached ones

menu -> Applications settings -> Additional options -> All settings, go to 'User defined options'
Title: TensortRT; Script name: rife_trt; Other values by default - "FRC profile", "ON or OFF"
Click 'Add option'

Go to the RIFE video profile, see the added TensortRT on/off swtich.
Make a copy of the RIFE video profile, one with TensorRT ON and another with OFF.

----------------
For a given video resolution the first-time start is super-slow (while it generates TRT net in a cache folder --> %APPDATA%\SVP4\cache\trt), so better do it in a "transcoding" mode.

Do I follow this guide? Or is there a more updated one?