You are not logged in. Please login or register.
SmoothVideo Project → Using SVP → New RIFE filter - 3x faster AI interpolation possible in SVP!!!
the onnx file is available now
Yep. But it does not work properly with SVP. It might look like it is working but the engine parse crashes. Looking at the new vsmlrt script there are a lot of changes and I'm not sure which ones are relevant. Especially since SVP moved some functionality into helpers.py.
RIFE 4.6 (very old model, fastest) - 112 fps
RIFE 4.15 lite (decent progress, exploring lite models) - 100 fps
RIFE 4.26 (finished trained 4.25) - 86 fps
RIFE 4.25 (very good breakthrough after 4.18) - 84 fps
RIFE 4.18 - 75 fps
I did speed test (with old SVP build)
How does 4.26 work for you with no changes?
oriento wrote:the onnx file is available now
Yep. But it does not work properly with SVP. It might look like it is working but the engine parse crashes. Looking at the new vsmlrt script there are a lot of changes and I'm not sure which ones are relevant. Especially since SVP moved some functionality into helpers.py.
i suppose you must change 32 into 64
multiple_frac = 32 / Fraction(scale)
dawkinscm wrote:oriento wrote:the onnx file is available now
Yep. But it does not work properly with SVP. It might look like it is working but the engine parse crashes. Looking at the new vsmlrt script there are a lot of changes and I'm not sure which ones are relevant. Especially since SVP moved some functionality into helpers.py.
i suppose you must change 32 into 64
multiple_frac = 32 / Fraction(scale)
Nope. Still crashes out. I've got the TRT error log and it says that broadcast dimensions must be comformable.
How does 4.26 work for you with no changes?
I didnt update yet
dawkinscm wrote:How does 4.26 work for you with no changes?
I didnt update yet
If the engine was created in less than 15 seconds then it crashed. It looks like it's working. SVP will tell you that it's working. But it is not coz there's an error log generated. When it works, no error logs are generated. BTW I did update but it makes no difference because the version number doesn't change.
Nope. Still crashes out. I've got the TRT error log and it says that broadcast dimensions must be comformable.
It seems 4.26 also crashes when the width or height of the resized video is not divisible by 64 with:
[ 1.419][f][vapoursynth] Script evaluation failed:
[ 1.419][f][vapoursynth] Python exception: vsmlrt.RIFEMerge: tile size must be divisible by 64 (416, 256)
[ 1.419][f][vapoursynth]
[ 1.419][f][vapoursynth] Traceback (most recent call last):
[ 1.419][f][vapoursynth] File "src\\cython\\vapoursynth.pyx", line 3121, in vapoursynth._vpy_evaluate
[ 1.419][f][vapoursynth] File "src\\cython\\vapoursynth.pyx", line 3122, in vapoursynth._vpy_evaluate
[ 1.419][f][vapoursynth] File "C:\Users\Flowr\AppData\Roaming\SVP4\scripts\1c2e810c.py", line 83, in <module>
[ 1.419][f][vapoursynth] smooth = interpolate(clip)
[ 1.419][f][vapoursynth] ^^^^^^^^^^^^^^^^^
[ 1.419][f][vapoursynth] File "C:\Users\Flowr\AppData\Roaming\SVP4\scripts\1c2e810c.py", line 62, in interpolate
[ 1.419][f][vapoursynth] smooth = RIFE_imp(input_rife,multi=rife_num,model=rife_mnum,backend=trt_backend)
[ 1.419][f][vapoursynth] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ 1.419][f][vapoursynth] File "C:\Program Files (x86)\SVP 4\rife\helpers.py", line 23, in RIFE_imp
[ 1.419][f][vapoursynth] return RIFE(clip,multi,1.0,None,None,None,model_num,backend,ensemble,False,implementation)
[ 1.419][f][vapoursynth] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ 1.419][f][vapoursynth] File "C:\Program Files (x86)\SVP 4\rife\vsmlrt.py", line 1215, in RIFE
[ 1.419][f][vapoursynth] output0 = RIFEMerge(
[ 1.419][f][vapoursynth] ^^^^^^^^^^
[ 1.419][f][vapoursynth] File "C:\Program Files (x86)\SVP 4\rife\vsmlrt.py", line 1049, in RIFEMerge
[ 1.419][f][vapoursynth] raise ValueError(
[ 1.419][f][vapoursynth] ValueError: vsmlrt.RIFEMerge: tile size must be divisible by 64 (416, 256)So we also need to change the code from "C:\Program Files (x86)\SVP 4\script\generate.js" from:
var pw = Math.floor((media.dst_w-1)/32+1)*32 - media.dst_w;
var ph = Math.floor((media.dst_h-1)/32+1)*32 - media.dst_h;to
var pw = Math.floor((media.dst_w-1)/64+1)*64 - media.dst_w;
var ph = Math.floor((media.dst_h-1)/64+1)*64 - media.dst_h;Attaching files that load both v1 and v2 correctly below.
dawkinscm wrote:Nope. Still crashes out. I've got the TRT error log and it says that broadcast dimensions must be comformable.
It seems 4.26 also crashes when the width or height of the resized video is not divisible by 64 with:
...
So we also need to change the code from "C:\Program Files (x86)\SVP 4\script\generate.js" from:var pw = Math.floor((media.dst_w-1)/32+1)*32 - media.dst_w; var ph = Math.floor((media.dst_h-1)/32+1)*32 - media.dst_h;to
var pw = Math.floor((media.dst_w-1)/64+1)*64 - media.dst_w; var ph = Math.floor((media.dst_h-1)/64+1)*64 - media.dst_h;Attaching files that load both v1 and v2 correctly below.
I got nowhere close in figuring this out lol. I will have to take a look at the changes but in the mean time: Damn good job buddy 
dawkinscm wrote:Nope. Still crashes out. I've got the TRT error log and it says that broadcast dimensions must be comformable.
It seems 4.26 also crashes when the width or height of the resized video is not divisible by 64 with:
Attaching files that load both v1 and v2 correctly below.
Just a quick update. I don't like the way v4.26 and the new code is performing. v4.25 with the old code works better. I'm just double checking to make sure.
Wow, 4.26 has almost no interference artifacts.
Wow, 4.26 has almost no interference artifacts.
And its faster than 4.25. Fucking magic
The artefacts I saw before v4.26 are still there but improved to the same level as v4.25. But when I tested DR Strange 2 with v4.26 with the new code, it makes IC behave identically to SVP and NVOF with intermediate Rife frames being created.
Updates:
So far using SVP motion detection for SC seems to work very well with v4.26 and the mod code. NVOF is very similar. What is a little annoying is that you can run the same scene multiple times and see varying degrees of the same artefact.
I can also forward and reverse some scenes when using SVP that previous got stuck.
Firstly flowreen91, great work on the changed files.
I downloaded and changed the default ones in rife and script folders.
.
RIFE 4.6 (very old model, fastest) - 112 fps
RIFE 4.15 lite (decent progress, exploring lite models) - 100 fps
RIFE 4.26 (finished trained 4.25) - 86 fps
RIFE 4.25 (very good breakthrough after 4.18) - 84 fps
RIFE 4.18 - 75 fpsI did speed test (with old SVP build)
Thanks for this, great work.
Wow, 4.26 has almost no interference artifacts.
4.25 v2 remains the best one for artifacts with my testing.
4.26 v2 failed the No Time to Die test at around minute 3 with the sliding glass door (most difficult scene I test). In fact it was the worst I have seen during that specific scene, but in other scenes it performed better than others.
I have reverted back to 4.25 v2 personally. Interesting that after it finished training (became 4.25), artifacts became worse.
From the vsmlrt script dev about the issue with SVP using v4.26:
It's not a problem with the official RIFE repository. I chose 32 to be the minimum requirement for the RIFE wrapper in vs-mlrt, and then the SVP developer adapted this requirement. The v4.26 model now requires mod64 input, and I have updated the code to explicit notify the change, and therefore it is the SVP developer that should update the code.
(this will only affect the "v1" variants, the "v2" variants handle this kinds of resolution requirement automatically
For me 4.26 works properly without change any files, I use V1 models but I test V2 only to make shure,and work properly too.
Whats the default align/valign values for SVP4 Pro? I forgot that I used 2x2 all the time, maybe thats why all models starting 4.16 looks fragmentary smooth!
Fragmentary smooth - when a small parts of one large object can be interpolated differently, even being a same large object! For example: faces, 4.4 to 4.16 faces looks buttery smooth, from 4.17 to 4.26 small parts like lips, cheeks, ears and other objects can be separated from the frame! There were cases when the lips moved apart alone, never happens for 4.4 to 4.16 before, also happens when the main object in the frame is interpolated correctly, but all the background moving both axis, looks like one step forward-two steps back frame!
The 4.26 new model is worse than 4.15 lite in Matrix rooftop 3 agents opening the doors, pattern errors on door textures, no door errors in 4.15v2 (both mod32 and mod64 input)!
Too noticeable door errors 4.4 (worst)
Less noticeable door errors 4.9 (okay)
No door errors 4.11 to 4.16 (perfect)
I forgot that I used 2x2 all the time
How do you do that? I use only v2 models (for better rewind speed) and wanna try. How does it affect model performance?
Whats the default align/valign values for SVP4 Pro? I forgot that I used 2x2 all the time, maybe thats why all models starting 4.16 looks fragmentary smooth!
Fragmentary smooth - when a small parts of one large object can be interpolated differently, even being a same large object! For example: faces, 4.4 to 4.16 faces looks buttery smooth, from 4.17 to 4.26 small parts like lips, cheeks, ears and other objects can be separated from the frame! There were cases when the lips moved apart alone, never happens for 4.4 to 4.16 before, also happens when the main object in the frame is interpolated correctly, but all the background moving both axis, looks like one step forward-two steps back frame!
The 4.26 new model is worse than 4.15 lite in Matrix rooftop 3 agents opening the doors, pattern errors on door textures, no door errors in 4.15v2 (both mod32 and mod64 input)!
Too noticeable door errors 4.4 (worst)
Less noticeable door errors 4.9 (okay)
No door errors 4.11 to 4.16 (perfect)
Are you talking about 4.26 lite only because the lite models do behave differently to the normal ones.
RickyAstle98 wrote:Whats the default align/valign values for SVP4 Pro? I forgot that I used 2x2 all the time, maybe thats why all models starting 4.16 looks fragmentary smooth!
Fragmentary smooth - when a small parts of one large object can be interpolated differently, even being a same large object! For example: faces, 4.4 to 4.16 faces looks buttery smooth, from 4.17 to 4.26 small parts like lips, cheeks, ears and other objects can be separated from the frame! There were cases when the lips moved apart alone, never happens for 4.4 to 4.16 before, also happens when the main object in the frame is interpolated correctly, but all the background moving both axis, looks like one step forward-two steps back frame!
The 4.26 new model is worse than 4.15 lite in Matrix rooftop 3 agents opening the doors, pattern errors on door textures, no door errors in 4.15v2 (both mod32 and mod64 input)!
Too noticeable door errors 4.4 (worst)
Less noticeable door errors 4.9 (okay)
No door errors 4.11 to 4.16 (perfect)Are you talking about 4.26 lite only because the lite models do behave differently to the normal ones.
The 4.26 lite? I dont see 4.26 lite model, where is it? Youre wrong!
dawkinscm wrote:RickyAstle98 wrote:Whats the default align/valign values for SVP4 Pro? I forgot that I used 2x2 all the time, maybe thats why all models starting 4.16 looks fragmentary smooth!
Fragmentary smooth - when a small parts of one large object can be interpolated differently, even being a same large object! For example: faces, 4.4 to 4.16 faces looks buttery smooth, from 4.17 to 4.26 small parts like lips, cheeks, ears and other objects can be separated from the frame! There were cases when the lips moved apart alone, never happens for 4.4 to 4.16 before, also happens when the main object in the frame is interpolated correctly, but all the background moving both axis, looks like one step forward-two steps back frame!
The 4.26 new model is worse than 4.15 lite in Matrix rooftop 3 agents opening the doors, pattern errors on door textures, no door errors in 4.15v2 (both mod32 and mod64 input)!
Too noticeable door errors 4.4 (worst)
Less noticeable door errors 4.9 (okay)
No door errors 4.11 to 4.16 (perfect)Are you talking about 4.26 lite only because the lite models do behave differently to the normal ones.
I'm not exactly right or wrong. I was just suggesting that there is a difference between a lite model. But yes you are
The 4.26 lite? I dont see 4.26 lite model, where is it? Youre wrong!
I've seen this happen before but if there is no lite model then this is not the case here.
Whats the default align/valign values for SVP4 Pro? I forgot that I used 2x2 all the time, maybe thats why all models starting 4.16 looks fragmentary smooth!
Fragmentary smooth - when a small parts of one large object can be interpolated differently, even being a same large object! For example: faces, 4.4 to 4.16 faces looks buttery smooth, from 4.17 to 4.26 small parts like lips, cheeks, ears and other objects can be separated from the frame! There were cases when the lips moved apart alone, never happens for 4.4 to 4.16 before, also happens when the main object in the frame is interpolated correctly, but all the background moving both axis, looks like one step forward-two steps back frame!
The 4.26 new model is worse than 4.15 lite in Matrix rooftop 3 agents opening the doors, pattern errors on door textures, no door errors in 4.15v2 (both mod32 and mod64 input)!
Too noticeable door errors 4.4 (worst)
Less noticeable door errors 4.9 (okay)
No door errors 4.11 to 4.16 (perfect)
For testing what you stated, can you provide or indicate the exact scene.thanks
RickyAstle98 wrote:Whats the default align/valign values for SVP4 Pro? I forgot that I used 2x2 all the time, maybe thats why all models starting 4.16 looks fragmentary smooth!
Fragmentary smooth - when a small parts of one large object can be interpolated differently, even being a same large object! For example: faces, 4.4 to 4.16 faces looks buttery smooth, from 4.17 to 4.26 small parts like lips, cheeks, ears and other objects can be separated from the frame! There were cases when the lips moved apart alone, never happens for 4.4 to 4.16 before, also happens when the main object in the frame is interpolated correctly, but all the background moving both axis, looks like one step forward-two steps back frame!
The 4.26 new model is worse than 4.15 lite in Matrix rooftop 3 agents opening the doors, pattern errors on door textures, no door errors in 4.15v2 (both mod32 and mod64 input)!
Too noticeable door errors 4.4 (worst)
Less noticeable door errors 4.9 (okay)
No door errors 4.11 to 4.16 (perfect)For testing what you stated, can you provide or indicate the exact scene.thanks
Nevermind, just say default align/valign values, thanks! The rendered results will look different on different PCs!
RickyAstle98 wrote:I forgot that I used 2x2 all the time
How do you do that? I use only v2 models (for better rewind speed) and wanna try. How does it affect model performance?
What do you mean?
Nevermind, just say default align/valign values, thanks!
Renaming the SVP folder from Roaming, generates a fresh config that looks like this:
https://gyazo.com/d8f75344daafabb8ecdcaeaecc2db2d3
SmoothVideo Project → Using SVP → New RIFE filter - 3x faster AI interpolation possible in SVP!!!
Powered by PunBB, supported by Informer Technologies, Inc.