1,501 (edited by flowreen91 26-04-2024 23:16:58)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

RickyAstle98 wrote:

Just this issue happens no matter what TRT installed, I have exactly same issue, and my friend with almost same PC build too, but reencoding some HEVC sources back AVC again, I said earlier its user related issue a bit, its only realtime playback bug, which mostly use, but not prerendered, I dont want render any movie or TV episodes, you maybe not facing this issue!
This is how issue look like (video is predelayed for showcase purposes) >
https://drive.google.com/file/d/1tGmkW6 … ojEPp/view

pensioner600 wrote:

I noticed that setting the scene change threshold (rife_sc) to 100% is a very bad idea. And for some reason I noticed this specifically in The Last of Us)) At the change of scenes from dynamic to dynamic, it happens, I don’t even know what to call it, back and forth for a moment. This happens at any high values

Ricky did u try to fix it with lower rife_sc values as pensioner600 mentioned?

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I also noticed that ALL versions of V2 give the entire picture a shake, which is very clearly visible on a static scene on small light objects on a dark background. I noticed a long time ago that something was wrong, but I didn’t have time to watch movies. Versions NOT V2 and lite no jitter. I’m definitely switching to 15lite or 16lite, even if it’s x3, not x4. V2 is complete nonsense. Today I devoted half a day to tests, a lot became clear))

1,503 (edited by RAGEdemon 27-04-2024 03:28:14)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

RickyAstle98 wrote:
RAGEdemon wrote:
RickyAstle98 wrote:

Something is wrong, because 4.16 lite working at 4K with my RTX 4070 without dropped frames
But 4.15v2 (nonlite) is very strong, I down 2160p to 1800p - works!

Bro, you can easily get 4K RIFE: use 4.4 to double your fps and sync to 48Hz refresh (for 24fps source). Honestly, it looks butter smooth. I would be hard-pressed to notice this vs 60/120. The magic is ensuring the output FPS matches your refresh.

I know about my 4070 capabilities, but in 4.4 too much artefacts (not critical but noticeable), secondly I dont need sync anything, GSync matches movies automatically! No magic at all!
My standard solutions for movies >
1) 4.4v2 1280x720 24>144
2) 4.9v2 1280x720 24>144
3) 4.15v2l 1280x720 24>144

Each iteration after 4.4 causes a massive performance hit @~16% for minimal artefact improvement. Above 4.9 is trained mostly on Anime.

What you say doesn't make sense to me. You say you don't use 4.4 because "too much artefacts (not critical but noticeable)", but then you are happy to force downscale all 4K and 1080p content to 1280x720p?! And you don't notice the huge degradation in quality?

I am guessing you don't have a 4K display, or else you wouldn't be saying something so seemingly bizarre smile

To each their own, I guess...

1,504 (edited by cws 27-04-2024 05:47:31)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

pensioner600 wrote:

I also noticed that ALL versions of V2 give the entire picture a shake, which is very clearly visible on a static scene on small light objects on a dark background. I noticed a long time ago that something was wrong, but I didn’t have time to watch movies. Versions NOT V2 and lite no jitter. I’m definitely switching to 15lite or 16lite, even if it’s x3, not x4. V2 is complete nonsense. Today I devoted half a day to tests, a lot became clear))

I think this has been fixed in newer TensorRT 9.x testing releases (note the current release that is part of SVP is 8.x.) If you can still see it with 9.x, it should be reported so developers are aware and hopefully it can be addressed.

1,505 (edited by RickyAstle98 27-04-2024 08:05:44)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

RAGEdemon wrote:
RickyAstle98 wrote:
RAGEdemon wrote:

Bro, you can easily get 4K RIFE: use 4.4 to double your fps and sync to 48Hz refresh (for 24fps source). Honestly, it looks butter smooth. I would be hard-pressed to notice this vs 60/120. The magic is ensuring the output FPS matches your refresh.

I know about my 4070 capabilities, but in 4.4 too much artefacts (not critical but noticeable), secondly I dont need sync anything, GSync matches movies automatically! No magic at all!
My standard solutions for movies >
1) 4.4v2 1280x720 24>144
2) 4.9v2 1280x720 24>144
3) 4.15v2l 1280x720 24>144

Each iteration after 4.4 causes a massive performance hit @~16% for minimal artefact improvement. Above 4.9 is trained mostly on Anime.

What you say doesn't make sense to me. You say you don't use 4.4 because "too much artefacts (not critical but noticeable)", but then you are happy to force downscale all 4K and 1080p content to 1280x720p?! And you don't notice the huge degradation in quality?

I am guessing you don't have a 4K display, or else you wouldn't be saying something so seemingly bizarre smile

To each their own, I guess...

Who said my monitor is 4K? I dont watch 4K at all, my monitor is standard chinese Full HD 165Hz GSync with builtin upscalers, I just test 4K source for performance, hehe! I dont see massive quality degradation only because of monitor upscalers, but my solutions shown are exclusively for 6x movie fps targets, obviously I watch 1080p with 3x for sure!

1,506 (edited by RickyAstle98 27-04-2024 08:10:55)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

pensioner600 wrote:

I also noticed that ALL versions of V2 give the entire picture a shake, which is very clearly visible on a static scene on small light objects on a dark background. I noticed a long time ago that something was wrong, but I didn’t have time to watch movies. Versions NOT V2 and lite no jitter. I’m definitely switching to 15lite or 16lite, even if it’s x3, not x4. V2 is complete nonsense. Today I devoted half a day to tests, a lot became clear))

Picture shake happens depending on the model and fps targets, if you let RIFE profile do 60fps exact, picture shake may occur, also different TRT behave differently!
Example : 4.4v2 fixed 60 fps 25>60 (picture shake) / 4.4v2 2.5x 25>62.5 (picture shake fixed) / 4.9v2 (no picture shakes)
I have no idea if you see picture shake on 2x 3x 4x factors or something, I dont see picture shake, I use TRT 9.2 library!

1,507 (edited by dawkinscm 27-04-2024 10:38:23)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

pensioner600 wrote:

I also noticed that ALL versions of V2 give the entire picture a shake,

There was a bug with Rife v2 which has been fixed since vslmrt script 3.18.4 released November last year. I think this bug caused picture shake issue.


RAGEdemon wrote:

Each iteration after 4.4 causes a massive performance hit @~16% for minimal artefact improvement.

I know you are not replying to me but I think your comments are interesting so I hope you don't mind me responding directly to you. As you say, there is a massive performance hit. But you have been very lucky if you haven't come across the horrible artefacts that appear below 4.9.

RAGEdemon wrote:

Above 4.9 is trained mostly on Anime.

Being "trained on Anime" or not doesn't matter if it still results in improved live action motion. Rife v4.9 and above are all smoother and have much less visible artefacts than below 4.9.

RAGEdemon wrote:

What you say doesn't make sense to me. You say you don't use 4.4 because "too much artefacts (not critical but noticeable)", but then you are happy to force downscale all 4K and 1080p content to 1280x720p?! And you don't notice the huge degradation in quality?

I am guessing you don't have a 4K display, or else you wouldn't be saying something so seemingly bizarre smile

To each their own, I guess...

Resolution is important but only up to a point. Although I personally would not downscale to 720p I do now downscale my 1920x2160 3D blu-rays to 1080p.   On a near IMAX VR screen, I've tried both upscaling to 4K and downscaling to 1080p and found no "visible" difference between the two. For actual 4K content there is a difference, but it will only be a problem on very large of screens. For most people watching on a TV, there will be little visible difference unless you decide to stand in directly in front of the screen.

1,508 (edited by flowreen91 27-04-2024 11:40:38)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dawkinscm wrote:
pensioner600 wrote:

I also noticed that ALL versions of V2 give the entire picture a shake,

There was a bug with Rife v2 which has been fixed since vslmrt script 3.18.4 released November last year. I think this bug caused picture shake issue.

Should SVP devs upgrade the out of the box vsmirt script so they will fix it for all users?
(knowing that non-lazy users might play around with v2 versions for that extra performance and run into that bug)


Chainik wrote:

one more time, what is the exact point of "v2"? hmm

it's not-faster or even slower, and it takes more RAM. so, why?
just because "v2" is cooler than "v1"? big_smile also tensort rt v10 is cooler than v8...

Never noticed, nevermind then. xD
If the upgrade to it would be safe and not create other issues, it would be nice for current SVP to be forward compatible with the future v2 models that they create.


dawkinscm wrote:

While it's no longer significantly faster, the release notes say it reduces "PCie traffic flow". From personal experience this helps a lot for real time processing and provides more GPU headroom.

I'm experiencing the same thing on my 4080. Our GPU is more bottlenecked by PCIe traffic flow than RAM at this point.
End goal should always be a stable product that users would brag about how good it is. "Random picture shake" effect when watching movies should not be one of it's downsides.

1,509

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

one more time, what is the exact point of "v2"? hmm

it's not-faster or even slower, and it takes more RAM. so, why?
just because "v2" is cooler than "v1"? big_smile also tensort rt v10 is cooler than v8...

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

flowreen91 wrote:
dawkinscm wrote:
pensioner600 wrote:

I also noticed that ALL versions of V2 give the entire picture a shake,

There was a bug with Rife v2 which has been fixed since vslmrt script 3.18.4 released November last year. I think this bug caused picture shake issue.

Should SVP devs upgrade the out of the box vsmirt script so they will fix it for all users?
(knowing that non-lazy users might play around with v2 versions for that extra performance and run into that bug)

lazy is harsh lol. But if the issue is vslmrt and you want to upgrade then I would recommend upgrading to the November 2023 script and not later because later vslmrt scripts made changes that require mods to the helper script too.

1,511 (edited by dawkinscm 27-04-2024 12:01:25)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

one more time, what is the exact point of "v2"? hmm

it's not-faster or even slower, and it takes more RAM. so, why?

While it's no longer significantly faster, the release notes say it reduces "PCie traffic flow". From personal experience this helps a lot for real time processing and provides more GPU headroom. Before I started to use downscaling, I was regularly hitting 100% GPU, but the v2 models would bring that down to 90% or less depending on the model.

But @Chainik thanks for the reminder because it's been a while since I used a non v2 model and unlike previous models, v4.15 seems to handle artefacts a "touch" better than the v2 version.

1,512

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I've been using V2 or V2 lite models for some time now (maybe 6+ months) and never noticed any shakiness. My vslmrt version is 3.18.16, says it was last modified in December 2023. Does that mean it includes the shake fix?

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

cws wrote:

I think this has been fixed in newer TensorRT 9.x testing releases (note the current release that is part of SVP is 8.x.) If you can still see it with 9.x, it should be reported so developers are aware and hopefully it can be addressed.

How can I change the TRT version? And how can I find out the current version? I really don’t like the scene change on any model at any 6%-100% values. If the dynamic scene changes to dynamic, then some kind of jerk occurs, it is instantaneous, but it is visible. Maybe they fixed this in other versions of TRT?

1,514 (edited by flowreen91 27-04-2024 22:25:13)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

pensioner600 wrote:

How can I change the TRT version?

pensioner600 if you would like to update vsmlrt to fix the v2 picture shake, follow the instructions from here:
https://www.svp-team.com/forum/viewtopi … 735#p83735

dawkinscm wrote:

There was a bug with Rife v2 which has been fixed since vslmrt script 3.18.4 released November last year. I think this bug caused picture shake issue.

Details about how they fixed it back then on 3.18.4:
https://github.com/AmusementClub/vs-mlr … 746eaad885

1,515

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

> How can I change the TRT version? And how can I find out the current version? I really don’t like the scene change on any model at any 6%-100% values.

this isn't related to the TRT version at all
TRT version can only affect build time, memory usage and overall performance

1,516 (edited by dawkinscm 27-04-2024 20:45:44)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Xenocyde wrote:

I've been using V2 or V2 lite models for some time now (maybe 6+ months) and never noticed any shakiness. My vslmrt version is 3.18.16, says it was last modified in December 2023. Does that mean it includes the shake fix?

Yes.


flowreen91 wrote:
pensioner600 wrote:

How can I change the TRT version?

pensioner600 if you would like to update vsmirt to fix the v2 picture shake, follow the instructions from here:
https://www.svp-team.com/forum/viewtopi … 735#p83735

No. As @Chainik stated above, this is nothing to do with TRT version.

1,517 (edited by flowreen91 28-04-2024 12:16:43)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Guide for updating RIFE to latest TensorRT from 3 days ago: https://github.com/AmusementClub/vs-mlrt/releases

Step 1: Download https://github.com/AmusementClub/vs-mlr … uda.v14.7z
Step 2: Go to C:\Program Files (x86)\SVP 4\rife\ and move the following to a backup folder: vsmlrt-cuda,vstrt.dll,vsmlrt.py
Step 3: Open vsmlrt-windows-x64-cuda.v14.7z and copy vsmlrt-cuda,vstrt.dll,vsmlrt.py in their place

Step 4: Edit vsmlrt.py replace line 1821
old:
tempfile.gettempdir(),
new:
os.path.expandvars("%APPDATA%\\SVP4\\cache\\"), #!SVP

Step 5: Edit helpers.py replace line 52
old:
return RIFE(clip,multi,1.0,None,None,None,model_num,backend,ensemble,implementation)
new:
return RIFE(clip,multi,1.0,None,None,None,model_num,backend,ensemble,True,implementation)

Step 6: Enjoy 10-15% performance drop caused by upgrading to latest code!

Noteable issues:
Your PC might not have the power to 2x a 4K video, and mpv might display desync errors that cancel SVP interpolation https://gyazo.com/a4714c85227b5b596a5f71084b5c2fae
so u should test it with a frc.frame.resize -19201080 or lower

1,518 (edited by dawkinscm 28-04-2024 15:19:20)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

If you have to use Rife v2 and have picture shaking issues then updating vslmrt makes sense.  I reset everything back to default (minus some script changes) then updated to TRT8.6. Not going beyond that unless someone can show me how TRT9 improves things*. Not sure why anyone would update to TRT 10.

Update:
*TRT9 is required if you are using Rife v2 and have the picture shaking issue.

1,519

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

flowreen91

step 6: "enjoy" what exactly? 10-15% performance drop? big_smile

another post that is actually a bad advice for people who don't understand what their doing, just following "guides"

----
dawkinscm
> If you have to use Rife v2 and have picture shaking issues then updating vslmrt makes sense.

only for TRT>=9
with TRT8 updating vsmlrt.py most likely does nothing
only difference between SVP's vsmlrt.py and current master is one line:

# TensorRT 9.0 or later
"ONNXTRT_Broadcast_*:fp32"

1,520 (edited by flowreen91 28-04-2024 15:11:31)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

dawkinscm
> If you have to use Rife v2 and have picture shaking issues then updating vslmrt makes sense.

only for TRT>=9
with TRT8 updating vsmlrt.py most likely does nothing
only difference between SVP's vsmlrt.py and current master is one line:

# TensorRT 9.0 or later
"ONNXTRT_Broadcast_*:fp32"

You are actually right! We were looking at commit messages only and we got confused. Sorry for that!
I need to check better what exactly fixed it.

1,521 (edited by dawkinscm 29-04-2024 08:05:15)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

dawkinscm
> If you have to use Rife v2 and have picture shaking issues then updating vslmrt makes sense.

only for TRT>=9
with TRT8 updating vsmlrt.py most likely does nothing
only difference between SVP's vsmlrt.py and current master is one line:

# TensorRT 9.0 or later
"ONNXTRT_Broadcast_*:fp32"

Noted thanks. I'm currently using v4.15 but if I go back to a v2 model then I will update TRT. Cheers smile

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

> How can I change the TRT version? And how can I find out the current version? I really don’t like the scene change on any model at any 6%-100% values.

this isn't related to the TRT version at all
TRT version can only affect build time, memory usage and overall performance

Are there any other settings besides rife_sc that affect scene changes? I really don’t like how it works, I’ve already tried many options. It's as if the algorithm in the next scene is confused and momentarily doesn't know what to do. Everything is perfect, smooth as butter, except for this trouble with the scene change.

1,523 (edited by dawkinscm 28-04-2024 17:22:31)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

pensioner600 wrote:
Chainik wrote:

> How can I change the TRT version? And how can I find out the current version? I really don’t like the scene change on any model at any 6%-100% values.

this isn't related to the TRT version at all
TRT version can only affect build time, memory usage and overall performance

Are there any other settings besides rife_sc that affect scene changes? I really don’t like how it works, I’ve already tried many options. It's as if the algorithm in the next scene is confused and momentarily doesn't know what to do. Everything is perfect, smooth as butter, except for this trouble with the scene change.

You're just going to have to experiment and see what works best for you. Most of my movies work well with almost any SC or even turning it off. But I've got a couple that need SC to be set to I had to find a balance.

1,524 (edited by flowreen91 28-04-2024 17:54:11)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

dawkinscm
> If you have to use Rife v2 and have picture shaking issues then updating vslmrt makes sense.

only for TRT>=9
with TRT8 updating vsmlrt.py most likely does nothing
only difference between SVP's vsmlrt.py and current master is one line:

# TensorRT 9.0 or later
"ONNXTRT_Broadcast_*:fp32"

After further investigation it seems the v2 issue is caused by old .dll files.

Out of the box svp with rife model 4.9 v2 with tree wiggle present:
https://gyazo.com/67445b08ae0aa2a214e8cb054e051b1b
Out of the box svp with rife model 4.9 v2 with tree wiggle fixed:
https://gyazo.com/61d5906c4bbbe4887171fd5724c6d4ed

If u override vsmlrt-cuda and vstrt.dll from rife folder with:
https://github.com/AmusementClub/vs-mlr … 4.test2.7z
Issue gets fixed.
If u downgrade back to:
https://github.com/AmusementClub/vs-mlr … 14.test.7z
wiggle issue will reappear.

You can check the example here:
https://drive.google.com/drive/folders/ … sSi1Kl8tSn

Please compare the differences and address this Rife v2 issue when u have time.
Thanks a lot @Chainik!

1,525 (edited by dawkinscm 29-04-2024 08:10:41)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

flowreen91 wrote:

After further investigation it seems the v2 issue is caused by old .dll files.

Thanks for the extra testing and I remember now that you also said this in January when I asked why I should upgrade to TRT9. But after downscaling is it still necessary to use v2?