1,676 (edited by Kaiji 25-06-2024 00:31:17)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

> but on testing the results seem worse than using image comparison

you'd better provide a sample video if you want this fixed

Settings:

https://i.imgur.com/3JRwQlP.jpeg

SVPMV:

https://mega.nz/file/BmdzmKhQ#ytRf15Jch … hAXq1opU6k

NVOFMV:

https://mega.nz/file/hy9RUYhB#Ul6ZFjKI9 … BLzrkFEYfY

Source clip:
https://mega.nz/file/8uEyTagS#XG79Gk0Ct … mggtR9kvoI

There are examples of scene changes from fast motion to still image at 14 and 35 seconds. Neither NVOF or SVP identified the one at 14 seconds, but SVP did catch the one at 35 seconds while NVOF didn't. Interestingly, there's a cut at 1:02 (after Ang puts his arms out to gain speed) that SVP didn't detect but NVOF did.

Thanks for the link you provided. I'll look into it.

1,677 (edited by flowreen91 24-06-2024 20:29:50)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

jdg4dfv7 wrote:

So I understand it was the correct way to do?

Sounds like you made it work, well done!

From what i see if i revert the following step, the old SVP vsmlrt.py file still works for all models including latest 4.17_lite.

Step 5: Edit helpers.py replace line 52
old TensorRT 8.5:
return RIFE(clip,multi,1.0,None,None,None,model_num,backend,ensemble,implementation)
new TensorRT 10.1:
return RIFE(clip,multi,1.0,None,None,None,model_num,backend,ensemble,True,implementation)

And i didn't see any signficant improvements if i replace it with latest vsmlrt.py file from last week:
https://github.com/AmusementClub/vs-mlr … /vsmlrt.py

So for now we can continue using the old file until somebody notices something.

1,678

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

jdg4dfv7

you don't need anything to use 4.16+ models, just put it into the models folder

if you don't know exactly _why_ you need updated TensorRT and/or vsmlrt.py files then you don't need them

Kaiji

please upload the source clip

1,679

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

@Chainik
Maybe it's me being a laymen, but will you respond more detailed please?
I asked multiple questions, also in comment 2.
If "no time" or "don't want to", than please say so, but completely ignoring is not polite.

3)

you don't need anything to use 4.16+ models, just put it into the models folder

So this

vsmlrt.py
Added support for RIFE v4.17 models.

can be completely ignored, and has no advantages/disadvantages (problems, issues, performance, quality etc.), if using the older Tensort RT 8.5.1 provided with svp, in
combination with newer RIFE models as 4.16 or 4.17?

1,680 (edited by jdg4dfv7 24-06-2024 22:27:28)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

@flowreen91
@Chainik
Thanks for providing a tip.

4) Switching to Tensor RT 15.x, it works swapping "vstrt.dll and the folder vsmlrt-cuda" only.
I noticed a bump of graphic card total power consumption (undervolted rtx 4090), about 5 - 10 % (differs in resolution, framerate, upscaling etc.)

5) If swapping the newer "vsmlrt.py" also from the TensorRT 15 file also, nothing works anymore; that means any machine learning based RIFE model and SVP's own interpolation. See image attached "error swapping vsmlrt_py".
Any way to fix from my side, you SVP developers side, or simply not swap the file?

6)
@flowreen91
Made the line change ("True"), nothing works.
See image attached.

Thanks in advance!

Post's attachments

error changing line 52.jpg, 69.47 kb, 902 x 309
error changing line 52.jpg 69.47 kb, 26 downloads since 2024-06-24 

error swapping vsmlrt_py.jpg 171.52 kb, 16 downloads since 2024-06-24 

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

jdg4dfv7 wrote:

@flowreen91
Made the line change ("True"), nothing works.

Try with these:
https://drive.google.com/drive/folders/ … SzUp1SAzbx

1,682

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

> please write down here, for laymen to understand, what's the best opion to use

the default one! (which is "SVP motion vectors")

> So this - "vsmlrt.py - Added support for RIFE v4.17 models." - can be completely ignored

yes, it can be completely ignored

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

With the new update downscaling to 720p is now gone and i need help to downscale my anime to 720p.

1,684 (edited by dawkinscm 25-06-2024 16:39:05)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

> please write down here, for laymen to understand, what's the best opion to use
the default one! (which is "SVP motion vectors")

The default one works well with everything except the intro to Hugo where it stutters considerably.

1,685

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

> With the new update downscaling to 720p is now gone

long-press the button wink

1,686

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dawkinscm wrote:
Chainik wrote:

> please write down here, for laymen to understand, what's the best opion to use
the default one! (which is "SVP motion vectors")

The default one works well with everything except the intro to Hugo where it stutters considerably.

I found other cases when svp motion vectors using blend struggles with a complex scene, for example in Doctor Strange Multiverse of madness, after defender Strange is killed by the Ribboned Creature.

1,687 (edited by Xenocyde 26-06-2024 09:19:27)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Did some testing last night to see how the new scene change detection options perform. With blend adjacent frames activated, SVP motion vectors shows the most artifacts with occasional microstuttering. NVOF and Image Comparison with SCT 100% seem to yield very similar if not identical results and show less artifacts with very little to no microstuttering.

1,688 (edited by dawkinscm 26-06-2024 11:03:24)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I think the previous comments are proof that this is setup dependant. SVP MV works best for @Chainik and works well overall but also has clear micro stutters. Some of the stutters are not so "micro". NVOF has always had clear stutters for me but works well for some. All except NVOF are siimlar, including Disable. But for me Image Comparison seems to work best with no obvious microstutters.

1,689

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I totally agree with Dawkinscm, of all the SCD methods, for me image comparison is the one that I think gives the best overall viewing experience. By the way, it's worth mentioning that we have this  new option  not only thanks to the work of Chainik and others. But also because the push and desire of members like Dawkinscm for bringing SVP/Rife to the ultimate potential. Thanks for that!!

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I got so excited lol

Then I tested, and the blinking was still there, I assume this is not fixed for RIFE? But only for motion vector or NVOF methods? @Chainik

I tried with both my custom mpv config, and then I removed my config file to test with default, and both had heavy blinking with Dolby Vision exclusive files.

https://i.ibb.co/Kx6PPvG/Screenshot-2024-07-03-044915.png

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

This is soooo cool!
https://gyazo.com/cd0cc66a7e77fc893d0d3cc4e8298b89

1,692

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I will like to ear others members results after this update.I still think that image comparison work the best , but i kinda like in some scenes NVOF motion vectors, the improve its subtle but its there.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

@Chainik I saw your comment earlier so I was getting a video for you, but as it's deleted now, here is an example below (video included).

Scene Change is on repeat, I do not use blend with RIFE. So that is not an issue.

The below happens with exclusively Dolby Vision content always.

Constant flickering always. Default config, custom config, tried many combinations, doesn't stop it.

I say exclusively DV above because there are many examples of DV / HDR content, where it comes with both DV and HDR10 or 10+ metadata, so mpv automatically plays the one that works correctly. An example of this is the latest 4K HDR DV episodes of House of the Dragon, they all work perfectly fine. But Disney+ DV content will always flicker because it is DV exclusive I believe (at least this is how I understood why some DV work and others never work).

https://streamable.com/5fyn6a

1,694

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

apparently, DV blinking _was_ fixed, but it seems that now, with a processing logic changed for "motion vectors scene change", the blinking is back, but for a different reason big_smile and it's a different kind of blinking now...

still the latest "0.38" changes were necessary, but not enough

1,695

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Drakko01

>I still think that image comparison work the best

to be "the best" it must give less false positive scene changes, and less missed scene changes
which is not true

1,696 (edited by dawkinscm 03-07-2024 11:46:23)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

In all my tests of problem scenes, SVP algorithm at it's best can be better than Image Comparison, but at it's worst can give artefacts similar to NVOF. Image Comparison seems to be the best compromise giving the cleanest output with the least artefacts.

Chainik wrote:

Drakko01

>I still think that image comparison work the best

to be "the best" it must give less false positive scene changes, and less missed scene changes
which is not true

Prior to this latest update, the SVP algorithm might provide the best objective test results for false positives etc, just like a scaling algorithm might have the best objective PSNR and SSIM image quality scores. But in both cases, the resulting output when viewed by a human being can be less preferable to a lower scoring algorithm. It has it's faults, but Image Comparison just performs better overall for every problem scene I have.

*Edit. This conclusion is prior to the more recent SVP update so I have updated comment. Actual test updates will follow.

1,697

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

> but Image Comparison just performs better overall for every problem scene I have.

probably you just prefer RIFE-generated "garbage" frames over repeated or blended ones

1,698 (edited by dawkinscm 03-07-2024 12:56:10)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

> but Image Comparison just performs better overall for every problem scene I have.

probably you just prefer RIFE-generated "garbage" frames over repeated or blended ones

By the nature of your response, I'm guessing by your comment that SVP motion is your baby and because of that maybe you are missing the point. I want SVP motion to be better.  All I'm trying to do here is provide feedback to help you make it so.

Anyway, I tested using an MPV build from this week with the latest SVP update. Maybe it's the latest mpv build or the MPV changes you made,  but SVP motion has improved since the previous update. But since I'm almost certainly watching on the largest screen here, I can see stutters and micro-stutters thats most may not. SVP motion has at least a couple of clear and repeatable stutters for certain scenes that IC does not. Other than that, IC at 15% and SVP motion are practically identical. YMMV.

1,699

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

> SVP motion has at least a couple of clear and repeatable stutters for certain scenes that IC does not.

meaning
1. it's a false positive SC, right? in which case you'd better share a sample with the problem... there're a lot of variables to adjust there.
2. you really prefer "RIFE-generated "garbage" frames" over frame repeating at a false-positive SC big_smile

1,700 (edited by dawkinscm 03-07-2024 16:00:04)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

> SVP motion has at least a couple of clear and repeatable stutters for certain scenes that IC does not.

meaning
1. it's a false positive SC, right? in which case you'd better share a sample with the problem... there're a lot of variables to adjust there.
2. you really prefer "RIFE-generated "garbage" frames" over frame repeating at a false-positive SC big_smile

No! I prefer a smooth 60fps motion where an action is clear without jitter or stutters. That's why I paid for SVP, but apparently I was mistaken. Good to know.