• Thanks: 64

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:
flowreen91 wrote:

We can see how the directly generated video of RIFE has no shake of the white lines on the right side of the screen.
But when transcoding it with SVP, every interpolated frame has different positioning of the white lines than the non-interpolated frames.
It's like you show the video normally on the non-interpolated frames and then reduce the height by a few pixels on the RIFE generated frames which makes the pixels not align with the original movie, adding a shake-like effect on the static white lines that is obvious for big screen users.

SVP devs please take a look

Looking through the magnifying glass on the 65'' OLED, I can see "shaking" in both of these converted samples (non-RIFE too), and moreover, in a real-time RIFE conversion.
I'd say this's because of color space converted back and forth, noticeable in a very high contrast areas only.
Probably hmm

Propably big_smile

2,027 Reply by Chainik

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

flowreen91

...and your "shaking" is not like Blackfyre's V2 shaking, which is quite obvious not only on those lines but in a whole frame

---
BTW: if you just replace vsmlrt.py with the latest git, then it will use V1 models instead of V2

2,028 Reply by flowreen91 (edited by flowreen91 09-10-2024 18:40:29)

  • Thanks: 99

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

BTW: if you just replace vsmlrt.py with the latest git, then it will use V1 models instead of V2

Yes, cause in december 2023 the dev updated the signature of the RIFE method:
https://gyazo.com/bde7edb2b4deb80294dbbaf71b757cc2
but he didn't add it as last element of the method so please fix it by updating helpers.py to send video_player True or False as 10th parameter:

return RIFE(clip,multi,1.0,None,None,None,model_num,backend,ensemble,False,implementation)
or
return RIFE(clip,multi,1.0,None,None,None,model_num,backend,ensemble,True,implementation)
(not sure which one is best for SVP)

Chainik wrote:

I'd say this's because of color space converted back and forth, noticeable in a very high contrast areas only.
Probably hmm
...and your "shaking" is not like Blackfyre's V2 shaking, which is quite obvious not only on those lines but in a whole frame

Yes, i think you're right. I tried to simplify the base.py and generate.js and ended up with this:
https://gyazo.com/5f8efd7ead2c79d6a3afeaa852161184
but micro shaking still persists. Which confirms that it's caused by the color space conversion required for Vapoursynth.
Easiest way to fix it would be to allow Vaporsynth to also apply the same conversion process on the source frame too.
This way all outputed frames should line up and 100% fix the micro-shaking.
Is this possible? (even if it will increase the computational processing needed)

Blackfyre's V2 whole-frame obvious shaking takes priority, though.
Thanks again for putting the time and effort to look through the magnifying glass.

  • Thanks: 200
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

flowreen91 wrote:
Chainik wrote:

BTW: if you just replace vsmlrt.py with the latest git, then it will use V1 models instead of V2

Yes, cause in december 2023 the dev updated the signature of the RIFE method:
https://gyazo.com/bde7edb2b4deb80294dbbaf71b757cc2
but he didn't add it as last element of the method so please fix it by updating helpers.py to send video_player True or False as 10th parameter:

return RIFE(clip,multi,1.0,None,None,None,model_num,backend,ensemble,False,implementation)
or
return RIFE(clip,multi,1.0,None,None,None,model_num,backend,ensemble,True,implementation)
(not sure which one is best for SVP)

Chainik wrote:

I'd say this's because of color space converted back and forth, noticeable in a very high contrast areas only.
Probably hmm
...and your "shaking" is not like Blackfyre's V2 shaking, which is quite obvious not only on those lines but in a whole frame

Yes, i think you're right. I tried to simplify the base.py and generate.js and ended up with this:
https://gyazo.com/5f8efd7ead2c79d6a3afeaa852161184
but micro shaking still persists. Which confirms that it's caused by the color space conversion required for Vapoursynth.
Easiest way to fix it would be to allow Vaporsynth to also apply the same conversion process on the source frame too.
This way all outputed frames should line up and 100% fix the micro-shaking.
Is this possible? (even if it will increase the computational processing needed)

Blackfyre's V2 whole-frame obvious shaking takes priority, though.
Thanks again for putting the time and effort to look through the magnifying glass.

I think False is the best because I remember reading that SVP already performs the same functions as True.

2,030 Reply by Drakko01 (edited by Drakko01 09-10-2024 22:38:31)

  • Thanks: 49

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

@jdg4dfv7
Ok, a lot to read. Let's start by making the following clarification, when I mentioned not being interested in 0.25 speed test, I meant not doing it. But in no way was my intention to minimize your efforts in testing it in a less flawed way. I doubt that this matters to you, since you are not here for the opinions of others. I really appreciate and see the value of this type of tests, but I am convinced that the real benefit of your tests is  directly with the Rife developers and I don't believe they see it as child's play, your findings may lead them to see something they can improve and extend the benefits to all of us.I encourage you, if you don't already do it, to send it to them in some way.
After reading everything else the only thing I can say is, thank you for taking the time to test and respond but from here on out I am not going to quote directly or respond to any of your post. Because I can see that it's just going to lead to endless, pointless arguments that benefit no one. And in a few days the thread will probably have 200 worthless pages.

For everyone else, I really interest on the "flawed"visual testing on normal speed playback of scene like a mentioned (Doc Op vs Spiderman under the bridge) or any other scene with obvious artifacts like this one,the kind that don't need to have superhuman vision to be able to spotted.

RickyAstle98 wrote:

4) The 4.26 has significant artifacts for me even at normal playback speed, which not happens with older models, sometimes even parts of big objects interpolating separately than full object, which still not happens for older models testing, no matter what playback speed was set, I dont say 4.26 will work the same for others, atleast thats how 4.26 works for me, period!

Looking at you, and comments like this(whichever your preferred model) plus some scene example.

If you dont have the file in cuestion, can use this on youtube in the resolution you prefer or downscaling via svp.
https://www.youtube.com/watch?v=CC5c8LyPyuk&t=13s

2,031 Reply by RickyAstle98 (edited by RickyAstle98 09-10-2024 23:14:27)

  • Thanks: 64

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Drakko01 wrote:

@jdg4dfv7
Ok, a lot to read. Let's start by making the following clarification, when I mentioned not being interested in 0.25 speed test, I meant not doing it. But in no way was my intention to minimize your efforts in testing it in a less flawed way. I doubt that this matters to you, since you are not here for the opinions of others. I really appreciate and see the value of this type of tests, but I am convinced that the real benefit of your tests is  directly with the Rife developers and I don't believe they see it as child's play, your findings may lead them to see something they can improve and extend the benefits to all of us.I encourage you, if you don't already do it, to send it to them in some way.
After reading everything else the only thing I can say is, thank you for taking the time to test and respond but from here on out I am not going to quote directly or respond to any of your post. Because I can see that it's just going to lead to endless, pointless arguments that benefit no one. And in a few days the thread will probably have 200 worthless pages.

For everyone else, I really interest on the "flawed"visual testing on normal speed playback of scene like a mentioned (Doc Op vs Spiderman under the bridge) or any other scene with obvious artifacts like this one,the kind that don't need to have superhuman vision to be able to spotted.

RickyAstle98 wrote:

4) The 4.26 has significant artifacts for me even at normal playback speed, which not happens with older models, sometimes even parts of big objects interpolating separately than full object, which still not happens for older models testing, no matter what playback speed was set, I dont say 4.26 will work the same for others, atleast thats how 4.26 works for me, period!

Looking at you, and comments like this(whichever your preferred model) plus some scene example.

If you dont have the file in cuestion, can use this on youtube in the resolution you prefer or downscaling via svp.
https://www.youtube.com/watch?v=CC5c8LyPyuk&t=13s

The problem is with mpv and SVP both, because this issue happens too often realtime playback, doesnt necessary for example videos...

2,032 Reply by Drakko01

  • Thanks: 49
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

RickyAstle98 wrote:

The problem is with mpv and SVP both, because this issue happens too often realtime playback, doesn't necessary for example videos...

Do you try it with this scene?
In my case for the example scene, happen exactly same position same way every time except on v4.25

2,033 Reply by RickyAstle98 (edited by RickyAstle98 10-10-2024 06:51:41)

  • Thanks: 64

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Drakko01 wrote:
RickyAstle98 wrote:

The problem is with mpv and SVP both, because this issue happens too often realtime playback, doesn't necessary for example videos...

Do you try it with this scene?
In my case for the example scene, happen exactly same position same way every time except on v4.25

Yes I tried with this scene too, some areas look pretty good with new model, but half of the video is better handles with older models!
Easier explanation (10 months old RTX 4070)
4.4 to 4.10 models: smooth experience with mid to average level of artifacts
4.10 to 4.16 models: choppy experience with low level of artifacts
4.15 lite sometimes better than newer models too much (100 movies tested)
4.17 to 4.25 models: chopper experience with low to mid level of artifacts
4.26 is worse than 4.25 (atleast for me)
Where level of artifacts emphasizes the level of their perception!

  • Thanks: 14

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I checked it many more times 4.26. It's surprising that many people don't see her mistake. No, I stay at 4.18, that's the best for me.

2,035 Reply by abraxas (edited by abraxas 10-10-2024 09:52:39)

  • Thanks: 6

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

pensioner600 wrote:

I checked it many more times 4.26. It's surprising that many people don't see her mistake. No, I stay at 4.18, that's the best for me.

If you have The Martian: at timestamp 18:33 and later, you can see bars between the bed and his workplace in the background. At 4.18 there are significant artifacts there. Not at 4.26, not at 4.22v2 lite, not at 4.9(!!). All the others I tested (4.15v2 lite, 4.16v2 lite, 4.18, 4.18v2) have artifacts.

Of course, this is only one scene, but for me these kinds of artifacts are the worst, because they catch my eye immediately.

For THIS test case, 4.26v2 and 4.9 are visually identical at normal speed.

Testing parameter: 4k source material, scaling to -19201080, upscaling with MPV and shaders to 4k LED screen.

2,036 Reply by RickyAstle98 (edited by RickyAstle98 10-10-2024 11:20:45)

  • Thanks: 64

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

abraxas wrote:
pensioner600 wrote:

I checked it many more times 4.26. It's surprising that many people don't see her mistake. No, I stay at 4.18, that's the best for me.

If you have The Martian: at timestamp 18:33 and later, you can see bars between the bed and his workplace in the background. At 4.18 there are significant artifacts there. Not at 4.26, not at 4.22v2 lite, not at 4.9(!!). All the others I tested (4.15v2 lite, 4.16v2 lite, 4.18, 4.18v2) have artifacts.

Of course, this is only one scene, but for me these kinds of artifacts are the worst, because they catch my eye immediately.

For THIS test case, 4.26v2 and 4.9 are visually identical at normal speed.

Testing parameter: 4k source material, scaling to -19201080, upscaling with MPV and shaders to 4k LED screen.

Nope!

2,037 Reply by Chainik

  • Thanks: 1622
  • Thanks for the post: 6

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

updated vsmlrt.py to fix "Blackfyre's V2 shaking"
basically it's a latest git version plus a few mods:
- altered engine path
- keep the same engine file names as with old version (to not rebuild everything after updating)

2,038 Reply by Honza

  • Thanks: 3

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

updated vsmlrt.py to fix "Blackfyre's V2 shaking"
basically it's a latest git version plus a few mods:
- altered engine path
- keep the same engine file names as with old version (to not rebuild everything after updating)

Thanks for the fix.

I'm new on the forum so bear with me for asking perhaps obvious question.
Is there a place to download this fix files and try/use it or do we need to wait (couple of months) for new version of SVP?

  • Thanks: 99
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Honza wrote:

new version of SVP?

Just click the SVP icon to update lol
https://gyazo.com/4453f660b971c9b35a3176cac6c106b1

2,040 Reply by Honza

  • Thanks: 3
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

flowreen91 wrote:
Honza wrote:

new version of SVP?

Just click the SVP icon to update lol
https://gyazo.com/4453f660b971c9b35a3176cac6c106b1

Right, update components, thanks

2,041 Reply by Magly

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

FWIW I still think 4.15v2 (not lite) looks the best, especially for stuff like the buildings in the opening of The Dark Knight

2,042 Reply by abraxas

  • Thanks: 6

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I switched back to 4.22v2 lite. 4.26v2 has too many artifacts IN MY MOVIES (newest example: Jason Bourne, 8:45, Window on the right side). Could switch back to 4.9 I think, but 4.22v2 lite has great performance (FOR ME).

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

updated vsmlrt.py to fix "Blackfyre's V2 shaking"

https://media0.giphy.com/media/v1.Y2lkPTc5MGI3NjExcmlnOWgwZ2dtdXNwNGVjcmhydWd5cHV0ZjZyZ2o2bzExOXhiOHZmNSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/7mgLf9RdKnhTy/giphy.webp

2,044 Reply by Mardon85

  • Thanks: 13

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I've been using RIFE and SVP for a while now and love it. Of late i've had all sorts of strange issues. The latest one has me questioning my sanity. SVP is running ether x2 or x3 of the resolution under rife MSI afterburner will report 74/75 FPS, the machine is loaded like its running RIFE as it always has but the video doesn't look like its been smoothed. It looks the same as a 24FPS video. Have I lost the plot? Is this even possible and has anyone else had a similar issue?

https://i.ibb.co/JBZjdnZ/svp.png

2,045 Reply by flowreen91 (edited by flowreen91 16-10-2024 22:45:11)

  • Thanks: 99

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Mardon85 wrote:

video doesn't look like its been smoothed

Press ` to open mpv console, any errors there?
Install Nvidia App and check the statistics overlay fps too? https://www.nvidia.com/en-us/software/nvidia-app/
Update SVP again?
Try the v2 of the same model for a bit of an fps boost? https://github.com/AmusementClub/vs-mlr … nal-models
Maybe Nvidia motion vectors glitching out cause 4K image and prevents interpolation, try disable that too xD
Try to comment hwdec in mpv.conf like #hwdec=auto-copy and compare if changes anything?

Maybe Windows reseted again "Power Mode" back to to Balanced instead of Performance?
https://gyazo.com/1925cb514aae0c50562db7a06a7fa6ab
Rather than doing a full reinstall, just play with the resize setting until it feels smooth. U don't actually need to see all 3840 x 2160 pixels.

2,046 Reply by Mardon85

  • Thanks: 13
  • Thanks for the post: 1

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Nope no dice. It looks like its dropping frames doesn't report that it is. I've just tried an AI Enhanced version of Wolverine Deadpool and I can notice the smooth motion immediately. At the same settings reported at 60FPS (Rife) it doesn't look smooth. If I turn on the OG frame interpolation, again I notice it (and the artifacts straight away).

I'm coming close to doing a clean windows install at this point.

2,047 Reply by Drakko01

  • Thanks: 49

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Mardon85 wrote:

Nope no dice. It looks like its dropping frames doesn't report that it is. I've just tried an AI Enhanced version of Wolverine Deadpool and I can notice the smooth motion immediately. At the same settings reported at 60FPS (Rife) it doesn't look smooth. If I turn on the OG frame interpolation, again I notice it (and the artifacts straight away).

I'm coming close to doing a clean windows install at this point.

You try with other player, to see if not mpv ?

  • Thanks: 2

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Mardon85 wrote:

Nope no dice. It looks like its dropping frames doesn't report that it is. I've just tried an AI Enhanced version of Wolverine Deadpool and I can notice the smooth motion immediately. At the same settings reported at 60FPS (Rife) it doesn't look smooth. If I turn on the OG frame interpolation, again I notice it (and the artifacts straight away).

I'm coming close to doing a clean windows install at this point.

I wonder if that enhanced version is my doing? ;D  DirtyHippie

Sorry you are having issues with RIFE not interpolating as it should.

If I set MPC-HC to exclusive fullscreen and lock it to 60hz, it seems to play more smoothly from my PC.  I always transcode the movies to 60fps, then play them from my android tv using KODI, as for some reason it plays better than from my PC. Another reason I prefer playing the transcoded movies from my TV is I can use true Dolby Vison vs the tone mapping from MPC Video Player in MPC-HC via Windows.

I don't think your issue is due to not having exclusive fullscreen set. Just though I'd share what I have discovered about that.

Post's attachments

Screenshot (1981).png 129.02 kb, 10 downloads since 2024-10-17 

Screenshot (1982).png 106.45 kb, 9 downloads since 2024-10-17 

2,049 Reply by scb

  • scb
  • Member
  • Offline
  • Thanks: 8

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

In context of using the RIFE filter, can anyone explain the differences between selecting (for 'Scene change detection'), 'Image comparison', 'NVOF motion vectors', 'SVP motion vectors', and 'disabled'?

2,050 Reply by Chainik

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

different scene change detection methods work differently big_smile