1 Topic by Super4Jet (edited by Super4Jet 08-08-2021 20:32:38)

Topic: Flickering videos with RIFE AI (likely caused by TTA)

Hi everyone!
I don't know where to start, so let me just start with the background.
I recently switched from using Linux to using Windows for SVP (Evaluation version). The reason I switched to Windows is pretty easily explained, RIFE needs version 210 which is not yet on Linux, and after I tried it on a Windows VM, it explained me it needed a GPU with Vulkan API which VMWare failed to provide. After spending hours gaining that knowledge, I installed Windows on a free partition and SVP installed without any issues. Also: The normal vapoursynth (non-RIFE) script never caused any issues, neither on Linux, nor on Windows VM nor on native Windows. It's the safe bet, but who's not willing to try something new?

-The Problem-

I configured the profile for higher quality (TTA on, FP32) and left everything as is, started the transcode and voila, it did actually start (the magic 1%)! Curious as I am, I opened the not-yet-finished file (read-only) in VLC, and I was surprised to see that it was flickering, and looking at it frame by frame told me every second frame was just black. Though the water mark was still there, to help troubleshooting smile . VLC told me the framerate is 24000 (sic!) with the original having 23.976024 fps. So I closed VLC, waited for it to be done transcoding, hoping that it would add these frames later on. But nothing changed. Despite the seemingly high framerate, the playback speed and seek bar times were correct, and I only had to go 48 frames forward to get to the next second. The log (attached) told me about some issues that I have no idea of. Task Manager told me 0~5% GPU load, 95~100% CPU load. Isn't that odd?

Also while writing this, I tried using the default settings and I think TTA causes this issue. Turning it off seems to get me a better video (I see interpolated frames, no flickering) and it takes a lot longer than before. If anyone needs snapshots in case they can't reproduce my error, I'll happily share some frames. Same goes for any logs/system information etc.

System Specs:
CPU: 3600
GPU: RX 570
M.2 SSD
RAM: 16GB DDR4
Windows 10, drivers and everything up to date
SVP 4 version 210

Thanks a lot big_smile
Super4Jet

Also since the attachements don't seem to work as far as I can see, here's the log on pastebin: https://pastebin.com/eTuX27Wz

2 Reply by Super4Jet (edited by Super4Jet 10-08-2021 17:45:34)

Re: Flickering videos with RIFE AI (likely caused by TTA)

A little update on my side: The only configuration that seems to work well is:
(AI Model: not tested, only used generic)
TTA: disabled
(Floating Point: no difference)
GPU Threads: 1 (2 caused issues, 4 and 6 failed directly after start)
GPU device: id0

Should I file a bug report for the configurations that are not working?

Cheers

Re: Flickering videos with RIFE AI (likely caused by TTA)

Also quick note since there is no button to close a topic, I successfully moved on to Flowframes. I know it's been more than a year now, but I just want to say that no further assistance is required (though there was none yet to speak of tbh).

Flowframes it is now. It's much easier to install, has to be compiled by Visual Studio once and then just works from this point on. SVP struggled a bit with everything, installing was complicated, on my native Windows it had issues running correctly, in a VM things were running a bit better but was a bit more difficult to install (I think at some point it wants to read the CPU core count and fails because it's virtual after all, and of course no useful error message), Linux didn't have the feature at the time and aaahh it was a tough time. I did get out a few videos with Vaporsynth which were quite good in the beginning, and as I said in the previous reply, I got RIFE to work eventually and it looked really good, so - yes. Things might be different now and I would still recommend SVP as an interpolation program for beginners, while FF is more for the tech-savvy.

Cheers!

Re: Flickering videos with RIFE AI (likely caused by TTA)

Whether I'm using Optical Flow or not, I get major flickering for some space scenes in Gravity when using Rife that I don't get on Automatic. It's similar to the effect you get when over sharpening.

Re: Flickering videos with RIFE AI (likely caused by TTA)

For me it was that every second frame was black, as in pitch-black, RGB #000000. I sadly don't have the video at hand anymore, wasn't worth the storage big_smile

This only happens with Gravity? You might want to try another RIFE-model, the most recent is v4.6 (afaik not in the current version of SVP), other good ones are plain v4 and v2.3. I'm not entirely sure if you have all these options, I'll probably have to spin up a VM and look which options are available.
Just for reference, you could also try Flowframes for once, the most recent precompiled version seems to already include RIFE AI v4. I'm not sure if 'recommending' competitors goes well here, so we'll see XD

Also I'm rate limited here, so I'm sorry my responses come in so slow... 2h/reply apparently

6 Reply by dawkinscm (edited by dawkinscm 20-03-2023 16:02:30)

Re: Flickering videos with RIFE AI (likely caused by TTA)

Super4Jet wrote:

For me it was that every second frame was black, as in pitch-black, RGB #000000. I sadly don't have the video at hand anymore, wasn't worth the storage big_smile

This only happens with Gravity? You might want to try another RIFE-model, the most recent is v4.6 (afaik not in the current version of SVP), other good ones are plain v4 and v2.3. I'm not entirely sure if you have all these options, I'll probably have to spin up a VM and look which options are available.
Just for reference, you could also try Flowframes for once, the most recent precompiled version seems to already include RIFE AI v4. I'm not sure if 'recommending' competitors goes well here, so we'll see XD

Also I'm rate limited here, so I'm sorry my responses come in so slow... 2h/reply apparently

The 4.6 model is available in SVP and that's the version I use. The 4.4 model is a little worse. I had a look into that other matter you mentioned and I'm not sure it's better. The latest version doesn't seem to support audio plus unlike SVP there's no way to test it, no free trial etc. So why would I want to test a supposedly inferior version when SVP (with some glitches) works pretty well?

Re: Flickering videos with RIFE AI (likely caused by TTA)

I've - just given up on re-installing SVP. Just doesn't work on my machine. It worked last time though, and I think there might be something fishy going on in the code regarding VM-Detection. Or it's just the lack of a physical GPU that's bothering the SVP Manager at startup. Either way it would require me to take a way more complicated approach than I would get with FlowFrames at the moment. Still, it isn't ideal, updating as a non-paying user also takes about half an hour which isn't great either. So I'll leave it be for now.
As for 4.6, I've just tested it with some black and white scenes and though I didn't compare to 4.4, it looks a lot better than v4. B&w seems to be really difficult for interpolation as it doesn't know where that specific dark spot came from, from above or from the side etc., whereas before it could just tell them apart by color.

Back to the original question: Since I think the same AI fed with the same image sequence will always return the same output, maybe cut out that scene, use a different RIFE version und paste it back in? A little complicated, I know, or you could just concede and stick with v4.4 for the whole movie? To me it doesn't look like a technical issue with neither SVP nor RIFE AI, just that the training data is always a bit different and sometimes just gets worse with an update in some areas. And v4.4 isn't vaaastly inferior to v4.6 imo...

Also, I'm well aware this is not a FlowFrames troubleshooting forum, but I'd be happy to help there if needed :)

8 Reply by dawkinscm (edited by dawkinscm 21-03-2023 22:10:10)

Re: Flickering videos with RIFE AI (likely caused by TTA)

Super4Jet wrote:

I've - just given up on re-installing SVP. Just doesn't work on my machine. It worked last time though, and I think there might be something fishy going on in the code regarding VM-Detection. Or it's just the lack of a physical GPU that's bothering the SVP Manager at startup. Either way it would require me to take a way more complicated approach than I would get with FlowFrames at the moment. Still, it isn't ideal, updating as a non-paying user also takes about half an hour which isn't great either. So I'll leave it be for now.
As for 4.6, I've just tested it with some black and white scenes and though I didn't compare to 4.4, it looks a lot better than v4. B&w seems to be really difficult for interpolation as it doesn't know where that specific dark spot came from, from above or from the side etc., whereas before it could just tell them apart by color.

Back to the original question: Since I think the same AI fed with the same image sequence will always return the same output, maybe cut out that scene, use a different RIFE version und paste it back in? A little complicated, I know, or you could just concede and stick with v4.4 for the whole movie? To me it doesn't look like a technical issue with neither SVP nor RIFE AI, just that the training data is always a bit different and sometimes just gets worse with an update in some areas. And v4.4 isn't vaaastly inferior to v4.6 imo...

Also, I'm well aware this is not a FlowFrames troubleshooting forum, but I'd be happy to help there if needed smile

Thanks but both 4.4 and 4.6 have the same issue. There are a few movies that require Rife's extra smoothness but Gravity seems smooth enough with Automatic. So if it bothers me too much I can go back to using Automatic for this one movie and use Rife for the rest. The conditional statements don't work properly in this case but it really isn't a major issue, just annoying.