Topic: Reducing blending ratios

I just thought of a way of greatly enhancing image quality and eliminating ghost effects in artifact areas. Plus, it has 0 performance cost.

Reduce the frame blending ratio so that it blends 20/80 instead of 50/50!

I've implemented it here in FrameRateConverter and it could easily be added to SVP. No performance cost.

Re: Reducing blending ratios

This's about SAD masking aka "yushko" aka "artifacts masking" in SVP's UI.
OK, why just not mask it with a nearest frame w/o blending at all (like SVP does with 1st and 2nd shaders)?
Is "20/80" better than both "50/50" and "0/100" in your tests?

===
BTW, do you really mix two SAD masks together with the occlusion one? hmm

3 (edited by Mystery 02-06-2019 04:32:35)

Re: Reducing blending ratios

I wasn't getting good results with 0/100, it causes weird "fixed" areas to slide (relate to its surrounding) and then jump.

50/50 replaces artifacts with double-shadows.

So yes, 20/80 gives better result.

Chainik wrote:

BTW, do you really mix two SAD masks together with the occlusion one? hmm

Yes, I'm mixing two SAD masks as the artifact areas are slightly different depending on the DCT mode -- so although DCT=1 generally gives better results (but slow), some areas actually render better with DCT=0. This thus reduces the overall artifacts areas. Forget about doing this in real-time though!

Also adding StripeMask into it. This one probably could be added to SVP quite easily to remove artifacts due to stripes.

Re: Reducing blending ratios

I mean this:

EMocc = MaskOcc > 0 ? C_luma.MMask(bak, ml=MaskOcc, kind=2, gamma=1/gam, ysc=255, thSCD2=SkipOver).mt_inpand() : Blank
EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM

EM is "SAD mask (kind=1)" plus "Occ. mask (kind=2)".

===
> So yes, 20/80 gives better result.

Did you tried it for *5/2 FRC rate?

5 (edited by Mystery 02-06-2019 10:00:08)

Re: Reducing blending ratios

I haven't done too much testing but tested on a low-quality "torture clip" and on a 24fps video giving 5/2 FRC rate.

Chainik wrote:

EM is "SAD mask (kind=1)" plus "Occ. mask (kind=2)".

Is there a problem with this? This was part of the original code I started from, but Occ mask doesn't seem to be doing much but it helps a tiny bit.

Re: Reducing blending ratios

> Is there a problem with this?

It's like mixing apples and oranges in my opinion. Occlusion (i.e. "cover-uncover") mask is (theoretically smile) allows FlowFps to use only one src frame when processing areas covered in the other frame.
I believe that occ. mask in MMask is just wrong, SVP (SVSmoothFps) calculates totally different mask (look at frame edges while panning, they are always blurred with MMask's occ. mask, and are perfectly clear with SVP's 21/23th shaders),
BUT mixing it with SAD mask ...  I dunno hmm

Isn't your "blown-away quality" achieved by masking most of the frame area? smile

Re: Reducing blending ratios

It's doing a really good job at masking the right areas -- but that work was kind of diluted for the fact that it was replacing artifacts with double-shadows and blurry images. Tweaking the blurring ratio is what's now making a huge difference, leveraging the good work that was already done. In the way I implemented it I'm pretty sure Occ mask isn't taking the borders, in fact very few spots rise above the threshold so it has a minor effect if I remember.

Re: Reducing blending ratios

I must say that it definitely got better with this trick, especially in case of old / low res videos.
But it's still bad when SAD masking itself is bad, e.g. with high contrast animation.

Re: Reducing blending ratios

told ya smile

of course if masking is bad can't do anything; but in my "torture test" clip, with the scene with a moose moving in in a parade, I'm skipping interpolation altogether for all those frames. This trick makes the parade scene look a lot clearer, otherwise crowd is a blurry mess and anthers show as double-shadows.

Haven't done much testing to find the best ratio, so let me know what you find works best.