Topic: Ugly frames generated (need newbie help)

I'm happy I finally got my environment so that InterFrame2 executes without blowing up!  (Had to switch to a newer CPU than my P4.)

But I'm at a loss to know how to proceed.  Here's my script:

AviSource("Tom's-VB-test.avi"   )
ConvertToYV12()
Import("C:/Program Files/AviSynth/plugins/InterFrame2.avsi")
InterFrame(NewNum=30, NewDen=1, Cores=4)

I've uploaded the test AVI, which has 50 frames in Lagarith codec, to this link:
http://www.tomsgoodfiles.com/tom's-vb-test.avi
(About 16MB)
It is at 15fps, and I'm trying to increase to 30.  I expected that the frames generated (the ones inbetween the existing ones) would be as clear as the 2 frames surrounding.  But in most cases they're not.  Could someone please download my AVI and try it out?  It's a couple seconds in a volleyball game.  You'll see 3-legged women and sometimes a single ball in 2 places at once!  And in at least one case, the woman's head is missing!

I didn't realize this process would take a lot of tuning, and I don't know how to proceed.  I can't find any info to help me deal with this.  I did try all the "Tuning=" options, but none fixed it and I think some were worse.  From reading the InterFrame2.html, it appears there are lots of options but no guidance for how to pick them.  In the one case of "Tuning=" I was baffled by the stated tension between smoothness and "frame cohesiveness".  To me they are the same thing.  I simply would like the generated frames to have all elements exist with the same clarity that their 2 surrounding frames have.

I didn't see any problems in the "20110618-Sample-InterFrame.mkv" video I viewed, yet isn't that just as complicated as my clip??

If all I need is discussed somewhere else already, I'll be happy to do all reading there, if someone would kindly point me to it.

If it's easy for someone to "tune up" the parameters for the above clip of mine, perhaps you could do it, then I could read and study what went into it later, while I have GBs of AVIs being processed over days of VirtualDub runs.  (I have a lot of volleyball games in this format that I wish to convert.)  I'd really appreciate the jump-start.

Thank you so much for what I hope will be a tool that really does what I've been dreaming of.

Re: Ugly frames generated (need newbie help)

TCmullet
I had a look at your video. It is bad. It has framedrops. Motions are not "uniform".

How did you get this video?
15 fps is not standard.

Re: Ugly frames generated (need newbie help)

I captured it from my computer screen at 15fps because that's what I was able to do without the screen capture itself dropping frames on my less-than-modern PC (3ghz P4).

Exactly what do you mean by framedrops?  Yes, there are times that a frame repeats one or more times due to some glitch, but those instances are relatively rare.  In most cases, each frame (in my source) is different than the prior one.  (No framedrop.)  Yes, half the frames from the 30fps source are dropped, but by intention.  I'm not as worried about the occasional 2-frame freeze.  That will simply make a slightly bigger gap between the next 2 frames that are different.

I was under the belief that the interpolated frame needs only 2 frames and it generates the one in between with everything spatially equi-distant between it's starting point (in the 1st frame) and ending point (in the 2nd frame), no matter how far apart they are.

I realize that if a framerate was really low, so low that any wild actions could occur between 2 snapped frames, then of course no lost action can be truly created.  But if an arm moves from, say, pointing down 45 degrees from horizontal, then the next frame shows it pointing upward 45 degrees, I'd think InterFrame would be smart enough to generate an inter-frame with the arm pointing 0 degrees, i.e. horizontal.  And most of the actions in this video are not as severe at that 90 degree example.

Also why does the ball, when appearing very discretely (round and clearly formed) sometimes get interpolated as 2 balls in one interpolated frame?  This, and the 3-legged women, seem more like a frame-blend of the 2 frames than of an interpolated leg.

You probably have explanations in response to all this, which I'd love to hear.  But I also hope that even if my video is characterized as "bad" by some, that you all can help me redeem this and get it up to 30fps better than the bare-bones script I used where I had everything at defaults.  These frames are not much different than if I had captured the 60 field/sec. video from TV, then decimated in half two times to get down to 15fps.  It is actually my goal (assuming InterFrame can do the trick) to first generate 30fps.  Then do it again to get back to 60.  (I have use for both speeds.)

Re: Ugly frames generated (need newbie help)

TCmullet
In most cases, each frame (in my source) is different than the prior one
That's not enough. You need equidistant frames.

Good video is constant frame rate video with one time-delta between frames. Your video has not one time-delta. They are various at each frame.
If source have 25 fps then no way to get 15 fps without harm to time-delta. Such irregularity are present in your video it not at one place, it is everywhere.

no matter how far apart they are.
You are right. In your case you can get some result. But to get good result you need good source. Your source is bad.

I realize that if a framerate was really low, so low that any wild actions could occur between 2 snapped frames, then of course no lost action can be truly created.
Yes. You are right.

But if an arm moves from, say, pointing down 45 degrees from horizontal, then the next frame shows it pointing upward 45 degrees, I'd think InterFrame would be smart enough to generate an inter-frame with the arm pointing 0 degrees
No. Interframe can get 0 degrees arm at out only if sources has -5 and 5 degrees. 45 degrees are too much for this task. No rotation predicted. Only global shift and local shifts.

why does the ball, when appearing very discretely (round and clearly formed) sometimes get interpolated as 2 balls in one interpolated frame?
Because ball is small and his moves are large.

I have no idea how to help you. You need a lot of hard hand work frame-by-frame to reconstruct dropped original frames.
Such big intervals between frames for sports is very hard to any interpolator. Too small details, too large moves and differencies between frames.

Sorry. I have no time to do your job with your video. I must to work at new SVP version. Maybe after SVP release I will find a time to help you more.

Re: Ugly frames generated (need newbie help)

> Interframe can get 0 degrees arm at out only if sources has -5 and 5 degrees.

Ah HA!  I see now that I was expecting too much.  While waiting for your kind and thoughtful reply, I did two things.

1.  I reflected on that you can take 24fps and get smooth action.  I figured going down to 15fps, while significant, wasn't that drastic a lowering.  I now see that going that low frequently causes an excess of 5 degrees.  I think 5 degrees is exceeded often even at 24fps, however, in those cases it would be only very briefly, say, during explosions, etc. in which case the roughness of interpolation would not be objectionable.

2.  I also realized that while not preferred, I can acceptably generate this at a higher rate.  (The source exists and can be recaptured again, albeit "no fun".)  If you knew I wanted to end up with both 30 and 60fps editions, which source rate would be best, 24fps or 25fps?  What I actually can do is use CamStudio at either 100fps or 200fps, then decimate to 24 or 25.  If I do 200, I *may* be able to reliably get 30 after decimation.  If that scenario was to work, then I'd only have the 30-to-60 task for InterFrame, which you'd probably say is a snap.

> You need a lot of hard hand work frame-by-frame to reconstruct dropped original frames.
Such big intervals between frames for sports is very hard to any interpolator. Too small details, too large moves and differencies between frames.

No I can't bother with the numerous (but not constant) dropped frames.  I was going to merely "live with with the dropouts".  And I now grasp what you're saying about "too" this or that.  Your 5 degree comment really put it in perspective.  (Thank you.)  I made a big mistake by believing that 15fps was enough temporal info.  I do still look forward to converting movies from 29.97 telecined to 24 (which I've already done many times), then up it to 60.

> Sorry. I have no time to do your job with your video. I must to work at new SVP version. Maybe after SVP release I will find a time to help you more.

I was not expecting more than suggested values for the various parameters, a small task.  I see now that I was exceeding the technology beyond any tweaking.

So at this point, my questions are limited to what source rate would be best to generate 30, if 24 and 25 are the choices?   (30 might not be reliable for me, but I will experiment.)  Hope you can advise with that.

Re: Ugly frames generated (need newbie help)

TCmullet
The source exists and can be recaptured again, albeit "no fun"
It is very nice. Why you captured it to PC? I think it is better to record it to DVD-recorder at full NTSC frame size and full NTSC 30p/30i frame rate.
What source is? VHS/DV-tape?

what source rate would be best to generate 30, if 24 and 25 are the choices?
No one of them. Only 30 fps. Because in case with 25 fps you will have 5 drops per second. And 6 drops for 24 fps. It is the same irregulation of frametimes.
15 fps at PC after recording is not so good because no guarantee what every odd frame is decimated. Really decimated ANY 15 frames of 30 each second. And it is the problem. Getted video is not "regular".

Re: Ugly frames generated (need newbie help)

Why you captured it to PC? I think it is better to record it to DVD-recorder at full NTSC frame size and full NTSC 30p/30i frame rate. What source is? VHS/DV-tape?

I agree full NTSC is better, then I wouldn't need InterFrame.  But the source is internet-streamed, only, sadly.

what source rate would be best to generate 30, if 24 and 25 are the choices?
No one of them. Only 30 fps. Because in case with 25 fps you will have 5 drops per second. And 6 drops for 24 fps. It is the same irregulation of frametimes.

But I may have to live with some droppage, and I may be able to do irregular droppage a bit more evenly.  I can capture at 30fps, but the source drops out sometimes (freezes) causing the actual rate to fluctuate, say 27.8 - 29.2.  I think it that case, it'd be better to further decimate to a round integer.   But what I was really asking was which of 24 and 25 are better suited to compute frames at 60fps?  You take 24fps film and go right to 60, right?  (I would have thought you'd have to go from 24 ( times 5) to 120, then half-decimate to 60.)  So I was wondering that if I have a choice between providing original material at 24 vs 25, would one of them be better suited to convert to 60.  (Leaving the factor of dropouts aside.)

I hope you'll answer that question (24 vs 25 from the InterFrame feasibility standpoint).  I *may* (with extra difficulty) get the cap rate up to 30.x or something, then decimate down to 30, but can't guarantee.

Re: Ugly frames generated (need newbie help)

In usual case 24 fps is better than 25 fps to be interpolated to 60.
You are almost right. 24 fps encreased to 60 fps by added several interframes. Formula is 5:2. That mean to add 4 new frames and decimate 1 from each 2 sorce frames.

But I tell about real 24 fps.

Your fps after capturing is not real. It is fake fps. No guarantee that all frames are at it's own positions. You need not to capture. You need to save video stream with no changes or with minimal changes.

Re: Ugly frames generated (need newbie help)

No guarantee that all frames are at it's own positions. You need not to capture. You need to save video stream with no changes or with minimal changes.

It's true, no guarantee. Have to go w/the best I can get.  BTN (better than nothing).  Yes, need actual stream, but don't think it's possible.  Will go with highest rate possible, so that multi-frame freezes will be seldom.  With CamStudio at 100fps, rate fluctuates 25.x to about 28.y, hence my desire to have 24 or 25 as target decimation.  But with CS at 200fps, I see rates 30 to 31.  With that I can decimate to 30.  I am assuming in all this that the stream is being shown at 30 or 29.97 fps, but only an educated guess.

You are almost right. 24 fps encreased to 60 fps by added several interframes. Formula is 5:2. That mean to add 4 new frames and decimate 1 from each 2 sorce frames.

Are you saying that internally you change it to 120fps and (also internally) decimate by 2?

I've noticed that the instructions don't give a way to specify to InterFrame what the input FPS is.  So I'm assuming it gets it from the file header itself.  But I've been wondering if NewNum/NewDen can be anything regardless of what the input FPS is.  I figured out that for 24 to 60, I need x5 then /2.  Does InterFrame come up with whatever is needed to allow any combination of input and output FPS?  And if so, does that mean I could come up with an outlandish change like say, 30fps to 59.94 or 30fps to 29.97, and it would correctly do it?

Re: Ugly frames generated (need newbie help)

TCmullet
I think that you have very long chain of video recording from net:

WebStream -> decode -> render -> capture -> encode -> save to HDD

And I think you need more short chain like this:

WebStream -> save to HDD

No decode. No encode. No Losses and no CPU load. Save stream "As Is".
What will you say? smile

you saying that internally you change it to 120fps and (also internally) decimate by 2?
Yes. It processed internally. And it is optimized. It works without rendering decimated frames.

Does InterFrame come up with whatever is needed to allow any combination of input and output FPS?
Yes. You can use any combinations of these FPS. But you can get maximum quality and minimum CPU load if you using multiply FPSs: from 25 to 50 fps, from 30 to 60 fps, from 23.976 to 47.952 fps.
Or fraction multiply with low devider: 24 * (5:2) = 60, 23.976 * (5:2) = 59.94.

In first case all source frames are saved in the output video, in second case only - only part of source frames will be saved.