Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Results of second analyze MOV-(yamb)->MP4-(mp4dump)->TXT (only part of header)
wide ipd test cam1 t02.txt:

type moov
  type mvhd
   timeScale = 600 (0x00000258)
   duration = 102408 (0x00019008)
   type mdia
    type mdhd
     timeScale = 3000 (0x00000bb8)
     duration = 512041 (0x0007d029)

wide ipd test cam2 t02.txt:

type ftyp
type moov
  type mvhd
   timeScale = 600 (0x00000258)
   duration = 102390 (0x00018ff6)
   type mdia
    type mdhd
     timeScale = 3000 (0x00000bb8)
     duration = 511951 (0x0007cfcf)

Put it into the table and calculate real duration in h:m:s,00 = duration / timeScale.

http://www.svp-team.com/forum/misc.php?item=1973

Good. Both video has identical length. Difference in 0.03 sec is very minimal to notice. But frameCount is different: 5111 and 5109 (from first log).

Post's attachments

real_durations.png, 3.44 kb, 236 x 146
real_durations.png 3.44 kb, 634 downloads since 2012-11-19 

27 (edited by Skaven252 19-11-2012 12:09:31)

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Interesting! smile  (also, more programs and utilities I've never heard of!). Um, that frame count difference is caused by the dropped frames, right?

I used a wireless remote control to start and stop both cameras simultaneously. They won't start or stop *exactly* at the same time, but at least 'close enough' so that the sync can be done manually in Vegas. Usually they're only 1-2 frames off from each other.

It's worked well so far, but those dropped frames are a big problem, as even if the videos are in sync in the beginning, they will end up drifting off sync towards the end as the frame drops wreak havoc...

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Continue of second analyze MOV-(yamb)->MP4-(mp4dump)->TXT (only part of body).
What about timecodes of each frame. It is floating framerate. sad

wide ipd test cam1 t02.vfr:

      type stts
       version = 0 (0x00)
       flags = 0 (0x000000)
       entryCount = 2136 (0x00000858)
        sampleCount = 2 (0x00000002)
        sampleDelta = 100 (0x00000064)
        sampleCount[1] = 1 (0x00000001)
        sampleDelta[1] = 101 (0x00000065)
        sampleCount[2] = 8 (0x00000008)
        sampleDelta[2] = 100 (0x00000064)
        sampleCount[3] = 1 (0x00000001)
        sampleDelta[3] = 101 (0x00000065)
        sampleCount[4] = 1 (0x00000001)
        sampleDelta[4] = 99 (0x00000063)
        sampleCount[5] = 1 (0x00000001)
        sampleDelta[5] = 101 (0x00000065)
        sampleCount[6] = 4 (0x00000004)
        sampleDelta[6] = 100 (0x00000064)
        sampleCount[7] = 1 (0x00000001)
        sampleDelta[7] = 101 (0x00000065)
        ...

wide ipd test cam2 t02.vfr:

      type stts
       entryCount = 2229 (0x000008b5)
        sampleCount = 1 (0x00000001)
        sampleDelta = 100 (0x00000064)
        sampleCount[1] = 1 (0x00000001)
        sampleDelta[1] = 101 (0x00000065)
        sampleCount[2] = 6 (0x00000006)
        sampleDelta[2] = 100 (0x00000064)
        sampleCount[3] = 1 (0x00000001)
        sampleDelta[3] = 101 (0x00000065)
        sampleCount[4] = 1 (0x00000001)
        sampleDelta[4] = 100 (0x00000064)
        sampleCount[5] = 1 (0x00000001)
        sampleDelta[5] = 101 (0x00000065)
        sampleCount[6] = 2 (0x00000002)
        sampleDelta[6] = 100 (0x00000064)
        sampleCount[7] = 1 (0x00000001)
        sampleDelta[7] = 101 (0x00000065)
        ...

Real interframe delays are 99, 100 and 101 of 1/3000 sec. It is not strange that synchro is missing if we not using VFR info.

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Um... now I fell off the tracks. N00b not understand what this mean. Do you mean the framerate is consistent after all?

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Skaven252
Ough! Yes! I was wrong about floating VFR.
It is constant framerate with a little deviation and one hard drop in second clip.

        sampleCount[919] = 2 (0x00000002)
        sampleDelta[919] = 100 (0x00000064)
        sampleCount[920] = 1 (0x00000001)
        sampleDelta[920] = 200 (0x000000c8)
        sampleCount[921] = 1 (0x00000001)
        sampleDelta[921] = 100 (0x00000064)

Base framerate is 29.95 fps (or 29.95048 to be precise).
You can open it and work with it as CFR:

DirectShowSource("[filename]", fps=29.95, convertfps=true)

After that you can get synchro by 0.003 sec (3 msec) wink

Later I will tell about how I checked it.

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Looks like you came to the right place Skaven...   wink

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Indeed. smile  Thanks for your help!

So... when AviSynth does the "convertfps", will it correct the drops by duplicating frames so that the hard drop doesn't throw the clips off sync?

And, um, how do I get this to work with QTSource, or do I need to convert those .movs to some other format first?

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Skaven252
when AviSynth does the "convertfps", will it correct the drops by duplicating frames so that the hard drop doesn't throw the clips off sync?
Yes. In your case it will be added no one duplicate frames to the first clip and only one to the second clip.

Try to open MOVs by DirectShowSource. If it works then it is nice (on my system it can't open MOVs, I don't know why.., I'm searching hmm ).
Otherwise point in QTSource right framerate (29.95). And with only one hard-drop you can forget about sync issue wink

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Thanks! smile

MAG79 wrote:

Otherwise point in QTSource right framerate (29.95). And with only one hard-drop you can forget about sync issue wink

Um, "pointing the framerate", I presume, doesn't do the same as AssumeFPS? Because with AssumeFPS (which I used to force both clips to the same framerate when re-encoding) they drifted pretty badly out of sync, like at least 0.15 seconds or so.

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Skaven252
Um, "pointing the framerate", I presume, doesn't do the same as AssumeFPS?
No. It is different things. If you open video with wrong FPS you will lose original frames or will get unwanted duplicates. For example opening 30 fps video with framerate 1 frame/sec will lead to losing 29 frames each second. And AssumeFPS in this case can't help absolutely.

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Skaven252
I checked how works QTInput. It works good with different value of mode parameter.
QTInput with mode=0 works good. Frame # 2108 is copy of 2107 in second clip and FramesCount is 5111.

QTInput("wide ipd test cam2 t02.mov", vfrFPS=29.95)

This script gives perfect sync with true length.

But with used by You mode=3 I can't see duplicate frame in the region of #2107 frame. This mode is not frame accurate. It potentially can lead to lose synchro.

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Great, thanks for checking this for me! I was just googling for the QTInput wiki a while ago, when a notification of this reply appeared. smile

I remember having some other difficulties with other modes than mode=3 (AviSynth crash? I don't remember) that's why I was using it.

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

BTW, those files I sent you are not the original files from the cameras. They give them wrong moov atom header info (which makes AviSynth stop at about 75% through the video), so I have to open them in QuickTime Pro and save them again as "stand alone movie". This rebuilds the header data.

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Skaven252
rebuilds the header data
Are you sure this operation is correct. Results still have synchro?

Can you share source to me?
Second file with one drop will be enough. wink

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Sure! I'm sending it to WeTransfer now.

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Skaven252
Original file showed the same statistics as previous MOV: mkv-vfr and mp4-dump are identical in part with timeCodes.
It is mean your processing between MOVs is correct wink

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

About CFR calculation. This is graphical result (I computed it in MS Excel): deviations of frames timeCodes. Data of clip2 used.

http://www.svp-team.com/forum/misc.php?item=1977

Graphic shows how early (lead) or later (lag) each frame will be dispayed if play frames at constant frame rate 29.95 fps.
As you can see maximum deviation is about 2 ms. MKV and MP4 data is close to each other. And we have one drop at the middle of the video. We must to insert one duplicate of frame to fix frames lead after drop.
QTInput(vfrFPS=29.95) and DirectShowSource(fps=29.95, convertfps=true) will do it automatically. All frames will opened. Without losses. All frames are at its own places.

Post's attachments

deviations_clip2.png, 19.32 kb, 860 x 449
deviations_clip2.png 19.32 kb, 653 downloads since 2012-11-20 

43 (edited by Skaven252 21-11-2012 07:06:08)

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Wow, how did you generate that report? Just Excellence? That's a lot of work again, many thanks! smile

I ran into an odd issue with QTSource... if I use "mode=0", it halves the resolution of the video to 960x540.  hmm This happens both in VirtualDub or if I open the .avs in any media player.  Only with "mode=3" I get full resolution.

With DirectShowSource, when opening the .avs, VirtualDub just freezes and refuses to open the video. I wonder if I need to install some codec or such to get .movs to work through DirectShow...

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Skaven252
how did you generate that report? Just Excellence?
Just MS Excel and couple of hours wink

if I use "mode=0", it halves the resolution of the video to 960x540
It shows to me the same.

I can't find any success way to open MOV via DirectShowSource too.

Only with "mode=3" I get full resolution.
Then I recommend to convert to any other container and open not through QTInput. QTInput(mode=3) adds fake dropped frames.

For example.
You can use yamb to produce MP4 (without re-encoding). Or you can use gdsmux to produce MKV (without re-encoding too).

45 (edited by Skaven252 21-11-2012 10:06:08)

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

My camera can also record straight into mp4 container. I wonder what the difference between its .mp4 and .mov files is.

Again, I remember I had some difficulties with .mp4 so I switched to .mov - but I can't remember what those difficulties were...  yikes  ...likely it was the "AviSynth stopping at 75% through the video" issue. And I had to switch to .mov because QuickTime couldn't re-save (to rebuild header data) the .mp4 files.

I'll have to try make some test recordings into .mp4 next weekend ( = more free time to play around with video files, this is slow stuff smile ).

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

I'm VERY pleased that someone has addressed this problem of dropped (duplicated) frames in a video.  Thank you!

Even with Russian to English translation (courtesy of Google Chrome), it's very difficult to wade thru that Russian thread.  Could you please reproduce here the most current DoubleDropFix script and maybe the GameDropFix script too, and tell us here why GameDropFix is needed?  I would think that it wouldn't matter what the video source was.  When a frame is dropped, the capture utility duplicates the last frame it had gotten.  This being so, it's puzzling why source=game would be any different than source=captured video.

What I have is captured video at 29.97fps and in some seconds (but not all) a frame will freeze for 1 or 2 frames, meaning there are either 2 frames identical or 3.  I'd be very happy if even the 2-identical-frame occurrences could be fixed.  (Need 1 generated frame.)  But if 3-identical-frame occurrences (needing 2 generated frames) could also be fixed, that would be SUPER!

(While waiting for your appreciated answer, I'll research getting AVISynth and SPV installed on my target machine.  I'll be feeding the script into VirtualDub.  I haven't touched SVP in a couple years.)

47 (edited by TCmullet 23-12-2014 03:41:40)

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

I've found and installed the current SVP and AVISynth for SVP.  And here is (what I believe is) the current DoubleDropFix (v3) with my own file referenced:

DSS2("D:\Video-Work\2014-NCAA-wv\2014-11-14.wv.KY-v-MS.lag.ac3-320.avi").ConvertToYV12()
sc = msup (for = 1)
vf = MAnalyse (sc, ISB = false)
vb=MAnalyse(sc,isb=true)
global frame_50 = MFlowInter(sc,vb,vf,time=50).SubTitle("Fix-50", size=12, align=3)
global frame_33 = MFlowInter(sc,vb,vf,time=33).SubTitle("Fix-33", size=12, align=3).trim(1,0)
global frame_66 = MFlowInter(sc,vb,vf,time=66).SubTitle("Fix-66", size=12, align=3)
global frame_src = last
global mini=sc.crop(8,height+8*3,width/2,height/2).TemporalSoften(1, 1, 0, scenechange=1, mode=2)
mini.mt_lut global luma_drop = (y = -1, you = 128, v = 128)
drop_clip = mini.scriptclip ("mini.ydifferencefromprevious dif = ()
\ dif_next1=mini.trim(1,0).ydifferencefromprevious()
\ dif_prev1=(mini.trim(1,1)+mini).ydifferencefromprevious()
\ dif_nextMax=max(max(dif,dif_next1)/8,0.01)
\ Dif_prevMax = max (max (dif, dif_prev1) /8,0.01)
\ Dif> min (dif_prevMax, dif_nextMax)? Short: luma_drop
')
scriptclip ("
\ Fix_prev2=(drop_clip.trim(1,2)+drop_clip).AverageLuma
\ Fix_prev1=(drop_clip.trim(1,1)+drop_clip).AverageLuma
\ Fix=drop_clip.AverageLuma
\ Fix_next1=drop_clip.trim(1,0).AverageLuma
\ Fix_next2=drop_clip.trim(2,0).AverageLuma
\ Fix!=1
\ ? frame_src
\ : Fix_next1!=1
\   ? Fix_prev1!=1
\     ? frame_50
\     : Fix_prev2!=1
\       ? frame_66
\       : frame_src
\   :  Fix_next2!=1
\     ? Fix_prev1!=1
\       ? frame_33
\       : frame_src
\     : frame_src")

It's bombing on the DSS2 statement.  (Virtualdub gives message, "Unable to open file ___ (my .avs script)".)  I've never used that, only having used avisource.  (And I haven't used AVISynth in a long time, either.)

I have these .dll files in the folder, C:\Program Files (x86)\AviSynth 2.5\plugins:

DirectShowSource.dll
libflowsse.dll
libflowgpu.dll
mvtools2.dll
svpflow1.dll
svpflow_cpu.dll
svpflow2.dll
svpflow_gpu.dll  (I can supply date stamps on these if necessary)

As they are in the plugins folder, they should be loaded.  Yet it seems to be acting like they're not.  What am I missing??

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

TCmullet
For avi-files you can use AviSource instead DSS2.
DSS2 is DirectShowSource2 filter that you can find in Matroska Haali splitter installed folder. You can copy avss.dll from there to avisynth\plugins folder to make script working. DSS2 is needed to non avi files to get frame-accurate processing.

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Thanks, Mag.  I'd never heard of DSS2, so I didn't realize it merely opens a file.  I replaced that line with this:

avisource("D:\Video-Work\2014-NCAA-wv\2014-11-14.wv.KY-v-MS.lag.ac3-320.avi").ConvertToYV12()

When I run it in a script by itself (that line being the only line), it works fine.  (I'm opening the .avs file as a video input file in VirtualDub.  But when I have it at the top of your entire script, the script still bombs.  The first few lines look like this:

avisource("D:\Video-Work\2014-NCAA-wv\2014-11-14.wv.KY-v-MS.lag.ac3-320.avi").ConvertToYV12()
sc = msup (for = 1)
vf = MAnalyse (sc, ISB = false)
vb=MAnalyse(sc,isb=true)
global frame_50 = MFlowInter(sc,vb,vf,time=50).SubTitle("Fix-50", size=12, align=3)
global frame_33 = MFlowInter(sc,vb,vf,time=33).SubTitle("Fix-33", size=12, align=3).trim(1,0)
.
.
.

It's still bombing (with VD's unhelpful message) and I know of no way to debug these scripts, esp. w/o knowing all the elements used.  (I realize VD shouldn't be blamed for the cryptic message, as it's only receiving frameserved output.)  Can you please look to see if you see some error in the rest of your script AS duplicated by me here?  I pasted it from your Russian thread into Notepad, then into here (and my own script).

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

TCmullet
Try to use original DoubleDropFix_v3.avs:

AviSource("D:\Video-Work\2014-NCAA-wv\2014-11-14.wv.KY-v-MS.lag.ac3-320.avi").ConvertToYV12()
sc=MSuper(pel=1)
vf=MAnalyse(sc,isb=false)
vb=MAnalyse(sc,isb=true)
global frame_50 = MFlowInter(sc,vb,vf,time=50).SubTitle("Fix-50", size=12, align=3)
global frame_33 = MFlowInter(sc,vb,vf,time=33).SubTitle("Fix-33", size=12, align=3).trim(1,0)
global frame_66 = MFlowInter(sc,vb,vf,time=66).SubTitle("Fix-66", size=12, align=3)
global frame_src = last
global mini=sc.crop(8,height+8*3,width/2,height/2).TemporalSoften(1, 1, 0, scenechange=1, mode=2)
global luma_drop=mini.mt_lut(y=-1, u=128, v=128)
drop_clip=mini.scriptclip("dif=mini.ydifferencefromprevious()
\ dif_next1=mini.trim(1,0).ydifferencefromprevious()
\ dif_prev1=(mini.trim(1,1)+mini).ydifferencefromprevious()
\ dif_nextMax=max(max(dif,dif_next1)/8,0.01)
\ dif_prevMax=max(max(dif,dif_prev1)/8,0.01)
\ dif>min(dif_prevMax,dif_nextMax) ? mini : luma_drop
")
scriptclip("
\ Fix_prev2=(drop_clip.trim(1,2)+drop_clip).AverageLuma
\ Fix_prev1=(drop_clip.trim(1,1)+drop_clip).AverageLuma
\ Fix=drop_clip.AverageLuma
\ Fix_next1=drop_clip.trim(1,0).AverageLuma
\ Fix_next2=drop_clip.trim(2,0).AverageLuma
\ Fix!=1
\ ? frame_src
\ : Fix_next1!=1
\   ? Fix_prev1!=1
\     ? frame_50
\     : Fix_prev2!=1
\       ? frame_66
\       : frame_src
\   :  Fix_next2!=1
\     ? Fix_prev1!=1
\       ? frame_33
\       : frame_src
\     : frame_src")

Where did you get your version?