1 (edited by oviano 12-01-2014 11:07:43)

Topic: Audio sync issue with capture device and SVP 'Auto' threads

I am capturing via a USB capture device and experiencing major audio sync the more threads I set SVP to use.

At 2 threads there is no discernible issue, but at Auto (which I think means 8 threads in my case) the audio leads the video by something like 750ms.

I'm using MPC-HC, SVP-core, ffdshow and AVIsynth.

I've tried different renderers but it made no difference.

The capture is uncompressed video and audio.

Tried setting the audio delay in the built in audio switcher but it made no difference.

Any experts out there have a clue how I can further diagnose this - maybe something to do with my capture being uncompressed or something?

The problem is that I can use more effective SVP settings by allowing more threads - at 2 threads things start to stutter on screen sooner than when I use Auto.

Intuitively it feels as if the audio isn't being buffered/processed at all - for example when SVP switches profile/activates smoothing I see the video freeze momentarily while it buffers the video but there is no such pause on the audio, so unsurpringly the audio leads the video.

Re: Audio sync issue with capture device and SVP 'Auto' threads

oviano
You are right. Big threads number can lead to audio delay. In my practice it must be more than 40 threads to get 500+ ms offset of audio. But I don't use capture. Just play with smoothing.

Two fast solutions:
1. manually find quite enough value for threads number. It must be between 2 and 8.
2. try to use audio delay setting in your player (grey +/grey - at keyboard in MPC-HC)

Re: Audio sync issue with capture device and SVP 'Auto' threads

Thanks, the audio delay in MPC-HC is having no effect though.

Other options on the audio switcher screen work though, such as Boost.

Re: Audio sync issue with capture device and SVP 'Auto' threads

oviano
Did you try option "Video delay" in SVP tray menu?

Re: Audio sync issue with capture device and SVP 'Auto' threads

Yes, and I can't really tell if it makes a difference because it only allows +/- 250ms whereas my audio is a fair bit more out of sync.

Re: Audio sync issue with capture device and SVP 'Auto' threads

oviano
You can add more values to list manually:
1. open SVPMgr.config in notepad.
2. find strings:

VDelay#1#2;Задержка видео
--250;-250 мсек
--200;-200 мсек
--150;-150 мсек
--100;-100 мсек
--50;-50 мсек
-0;0 мсек;default
-50;+50 мсек
-100;+100 мсек
-150;+150 мсек
-200;+200 мсек
-250;+250 мсек

add your values (for example):

VDelay#1#2;Задержка видео
--750;-750 ms
--500;-500 ms
--250;-250 мсек
--200;-200 мсек
--150;-150 мсек
--100;-100 мсек
--50;-50 мсек
-0;0 мсек;default
-50;+50 мсек
-100;+100 мсек
-150;+150 мсек
-200;+200 мсек
-250;+250 мсек
-500;+500 ms
-750;+750 ms

3. close and save
4. close SVPMgr and run again

Re: Audio sync issue with capture device and SVP 'Auto' threads

Thanks, I added those values.

Unfortunately they have no affect whatsoever, for my setup, either + or -.

Re: Audio sync issue with capture device and SVP 'Auto' threads

oviano
Hmmm.
You need to emulate audio delay to get synchro with video. Video delay appears because of video processing in several threads and video buffer before and after processing. So. Try any audio filter to emulate audio delay with internal audio buffer. Start from AC3Filter. Maybe you know audio filter better than AC3Filter.

Re: Audio sync issue with capture device and SVP 'Auto' threads

Thanks - well I tried the ffdshow audio processor, which support a Delay but ran into another problem - basically it seems to be set up ok in the filter graph, ie it says that it is taking the correct input and outputting it to the TV but no sound comes through.

The audio processor is definitely receiving the signal though because in the configuration panel if I tick 'Volume' and show levels I can see the L and R channels fluctuating.

For some reason though no sound actually goes to the audio device.

I will try the AC3Filter then.

Maybe the issue is because the capture is uncompressed audio or something.

10 (edited by oviano 03-02-2014 10:02:40)

Re: Audio sync issue with capture device and SVP 'Auto' threads

Further update to this.

I noticed that the issue only occurs with a 25fps capture source and not with the 29.97fps capture source.

With the latter, it appears to remain in sync. Further, the actual latency of the video seems about half when the source is 29.97.

I don't think it's SVP though - I quit SVP manager and I can reproduce the issue simply by toggling the AVISynth Forward buffer. Turn off this buffering and the audio syncs, turn it on it gets messed-up as described.

Should/does AVISynth 2.6 alpha 5 work with SVP - maybe I could try that? Though I checked their changelists and couldn't see a relevant fix.

None of the audio processors or built-in MPC-HC audio switcher has any effect on the audio delay, simply gets by passed. Presumably that's because it's a capture rather than a file.

My ugly workaround was to to stop capturing audio on MPC-HC and instead use VAC's Audio Repeater to capture the audio from the device and forward it to the output. Audio repeater allows the buffer to be set. Better still it can be launched and iconised to the taskbar from the command line so I managed to hack in a call to ShellExecute in MPC-HC source to make MPC-HC launch it when the capture starts.

Works well but like I say, ugly solution and I'd prefer it if the latency for 25fps capture sources was similar to 29.97, rather than be 750ms with Auto threads (4 threads I think).

Trouble with reducing to 2 threads is it means I have to dial down the SVP settings to avoid glitches. With Auto threads I can get a better result by being less conservative.

Re: Audio sync issue with capture device and SVP 'Auto' threads

Turns out that if I use a different USB capture device things change.

I was using one made by Inogeni and that was giving me the 750ms video latency with 25-to-50fps interpolation and hence it needed the same audio delay. Tried one made by Magewell and this reduces the latency to about 400ms, similar to 29-to-59fps interpolation.

That's strange because when I turn off SVP completely, both devices exhibit similar and very small latency.

Anyway, it got me thinking - I believe there is an AVISynth SetAudioDelay setting, why not add a setting for this for a given SVP profile? Even in my current scenario with the new device I get video latency of about 400ms which is enough to need an audio delay.

It would be much tidier to be able to tie this setting into the profile, rather than my evil hack to launch the separate Audio Repeater app.

Although, given my experience with setting the audio delay via MPC-HC's own settings I'm not convinced avisynth SetAudioDelay would work - perhaps one of the devs can point me in the direction of the AVISynth script that SVP creates so I can experiment with hardcoding some calls?

Re: Audio sync issue with capture device and SVP 'Auto' threads

oviano
Video delay from audio occurs because of video buffer size and threads number.
1) You can decrease threads number via SVP menu but as you said it is not suitable to you.
2) You can decrease number of frames in ffdShow ahead buffer via SVP hidden setting "ExBuffAheadAdd=3". Did you try it?
3) You can decrease number of frames in Avisynth internal buffer via AVS-script. See line "global svp_cache_fwd=threads+2". You can change it manually in generate.js file.

Re: Audio sync issue with capture device and SVP 'Auto' threads

2) yes, I have reduced this from 3 to 1, as I found (by enabling AVISynth OSD) that it didn't need the extra two. It didn't make a significant difference though.

3) I will take a look. Is that a different buffer to the ffdshow ahead buffer?

But even so, is it not a potential flaw that depending on the hardware/threads etc the audio can be delayed due to the video latency?

I'm not really concerned about having a video latency of even up to 1s, if it makes things for SVP as good as possible, but I feel there needs to be a way of controlling the audio delay - but in such a way that it can be tuned on a per-profile basis, and also works for capture cards not just video files.

Would generate.js be the place to put in a call to SetAudioDelay too? Is that the place where it creates the AVISynth script that gets used?

I appreciate your help by the way!

Re: Audio sync issue with capture device and SVP 'Auto' threads

do not try 3rd option big_smile

Re: Audio sync issue with capture device and SVP 'Auto' threads

oviano
Yes. generate.js is place where the Avisynth script is created. Chainik knows more about internal Avisynth buffer than me. He knows why 'do not try 3rd option' big_smile

In any way SVP needs video buffer to process frames and to produce intermediate ones. Video delay is a fact.
So you need to find a way to add the same delay to audio.

I don't know why it is not important when play file from disk and important when capture video and process frames on-the-fly.
Maybe answer is hidden here.

Audio track not processed by SVP. It is even not used in AVS script. Audio data is bypassing from ffdShow and Avisynth. So you need separate solution for audio.

Re: Audio sync issue with capture device and SVP 'Auto' threads

Thanks, it seems like my 'ugly' solution is maybe as good as I'll get then.

17 (edited by mark007 12-02-2014 19:51:25)

Re: Audio sync issue with capture device and SVP 'Auto' threads

This is something I have also battled with in my setup. Its difficult to get a svp threads + ffdshow buffer value that doesn't cause video desync. In my opinion because SVP is adding frames and can cause these delays, it should have the potential to correct any delay it adds by changing the frames timestamps. Or do you think its ffdshow / avisynth itself thats at fault?

Things like LAV filters will sync audio to video no matter how much the video gets delayed, once the video timestamps are correct coming out of svp / avisynth / ffdshow, but in some situations as described above, the video timestamps must be getting set very very badly wrong? Is this correct Chainik, MAG79? Is it possible to see on your end of things whether situations like desribed above produces timestamps that are forward or backward from where they should be?

Re: Audio sync issue with capture device and SVP 'Auto' threads

Well I've found that by quitting SVP manager and just toggling the ffdshow ahead buffers on and off you can enable/disable the audio sync problem so I suppose the error is inside AVIsynth/ffdshow.

I agree with you though, whatever is delaying the video frames should delay the audio too, otherwise an implicit assumption is being made that the hardware/threads/script combination isn't causing a noticeable video delay - but under some circumstances it is.

Re: Audio sync issue with capture device and SVP 'Auto' threads

No I don't think audio has to be touch, just if video gets delayed, the video frames timestamps should be updated to indicate that. The splitter like LAV splitter should be able to handle any delay in audio / video and still sync them together, once the timestamps are correct.

I assume in this case the video timestamps are being broken somewhere.

Re: Audio sync issue with capture device and SVP 'Auto' threads

Ah right, yeah that makes sense.

Not sure how that would work with capture vs file though.

In my setup the only filters active are:

Enhanced Video Renderer (cp)
Smart Tee (Audio)
ffdshow raw video
Audio capture filter (for the capture device)
Smart tee (Video)
Video capture filter (for the capture device)

Ie there isn't a LAV splitter.

21 (edited by mark007 13-02-2014 11:47:11)

Re: Audio sync issue with capture device and SVP 'Auto' threads

hmm, what ties it all together at the end into the final output, I guess its that components job to take audio / video and sync them together. Is it saving their timestamps in the captured output? If so again it will be up to the player to understand those and it will keep them in sync if the timestamps are right.

Thats my understanding anyways. I might try to see can I find an application that outputs timestamps of frames without avisynth / svp, and with, and see what happens to them, ie are they all offset by a certain amount depending on the buffering / threads.

22 (edited by oviano 13-02-2014 12:08:41)

Re: Audio sync issue with capture device and SVP 'Auto' threads

Well I don't know to be honest, I'm not knowledgeable on how this stuff all fits together.

All I know is that in my configuration it behaves as if the audio is being passed straight through by the player from the capture device to the audio output device with no consideration given to any video delays that may be introduced directly or indirectly by the player along the way.

So that's one issue.

But if you're having issues with a file-based setup, then it seems like there must be another issue in that even where there is a filter that is supposed to sync things up, it's not working, as you say maybe the time stamps for the video frames are not being adjusted properly or something so whatever is meant to keep them in sync doesn't know they are even out of sync.

Re: Audio sync issue with capture device and SVP 'Auto' threads

Thats what I assume is happening. I'm hoping someone like Chainik or Mag79 might have tools to be able to see if this is happening or if SVP is in control of the timestamps. (I assume it is if its creating new frames).

Re: Audio sync issue with capture device and SVP 'Auto' threads

i dunno may it's stupid but what if try to play with ReClock - setting it as "audio renderer" in video player?
it's intended for doing some audio/video sync magic so why not? smile

the other thing is to prove that audio and video really are "independent" - you can change video delay in "ffdshow video" (w/o svp at all)
the next step can be to pass audio through "ffdshow audio" and set some fixed offset

Re: Audio sync issue with capture device and SVP 'Auto' threads

mark007
if SVP is in control of the timestamps. (I assume it is if its creating new frames).

nope, it's totally up to ffdshow/avisynth