i dunno may it's stupid but what if try to play with ReClock - setting it as "audio renderer" in video player?
it's intended for doing some audio/video sync magic so why not?
the other thing is to prove that audio and video really are "independent" - you can change video delay in "ffdshow video" (w/o svp at all)
the next step can be to pass audio through "ffdshow audio" and set some fixed offset