1 Topic by m45t3r (edited by m45t3r 23-02-2017 01:30:08)

  • Thanks: 5
  • Thanks for the post: 4

Topic: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

EDIT 22/02/2017: contrary to what I said before, this is still a problem. However, it actually depends of the type of video detected by SVPtube: M3U8 streams are parsed directly by mpv, works perfectly, and this seems the format used by more recently releases at Crunchyroll. RTMP streams needs to be muxed in a "video.mp4" file and brings the wrong FPS problem.

EDIT 19/02/2017: since latest SVPtube update, the procedure described in this topic does not seem to be necessary anymore. The problem seems to be fixed since SVPtube know uses a very long string as filename (with ".m3u"8 extension), instead if the old "video.mp4". They probably started to feed the HLS stream directly to mpv instead of trying to remux the video, solving the problem.

So I finally solved the problem of audio desync when using mpv+SVP4 and Crunchyroll (using SVPTube or directly by mpv+youtube-dl) or HorribleSubs. This seems to be a common problem:

https://www.svp-team.com/forum/viewtopic.php?id=3563
https://www.svp-team.com/forum/viewtopic.php?id=3584
https://www.svp-team.com/forum/viewtopic.php?id=3357

The problem is basically what is described in this post: Crunchyroll encoded videos reports a FPS of 23.810, instead of the more common 23.976. Actually, this seems to be a lie: the video is actually encoded with 23.976FPS, however the container reports a lower FPS for some reason. This doesn't cause any problem for normal playback, however when we use SVP this makes the video go faster than the audio, causing a desync.

The cause of the problem seems to be in the vapoursynth support in mpv: vapoursynth does not have any information about the video, so mpv fills some variables accesible to vapoursynth scripts with some information about the video. The variable that stores the video FPS is container_fps, however this variable reports the container FPS instead of the actual video FPS (I don't know if mpv actually has this information btw). So vapoursynth gets the wrong FPS, and SVP by instance gets the wrong FPS too, scalling the video to 59.6852 or 60.6073 instead of the correct 59.97FPS that we see in videos with proper FPS.

FYI, HorribleSubs basically dumps and mux Crunchyroll streams, so they get it wrong too.

So, how to fix it? We will create a special mpv.conf file that will force the correct FPS in the video, allowing us to watch anime from Crunchyroll without audio desync (however there are still some minor problems, I will describe it better later).

Press Win+R and type "%appdata%\mpv", this will open a new Windows Explorer window. Create a new .txt file there, name it "mpv.conf" and edit it as below:

[svp-crunchyroll]
profile-desc="Crunchyroll fix for SVPTube"
fps=23.976
no-correct-pts
sub-delay=0.166
#sub-speed=23.976/23.810

[extension.mp4]
profile=svp-crunchyroll

So basically what we're doing is create a new profile that makes mpv to assume the video is running at 23.976FPS, and now SVP correctly scales the video to 59.97FPS. no-correct-pts makes mpv to assume that the video metadata is broken so it should assume the FPS we set before. Since we are using [extension.mp4] it will only apply to .mp4 files, since SVPTube mux the output to a video.mp4 file. If you're watching from HorribleSubs, you probably should add a [extension.mkv] too.

Now for the problems: the subtitles still assume a video playing with 23.810, so they will be slightly behind and sometimes they will be too fast. I fixed the first problem with sub-delay, however the second problem I tried to fix with the sub-speed option without success. If someone can fix the second problem it would be interesting, so I leave it to someone to try. However even with this problem, the videos from Crunchyroll/HorribleSubs are watchable.

The other problem is, since we use this profile for every video file with mp4/mkv extension, this will break other videos you try to play with mpv. There isn't a simple way to fix this: you can comment those lines in mpv before playing other type of video, however this a pretty bad solution.

I hope the SVP team comes with a better workaround for this problem. If we could pass parameters for mpv from SVPTube, for example, we could simple pass mpv --profile=svp-crunchyroll when playing Crunchyroll videos, and mpv --profile=default when playing normal, non-broken content. For now at least this is a workaround.

Edit: MPC-HC does not seem to suffer from this problem at all. However mpv, at least in my experience, is much smoother than MPC-HC. And MPC-HC does not support RTMP streams like mpv, so your only option is to use HorribleSubs or download the video with SVPTube.

  • MAG79
  • SVP developer
  • Offline

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

m45t3r
Thank you for explanation.
Good suggestion about --profile parameter to mpv. It is good but partial solution.
But the best way is to fix the real problem.
I will look what we can do in common case. We need to recheck real fps for video with strange framerate and pass real fps instead of wrong value.

3 Reply by myfakeacc (edited by myfakeacc 24-12-2016 12:41:52)

  • Thanks: 7

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

The cause of the problem is a damaged file (incorrectly tagged FPS/duration). Therefore, in my opinion, the only sane way to fix this is by fixing the damaged file: remux it with the correct FPS value. Remuxing takes like 2 seconds, it won't change the video or audio and fixes the problem mentioned here, for every player.
I don't think that configuring/changing behavior of your player is a good solution for fixing an error in a file that can be corrected so easily just by remuxing.

A incorrect FPS value in the container is actually so common that every decent video player simply ignores it and does playback without. E.g. many .wmv files have a tagged FPS value of 9999. mpv offers this container fps value as property for scripts etc.
mpv contains one more fps property: estimated fps. This is the real FPS value which is also correct for incorrectly tagged videos. There is one drawback: it is a measured value, averaging over the last n frames. Consequently, it is only correct after n subsequent samples (i.e. only after a few frames were rendered) and it might change during playback (e.g. variable frame rate videos). Therefore it is unsuitable for a script that calculates stuff on startup like SVP

  • Thanks: 5

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

myfakeacc wrote:

The cause of the problem is a damaged file (incorrectly tagged FPS/duration). Therefore, in my opinion, the only sane way to fix this is by fixing the damaged file: remux it with the correct FPS value. Remuxing takes like 2 seconds, it won't change the video or audio and fixes the problem mentioned here, for every player.
I don't think that configuring/changing behavior of your player is a good solution for fixing an error in a file that can be corrected so easily just by remuxing.
SVP

If I was watching HS files this would be a acceptable (however a PITA) solution, however I mostly watch directly from Crunchyroll. I don't know if whatever mux SVPTube/youtube-dl uses could fix do the right thing.

myfakeacc wrote:

A incorrect FPS value in the container is actually so common that every decent video player simply ignores it and does playback without. E.g. many .wmv files have a tagged FPS value of 9999. mpv offers this container fps value as property for scripts etc.
mpv contains one more fps property: estimated fps. This is the real FPS value which is also correct for incorrectly tagged videos. There is one drawback: it is a measured value, averaging over the last n frames. Consequently, it is only correct after n subsequent samples (i.e. only after a few frames were rendered) and it might change during playback (e.g. variable frame rate videos). Therefore it is unsuitable for a script that calculates stuff on startup like SVP

Maybe trying to match the container_fps to a list of known, sane FPS values, like 23.976, 25, 29.97 and so on. A value of 23.810 would be fixed to 23.976, a value of 29.90 would be fixed to 29.97 etc etc. However, of course, this would break in videos that actually have crazy FPS settings.

Well, since this is kinda of a hard problem I think my proposal in OP is at least one way to fix the most annoying case, that is SVPTube+Crunchyroll. I think other video providers don't have wrong FPS settings for the majority of their video library. It would be nice if CR actually fixed their videos, however I don't think this will happen anytime soon.

5 Reply by myfakeacc (edited by myfakeacc 24-12-2016 23:31:38)

  • Thanks: 7
  • Thanks for the post: 1

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

Ah, streams. Well, I think I might have a pretty good solution for you.
Get this script and put it into your mpv scripts folder: https://github.com/wm4/mpv-scripts/blob … ofiles.lua
Now you can create profiles that are applied based on lua expressions. Additionally, you can use mpv properties like normal variables.

E.g.:

[svp-crunchyroll]
profile-desc=cond:get('container_fps', 0) > 23 and get('container_fps', 0) < 23.9
fps=23.976
...

It's not a perfect solution. The profile expression is re-evaluated every time the used properties change and applied/activated when it evaluates to true. Therefore, when watching streams with variable fps it could activate the profile (e.g. when the fps suddenly drop below 23.9 but stay above 23 (in this case)).
Of course you can make your own, even more precise conditions.

Note: profiles do not get "unapplied". In case you want to "unapply" a profile you need an inverted version, e.g.:

[4K]
profile-desc=cond:get('width', -math.huge) >= 3840
vd-lavc-threads=32

[4K-inverted]
profile-desc=cond:get('width', -math.huge) < 3840
vd-lavc-threads=0
  • Thanks: 5

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

myfakeacc wrote:

Ah, streams. Well, I think I might have a pretty good solution for you.
Get this script and put it into your mpv scripts folder: https://github.com/wm4/mpv-scripts/blob … ofiles.lua
Now you can create profiles that are applied based on lua expressions. Additionally, you can use mpv properties like normal variables.

E.g.:

[svp-crunchyroll]
profile-desc=cond:get('container_fps', 0) > 23 and get('container_fps', 0) < 23.9
fps=23.976
...

It's not a perfect solution. The profile expression is re-evaluated every time the used properties change and applied/activated when it evaluates to true. Therefore, when watching streams with variable fps it could activate the profile (e.g. when the fps suddenly drop below 23.9 but stay above 23 (in this case)).
Of course you can make your own, even more precise conditions.

Note: profiles do not get "unapplied". In case you want to "unapply" a profile you need an inverted version, e.g.:

[4K]
profile-desc=cond:get('width', -math.huge) >= 3840
vd-lavc-threads=32

[4K-inverted]
profile-desc=cond:get('width', -math.huge) < 3840
vd-lavc-threads=0

This is a good idea, however it does not seem to work. Maybe because when auto-profiles applies the profile, the wrong container_fps is already reported to SVP, so it does not matter if I change it.

7 Reply by dlr5668 (edited by dlr5668 25-12-2016 05:18:00)

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

Its kinda works

input-ipc-server=mpvpipe
vo=opengl-hq:backend=dxinterop:scale=ewa_lanczossharp:cscale=ewa_lanczossoft:interpolation:tscale=mitchell:tscale-clamp:temporal-dither
video-sync=display-resample
video-sync-max-video-change=10
hwdec=d3d11va-copy
slang=enUS,en,eng,ptBR,pt
alang=jp,jpn,enUS,en,eng,ptBR,pt
save-position-on-quit=yes
autosync=10
script-opts=autospeed-nircmd=true,autospeed-speed=false,autospeed-nircmdc="C:\Users\vadash\AppData\Roaming\mpv\nircmdc.exe",autospeed-rates="50,60,71",autospeed-osd=true
log-file="C:\tmp\mpv.log"

[30_fps]
profile-desc=cond:get('container_fps', 0) > 29.5 and get('container_fps', 0) < 30.5
fps=30

[25_fps]
profile-desc=cond:get('container_fps', 0) > 24.5 and get('container_fps', 0) < 25.5
fps=25

[23_976_fps]
profile-desc=cond:get('container_fps', 0) > 23.9 and get('container_fps', 0) <= 24
fps=23.976

[23_810_fps]
profile-desc=cond:get('container_fps', 0) > 23 and get('container_fps', 0) <= 23.9
fps=23.976

But u need to press "_" twice to reload video (#_ cycle video setting). Thanks for new plugin mate.

Post's attachments

mpv_2016-12-25_08-09-43.png 943.45 kb, 298 downloads since 2016-12-25 

8 Reply by m45t3r (edited by m45t3r 25-12-2016 17:09:00)

  • Thanks: 5

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

dlr5668 wrote:

But u need to press "_" twice to reload video (#_ cycle video setting). Thanks for new plugin mate.

Nice one. This seems to work automatically:

[svp-crunchyroll]
profile-desc="Crunchyroll fix for SVPTube"
fps=23.976
no-correct-pts
sub-delay=0.166

[23_810_fps]
profile-desc=cond:get('container_fps', 0) > 23 and get('container_fps', 0) <= 23.9
vid=no
profile=svp-crunchyroll
vid=1

Edit: nope, said too soon. Even toggling manually still causes desyncs. Looking at SVP itself it still reports a wrong FPS, so I think it didn't update its state.

  • Thanks: 5
  • Thanks for the post: 1

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

Well, this seems to work:

[svp-crunchyroll]
profile-desc="Crunchyroll fix for SVPTube"
fps=23.976
no-correct-pts
sub-delay=0.166

[normal]
fps=0
correct-pts
sub-delay=0

[default]
profile=svp-crunchyroll

[non-broken]
profile-desc=cond:get('container_fps', 0) < 23 and get('container_fps', 0) >= 23.9
profile=normal

However I don't have non 23.976 FPS videos to test, so I don't know if this work for the non broken case.

  • Thanks: 7

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

I thought SVP adapts to changing FPS (maybe after a few seconds)?

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

> I thought SVP adapts to changing FPS

Yep, but not with mpv, because SVP doesn't even know about actual FPS change.
That "estimated-vf-fps" property mentioned above shows "estimated/measured FPS of the video filter chain output", i.e. after the vapoursynth filter, and since vapoursynth script have to call AssumeFPS(), estimated-vf-fps just stays fixed.

  • Thanks: 5

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

Just coming to report that even in last SVP (that at least according to changelog, did fix this problem), I still get wrong FPS in mpv.

I am going to ask, can't we have an option in SVP configuration to force the FPS from input video? It seems to be better to fix this in SVP side than mpv side.

  • Thanks: 5

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

Reporting that the lastest SVPtube update did seem to fix this issue. Now I can watch Crunchyroll videos without the hack described in this topic. Just FYI, in this latest version SVPtube seems to use .m3u8 playlist files, instead of the older video.mp4 muxed file.

The FPS reported by SVP is still somewhat strange, though, 59.9361 FPS, however I already watched two problematic series in Crunchyroll since the update (Fuuka and Rewrite) and didn't have any desync problem yet.

14 Reply by m45t3r (edited by m45t3r 23-02-2017 01:26:03)

  • Thanks: 5

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

Well, I said too soon. It seems that some videos in Crunchyroll are still using RTMP protocol, so they're still having wrong FPS. For example, this one: http://www.crunchyroll.com/konosuba-god … ead-727601

HLS (identified as M3U8 in SVPtube) videos seems fine, since they're decoded by mpv itself. Well, back to my old hack.

Something interesting I discovered today too, the FPS reported by SVP when I use M3U8 streams (59.9361 FPS) actually seems correct. The videos using this FPS seeks correctly, while I still have some desync when watching RTMP streams (that maybe problematic in longer videos like movies, however since the majority of animes episodes are only ~25 minutes it is not problematic).

  • Thanks: 3

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

I couldn't get the auto-profile script to work. So I just changed the extension of HorribleSubs files to .hs and appended this to my config.


[extension.hs]
fps=23.976
no-correct-pts
sub-delay=0.166

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

I was having the same problem with a couple of anime chapters that i downloaded from another site. It seems that they were based on HorribleSubs release because i fixed the audio desync with the solution that you posted here. Thx for the info!
I'm using mpv + svp pro 4.2.0.122

  • Thanks: 1

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

Latest changelog:

+ "Is variable frame rate" profile condition
+ attempt to fix incorrectly reported source frame rate: "Treat 23.810 as 23.976" option

18 Reply by adamkex (edited by adamkex 05-05-2018 04:40:41)

  • Thanks: 3

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

starks wrote:

Latest changelog:

+ "Is variable frame rate" profile condition
+ attempt to fix incorrectly reported source frame rate: "Treat 23.810 as 23.976" option

Still doesn't work for me. I still need to use my dirty hack.

Tested on: "[HorribleSubs] Eromanga-sensei - 01 [1080p].mkv"

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

> Tested on: "[HorribleSubs] Eromanga-sensei - 01 [1080p].mkv"

Ah. In this case, MediaInfo gives correct 23.976, but mpv for some reason still thinks it's 23.810, thus "container_fps" script variable is 23.810 and the target rate in AssumeFPS is incorrect.

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

should be fixed now (Rev.137), at least for this particular "Eromanga-sensei - 01" video

  • Thanks: 1
  • Thanks for the post: 1

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

I've thrown a few similarly-encoded files at it. Looks good.

Lip flaps match up and there's no drift in the A/V sync.

22 Reply by mashingan (edited by mashingan 09-10-2019 05:46:48)

  • Thanks: 37

Re: Solving Crunchyroll/HorribleSubs audio desync with mpv+SVP4

Late but I painfully just had this case, I solved it by checking whether the container_fps is 0.00 or 23.81, if its value is one of those, I then change it 23.976.

Apparently the problematic video reported wrong fps, checked with MediaInfo resulted in correct fps but using ffprobe returned 500/21 (23.81 fps).
Since mpv using ffmpeg, so mpv got the wrong fps too.

Another way maybe by checking video meta info with MediaInfo, get the correct fps and run the mpv with options --no-correct-pts --fps=<fps-from-mediainfo>

I'm still finding a way whether it's possible to run a script once before actually running the video/mpv player itself. (Well, using bash is pretty much possible, but I prefer double-clicking my video file tongue )