• Thanks: 47

Topic: SVP + RIFE transcoding audio desync

Hi, guys! Started using RIFE in SVP with MPV a few weeks ago. My laptop specs are Ryzen 5800H with 32 GB DDR4-3200 and an RTX 3070 with 145 W TGP. First time I tried RIFE with the movie Bullet Train in 1080p and it loaded instantly. I messed with the settings a bit and everything was applying instantly. Then after 2 days I tried to watch an episode of Star Girl in 1080p and loading times were a bit longer. Saw that the video had low framerate and I messed with the settings a bit. After a few minutes of messing around and changing frame size to 1080 or lower, the video started playing at 60 fps. After that, could not manage to load RIFE with 60 fps on any other video.

I understand it takes some time to load for on-the-go interpolation, but it seems like I might have messed something up. Tried to add RIFE anime model, but that does not seem to do anything. Also installed RIFE 4.6... still does not play at 60 fps. Probably need a more powerful GPU. Anyway, figured I should try transcoding the videos. I managed to transcode the latest Peripheral episode without problems (took around 4 hours). Then I transcoded the latest episode from Star Girl in around 2 h and 15 minutes before the end of the episode the video stuttered and the audio desynced a bit. Not sure what caused this. Peripheral used scene change threshold set to max, and Star Girl used threshold set to minimum... maybe this messed up things? Is there a way to fix audio desync after transcoding is done? Or maybe a way to avoid this altogether?

  • UHD
  • Member
  • Offline
  • Thanks: 39
  • Thanks for the post: 1

Re: SVP + RIFE transcoding audio desync

Xenocyde wrote:

Then I transcoded the latest episode from Star Girl in around 2 h and 15 minutes before the end of the episode the video stuttered and the audio desynced a bit. Not sure what caused this.

This can be caused by a variable frame rate: https://en.wikipedia.org/wiki/Variable_frame_rate

  • UHD
  • Member
  • Offline
  • Thanks: 39
  • Thanks for the post: 1

Re: SVP + RIFE transcoding audio desync

It can help you:
https://github.com/styler00dollar/VSGAN … docker#vfr

  • Thanks: 1622
  • Thanks for the post: 3

Re: SVP + RIFE transcoding audio desync

yeah, RIFE plugin may not pass variable frame durations correctly in case of non-integer interpolation factor (like 24 *2.5 = 60)
good news I know how to fix this big_smile

5 Reply by Xenocyde (edited by Xenocyde 08-11-2022 09:22:12)

  • Thanks: 47

Re: SVP + RIFE transcoding audio desync

UHD wrote:

It can help you:
https://github.com/styler00dollar/VSGAN … docker#vfr

I tried to figure out how this works, but it being only command line is kinda tedious for me. Hope we can get some sort of basic UI soon, similar to how some text-to-image AI renderers have an internet browser-based UI.

  • Thanks: 47

Re: SVP + RIFE transcoding audio desync

Chainik wrote:

yeah, RIFE plugin may not pass variable frame durations correctly in case of non-integer interpolation factor (like 24 *2.5 = 60)
good news I know how to fix this big_smile

Hopefully this is coming soon? smile

Re: SVP + RIFE transcoding audio desync

Chainik wrote:

yeah, RIFE plugin may not pass variable frame durations correctly in case of non-integer interpolation factor (like 24 *2.5 = 60)
good news I know how to fix this big_smile

Получилось у Вас это исправить?

Re: SVP + RIFE transcoding audio desync

In fact, if someone have a sample VFR video that is not de-synced after "normal" transcoding and de-synced with RIFE - please share it
I can't see any problems with VFR videos I have.

9 Reply by Xenocyde (edited by Xenocyde 18-11-2022 15:37:36)

  • Thanks: 47

Re: SVP + RIFE transcoding audio desync

Chainik wrote:

In fact, if someone have a sample VFR video that is not de-synced after "normal" transcoding and de-synced with RIFE - please share it
I can't see any problems with VFR videos I have.

The problem might only arise with full length videos of at least 40 minutes. Don't think I can reproduce this with a short sample.

Since my original post, I have encountered the desync problem again with another video, but that one did not show any particular stuttering to trigger the desync. It looked like the desync happened gradually over the course of at least 10 minutes.