Topic: Slow video / audio of sync, RIFE AI.

Hello,

I am new to motion interpolation and this software.

I just want to watch anime and some movies at 60 fps.

i setup tensorRT and watched a few movies, it worked flawlessly.

However, now I am watching an anime and the audio is completely out of sync. Audio comes first and the scene happens after.

My Specs are:
5600X
32GB
3060

Rife Options
Threads: 2
Model: 4.6
Performance Boost: Enabled

Scene Change Threshold: Custom:36, I tried all different values. 10, 15, 20 etc
and Do not remove duplicate frames.

2 Reply by flowreen91 (edited by flowreen91 05-05-2024 21:56:31)

  • Thanks: 99
  • Thanks for the post: 1

Re: Slow video / audio of sync, RIFE AI.

aliyansaleem2013 wrote:

the audio is completely out of sync. Audio comes first and the scene happens after.

This is usually caused by watching videos higher than 1920x1080.
Since SVP needs to pass all the video frames to Rife to interpolate them, if the frames are big it will take more processing time.
This makes it to run in 10 fps or less since it can't output them fast enough to catch up to the audio.
The solution is to just add a resize setting in SVP configuration menu to lower the frame to a size that your 3060 can process.
Go to Application Settings, search resize and add:
-19201080
or
-12800720
https://gyazo.com/cac3a2a8f4b8c384fcce1bbcf3e8dd1d

Re: Slow video / audio of sync, RIFE AI.

flowreen91 wrote:
aliyansaleem2013 wrote:

the audio is completely out of sync. Audio comes first and the scene happens after.

This is usually caused by watching videos higher than 1920x1080.
Since SVP needs to pass all the video frames to Rife to interpolate them, if the frames are big it will take more processing time.
This makes it to run in 10 fps or less since it can't output them fast enough to catch up to the audio.
The solution is to just add a resize setting in SVP configuration menu to lower the frame to a size that your 3060 can process.
Go to Application Settings, search resize and add:
-19201080
or
-12800720
https://gyazo.com/cac3a2a8f4b8c384fcce1bbcf3e8dd1d


The videos are indeed 1920x1080, the movies that I watched were also in 1080p. I've never download anything higher because my monitor is 1080p.
I don't think its running in 10 fps, the video is smooth. (is there a way to check at what fps is it running in real time (like msi afterburner for games))?
I still tried the solution though. tried both res, resize 1080p and 720p, didn't work. audio is still out of sync. It works fine on Automatic but RifeAI just looks better.
Also what does resize do exactly, downscale my video?

4 Reply by Drakko01 (edited by Drakko01 06-05-2024 02:08:58)

  • Thanks: 49

Re: Slow video / audio of sync, RIFE AI.

aliyansaleem2013 wrote:

Hello,

I am new to motion interpolation and this software.

I just want to watch anime and some movies at 60 fps.

i setup tensorRT and watched a few movies, it worked flawlessly.

However, now I am watching an anime and the audio is completely out of sync. Audio comes first and the scene happens after.

My Specs are:
5600X
32GB
3060

Rife Options
Threads: 2
Model: 4.6
Performance Boost: Enabled

Scene Change Threshold: Custom:36, I tried all different values. 10, 15, 20 etc
and Do not remove duplicate frames.

Some more information
- What video player do you use?
- What is the frequency of your monitor?
-What is the frame rate of the video?
-Frame rate variable?

Re: Slow video / audio of sync, RIFE AI.

Drakko01 wrote:

- What video player do you use?
- What is the frequency of your monitor?
-What is the frame rate of the video?
-Frame rate variable?

1)MPC-HC

2)75hz

3)
Information about the video:
Format: HEVC[x265] (Matroska) (Overall bitrate - S01: 1 488 kb/s |S02: 1 407 kb/s |S03: 1 405 kb/s |S04: 1 304 kb/s)
Resolution: 1920x1080 pixel
Frame Rate: 23.976 FPS
Bit Depth: 10 bits

Audio 1: OPUS (Sampling rate: 48.0 kHz|Bitrate: S01: 107 kb/s |S02: 109 kb/s |S03: 112 kb/s |S04: 110 kb/s)
Channel(s) : 2 channels
Language: English

4)Ive set it to fixed 60 fps in SVP.

  • Thanks: 99
  • Thanks for the post: 1

Re: Slow video / audio of sync, RIFE AI.

aliyansaleem2013 wrote:

Also what does resize do exactly, downscale my video?
4)Ive set it to fixed 60 fps in SVP.

Yes, if that didn't fix it, it means you forgot to add the minus in front of the value.
Go to Application Settings, search resize and add:
-19201080
or
-12800720
or
-9600540

In some scenarios using fixed fps will make it go slower, so make sure you uncheck "Force the exact value"
For example setting a 240 fixed fps on a 23.976 video will make it generate 239.76 + 23.976 frames frames instead of rounding itself down to only 239.76 frames

Maybe try to select madVR in MPC-HC options and see if issue persists?
https://gyazo.com/509b01a01eb6dd413667b4dd3f6d819a

Don't think it's caused by a badly encoded HEVC[x265] file. Try to download other versions of the same video encoded in [x264] and compare.

7 Reply by Drakko01 (edited by Drakko01 06-05-2024 21:47:56)

  • Thanks: 49
  • Thanks for the post: 1

Re: Slow video / audio of sync, RIFE AI.

aliyansaleem2013 wrote:

1)MPC-HC

2)75hz

3)
Information about the video:
Format: HEVC[x265] (Matroska) (Overall bitrate - S01: 1 488 kb/s |S02: 1 407 kb/s |S03: 1 405 kb/s |S04: 1 304 kb/s)
Resolution: 1920x1080 pixel
Frame Rate: 23.976 FPS
Bit Depth: 10 bits

Audio 1: OPUS (Sampling rate: 48.0 kHz|Bitrate: S01: 107 kb/s |S02: 109 kb/s |S03: 112 kb/s |S04: 110 kb/s)
Channel(s) : 2 channels
Language: English

4)Ive set it to fixed 60 fps in SVP.

-Try Resize like flowreen91 said
-Try to set X2 o x3 instead fixed
-If you use Madvr try set default settings
-Ctrl J shows your stats , see if your playback drops frames
- Try other models V2 its an option
- This also happens to you with MPV?

8 Reply by aliyansaleem2013 (edited by aliyansaleem2013 07-05-2024 01:26:57)

Re: Slow video / audio of sync, RIFE AI.

flowreen91 wrote:

In some scenarios using fixed fps will make it go slower, so make sure you uncheck "Force the exact value"
For example setting a 240 fixed fps on a 23.976 video will make it generate 239.76 + 23.976 frames frames instead of rounding itself down to only 239.76 frames

That's it! I got another issue now, there are black bars, top and bottom, How to get rid of them? Detect and remove black bars is checked.

I did add a hyphen in resize, that wasn't the issue, the issue was what you said above, using 2x and 3x, fixed it for me.

Also what does resize actually do? downscale the video? or something else? because It looked the same to me on 720 and 1080 after resizing.

my settings:https://imgur.com/a/OkRzbQ8

9 Reply by Drakko01 (edited by Drakko01 07-05-2024 02:04:48)

  • Thanks: 49
  • Thanks for the post: 1

Re: Slow video / audio of sync, RIFE AI.

-Select to Screen in the options of the image

-Resize the video, that way the interpolation is less demanding.

-You always get an upscalling via madvr or spv, that is why the video is not that different, this being a very good option for many,a good trade off, interpolation and performance.

Re: Slow video / audio of sync, RIFE AI.

Drakko01 wrote:

-Select to Screen in the options of the image


Still there are borders, top and bottom, weren't there before(in the same video), don't know what caused this.
https://imgur.com/a/4izwa9m

  • Thanks: 99
  • Thanks for the post: 1

Re: Slow video / audio of sync, RIFE AI.

aliyansaleem2013 wrote:

Still there are borders, top and bottom, weren't there before(in the same video), don't know what caused this.

Video Frame -> Stretch To Window fixes it?
https://gyazo.com/24107703db5dec46e0f9f49d95274878

  • Thanks: 49
  • Thanks for the post: 1

Re: Slow video / audio of sync, RIFE AI.

flowreen91 wrote:
aliyansaleem2013 wrote:

Still there are borders, top and bottom, weren't there before(in the same video), don't know what caused this.

Video Frame -> Stretch To Window fixes it?
https://gyazo.com/24107703db5dec46e0f9f49d95274878

I preferred the option touch window from inside /outside, try all an see which it's best for you

Re: Slow video / audio of sync, RIFE AI.

Thank you guys, much love.