1 (edited by PsykoTenshi 01-01-2013 17:29:35)

Topic: High memory usage with dynamic subtitles

Originally posted to report memory usage peaking, dynamic subtitles confirmed as the cause.
This bug seems to have been fixed on SVP 3.1.3! big_smile

==============
Original Post below
==============
But first, bear with me while I attempt to put in words how grateful I am for this stunningly amazing piece of software. If you ask me, SVP is a prime example of hardworking independent developers pulling out freeware with capabilities commercial software can't even dream of.
Why such a difference? Because software such as SVP is intended primarily to work, not to sell. And that's why I love the thing. It's efficient, unlike most of everything you see out there.
What really moves me is all the effort the SVP developers put into it. I'm not exactly trained on coding, I can barely do some kiddie-level scripting, but just using the UI, reading the documentation and logs, and most importantly: The results. Steady 60 fps on 720p and just 30% cpu load?! YES PLEASE.

Of course, no software is perfect. Yesterday, I got a warning sign while watching an anime episode. I didn't take note of what it said, mostly because of the shock I think, it was the first unexpected issue I ever had with SVP after all.
Fortunately the same message is on the attached errorlog, apparently it said this: "Error: Warning: there's a memory leak in the video player: 1170 MB
To prevent the closing of the player due to lack of memory a script restarted improve smoothness."
Gotta love "good guy SVP" xD
"Detects memory leak. Fixes, warns user and doesn't even break playback."

My 1280x720@24 profile is as follows:
Frames interpolation mode: "2m" (min artifacts)
SVP shader: 2. Sharp (for anime)
Motion vectors interpolation mode (grayed out for some reason): Bilinear
Target frame rate: To screen refresh rate (default)
Motion vectors grid: 32 px. Large 0
Decrease grid step: By two with global refinement
Search radius: Small and fast
Motion vectors precision: Two pixels
Wide search: Disable
Artifacts masking: Strongest
Processing of scene changes: Repeat frame (default)
Decrease frame size: Disable (default)
btw, I've extensively tested these settings and I dare to say it works about perfectly with not just anime, but with live action videos as well.

My system specs:
AMD Phenom II 955 (x4 @ 3.2 Ghz)
Gigabyte 870A-USB3
Sapphire Radeon HD 4890 Vapor-X
2x G-Skill F3-12800CL8-2GBTD (2x2Gb 1333Mhz)

Additionally, the video-related MediaInfo on the anime episode I was watching during the memory leak. Not sure if it'll be of any help but, best to be sure.
[container and general information]
Writing application: mkvmerge v5.0.1 ('Es ist Sommer') built on Oct 9 2011 11:55:43
Writing library: libebml v1.2.2 + libmatroska 1.3.0
[first video stream]
ID: 1
Format: AVC
Format/Info: Advanced Video Codec
Format profile: High 10@L5.0
Format settings, CABAC : Yes
Format settings, ReFrames: 16 frames
Muxing mode: Header stripping
Codec ID: V_MPEG4/ISO/AVC
Duration: 23mn 39s
Width: 1 280 pixels
Height: 720 pixels
Display aspect ratio : 16:9
Frame rate mode: Constant
Frame rate: 23.976 fps
Color space: YUV
Chroma subsampling : 4:2:0
Bit depth: 10 bits
Scan type: Progressive
Title: H.264
Writing library: x264 core 125 r2200+677M 3a754e7
Encoding settings: cabac=1 / ref=16 / deblock=1:1:1 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / fade_compensate=0.80 / psy_rd=0.92:0.02 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-3 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / fgo=0 / bframes=10 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=17.2000 / qcomp=0.60 / qpmin=0 / qpmax=81 / qpstep=4 / ip_ratio=1.40 / aq=3:0.92
Default: Yes
Forced: No
Matrix coefficients: BT.709

Apologies for the gigantic post, I didn't find a "compacting spoiler tag" so I couldn't do much about the wall of text.

Post's attachments

LastErrorLog.txt 28.5 kb, 692 downloads since 2012-12-10 

Re: High memory usage with dynamic subtitles

PsykoTenshi
Thank you for detailed description.

memory leak in the video player: 1170 MB

It is high value of memory use at your settings.

From log I see it took 17 minutes from start playing after rewind to used memory grown to 1170 MB:

10:36:19.085; ===== rewind backward 00:05:24 / 00:23:38
10:53:44.726; Error: Warning: there's a memory leak in the video player: 1170 MB

I met used memory growth when used wrong subtitle filter.
1. What filter are you using for subtitles?
2. Show please used memory graph from SVP performance graphs window while error appears again?

Re: High memory usage with dynamic subtitles

Or you can use this modification of SVP Manager. It contains details in log file about memory leaks.

Show your log file after error occurs again.

Download: SVPMgr_MemoryLeaksDetails_mod.zip

1. Exit SVP Manager
2. Unpack to SVP folder with files replacing
3. Run SVP Manager

Post's attachments

SVPMgr_MemoryLeaksDetails_mod.zip 374.86 kb, 594 downloads since 2012-12-10 

4 (edited by PsykoTenshi 11-12-2012 10:29:45)

Re: High memory usage with dynamic subtitles

MAG79 wrote:

What filter are you using for subtitles?

Well, this is embarrassing. I have no idea. Overall, video decoding isn't my specialty, I usually fiddle with player settings until I get it working as I want to.

If I understood Media Player Classic's tooltip's correctly, I'm using its internal subtitle renderer, and EVR Custom Pres. for Direct Show video.
I think however, that the MemoryLeaksDetails mod.zip fixed the issue, as I played both the proportionally biggest video I have (1.22Gb for 24 min! 1080p + 3 audio streams, 2 of them in flac lol) and a 51 min 720p video with no issues. Both had ASS (talk about an unlucky acronym) lossless compression subtitles.
Memory usage for Media Player Classic didn't even reach 600 Mb when playing the 1080p video @ 60fps.

P.S.: I should note that I've set all 4 profiles to the same settings.

Re: High memory usage with dynamic subtitles

PsykoTenshi
I think however, that the MemoryLeaksDetails mod.zip fixed the issue
No. It contain more detailed log when leak occurs. To easy understand what happend smile

6 (edited by PsykoTenshi 16-12-2012 23:03:26)

Re: High memory usage with dynamic subtitles

Well, just for the sake of science, I watched another episode of that particular anime where I experienced the "memory leak" (if it's fit to call it so).
I got a sign at the same spot where I got the memory leak ones before, I took a screenshot of it but I was so dumb of copying some text before pasting it on an image editor.
If my memory serves me right, it was something about an error when generating an errorlog (oh, the irony).
Afterwards I noticed the ending on that anime is rather merciless on motion interpolation. Here's a link to said ED (Youtube link). Up to 0:57, it's like it was designed to not be "SVP'd". They'll have to try harder >:3
SVP takes it like a boss and generates not only the frames needed for 60 fps but also no artifacts cool (well, no artifacts that I could see)
Then I did a quick test on that ending and watched it (on 720p) with the performance graphs running.
http://imageshack.us/a/img542/1307/performancegraphsanimee.png
Then, I did the same but on fullscreen (I have a 1080p display). That's when MPC's memory usage hit over 1100Mb, and in less than 1 minute! It peaks specifically on the part with a girl with a black dress. It's like a crazy motion interpolation test  hmm 
http://imageshack.us/a/img254/1307/performancegraphsanimee.png
The 400-something Mb is me being a fool and not noticing MPC continues to use memory while on pause.
I think I should note that while I wasn't exactly noticeable due to a lack of a clear movement pattern, the first half of the video had very unstable framerates (even below 24) according to Fraps.

Re: High memory usage with dynamic subtitles

PsykoTenshi
I have never seen such strange memory usage line. smile
Can you share this file to me or part of it what shows such behavior?
I will take a look

Re: High memory usage with dynamic subtitles

After some testing, I have determined it's the particular encoding of some particular episodes, plus my 1.5Tb Seagate hard drive (ST31500341AS) acting up as I moved the videos to my other drive, a 640Gb WD (WD6400AAKS-00E4A0) and things performed slightly... less bad.

Putting it on clearer terms: I have this video sequence https://www.youtube.com/watch?v=tUS7gSv2ZxE
On "encoding A" and "encoding B".
Encoding A performs excellently on both hard drives.
Encoding B has very poor performance on Seagate hard drive, slightly less bad on WD hard drive.

My internet connection isn not exactly fast for uploads, so uploading a ~300Mb video file for the final 1:30 minutes isn not exactly profitable, and if I re-encoded the troublesome bit and sent it I would lose the particular encoding that triggers this issue.
As I obtained the video file on a "t0rr3nt" site, I'm not entirely sure if I'm allowed to post a such a link here, in addition to the fact I have yet to find a "Forum Rules" section xP
I would PM you but I have found no way of doing so either so... what to do?

I will try to upload the video later today, though.

Re: High memory usage with dynamic subtitles

PsykoTenshi
I downloaded 720p MP4 version from YouTube by your link.
It played to me nice. With good memory use line.

http://www.svp-team.com/forum/misc.php?item=2056

You are right. To reproduce memory use bug I need right this "problem" file what you have.

Post's attachments

ED_SVP-performance.png, 5.23 kb, 448 x 233
ED_SVP-performance.png 5.23 kb, 596 downloads since 2012-12-13 

Re: High memory usage with dynamic subtitles

I just went ahead and uploaded it while I was out for a while.
http://filesmelt.com/dl/Motion_interpolation_test.mkv
It's a whole episode. I didn't check closely but apparently the whole thing plays ok, with the exception of the ending credits (starting around 22:20) that refuse to play at 60 fps.

Re: High memory usage with dynamic subtitles

PsykoTenshi
Thank you. I saw the same picture with "hill" at memory use graphic. It is dynamic subtitles. It was not enough time to make some tests with it.
I will compare different subtitle filters and renderers to find better.

12 (edited by PsykoTenshi 16-12-2012 23:02:25)

Re: High memory usage with dynamic subtitles

Thanks smile
For some reason I didn't even consider subtitles as a cause for the problem, so I didn't disable them to confirm. I will edit the first post title to more accurately tell what causes this issue.

Edit:
For science! I tried changing the renderer in MPC output settings as well as subtitles settings. I got some interesting results.
I quickly found out that I could get stable framerate (almost, it dropped to 59, 58 fps a couple times) on these troublesome dynamic subtitles by using these settings:
http://imageshack.us/a/img823/1244/subtitlesettings1.png

Instead of the default ones:
http://imageshack.us/a/img577/2819/subtitlesettings2.png

The texture size at 720p did not represent a noticeable loss on fullscreen (and I have a 1080p display).
I could not find a fix to the high memory usage, but I did notice that different output renderers had slightly different memory performance. I did not make enough runs to call this a truly reliable "scientific" test, but (always on a 720p window):
-VMR-9 (renderless) peaks to about 900 Mb.
-EVR Custom Presenter peaks barely over 900 Mb, and not always gets over 900.
-Haali has the lowest memory usage of all, doesn't reach 600 Mb, but it also had flashing subtitles on my settings sad
-madVR seems to hit a "limit" at 910 Mb, but always reaches that level.

Something noteworthy: Disabling "Allow animation when buffering" on MPC Subtitles options allows to free up memory much quicker.
I recalled one of these had a higher memory usage so I played the ending with all renderers again but with subtitle textures set to 1920x1080. It would seem Haali uses a lot more memory (even when the window size was 1280x720) with higher subtitle textures. I tried getting to an optimal setting with the Haali renderer with no avail, as it requires "Allow animation when buffering" enabled so subtitles are constantly visible.

Using EVR Custom Presenter and the subtitle settings I posted before I got to this performance graph and a stable framerate all along (on a 1280x720 window). Maximum memory usage was 906 Mb.
http://img405.imageshack.us/img405/9008/performancegraphbetter.png

Yes, I played the same bit of video like 20 times in a row, that music is going to resonate in my head for days.

Re: High memory usage with dynamic subtitles

PsykoTenshi
I got to this performance graph and a stable framerate all along (on a 1280x720 window). Maximum memory usage was 906 Mb
It is good news! Congratulations!

I played the same bit of video like 20 times in a row, that music is going to resonate in my head for days.
Yeah smile I think soon I will can sing this song by heart big_smile

Re: High memory usage with dynamic subtitles

If you're not going to be rendering subtitles at 1920x1080, there's no point in using MPC-HC's internal subtitle renderer. xy-VSFilter is significantly faster and better.

15 (edited by %username% 17-12-2012 05:31:48)

Re: High memory usage with dynamic subtitles

Solinus
as practice shown xy-VSFilter bring some instability and memory overuses
FFDShow subtitle mechanism is normal, stability and as I know works fine with all type of subtitles