1 (edited by Kodiack 22-04-2015 11:43:19)

Topic: Running out of memory, even with "4 GB fix"/large address aware

Hi all,

I'm trying to use SVP to watch footage in 4K. Unfortunately, MPC-HC and SVP simply cannot address enough memory, even with the large address aware flag set.

https://i.imgur.com/z605Jaq.png

There's a common misconception that the LAA flag allows programs to use 4 GB of commit space, although this simply isn't true; it actually just tells programs that they are able to use more than the standard 2 GB of commit space.

Is there any way that I can use SVP with 4K, preferably without needing to turn settings to horribly low values?

Specs:
Core i7-5820K
16 GB DDR4 RAM
Radeon R9 290

Cheers,
Kodiack

Re: Running out of memory, even with "4 GB fix"/large address aware

Kodiack
Is there any way that I can use SVP with 4K

try latest Avisynth 2.6 MT build - http://forum.doom9.org/showthread.php?t=148782

also upcoming SVP update will have better compatibility with 4K

Re: Running out of memory, even with "4 GB fix"/large address aware

No dice. sad MPC-HC climbs to ~2.9-3.2 GB of commit size, and then crashes.

I'm thrilled to hear that an SVP update is going to have better 4K compatibility, though!

4 (edited by dlr5668 23-04-2015 19:26:13)

Re: Running out of memory, even with "4 GB fix"/large address aware

I tested last beta 02.04 and it works for me
http://puu.sh/hn6jb/cd458e609e.jpg
However my amd fx 6300 @ 4400 Mhz cant process it. Need +10-20% power.
Src file is SONY F55 4K Blue Shanghai Zhen Wang Wei 150mbps.
I will retest it without madvr after cinema

new marvel movie HYPE

Edit: After reducing settings and switching to EVR:
http://puu.sh/hnpFt/901e0c90fe.jpg

Re: Running out of memory, even with "4 GB fix"/large address aware

4K content is somewhat stable with these settings, although my CPU's certainly got a lot of overhead left to improve image quality even further. This still isn't fully stable either; for most footage, it clocks in at just under 3 GB of commit space, and if it hits that magical 3 GB point, it will promptly crash. Really looking forward to the version of SVP that either lowers memory requirements or introduces 64-bit. smile

https://i.imgur.com/RSqecwG.png

Re: Running out of memory, even with "4 GB fix"/large address aware

set "motion vectors precision" to "two pixels" and "decrease grid step" to "disabled"and you'll save a lot of memory
also decreasing number of threads can help

although my CPU's certainly got a lot of overhead left

do you really have 4K screen? otherwise you're dealing with 4K source downsized to your screen wink

7 (edited by MistahBonzai 24-04-2015 22:24:56)

Re: Running out of memory, even with "4 GB fix"/large address aware

To satisfy my curiosity I conducted some comparisons with SVP running on AviSynth (AVS+ and AviSynth 2.6 MT). Using MPC-BE and MPC-HC on my Windows 7 64-bit OS.

  • The test video was the "SONY F55 4K Blue Shanghai Zhen Wang Wei 150mbps.mp4" file mentioned earlier.

    All tests were conducted using an AviSynth script with the SVP configuration embedded within that script. The SVP values were identical in all cases with the global resize_string="". The 4K video (3840x2160/25FPS) was the input to the SVP script. As I have a 1080P display (40" Sony Bravia) set at 60FPS the GPU downsized 3840x2160/60FPS output from SVP to 1920x1080/60FPS.

    The MPC-BE and HC video renderers were EVR-CP 8-bit Integer set to "Nearest Neighbor". I utilized that particular Video Rendering configuration because I quickly discovered that anything more would max my HD-7850. I believe this to be the case due to using "gpu:1" in the SVP values. I tried gpu:0 but it maxed my CPU (Intel i7-3770@3.4GHz).

    The same filters/latest revisions (LAv Video, LAV Splitter, ffdshow raw video, File source async and EVR-CP) were used in all cases.

    Global threads=8 with AviSynth 2.6MT. Prefetch(8) was used with AVS+. Note: As you know the # of 'threads' make a large difference with memory useage. I was a 'set threads to 15 and forget it' kinda guy with AvisYNTH 2.6MT but have now rethought that setting after running these tests.

    Both MPC.** executables were patched for "4K". I did note that I had to re patch them once when changing test configurations - more testing needed to see if that was a result of changing the AviSynth .DLLs. Speaking of which..I simply moved the appropriate AviSynth.dll to the requisite MPC-BE or MPC-HC folder along with changing the AviSynth script when reconfiguring them for AviSynth 2.6MT or AVS+.

Bottom Live:

All tests with MPC-BE and HC yielded similar GPU and CPU utilization (in my case ~50% GPU and 32% CPU).
MPC-BE consistently used ~ 20% more system memory with both versions of AviSynth.
MPC-BE was a bit smoother - less jitter (+/- ~1MS versus +/- ~2MS with MCC-HC)

Actual average memory utilization was:
MPC-BE ~1.84GB with AVS+ -- ~2.27GB with AviSynth 2.6MT
MPC-HC ~1.44GB with AVS+ -- ~1.82GB with AviSynth 2.6MT

In no cases did anything crash and system memory utilization would level out after 5-10 seconds and then remain fairly constant.

Edit: SetMemoryMax(1024). When using "SetMemoryMax(512) with either MPC-BE or HC it will crash.

Re: Running out of memory, even with "4 GB fix"/large address aware

MistahBonzai
EVR-CP is not good for 4k@60. Plain EVR works much better.

Re: Running out of memory, even with "4 GB fix"/large address aware

Chainik wrote:

MistahBonzai
EVR-CP is not good for 4k@60. Plain EVR works much better.

And so it does smile  Thanks.

Re: Running out of memory, even with "4 GB fix"/large address aware

another one - I suggest to set 'setmemorymax' to some value in 1500-2000 for 4K
pressing too hard on Avisynth memory dispatcher is not a good idea big_smile

11 (edited by Nintendo Maniac 64 24-04-2015 23:03:34)

Re: Running out of memory, even with "4 GB fix"/large address aware

MistahBonzai wrote:
Chainik wrote:

MistahBonzai
EVR-CP is not good for 4k@60. Plain EVR works much better.

And so it does smile  Thanks.

If it's anything like 1080p on old low-end Intel iGPUs, D3D Fullscreen may also work much better than EVR-CP.

Re: Running out of memory, even with "4 GB fix"/large address aware

There's still too much to make 4K video usable for any housewife.
And SVP is on the bleeding edge!  lol

Re: Running out of memory, even with "4 GB fix"/large address aware

Chainik wrote:

There's still too much to make 4K video usable for any housewife.
And SVP is on the bleeding edge!  lol

Using MPC-BE/AVS+ I tried your suggestion about D3D fullscreen. With EVR-CP I enabled D3D Fullscreen, 10-bit RGB output, 32-bit FP Surfaces and Michell-Netravali spline (PS 2.0). CPU unchanged at ~34%. GPU now ~67%. and I gotta tell ya..it's slick as snot on a door knob.

Not only that but being downsized from 4K to 2K on my 40" 1080P Sony Bravia at arms length (it's my monitor) it's looking good. This is starting to be fun  smile  However, I agree with Chainik, I doubt that the 'average' consumer would think as much.

Thanks Chainik and Nintendo - this little experiment has given me a lot of new stuff to contemplate.

Re: Running out of memory, even with "4 GB fix"/large address aware

If you're downscaling to 1080p, then make absolute sure you're getting 4:4:4 chroma since being able to get a full chroma resolution is one of the benefits of downsizing 4k video to 1080p.

Easiest way to do this is to open the following test image in MSpaint:
http://forum.doom9.org/showthread.php?p … ost1640299

Re: Running out of memory, even with "4 GB fix"/large address aware

My 40" Sony Bravia LCD 'monitor' (it has never displayed the likes of TV or movies) does 4:4:4 chroma (10-bit panel) and full range RGB. It has better white balance calibration controls than many workstation displays.

16 (edited by MistahBonzai 25-04-2015 05:06:43)

Re: Running out of memory, even with "4 GB fix"/large address aware

Something isn't playing nice... Swapped in the new svpflow1.dll and svpflow2.dll files.

  • Fired up SVP and configured a 3840x2160/25fps profile using the values I've been using right along while conducting my tests.

  • Rendered the 4K video file (it ran smoothly with ~25% CPU and GPU) and exported the SVP script (just to make sure that nothing had changed with the new SVPflow dll files).

  • Then copied the SVP script into the Avisynth scripts I've been using for AviSynth 2.6 MT and AVS+. Made sure the avisynth.dll files were in the proper folders and the SVP svpflow dlls were in the PluginPath.

Keep in mind I'm using the 4K file with out downsizing. Under AVS+ the CPU was ~65% and GPU ~60% (about what I have been seeing with the 'old' svpflow dll files after I jacked-up the SVP config and video rendering settings), System memory utilization was ~1.4GB. However I was dropping frames all over the place - it was a disaster.

Edit: Tried lowering Threads and Prefetch values from 8 to 6 but no difference with the performance - still dropping frames like crazy.

Swapping over to AviSynth 2.6 MT I experienced the same performance issues running both MPC-BE and MPC-HC as well.  Swapping back to the 'old' svpflow dll files and SVP script solved the issue.

Something ain't quite right. Got any ideas? I'll gladly supply system config details and script samples.

Re: Running out of memory, even with "4 GB fix"/large address aware

Fresh SVP 3.1.6 installation + "hotfix" patch.
4K musik video from YouTube, h.264 @24 fps
MPC-HC, EVR video renderer
Latest Avisynth 2.6 MT

"old" libs: 88% CPU load average
"new" libs: 69% CPU load average

No dropped frames in both cases.

For whatever reason any OSD message displayed by SVP drops performance by 2-3 times  hmm
So turn them off (SVP -> Settings -> Show OSD messages)
Looks like ffdshow issue, 2K (1440p) videos aren't affected.


I'll gladly supply system config details and script samples.

yes, please

Re: Running out of memory, even with "4 GB fix"/large address aware

Did you use AVS+ ffdshow-fix r1800? Please try. https://cloud.pados.hu/index.php/s/0e0d … 11731699b6

19 (edited by MistahBonzai 26-04-2015 20:08:38)

Re: Running out of memory, even with "4 GB fix"/large address aware

James D wrote:

Did you use AVS+ ffdshow-fix r1800? Please try. https://cloud.pados.hu/index.php/s/0e0d … 11731699b6

Yes, I have been using AviSynth+ (0.1 - r1800) with x86 along with the new SVP 3.1.6 installation and the SVP "hotfix" patch with all my testing.  I've been away this weekend but am now ready to give it another run through. I *think* I only have the dropping frames issue with the new svpfolw* dll files with rendering native (not downsized) 4K/25P file. The old svpflow* dll files ran the test file smooth as could be with latest MPC-BE/ffdshow/lavfilters and Avisynth/+ SetMemoryMax with 1024 (now using 1536) and EVR-CP 32-bit FPS, Mitchell-Netravali4, D3D Fullscreen. I could back-off on the configuration but it did run smooth with the old svpflow dlls.  CPU and GPU utilization looked file with <70% but the frame drops weren't random/occasional - they were 'gawd awful' sad

Edit: Keep in mind that I'm running the SVP script within the 'raw' AviSynth script and not as SVP standalone app - if that makes a difference.  That way I can override down scaling the video within SVP and pass it along to the GPU full size (3840x2160/60fps).

Re: Running out of memory, even with "4 GB fix"/large address aware

PROBLEM SOLVED: When reloading SVP 3.1.6 Core to get an old version of the svpflow dlls I overlooked reapplying the hotfix (I screwed-up).  All fixed now. Works very well - perhaps 'smoother' (less jitter) than with the old svpflow dlls with MPC-BE.  I'll perform some controlled testing and report back with CPU/GPU utilization values.

Re: Running out of memory, even with "4 GB fix"/large address aware

MistahBonzai
PROBLEM SOLVED: When reloading SVP 3.1.6 Core to get an old version of the svpflow dlls I overlooked reapplying the hotfix (I screwed-up).  All fixed now. Works very well - perhaps 'smoother' (less jitter) than with the old svpflow dlls with MPC-BE.  I'll perform some controlled testing and report back with CPU/GPU utilization values.
this probably doest matter but delete cpu gpu dlls from plugin folder

Re: Running out of memory, even with "4 GB fix"/large address aware

Again, keep in mind that I'm running the SVP script within the 'raw' AviSynth script and not as SVP standalone app.  That way I can override down scaling the video within SVP and pass it along to the GPU full size (3840x2160/60fps).

So, when using the new SVP script (extracted from SVP) I overlooked that there was no overlap: qualifier. After adding overlap:0 to global analyse_params the dropped frames issue cleared up.

What I did discover while comparing AviSynth 2.6 MT to AVS+ was that although the CPU utilization was similar, AVS+ didn't drop frames in the most difficult sections of the video. AviSynth 2.6 MT would drop frames for me when CPU utilization topped 63% while AVS+ would hit 70% in the same section but not drop the frames that AviSynth 2.6 MT did. Perhaps I could eliminate the difference by tweaking threads in AviSynth 2.6 MT... In any case AVS+ coupled with the new SVPflow DLLs works fine as far as I'm concerned.

Re: Running out of memory, even with "4 GB fix"/large address aware

MistahBonzai
That way I can override down scaling the video within SVP and pass it along to the GPU full size (3840x2160/60fps).

I don't get it. There's a built in option in SVP to manage downscaling.

Re: Running out of memory, even with "4 GB fix"/large address aware

Yes I can set decrease frame size to disabled. I could use some help finding where I can set overlap: to 0.

The reason I run SVP within my main AviSynth script is because I have other filters I use in conjunction with SVP. Also doing it that way allows me to balance CPU/GPU utilization. For instance when using MadVR I can run nnedi3 on the CPU thus freeing up the GPU for additional video processing. I hate to see either the CPU or GPU loafing along while the other is maxed out smile

Re: Running out of memory, even with "4 GB fix"/large address aware

was this ever fixed?