1 Topic by smooth-motion (edited by smooth-motion 05-07-2018 14:36:36)

  • Thanks: 1

Topic: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

1.)Clone the latest git of mpv, which will contain build scripts for libass and ffmpeg.
    Follow the instructions on this website, regarding the necessary dependencies to compile mpv.
https://github.com/mpv-player/mpv-build

2.)In order for nvenc to be supported, the option must be enabled in ffmpeg.
   cd into /mpv-build and execute the following commands. (The first three are mandatory for encoding to work in general)

    echo --enable-libx264 >> ffmpeg_options
    echo --enable-libmp3lame >> ffmpeg_options
    echo --enable-libfdk-aac >> ffmpeg_options
    echo --enable-nvenc >> ffmpeg_options
    echo --enable-cuvid >> ffmpeg_options

3.)You must clone the nvenc headers git and make/install

git clone https://git.videolan.org/git/ffmpeg/nv-codec-headers.git
make
sudo make install

4.) Install libvapoursynth-dev from the djcj/vapoursynth repository
https://launchpad.net/~djcj/+archive/ubuntu/vapoursynth

5.)Add vapoursynth support by executing the following:

     echo --enable-vapoursynth > mpv_options

6.)Execute the following:

./rebuild -j4

Don't panic, the script might fail because you forgot to install a dependency.
This output will pop up, detailing the options/packages that have been enabled/disabled or not found
Should an essential package not have been found, check which one and it's version number, install it and run ./rebuild -j4 again.

Checking for compiler flags -Werror=implicit-function-declaration : yes 
Checking for compiler flags -Wno-error=deprecated-declarations    : yes 
Checking for compiler flags -Wno-error=unused-function            : yes 
Checking for compiler flags -Wempty-body                          : yes 
Checking for compiler flags -Wdisabled-optimization               : yes 
Checking for compiler flags -Wstrict-prototypes                   : yes 
Checking for compiler flags -Wno-format-zero-length               : yes 
Checking for compiler flags -Werror=format-security               : yes 
Checking for compiler flags -Wno-redundant-decls                  : yes 
Checking for compiler flags -Wvla                                 : yes 
Checking for LGPL (version 2.1 or later) build                    : disabled 
Checking for GPL (version 2 or later) build                       : yes 
Checking for internal audio filter chain                          : yes 
Checking for mpv CLI player                                       : yes 
Checking for shared library                                       : disabled 
Checking for static library                                       : disabled 
Checking for static build                                         : disabled 
Checking for whether to include binary compile time               : yes 
Checking for whether to optimize                                  : yes 
Checking for whether to compile-in debugging information          : yes 
Checking for manpage generation                                   : no (missing RST2MAN) 
Checking for html manual generation                               : disabled 
Checking for pdf manual generation                                : disabled 
Checking for dynamic loader                                       : yes 
Checking for C plugins                                            : yes 
Checking for zsh completion                                       : disabled 
Checking for inline assembly (currently without effect)           : yes 
Checking for test suite (using cmocka)                            : disabled 
Checking for generate a clang compilation database                : disabled 
Checking for compiler support for noexecstack                     : yes 
Checking for linker support for --nxcompat --no-seh --dynamicbase : no 
Checking for -lm                                                  : yes 
Checking for MinGW                                                : os-win32 not found 
Checking for POSIX environment                                    : yes 
Checking for Android environment                                  : no 
Checking for development environment                              : yes 
Checking for Universal Windows Platform                           : disabled 
Checking for win32 desktop APIs                                   : os-win32 not found 
Checking for internal pthread wrapper for win32 (Vista+)          : posix found 
Checking for POSIX threads                                        : yes 
Checking for GNU C extensions                                     : yes 
Checking for stdatomic.h                                          : yes 
Checking for stdatomic.h support or slow emulation                : yes 
Checking for linking with -lrt                                    : yes 
Checking for iconv                                                : yes 
Checking for w32/dos paths                                        : os-win32 not found 
Checking for spawnp()/kill() POSIX support                        : yes 
Checking for spawnp()/kill() Android replacement                  : posix-spawn-native found 
Checking for any spawnp()/kill() support                          : yes 
Checking for Windows pipe support                                 : posix found 
Checking for glob() POSIX support                                 : yes 
Checking for glob() win32 replacement                             : posix found 
Checking for any glob() support                                   : yes 
Checking for fchmod()                                             : yes 
Checking for vt.h                                                 : yes 
Checking for gbm.h                                                : no 
Checking for GLIBC API for setting thread name                    : yes 
Checking for OSX API for setting thread name                      : glibc-thread-name found 
Checking for BSD API for setting thread name                      : glibc-thread-name found 
Checking for BSD's fstatfs()                                      : no 
Checking for Linux's fstatfs()                                    : yes 
Checking for Samba support (makes mpv GPLv3)                      : disabled 
Checking for Lua                                                  : no ('luajit >= 2.0.0' not found) 
Checking for Javascript (MuJS backend)                            : no ('mujs >= 1.0.0' not found) 
Checking for SSA/ASS support                                      : yes 
Checking for libass OSD support                                   : yes 
Checking for dummy OSD support                                    : libass-osd found 
Checking for zlib                                                 : yes 
Checking for Bluray support                                       : no ('libbluray >= 0.3.0' not found) 
Checking for dvdread support                                      : disabled 
Checking for dvdnav support                                       : disabled 
Checking for DVD/IFO support                                      : dvdread not found 
Checking for cdda support (libcdio)                               : disabled 
Checking for uchardet support                                     : no ('uchardet' not found) 
Checking for librubberband support                                : no ('rubberband >= 1.8.0' not found) 
Checking for LCMS2 support                                        : no ('lcms2 >= 2.6' not found) 
Checking for VapourSynth filter bridge (Python)                   : yes 
Checking for VapourSynth filter bridge (Lazy Lua)                 : lua not found 
Checking for VapourSynth filter bridge (core)                     : yes 
Checking for libarchive wrapper for reading zip files and more    : no ('libarchive >= 3.0.0' not found) 
Checking for SDL2                                                 : disabled 
Checking for OSS                                                  : yes 
Checking for RSound audio output                                  : no 
Checking for sndio audio input/output                             : disabled 
Checking for PulseAudio audio output                              : no ('libpulse >= 1.0' not found) 
Checking for JACK audio output                                    : no ('jack' not found) 
Checking for OpenAL audio output                                  : disabled 
Checking for OpenSL ES audio output                               : no 
Checking for ALSA audio output                                    : no ('alsa >= 1.0.18' not found) 
Checking for CoreAudio audio output                               : no 
Checking for AudioUnit output for iOS                             : no 
Checking for WASAPI audio output                                  : os-win32 not found 
Checking for Cocoa                                                : no 
Checking for DRM                                                  : yes 
Checking for DRM Prime ffmpeg support                             : yes 
Checking for GBM                                                  : gbm.h not found 
Checking for wayland-scanner                                      : yes 
Checking for wayland-protocols                                    : no ('wayland-protocols >= 1.14' not found) 
Checking for Wayland                                              : wayland-protocols not found 
Checking for X11                                                  : no ('x11 >= 1.0.0 xscrnsaver >= 1.0.0 xext >= 1.0.0 xinerama >= 1.0.0 xrandr >= 1.2.0' not found) 
Checking for Xv video output                                      : x11 not found 
Checking for OpenGL Cocoa Backend                                 : cocoa not found 
Checking for OpenGL X11 Backend                                   : x11 not found 
Checking for OpenGL X11 EGL Backend                               : x11 not found 
Checking for OpenGL DRM EGL Backend                               : gbm not found 
Checking for OpenGL Wayland Backend                               : wayland not found 
Checking for OpenGL Win32 Backend                                 : win32-desktop not found 
Checking for OpenGL/DirectX Interop Backend                       : gl-win32 not found 
Checking for OpenGL ANGLE headers                                 : os-win32 not found 
Checking for OpenGL Win32 ANGLE Library                           : egl-angle not found 
Checking for OpenGL Win32 ANGLE Backend                           : egl-angle not found 
Checking for VDPAU acceleration                                   : x11 not found 
Checking for VDPAU with OpenGL/X11                                : vdpau not found 
Checking for VAAPI acceleration                                   : x11 not found 
Checking for VAAPI (X11 support)                                  : vaapi not found 
Checking for VAAPI (Wayland support)                              : vaapi not found 
Checking for VAAPI (DRM/EGL support)                              : vaapi not found 
Checking for VAAPI GLX                                            : vaapi-x11 not found 
Checking for VAAPI EGL on X11                                     : vaapi-x11 not found 
Checking for VAAPI EGL                                            : vaapi-x-egl not found 
Checking for CACA                                                 : no ('caca >= 0.99.beta18' not found) 
Checking for JPEG support                                         : no 
Checking for Direct3D support                                     : win32-desktop not found 
Checking for libshaderc SPIR-V compiler (shared library)          : no 
Checking for libshaderc SPIR-V compiler (static library)          : no 
Checking for libshaderc SPIR-V compiler                           : shaderc-shared not found 
Checking for libcrossc SPIR-V translator                          : no ('crossc' not found) 
Checking for Direct3D 11 video output                             : win32-desktop not found 
Checking for Raspberry Pi support                                 : no 
Checking for iOS OpenGL ES hardware decoding interop support      : no 
Checking for OpenGL without platform-specific code (e.g. for libmpv) : libmpv-shared not found 
Checking for MALI via Linux fbdev                                    : no 
Checking for OpenGL context support                                  : gl-cocoa not found

7.) Once you have successfully compiled mpv, simply execute the following in order to install it locally, which is not necessary as you can simply copy /mpv-build/build/mpv into /usr/local/bin

sudo ./install

8.) Finally, the SVP command  for transcoding with NVENC will cause an error as it uses the option --ofps which has apparently been deprecated with the latest git of mpv.
I have been using this with a script to launch it for now.

mpv /home/donald/Downloads/sintel.mkv -o output --no-audio --no-sub --no-sub-auto --input-ipc-server=mpvencodepipe --vf=vapoursynth:/home/donald/60fps.vpy:4:16 --of=matroska --ovc=h264_nvenc --ovcopts=b=11055051,rc=vbr_hq,preset=slow,profile=high,threads=16
Playing: /home/donald/Downloads/sintel.mkv
 (+) Video --vid=1 (h264 1920x818 24.000fps)
 Audio --aid=1 --alang=eng 'AC3 5.1 @ 640 Kbps' (ac3 6ch 48000Hz)
VO: [lavc] 1920x818 yuv420p
[vo/lavc] Opening encoder: NVIDIA NVENC H.264 encoder [h264_nvenc]
[encode] Opening output file: output
[encode] Opening muxer: Matroska [matroska]
V: 00:04:36 / 00:14:48 (31%) {5.5min 111.6fps 2349.6MB}
V: 00:04:42 / 00:14:48 (31%) {5.4min 111.6fps 2354.5MB}
V: 00:04:55 / 00:14:48 (33%) {5.3min 111.3fps 2370.3MB}

Cheers!

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

> Finally, the SVP command  for transcoding with NVENC will cause an error as it uses the option --ofps which has apparently been deprecated with the latest git of mpv.

Just use mpv v0.28.2 git tag for now.

3 Reply by smooth-motion (edited by smooth-motion 06-07-2018 14:41:47)

  • Thanks: 1

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

Hey Chainik, I have some more details to report.

Transcoding works fine, with about 40% - 50% Video Engine Utilization and 80% - 90% CPU usage.
My goal now is to increase Video Engine Utilization to the maximum, while retaining quality.

Without using SVP Panel to modify the interpolation options, I have been editing ffff.py and override.js to change interpolation parameters in order to speed up transcoding.

With the following values I can achieve 60% Video Engine Utilization and an average of 174FPS during transcoding.

super_params     = "{scale:{up:0},gpu:1,rc:true}"
analyse_params   = "{block:{w:8,overlap:0},main:{search:{coarse:{distance:-12,bad:{sad:2000}},type:2}}}"
smoothfps_params = "{gpuid:11,rate:{num:5,den:2},algo:13,scene:{blend:true,mode:0}}"
 [vo/lavc] Opening encoder: NVIDIA NVENC H.264 encoder [h264_nvenc]
[encode] Opening output file: /home/donald/Downloads/default
[encode] Opening muxer: Matroska [matroska]
V: 00:00:27 / 00:02:06 (21%) {0.6min 172.2fps 334.6MB}

When using the command without --vf option, Utilization shoots up to 100% with about 230FPS, so there must still be room for improvement.

[vo/lavc] Opening encoder: NVIDIA NVENC H.264 encoder [h264_nvenc]
[encode] Opening output file: /home/donald/Downloads/default
[encode] Opening muxer: Matroska [matroska]
V: 00:02:06 / 00:02:06 (99%) {0.0min 235.2fps 131.9MB}

Do you have any suggestions on how to accomplish even more Engine Utilization by editing certain parameters in the various scripts?

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

It's quite obvious that the lower are SVP interpolation settings - the higher is transcoding speed.

  • Thanks: 1

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

Sure, but the primary focus is more GPU Utilization.
I have read the documentations and gathered all the info I could on the forum, regarding the script parameters in question, which led me to achieve 60% Utilization at max.

How to achieve 90% - 100% Video Engine Utilization?

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

faster encoding -> more GPU utilization

if your system (i.e. CPU) can only make interpolated video stream at 174 fps then it will be encoded exactly at 174 fps
if you lower SVP's settings and your CPU will be able to do interpolation at, let's say, 220 fps, then encoding will be at 220 fps and will use more of the "video engine"

  • Thanks: 1

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

You are absolutely right, sorry for picking up slow, I am still learning as I go about.

I really love this stuff though, is there anything I can do on the Linux end with/for SVP?
Not an expert, but I am trying, haha.

8 Reply by dlr5668 (edited by dlr5668 07-07-2018 09:56:12)

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

buy better Ryzen7

9 Reply by smooth-motion (edited by smooth-motion 08-07-2018 16:45:26)

  • Thanks: 1

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

Chainik wrote:

faster encoding -> more GPU utilization

Hey man, I wonder if your superman skills have met their kryptonite with this one, because I have lost years of my life over it.

Let's say you take 5 separate video clips with a 1080p resolution and framerate of 24FPS, put them together in a video editor and render them with a 1080p60FPS project profile.
The resulting output file will report 60FPS even though the clips are still 24FPS, due to a constant framerate forcibly having to be set up.

Now when one tries to interpolate/transcode this file with SVP, the interpolation/transcoding will fail as the script reads the reported 60FPS and exits.

Is there a way to make SVP recognize the parts with 24FPS material, even though the file reports 60FPS, or to commence interpolation of the entire file, regardless of the reported 60FPS?

Changing the "else" argument that decides whether or not to continue when the source is >59.9 doesn't work.

(The reason why the project profile has to be 60FPS is because of effects/transitions need to be rendered at 60FPS) (Rendering them also at 24FPS and having SVP interpolate the entire thing doesn't work, because there is text involved and it gets messed up, no matter how much artifact masking etc)

If you can solve this, then I would like to buy some shares of SVP and name my first born son Chainik.

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

why you can't just take those 5 clips, interpolate them separately and then put 60-fps clips to the video editor?

> and name my first born son Chainik

you don't want this, believe me! big_smile

11 Reply by smooth-motion (edited by smooth-motion 09-07-2018 18:32:35)

  • Thanks: 1

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

Ok, let me explain a little better.

I have a Youtube Channel where I upload the best sports highlights of the week. Basically, I take a bunch of clips, append them, add transitions, some music and do some minor editing.
Interpolating each clip individually would be a real pain, even though it might be possible with a script that would perform the mpv transcode command with every single clip in a folder......huh could that work actually?
Something like mpv *.mp4 or from a subdir, I guess it would fail with the output file, unless you can specify a static prefix ---> I will check the mpv documentation.

Anyways, I have to render the project in 1080p60FPS because the transitions/effects look choppy otherwise
But as I mentioned, the SVP script will not engage since  the source frame rate reports >59.9FPS
Now I have tried to enable the option of a variable framerate, even recompiled kdenlive from scratch, looking for any ever so tiny hint of that f*cking functionality, but nope. Only available in Adobe Premiere Pro, but the day I use Windows I must be in a hostage situation.

Would SVP even be able to recognize the portions with 24FPS in a file with a variable framerate?
I always thought that the script  inspects and intuitively interpolates scenes with motion, so basically editing the >59.9 FPS threshhold condition should have solved the problem, but I was wrong, apparently.

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

Last time I checked, SVPcode UI accepted several files for batch transcoding... Just switch to mpv 0.28 (not git-master).

> Would SVP even be able to recognize the portions with 24FPS in a file with a variable framerate?

Nope.

  • Thanks: 1

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

Yes, it works with SVP, just checked, but I had to add a second script which halts by n-2, renames the last output and recommences script 1 with a for loop.

I am now trying to do it cleaner, with a single bash script.
mpv allows multiple instances, I am now figuring out how to designate -o with a static pointer for each filename + an extra string like "1" or "a"  descending.

  • Thanks: 1
  • Thanks for the post: 1

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

Just wanted to share my script to execute the transcoding command against any file located in the specified working directory --> OP updated.
Note: You have to change the container so that it differs from the source, or choose a different directory for the output.

#!/bin/bash
for f in /home/donald/clips/*$f; do mpv "$f" -o "$f.mkv" --ovc=h264_nvenc  --vf=vapoursynth:/home/donald/60fps.vpy:4:16 --ovcopts=b=55055051,rc=vbr_hq,preset=slow,profile=high,threads=16; done

15 Reply by smooth-motion (edited by smooth-motion 29-07-2018 14:19:18)

  • Thanks: 1

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

Hey Chainik, I have a quick question about SVP Code.

I would like to edit the the mpv command SVPCode uses in order to remove "--ofps=60", as it is deprecated and I can't use the lower git version as I have embedded the newer release into a remastersys iso which already shipped out.

The SVP Control Panel Settings do not seem to offer this specific option and I couldn't find anything in the scripts.
Now I tried to negate --ofps by adding the "--null" in the mpv_args section, which should negate the mpv option which has been designated, but I get an error.

Is there any other way to accomplish this?

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

as mpv 0.29 is finally released, we have to update SVPcode/cast

  • Thanks: 1

Re: OFFICIAL Guide For SVP/NVENC Transcoding On Ubuntu 16.04 - 18.04

Cool!
Where exactly is the mpv command for SVPCode generated?
I found some stuff regarding bitrates etc. in the settings manager but no --ofps
Is that part of the code generally closed, or am I just not finding it in the scrtips