• Thanks: 1

Topic: 10-bit to 8-bit conversion colour banding

I've got some 10-bit stuff that I'm converting to 8-bit.  Mainly I'm just doing this because my HTPC is still on Win7, which doesn't properly support 10-bit playback.  I've noticed that the 8-bit colour conversion that SVP does during transcoding when 10-bit output is disabled is noticeably worse than what Handbrake manages to do when it converts to 8-bit colour.  There is a lot more banding and blockiness noticeable in the SVP conversion.  Might this be something simple to fix on SVP's side, or would it be complicated to fix?  Thanks.

  • Thanks: 1622
  • Thanks for the post: 1

Re: 10-bit to 8-bit conversion colour banding

https://mpv.io/manual/master/#options-deband

You may want put "--deband" into code.tools.mpv_args in All settings.

  • Thanks: 1

Re: 10-bit to 8-bit conversion colour banding

Thank you very much!  I'll give it a try, as well as read through that whole list as I find some time.  Appreciate it.

4 Reply by _Shorty (edited by _Shorty 19-12-2019 13:35:26)

  • Thanks: 1

Re: 10-bit to 8-bit conversion colour banding

Gave it a try on a small sample that I had noticed the issue on, and it doesn't really appear to be doing much.  Tried looking at the difference between the two SVP images in Photoshop with a difference blend, and the difference image was almost completely black, as in there was very little change, so perhaps the issue is not in the workflow position that --deband would affect things.  Here is a small portion of the sky in a particular frame that illustrates the banding issue.  If you mouse-over on the four different labels or press the corresponding number on your keyboard it will switch between those four samples.  One is the 10-bit original, one is the 8-bit conversion done by Handbrake, one is SVP 8-bit, and finally one is SVP 8-bit with --deband.  This is from the very first frame of the test sample, so the frame shouldn't be any different between each sample, other than colour conversion.

http://www.framecompare.com/image-compa … n/JMBMFNNU

edit:  Both Handbrake and SVP were using x264 medium CRF 18.

Re: 10-bit to 8-bit conversion colour banding

"allow 10-bit output" in SVP - on or off? try both values.

----
ahhh, this's stupid big_smile cause it will automatically select 10-bit profile for x264

  • Thanks: 1

Re: 10-bit to 8-bit conversion colour banding

I turned "Allow 10-bit output" in SVP off, as I thought that was the way to get it to convert the 10-bit source to 8-bit.

Re: 10-bit to 8-bit conversion colour banding

> Gave it a try on a small sample

could you share that small sample?

Re: 10-bit to 8-bit conversion colour banding

There're number of options to try.

1. Move 10-to-8-bits conversion to ffmpeg: turn ON "allow 10-bit output" and encode in "advanced mode":
- ovc: libx264
- ovcopts: preset=slow, crf=18, profile=high
- lavfi: ";format=pix_fmts=yuv420p" - w/o quotes, note ';' at the beginning!

  • Thanks: 1

Re: 10-bit to 8-bit conversion colour banding

This doesn't appear to work.  It looks the same as when disallowing 10-bit output.

lavfi: ;format=pix_fmts=yuv420p

I think there might be something wrong with that, or the lavfi documentation is incorrect.

https://ffmpeg.org/ffmpeg.html

The section on pixel format states:

"-pix_fmt[:stream_specifier] format (input/output,per-stream)
Set pixel format. Use -pix_fmts to show all the supported pixel formats."

So it appears -pix_fmt is for setting the format, and -pix_fmts is for getting a list of available formats.  I get an error if I remove the s from it, though.
Anyway, back on topic...

This worked, though!

ovcopts: pix_fmt=yuv420p

The output from that, with 10-bit output allowed, gives an 8-bit output file that looks very similar to the Handbrake output.  So that will work just great!  Here's the test sample if you're still curious.  Thanks again for the help.

https://www.dropbox.com/s/idedyznh71do0 … t.mkv?dl=0

Re: 10-bit to 8-bit conversion colour banding

> pix_fmts

it's a conversion filter, not an encoder option - https://ffmpeg.org/ffmpeg-filters.html#format-1

> This worked, though!
> ovcopts: pix_fmt=yuv420p

how? hmm

[vo/lavc] AVOption 'pix_fmt' not found.


===
1. How SVP works by default:
<10-bit after decoder> --> <10-to-8 Vapoursynth conversion w/o dithering> --> <motion interpolation in 8-bit> --> <8-bit encoder>
"allow 10-bit" OFF

2. Proposed method via lav filters:
<10-bit after decoder> --> <no Vapoursynth conversion> --> <motion interpolation in 10-bit> --> <10-to-8 lavfi conversion> --> <8-bit encoder>
"allow 10-bit" ON, lavfi: ";format=pix_fmts=yuv420p"

3. Other possible solution:
<10-bit after decoder> --> <10-to-8 Vapoursynth conversion with dithering> --> <motion interpolation in 8-bit> --> <8-bit encoder>
"allow 10-bit" OFF, needs editing SVP/script/generate.js, line 397:

res += bl+'input_um = clip.resize.'+(media.is420 ? 'Point':'Bicubic')+'(format=vs.YUV420'+(media.p10?'P10':'P8')+',dither_type="random")'+br;

4. One more...
<10-bit after decoder> --> <10-to-8 Vapoursynth conversion> --> <motion interpolation in 8-bit> --> <lavfi deband filter> --> <8-bit encoder>
"allow 10-bit" OFF, lavfi: ";deband"

5. And more...
<10-bit after decoder> --> <some super-duper Vapoursynth conversion script including deband via f3kdb plugin> --> <motion interpolation in 8-bit> --> <8-bit encoder>

(1) obviously the worst
(2) the best (?) but slowest
(3) better than (1) and worse than (2) in quality, but should be noticeable faster than (2)
(4) probably even better than (2)
(5) have not tried, the best possible quality (in theory) + slowest processing

===
Conclusion: we'll insert "deband" filter in case of 10-to-8 conversion in the next build

  • Thanks: 97

Re: 10-bit to 8-bit conversion colour banding

>(4) probably even better than (2)

Than (3) you mean?

  • Thanks: 1

Re: 10-bit to 8-bit conversion colour banding

As I said, lavfi: ";format..." does not work.  It produced the same incorrect output as "allow 10-bit" off did.  Putting "pix_fmt=yuv420p" in ovcopts works.  I've got output files that I am not imagining that show this to be the case.  They actually exist on my machine, hehe. wink  In case it wasn't clear, I am talking about transcoding, which means ffmpeg, no?  Doesn't vo/lavc have to do with the player and not the encoder?  Do you not use ffmpeg to encode?  Because I see ffmpeg.exe in SVP's utils directory.  Encoding with ffmpeg means --pix_fmt is a valid option, which also happens to provide correct output files.  And I thought that's what the ovcopts were when you were using libx264/libx265, ffmpeg command line options.  Sure seems to be the case, since it worked.

Re: 10-bit to 8-bit conversion colour banding

> Putting "pix_fmt=yuv420p" in ovcopts works

please post the transcoder log here

  • Thanks: 1

Re: 10-bit to 8-bit conversion colour banding

My mistake!  I am seeing the same thing you're seeing now.

16:46:36.162: ===== Starting mpv ======
16:46:36.162: Command line: C:\Program Files (x86)\SVP 4\mpv64\mpv.exe C:/temp5/test2.mkv -o C:/temp5/test2.SVP.temporary.mkv --no-audio --no-sub --no-sub-auto --input-ipc-server=mpvencodepipe --input-media-keys=no --no-msg-color --vf=vapoursynth:[C:\Users\Clay\AppData\Roaming\SVP4\scripts\ffff.py]:4:12 --of=matroska --ovc=libx264 --ovcopts=preset=superfast,crf=18,pix_fmt=yuv420p,threads=12
16:46:36.198: (+) Video --vid=1 (*) (hevc 3840x2160 25.000fps)
16:46:36.198: Audio --aid=1 (*) (ac3 6ch 48000Hz)
16:46:36.198: Audio --aid=2 (*) (eac3 6ch 48000Hz)
16:46:37.175: VO: [lavc] 3840x2160 yuv420p10
16:46:37.185: [vo/lavc] AVOption 'pix_fmt' not found.
16:46:37.189: [vo/lavc] Opening encoder: libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 [libx264]
16:46:37.190: [ffmpeg] libx264: MB rate (777600000) > level limit (16711680)
16:46:37.225: [encode] Opening output file: C:/temp5/test2.SVP.temporary.mkv
16:46:37.225: [encode] Opening muxer: Matroska [matroska]
16:46:37.250: V: 00:00:00 / 00:00:20 (0%) {0.0min 0.0MB}
16:46:40.972: Script exceeded memory limit. Consider raising cache size.
16:47:17.760: [encode] video: encoded 69755088 bytes
16:47:17.760: [encode] audio: encoded 0 bytes
16:47:17.760: [encode] muxing overhead 10694 bytes
16:47:17.764: Exiting... (End of file)
16:47:17.840: ===== Starting mkvmerge ======
16:47:17.840: Command line: C:\Program Files (x86)\SVP 4\extensions\code\mkvmerge.exe -o C:/temp5/test2.SVP.mkv C:/temp5/test2.SVP.temporary.mkv -D C:/temp5/test2.mkv
16:47:17.894: mkvmerge v35.0.0 ('All The Love In The World') 32-bit
16:47:17.906: 'C:/temp5/test2.SVP.temporary.mkv': Using the demultiplexer for the format 'Matroska'.
16:47:17.931: 'C:/temp5/test2.mkv': Using the demultiplexer for the format 'Matroska'.
16:47:17.931: 'C:/temp5/test2.SVP.temporary.mkv' track 0: Using the output module for the format 'AVC/h.264'.
16:47:17.931: 'C:/temp5/test2.mkv' track 1: Using the output module for the format 'AC-3'.
16:47:17.931: 'C:/temp5/test2.mkv' track 2: Using the output module for the format 'AC-3'.
16:47:17.932: The file 'C:/temp5/test2.SVP.mkv' has been opened for writing.
16:47:17.944: Progress: 9%
16:47:18.112: Progress: 100%
16:47:18.112: The cue entries (the index) are being written...
16:47:18.121: Multiplexing took 0 seconds.

So now I'm confused as to what I ended up doing to end up with the 8-bit file that turned out with a better conversion.  Going to have to see if I can remember what steps I ended up taking to get here from there.  I could've sworn that's all I did, pix_fmt=yuv420p, but apparently that can't be correct.  Hmm...

  • Thanks: 1

Re: 10-bit to 8-bit conversion colour banding

Oh, for crying out loud.  I see what I did now.  Ok, at some point during various file edit/copy operations I ended up saving some x264 lossless source files that were edited to just some short test segments.  And for some of them I had let Handbrake do the 10-bit to 8-bit conversion, which I was using just for testing various CRF values.  And when I got back to testing the SVP stuff again really late that night it appears I may have been selecting those pre-converted files as my source.  And that's apparently how I ended up with 8-bit conversions that looked ok with the non-existent ovcopts pix_fmt option.  Argh.  Sorry for that waste of a bit of time.

Re: 10-bit to 8-bit conversion colour banding

in this log you just re-encoded 10-bit source as 10-bit, w/o any conversion at all

---
update to the latest SVP rev.182 (it includes dithering in the Vaporsynth script) and put deband filter into lavfi field

  • Thanks: 1

Re: 10-bit to 8-bit conversion colour banding

Chainik wrote:

in this log you just re-encoded 10-bit source as 10-bit, w/o any conversion at all

---
update to the latest SVP rev.182 (it includes dithering in the Vaporsynth script) and put deband filter into lavfi field

Yes, I was confirming the behaviour you were saying I should see since the option did not actually exist, and how a mistake I made had me thinking it was successfully converted when in fact it was not.

Thanks!  I will try the update now.

  • Thanks: 1

Re: 10-bit to 8-bit conversion colour banding

That appears to be working just fine for me.  I disallowed 10-bit output and put ";deband" in the lavfi line, and I got an 8-bit file and it appears much better than the previous version.  Thank you very much for taking the time to look into this, and coming up with a fix.  Much appreciated.

00:37:43.507: ===== Starting mpv ======
00:37:43.507: Command line: C:\Program Files (x86)\SVP 4\mpv64\mpv.exe C:/temp5/test1-short.mkv -o C:/temp5/test1-short.SVP.temporary.mkv --no-audio --no-sub --no-sub-auto --input-ipc-server=mpvencodepipe --input-media-keys=no --no-msg-color --vf=vapoursynth:[C:\Users\Clay\AppData\Roaming\SVP4\scripts\ffff.py]:4:12,lavfi=[deband] --of=matroska --ovc=libx264 --ovcopts=preset=superfast,crf=18,threads=12
00:37:43.546: (+) Video --vid=1 (*) (hevc 3840x2160 25.000fps)
00:37:43.546: Audio --aid=1 (*) (ac3 6ch 48000Hz)
00:37:43.546: Audio --aid=2 (*) (eac3 6ch 48000Hz)
00:37:45.443: VO: [lavc] 3840x2160 yuv420p
00:37:45.444: [vo/lavc] Opening encoder: libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 [libx264]
00:37:45.444: [ffmpeg] libx264: MB rate (777600000) > level limit (16711680)
00:37:45.462: [encode] Opening output file: C:/temp5/test1-short.SVP.temporary.mkv
00:37:45.463: [encode] Opening muxer: Matroska [matroska]
00:37:45.495: V: 00:00:00 / 00:00:04 (1%) {0.0min 0.0MB}
00:37:54.503: [encode] video: encoded 28153705 bytes
00:37:54.503: [encode] audio: encoded 0 bytes
00:37:54.503: [encode] muxing overhead 2976 bytes
00:37:54.508: Exiting... (End of file)
00:37:54.581: ===== Starting mkvmerge ======
00:37:54.581: Command line: C:\Program Files (x86)\SVP 4\extensions\code\mkvmerge.exe -o C:/temp5/test1-short.SVP.mkv C:/temp5/test1-short.SVP.temporary.mkv -D C:/temp5/test1-short.mkv
00:37:54.635: mkvmerge v35.0.0 ('All The Love In The World') 32-bit
00:37:54.652: 'C:/temp5/test1-short.SVP.temporary.mkv': Using the demultiplexer for the format 'Matroska'.
00:37:54.668: 'C:/temp5/test1-short.mkv': Using the demultiplexer for the format 'Matroska'.
00:37:54.668: 'C:/temp5/test1-short.SVP.temporary.mkv' track 0: Using the output module for the format 'AVC/h.264'.
00:37:54.669: 'C:/temp5/test1-short.mkv' track 1: Using the output module for the format 'AC-3'.
00:37:54.669: 'C:/temp5/test1-short.mkv' track 2: Using the output module for the format 'AC-3'.
00:37:54.669: The file 'C:/temp5/test1-short.SVP.mkv' has been opened for writing.
00:37:54.682: Progress: 27%
00:37:54.741: Progress: 100%
00:37:54.741: The cue entries (the index) are being written...
00:37:54.751: Multiplexing took 0 seconds.