Encoding BluRays (2016)

The aim of this article is to consolidate and update echo's, parent's and HD x264 encoding guides. It is intended to provide straightforward instructions which will result in a decent encode, with links and additional resources in hide tags for people interested in more advanced topics.

This guide assumes you are starting with a m2ts image or a remux. If you are still holding a physical disc, follow Wiki > How to rip a BluRay (2016) first.

Required software
Avisynth v.2.6.0
Plugin Bundle (Updated)
AvsPmod
Simple x264 Launcher
MKVToolNix v8.3 + MkvExtractGUI
DTS-HD Master Audio Suite 2.60.22
eac3to + Haali Media Splitter
SupRip
SubtitleEdit

Overview

A typical workflow would look like this:

https://i.ibb.co/GcW5vhG/z75sfq.png

The numbers in the yellow bubbles correspond to the chapters of this guide.

1. Demux your source with eac3to

Extract BDMV/m2ts : Show

Extract a specific track : Show

Extract a remux : Show

2. Crop, filter & resize with AvsPmod

Open a new tab in avspmod and load your demuxed video file

ffvideosource("C:\workshop\Movie\demux.mkv")

Press F5 and wait a few minutes for the video to be indexed and loaded.

Crop

If your picture is surrounded by black bars, you need to crop them out.

1. Open Crop Editor from the Video menu.
2. Adjust the cropping so that you remove all the black bars. You can only crop in multiples of 2 pixels. Leave one black line in if you can't crop it to an even number, we'll fix it in the filtering stage.
3. Select 'Insert Crop() command at script end' and click Apply

Filter

Each filtering step is optional, and depends entirely on the quality of the source. Incorrect filtering can easily do more harm than good, feel free to ask for help in Encoding Workshop... ***Get Encoding Help Here*** and #ptp-encoding if you have any doubts.

Fix dirty lines : Show

Fix banding : Show

Fix edge enhancement artifacts : Show

Fix double levels compression : Show

Resize

If you are encoding 1080p, skip this part -- you are not doing any resizing, even if the cropped image is smaller than 1920x1080 (never upscale!).
If you are making a 720p or SD encode, you need to resize it accordingly:

https://i.ibb.co/CPCC9R2/sue885.png

1. Open Resize calculator from the Tools menu.
2. Click configure
3. Set Resize block constraints to 2x2 and change 'Avisynth resize' to 'Spline36ResizeMod(%width%, %height%)', click OK
4. Drag the slider until both the width and height of your scaled picture is either equal or smaller than the maximum dimensions in the table above. Example: 1920x1008 source will scale to 1280x672 but 1410x1080 will scale to 940x720. In other words, it must "fit inside" the maximum size for a given format in the table above.
5. Click Apply

If you duplicated a single line using FillMargins, you can omit it during scaling. For example, to omit one line on left and one on right side use the following:
Spline36ResizeMod(1278, 720,1,0,-1,0)

You cannot directly remove a single black line this way, as it would still be sampled during resizing and 'dirty up' the adjacent line. That is why in the filtering stage we used FillMargins to get rid of it.

Hint: Avisynth chroma resize bug and what is Spline36ResizeMod: Show

3. Test video encodes with AvsPmod and Simple x264 Launcher

Once you are happy with your filtering it is time to get started on making test encodes to optimize the x264 settings for your movie. There is no silver bullet here so you will need to evaluate a number of parameters (one at a time) as you compare your test encode to the filtered source.

Let's start with making a single test encode. At the end of your avisynth script append SelectRangeEvery(4000,100,10000)

This will pick 100 continuous frames every 4000 frames omitting the first 10000 frames (opening credits etc). Adjust the first parameter (4000) to get around 3000 frames to work with which are representative of the whole movie. You will want static scenes, high-motion scenes, close-ups, dark scenes and brightly lit scenes. Once you are happy with your choice, save the .avs file, open Simple x264 launcher and select Job - Create job.

Make sure that the drop down menus are selected as in the screenshot below (click the image to enlarge).
https://i.ibb.co/VpHXWDM/9jw7p7.png

For the baseline encoding settings, paste the following line under Custom Encoder Parameters:
--level 41 --vbv-bufsize 78125 --vbv-maxrate 62500 --merange 32 --bframes 16 --deblock -3:-3 --no-fast-pskip --rc-lookahead 250 --qcomp 0.60 --psy-rd 1.00:0.00 --aq-mode 2 --aq-strength 1.00

If you encode at a SD resolution, it is a good idea to append '--colormatrix bt709' to your custom parameters. This will ensure that compatible players will use the correct color matrix for BluRay sourced video. For more background information, check out Wiki > How to correct color matrices during playback

For now, let's make a single test encode and compare it to our source. Click Add Job and wait for the encode to finish.

Once done, open a new tab in avspmod and paste the following:

a=import("E:\workshop\example\example.avs").subtitle("Source", align=8).ffinfo(cfrtime=false,vfrtime=false) #Filtered source
b=ffvideosource("E:\workshop\example\18.mkv").subtitle("Encode crf18", align=8).ffinfo(cfrtime=false,vfrtime=false) # Test encode CRF18
#c=ffvideosource("E:\workshop\example\17.mkv").subtitle("Encode crf17", align=8).ffinfo(cfrtime=false,vfrtime=false)
interleave(a,b)

Hit F5. Avspmod will annotate the frames and display each frame from the source followed by the respective frame of the encode. You can use the keyboard arrow keys to step through the video one frame at a time. Your objective is to visually compare the quality of 'B-type' frames in the encode to 'P-type' frames in the source in order to find the best settings. And why's that?: Show

Having issues with frames not matching between your source and encode (off by 1)? Try downgrading to an earlier version of ffms2 (i.e. 2.20 from here).

But we are going to need much more than one test encode to determine that!

The first step is to find the optimal bitrate for your video. Make a series of encodes decreasing the CRF value by 1 until your test encode has subjectively the same quality as the source. Now make a series of encodes that increase CRF by 0.2 until you find the bitrate at which the quality begins to degrade. Look in the log of the last satisfactory test encode for the bitrate and note it down.

The second step is to switch to 2-pass mode and optimize the individual parameters. Create a new job, change 'Mode' to '2-pass' and enter the target bitrate obtained in the previous step. Click Add job - this will be your first reference test encode.

It is beyond the scope of this guide (and its author competency) to explain the theory behind each parameter that needs to be optimized. If you are interested, read Mosc's Guide to x264, Detail Loss & Desperation. Meanwhile, the parameters need to be empirically tested anyway.

Test --qcomp (0.60-0.80 by 0.05 increments), --aq-mode (1,2 or 3), --aq-strength (0.50-1.10 by 0.05 increments), --psy-rd (0.80:0.00-1.15:0.00 by 0.05 increments), and finally psy-trellis (0.00-0.15 by 0.05 increments) changing only one parameter at a time. Once you determine the optimal value for one parameter, keep it and move on to the next one. A well-tested encode will take dozens of tests.

This is a very thorough approach that should give you a good result. There may be cases where other values outside these ranges may be necessary (e.g. psy-rd 0.35-0.80 for animation), but these are usually more than sufficient. You can test at 0.1 increments and then further at 0.05 increments to save time.

In addition, test --no-mbtree and --no-dct-decimate. With --no-mbtree you might also want to lower --pbratio. Testing deblock -3-2, -2-2, -2-1 can be useful for SD sources.

Once you've found the best values for these settings, switch back to CRF mode and test until you find the value which produces the same (or lower) bitrate as your 2 pass encode at the same quality.

Hint: What to look for in image comparisons?: Show

You may find that certain scenes require more bitrate, while others consume too much. In that case it is recommended to 'zone' them, that is, adjust the crf value just for selected scenes. In order to do that, add e.g. --zones a,b,crf=15/c,d,crf=16 to the parameters of your final encode, where a and b (or c and d) are the first and last frames of a scene for which you want to override the crf value used for the rest of the movie.

4. Final video encode with Simple x264 Launcher

For the final encode, edit your .avs script to remove SelectRangeEvery(), save, and restart x264 Launcher using your optimized settings. The encoding process with the options suggested above will likely take several hours on a reasonably modern 4-core PC. If you have an exceptionally fast machine, or don't care about encoding time, you can add --me tesa --subme 11 to squeeze out the last tiny bit of improvement at the cost of likely doubling the encode time.

Hint: What about ref or bframes?: Show

5. Encode audio with eac3to or DTS-HD Audio Suite

You need to pick an appropriate audio codec for the resolution you want to encode. Refer to the table below:
https://i.ibb.co/Rc0gkvF/pz1514.png

Note: 24-bit FLAC should be reduced to 16 bits. See Wiki > Audio bit-depth reduction and sample rate conversion for details.

Transcoding TrueHD/FLAC to DTS Spoiler: Show
Transcoding DTS-HD MA to DTS Spoiler: Show
Transcoding to AC3 Spoiler: Show
Transcoding to AAC Spoiler: Show

6. OCR subtitles with Suprip and SubtitleEdit

Subtitles on a BluRay are saved in a bitmap format (PGS). They need to be OCR'd to a text file before you can include them in the final encode.

1. Open Suprip. Click Open... and load your .sup file that was extracted with eac3to.
2. Tick 'Automatically continue with next subtitle'
3. Press Ctrl-O. SupRip will show you the first character it is unable to identify. Enter that character(s), into the box next to the OK button. Hit Enter to display the next character that SupRip can't identify.

https://i.ibb.co/dptHzrq/9z3x77.png

3. Repeat until you reach the end of the subtitles. Remember to make use of the Italic checkbox when necessary.
4. When you are done, click the SRT tab and save your .srt file.
5. Open your .srt file in SubtitleEdit.
6. Go to Tools - Fix common errors and select the following options:

https://i.ibb.co/Lg6JCrQ/33dycc.png
7. Click Next, select the errors you want to fix, and click Apply selected fixes.
8. Go to Spell check - Spell check, select the correct language dictionary, and correct any remaining typos

https://i.ibb.co/2KKm0Kr/xvi167.png

9. Save the corrected .srt file

It is recommended to watch the movie with the ripped and corrected subtitles as some typos may slip through the process described above.

Note that Suprip may crash on certain subtitles. As an alternative, consider using subextractor.

7. Prepare chapters

You have obtained the chapter list when you extracted the source. The file will be called 'demux - Chapters.txt'. Open it to preview it.
Likely, the chapters will lack proper titles and contain timecodes only. Your options here are rather limited.

  • Check the BluRay menu using a commercial player or software. You might have some luck with VLC but it handles the menus poorly.
  • Check http://chapterdb.org/browse to see if the chapters for your movie have been submitted.
  • Check other encodes & online listings to see if the chapter list is available elsewhere.


If you find the chapter titles, edit the text file accordingly and save it in UTF-8 encoding.

8. Mux it all together with mkvmerge GUI

Time to put everything together using mkvmerge GUI.

1. Open mkvmerge GUI
2. Go to Muxing - Add command line options
3. Paste the following line under 'Command line options' to ensure maximum compatibility with standalone players and older software:
--engage no_cue_duration,no_cue_relative_position,no_simpleblocks --clusters-in-meta-seek --disable-lacing
4. Tick 'Save as default' and click OK
5. Drag the encoded files (video, audio & subtitles) to Input files
6. For each track set the correct language.
7. If you have multiple audio tracks, set the main track to 'default: yes'
8. If the default audio track is foreign, set English subtitles to 'default: yes' Hint: What about forced subs?: Show
9. Set Track names for special tracks as necessary (commentary audio, Atmos track, hard-of-hearing subtitles etc.)
https://i.ibb.co/mtqsY3Q/4hk11g.png
10. Switch to 'Global' tab
11. Load your chapters file and select the correct language and UTF-8 encoding
12. Set the output file name to Movie.Year.Resolution.BluRay.x264.Audio.Channels.mkv and click Start Muxing

At this point your encoding work is done, congratulations! Make final checks to ensure you didn't make a mistake somewhere -- watching the complete encode is always a good idea (if you are not sick of it yet). Once that's done, create a torrent and upload it!

Appendix 1: Special cases

Note: Many tools mentioned in this section are not included in the plugin pack.

1. "My source is interlaced."
A vast majority of BluRays is progressive (1080p), however, you may happen across a 1080i file with a framerate of 50 or 60fps and visibly misaligned odd and even lines. If you want to encode it to 720p resolution it is recommed to deinterlace it at the beginning of your script:
QTGMC("medium")
If you want to encode it to 1080p, you need to consider that Level 4.1 which is targeted on PTP for compatibility with hardware players allows for a maximum of 30 frames at 1080p. In this case, you may want to consider encoding 720p60/720p50, or decimating your deinterlaced source to 30/25fps for some loss of the temporal resolution:
QTGMC("medium").SelectEven()

If you are having trouble seeking in the interlaced source with ffms2, consider using DGDecNV/DGAVCDecDi.

2. "My source says it's interlaced, but it is not."
The Blu-ray standard does not allow for 1080p25. Some Blu-rays may use what's called a 'fake interlace' where progressive content is tagged as interlaced. In that case, a Blu-ray may appear to be 1080i50, but in fact have duplicated progressive frames. Step through your video in avisynth to verify that. If that's the case, you can decimate the duplicated frames with SelectEven() and treat the source as progressive.

3. "My source is encoded with VC-1."
Make sure that the VC-1 source is your only option, as often VC-1 is inferior to AVC releases. If you need to encode a VC-1 source, you need to use a different decoder than ffms2.

If you have a NVIDIA gpu, it's recommended to use DGDecNV. For an AMD card you can try DGAVCDecDI + DiAVC. If that fails, you can try to use dss2mod + ffdshow.

4. "My source uses MBAFF."
With MBAFF encoding, each macroblock can be either progressive or interlaced. It's recommended to use a decoder which plays nice with MBAFF/PAFF streams, to be on the safe side use DGDecNV/DGAVCDecDI. A correctly decoded source can be then deinterlaced with QTGMC.

With thanks to the creators of the original guides, nonshatter, Darky, Mosc, F00D and all the lurkers in #ptp-encoding.