X264: The open high-def codec
Nick Peers digs into the spaghetti-like jumble of codecs, containers and other digital video jargon to find out how you can rip video to your PC. All you need to know about codecs and converting video
The days where your media collection consisted of video cassettes or optical discs are long gone. Why clutter up your living room when entire digital video collections can be housed within a single hard drive? Pair this data with a media server such as Emby , formerly known as MediaBrowser, [see Features, p32, LXF204] and you can keep your entire collection at your fingertips – no more disc swapping or sitting through endless trailers.
It all sounds wonderful, but do you really want to purchase digital – and usually restricted – copies of movies and TV shows you already own? If you’re lucky enough to live in a country that permits you to transfer your media from one format to another, then not only is the process of building your digital media library legal, it’s also practical and – armed with the right tools and know-how – not so difficult. Sadly, copyright law is often in flux, so make sure you’re on the right side of the law before you begin – in the UK, eg digital copying for personal use was made legal in October 2014, but the High Court ruled against the legislation in July 2015, so it’s once again legally dubious to rip DVDs and CDs, but not for the many copyleft works out there, of course.
Whatever the legalities of your individual situation, read on to discover the practicalities, techniques and tips you need to start building your own digital media library.
The basics
In a perfect world, there would be a single digital video format guaranteed to work across all your devices. But as with audio files, there’s no single standard and there are multiple competing standards when it comes to digital video formats.
The matter is made more confused by the fact that video formats consist of two separate elements: the container and the codec. The container is the containing file format, which you can identify by its extension, from MKV to MP4 and AVI. The codec is the technology used to encode video and audio to a specific format, then decode it again in order to view it. A digital video player has to recognise both the container and the codec to play the file – and because it’s possible to mix and match the two, it’s possible to build a library of seemingly identical MP4 files only to find that half of them won’t play on a device because they’re encoded using different codecs.
Codecs come in all shapes and sizes, arranged into families or ‘standards’. Many popular standards have been set by the Motion Picture Experts Group (MPEG), each one evolving from its predecessors to reflect new technologies and the growth in both video resolution and the computational power required to process it. The most notable example here is H.264 (also known as MPEG-4 AVC). Other families have been developed by a specific body that retains the ownership of those codecs, restricting their use – Microsoft’s WMV (Windows Media Video) series of codecs is a good example.
Within each standard there’s room for multiple codecs to be developed. The key thing is that each codec is capable of delivering a file that meets the standard, ensuring it can be played on any device that supports that standard. Why so many codecs? Many are proprietary, meaning the owners retain control and can charge licensing fees for their use. These closed formats have led to the development of opensource rivals, designed to provide an open alternative.
The most notable example of a proprietary codec is Apple’s QuickTime H.264 codec. Its open source equivalent is the x264 codec, although it’s worth noting that like many open formats, x264 contains no decoding capabilities; it can only encode video to the H.264 standard, but not decode. The less well known open format OpenH264 can do both.
The predecessor to the H.264 standard – H.263/MPEG-4 Part 2 – contains two open-source codecs in the form of Xvid and FFmpeg MPEG-4, while the successor to H.264 – H.265/ MPEG-H HEVC – contains an embryonic open-source encoder in the form of x265.
Another standard worth mentioning – partly because it’s part of Google, but also because it’s been made open source – are the On2 codecs, usually associated with the WebM container. These were originally developed as proprietary codecs by On2 Technologies, but after Google acquired the company in 2010 it moved to make the latest codecs – VP8 and VP9 – open source in order to encourage take-up.
How codecs work
Most codecs take the opportunity during encoding to reduce the file size of the original digital video through a variety of
compression methods. These include ‘lossy’ techniques that discard some of the original video’s information to cut its size much further than would otherwise be possible. The result is a video that’s lower quality than the original, although you can strike a balance between reducing the file size and keeping the quality high enough so the difference is not perceptible.
Codecs reduce file size through a number of different techniques. First, because analogue video signals represent colour and brightness (luminosity) separately, the codec can store each separately too, which in turn allows the less perceptible colour information to be compressed to a lower resolution without the loss of detail being as noticeable.
Assuming that the video’s dimensions on-screen need to stay the same, the main way that video file sizes are reduced is through lower bit rates. Bit rate is a measure of the amount of data stored for each second of recording or playback. For HD Blu-ray discs, video can be stored at 20Mbps (megabits per second). For standard DVD, the typical bit rate is 6Mbps, and it’s 2Mbps for high-quality web video. Codecs attempt to reduce this bit rate through image compression, which like saving images as smaller JPEGs results in the image quality degrading. The trick is to find the lowest possible bit rate that reduces the file size without visibly ruining the image.
It’s worth noting that newer standards have developed compression techniques that allow them to improve image quality at lower bit rates. This translates into newer standards producing video at lower file sizes – H.265 compared to H.264, and H.264 compared to H.263 before it – while maintaining the same levels of image quality.
Choosing the right codec
Although the number of standards and codecs can be bewildering, your choices are actually quite simple. These days, it’s a straight fight between MPEG and Google’s standards – so H.264/H.265 versus VP8/VP9 respectively. If you want to produce video that will play back on just about any device you own, from mobile and desktop to smart TV and set-top box, then the H.264 standard is the one to go for – we’d recommend pairing the x264 open-source codec with the MP4 file container.
It’s not just about popularity – the x264 codec produces decent results and small(ish) file sizes, and it is packed with features that enable it to support various applications, such as TV broadcasts, web video and Blu-ray low-latency video. Although designed for HD video, H.264 has also proved extremely adaptable, enabling it to handle 4K video too. It is, however, worth keeping an eye on the next-generation battle being fought between MPEG (H.265) and Google (VP9) ( see The Future of Encoding box for details, below).
Tools to encode video
Okay, you’ve settled on x264 as your chosen video codec, so how do you go about converting video to this (practically) universal standard? The good news is that there are plenty of tools available to harness the computational power of your PC to convert video from a wide range of formats – including DVD – to MP4/x264 files.
The obvious first choice to look at is the x264 commandline encoder, available through the Ubuntu Software Store. Once it’s installed, type x264 --help into the Terminal to get a full list of commands. The basic syntax is this:
x264 [options] –o outputfile inputfile
This looks simple enough, but the truth is that trying to convert files using the x264 CLI can be quite tricky. The simplest mode to employ is ‘constant quality mode’, also known as ‘constant rate factor’ or crf. This sets a basic quality setting between 0 and 51. The lower the value, the better the video quality (but the larger the file size). As a rule of thumb, most people opt for values between 18 and 28, with the default set to 23. When starting out, it pays to experiment with different CRF values – start with 23, then if the quality’s good, try 24 or 25; if it’s bad, try 22 or 21. Keep tweaking the figure until you get the best balance of quality and file size, then use that going forward.
The more adventurous can get involved with tweaking bit rates and setting up two-pass filters, plus choosing between variable or constant frame rates. Type x264 --longhelp or even x264 --fullhelp and you’ll unlock even more controls you can tweak – for example, the ‘tune’ settings allow you to adjust a whole set of parameters to suit different types of source material (such as animation or film). For most people, though, the crf setting should suffice for the most part.
Two areas where the x264 CLI tool falls down are format support and filters. It can’t handle DVDs and requires other tools to decode from other formats. Filters exist to crop and resize the picture, but there aren’t any deinterlacing filters on offer( see the All About Interlacing box, below ). If you want to push the envelope further, you’ll need to look for another tool to do the job.
An easier and friendlier way to encode video lies with the open-source Handbrake tool. It’s available with a GUI and there’s a command line tool that’s simpler to use than the x264 utility too – see https://trac.handbrake.fr/wiki/
CLIGuide for a complete guide. Not only does Handbrake provide a friendlier way to encode video, it includes additional options not covered by the basic x264 CLI tool, most notably support for additional filters, including deinterlace and detelecine filters, which come into their own when encoding movies and TV shows. It also supports additional encoders in the form of open-source x265, MPEG2/4 (both FFmpeg), VP8 and Theora (VP3) codecs, making it easy to switch standards should you wish to do so.
Using Handbrake
Getting files into Handbrake is simple – if you’re simply converting from one digital format to x264, then place all your videos in a single directory, select it as the source and use Handbrake’s queueing function to process each video separately with the appropriate settings (select each video via the Title drop-down menu). Handbrake natively accepts a wide range of input formats – basically any supported by libavformat and libavcodec, which covers most bases.
Handbrake also excels as a tool for ripping media from DVD and some Blu-ray discs. You’ll need to install the libdvdcss2 package – if you’ve not already done this, then those running Ubuntu 15.10 or later can install it with a single line in the Terminal: sudo apt-get install libdvd-pkg
If you’re running Ubuntu 15.04 or earlier, you’ll need to type the following two lines (skip the first if you’ve previously installed *ubuntu-restricted-extras): sudo apt-get install libdvdread4 sudo /usr/share/doc/libdvdread4/install-css.sh
If necessary, reboot your PC, at which point you can now watch DVDs in Linux. When it comes to installing Handbrake, there’s an outdated version in the SoftwareCenter, but it’s better to download the build from ( https://handbrake.fr ), then open Terminal and type the following commands to get Handbrake: sudo add-apt-repository ppa:stebbins/handbrake-releases sudo apt-get update && sudo apt-get install handbrake-gtk
If you’d also like to access Handbrake from the command line, add the following command: sudo apt-get install handbrake-cli
Encode DVDs
If you’re looking for a quick, easy and effective way to encode video from a DVD using Handbrake, see the step-by-step guide below. When you come to select your filters from the app’s Pictures tab, choose the Decomb option and set it to Default. Decomb is an alternative to the older Deinterlace method, and is a better choice because it’ll only deinterlace frames that are visibly interlaced, leaving others untouched. If you’re ripping region 1 discs you may find setting Detelecine to Default helps improve results too. Like Decomb it scans each frame and applies the filter only when it’s needed.
You can also include subtitles from supported video tracks when encoding with Handbrake: in the Subtitle Defaults tab, set ‘Selection Behavior’ to ‘First Track Matching Selected Languages’. Scroll through the list to English, select it and click Add. Now switch to the Subtitle List tab and click Add. If subtitles are present, they’ll appear here; you’ll need to select English (VOBSUB) from the list. Tick ‘Forced Subtitles Only’ if you’re looking for subtitles to automatically display when someone is speaking a foreign language in the movie. Also consider ticking ‘Set Default Track’ if you’re encoding more than one set of subtitles. Once they’re encoded, you’ll be able to switch on subtitles for that movie via supported apps (ie Subtitles > Sub track in VLCMediaPlayer).
Experiment to perfection
If you’re striving to get the best possible quality from your Handbrake rips, then take our advice and experiment with all the settings. Rather than wait half an hour to re-encode an entire film, select one chapter using the Chapter drop-down menus and encode that. Save multiple encodings of the same section of film – clearly labelling each one as ‘movie-presetfast.mp4’ or ‘movie-tuning-film.mp4’ – and then compare them to see which one hits the mark. Also note that some settings may work better on specific titles than others, so try setting up multiple presets to cover different needs.