Project

General

Profile

About

vdr-convert is a tool to accurately transcode VDR1.x and VDR2.x TV recordings, including all valid streams - video, audio (including AC3/DTS 5.1), Audio Description (AD), and DVB subtitles - into more compressed and accessible formats, while maintaining perceived quality with good compatibility.

H264 and AAC are the default codecs for the main streams, but H265 is available, together with most common multimedia file formats for use in single-use/batch/podcast modes. Standard Definition (SD) recordings are reduced to anywhere from 35% - 90% of original size depending on content and compression settings. The average reduction is around 35% of original size over a range of SD recording types. The better and less noisy the content, the better the compression, for example current SD output from the BBC is regularly compressing to less than 50% of original size, typ. 550-600M/hr (in keep mode, H264 CRF=21).

When keeping files for use inside VDR, it is assumed that the user has VDR2.x with an HD capable setup, and so can play anything using the above codecs. A number of size, duration and compatibility tests are performed before (optionally) deleting the originals. When keeping VDR1.x recordings, a new directory is made in VDR2.x format and the metadata files renamed, amended with new tags, and moved over.

Rationale

Disk space is cheap - why bother?
  • Disk space isn't quite so cheap or easy to manage when it's in a RAID array, a phone or tablet or other mobile device with limited storage capacity.
     
  • A key driver was a desire to recover a substantial archive of rarely transmitted shows recorded over 8+ years, mostly using VDR 1.6. The older .vdr private file format contains subtitles that are encoded uniquely, so they won't show on anything other than certain legacy VDR configurations; these recordings can be difficult to view/transcode for 3rd party players (multiple files, file format not always properly recognised, accessibility features missing).
     
  • The need for a semi-automatic compression/FTP upload/podcast facility creating a more compact temporary copy for entertainment when working abroad, or travelling with a mobile with limited or no Internet access.

There are many other solutions (like the very original vdrconvert [http://vdrconvert.vdr-portal.de] after which this project is named) and simpler scripts/programs out there, but most do the much simpler job of single video/audio stream conversion without VDR 1 > 2 format modification, subtitle extraction, validity checks, and indexing. Additionally, vdr-convert applies a number of carefully selected x264/x265 and mpegts multiplexer optimisations appropriate to TV recordings, to make sure that the recordings look as good as possible, streams are well ordered and labelled. Usability, such as good skip forward/backwards performance, has also been considered (default ffmpeg settings are not ideal).

More importantly, due to the complexity of the steps required, in testing no other conversion method could successfully recover all valid streams (and deal with empty streams), particularly DVB subtitles in VDR 1.x files with their correct colours, clean fonts, properly synchronised for use by current VDR (2.2+) AND other players.

Usage

vdr-convert can be used in 3 main ways:

  1. "Keep" mode (-k) - after a recording is made - in exactly the same way as noad, markad and similar tools - run by VDR automatically. The recording files are compressed, re-indexed and recognised by VDR, but are significantly smaller. If already compressed with the target compression type (e.g. H264), the recording is not modified. The author uses this on all recordings with lifetimes > 7 days
     
  2. In its original "single-use" mode with approximately doubled compression and conversion speed, together with optional FTP integration to upload converted files to a remote FTP server. Unlike the "keep" mode, this will also further compress H264/H265 streams. By default, the output is MPEGTS (.ts) with AAC as main audio stream (as in keep mode), but most other common formats are supported - details below.
     
    A flavour of this mode is the Podcast-mode which outputs transcoded files in hierarchical directories organised by Title, providing more than double the compression for audio files with HE-AAC v2, plus inserting comprehensive metadata tags and providing indexing hooks for media servers such as a Squeezeserver, and common media players.
     
  3. In a batch, to convert as many or as few recordings as required, either for keeping in the VDR environment or for single-use.

There is no UI for batch mode as vdr-convert is often used just once in this mode. A text "todo" file is created, and then a small script called batch executes vdr-convert on each file. This allows the user to batch up as many or few recordings as preferred, filter and choose the order.

vdr-convert always outputs exactly ONE multimedia file as this was its original design for convenience. In single-use modes it will create an edit-decision-list(.edl) if a marks file is present to facilitate commercial skipping in players such as mplayer.

A file for single use is named using available VDR recording metadata: TITLE-SUBTITLE-EP-DATE-DOW_TIME-SD/HD/AUDIO.ext.
Example: Rick-Stein's-Long-Weekends-Palermo-EP-0.9-10.-2017-09-23-Sat_12-00-SD.mkv

See Howto and Options for full detail

Player and file compatibility

MPEGTS (.ts) is used as the default output container, as it is the native format now used by VDR, and the most robust one supporting all streams and player combinations. However other containers such as MKV are around 8-10% smaller (not having the mpegts packet overhead) and have other advantages such as supporting metadata tags.

The following containers/formats are supported with the -f option in single-use and batch modes: avi, mkv, mp4, mov, m2ts and ts, and for radio/audio aac, m4a, mkv, mp4, mp3 and ts. Others may work too, depending on your ffmpeg build, but are not tested.

H264, H265 and copy (MPEG2/H262 or higher) video codecs are supported using the -v and -m options, and native AAC, Fraunhofer AAC or libmp3lame(MP3) are supported as audio codecs using the -a option

Tested video players and platforms are shown in the following table for H264 and H265 video codecs.
Note that some containers have design limitations; mp4 and avi don't support DVB subtitles, ts and m2ts don't support usable metadata

In summary, some players are good for almost all formats, and if you don't need subtitles or automatic aspect ratio switching for 4:3 or other aspect ratios, most players will play most formats. The author mainly uses VDR 2.2 with Kodi/Raspberry Pi frontends. For use outside vdr/Kodi, the author recommends .mkv for best balance of size, stream support, metadata and compatibility, and default .ts for best overall compatibility.
vdr-convert format tests

Quality

See Conversion and Workflow for a discussion about conversion parameters, quality and file sizes

Issues / testing

vdr-convert has been run successfully and refined on 2000+ recordings from a variety of UK DVB-T and DVB-T2 (HD 1080i) channels, German DVB-S channels with MP2 and AC3 audio streams, made using VDR versions 1.3x - 1.6x and 2.2 over a period of 10 years. Please note however that the author has a headless VDR server, so native VDR playback onscreen has not yet been tested. Aside from any compatibility issues shown above, the following minor issues exist

  • Simultaneous AC3/DTS audio and DVB subtitles in VDR1.x recordings have yet to be tested, but are logically expected to work. Both AC3 and subtitles have been independently and extensively tested on a variety of recordings. Please contact the author if you have suitable samples !
  • The languages for multiple audio and subs streams of the same type may not be tagged correctly in the output files because ffmpeg does not specify the order that it detects the streams which have been observed varying between files, and so will not match VDR. Currently the language of the first matching stream type (stereo, mono, AD, 5.1, subs etc) is used for all streams of that type. This will not necessarily be correct for recordings with, for example, both German and English stereo streams.
  • Broken VDR1.x recordings comprising more than 1 file do not convert well to MP4 due to timestamp issues - use another format.

Software required

If keeping recordings for use inside a VDR environment, you need VDR 2.2x installed on the conversion machine, otherwise not.

All recordings

  • System running Linux/bash shell
  • Core Linux utilities such as nice, bc, timeout etc
  • FFMPEG 3.x, built with any required libraries (e.g. libx264, libx265, libfdk_aac, libsoxr). Libx264 is a minimum.

Additionally for VDR 1.x recordings

  • Modified GENINDEX (0.2 or later) to convert DVB subtitles to standard ETSI EN300743 format, remove initial short segment sequences that currently prevent ffmpeg reliably detecting subtitle streams, and remove substream headers from AC3/DTS streams for ffmpeg compatibility.
    (GENINDEX is included in this project in the repository: source:genindex)
  • MPLEX13818 from http://www.scara.com/~schirmer/o/mplex13818 to reliably convert recordings into an mpegts container. ffmpeg doesn't handle dvbsubs in a program stream (.vdr native format), and sometimes fails to recognise .vdr files correctly.

Optionally

  • NCFTP client if you want to upload files after conversion

See the source:Readme.txt for details of how to build and install the software components from the repository

Patching

  • You should consider patching ffmpeg per source:Readme.txt, because stock ffmpeg is intolerant of subtitles streams with any DTS timestamp errors. If you have broken recordings - e.g where there was reception interference during recording, or DTS timestamp errors, stock ffmpeg will exit. It should be noted that some second tier broadcasters regularly transmit programmes with DTS problems, probably caused by inserting commercials, so depending on your recordings, patching may be essential. source:FFMPEG patches
  • VDR benefits from a couple of minor patches (source:VDR patches) a) to suppress warnings when reindexing, and, at the time of writing, b) to allow re-indexing of audio-only recordings
  • See the source:Readme.txt for full details

formats.jpg - vdr-convert format tests (111 KB) keynet, 03/24/2017 01:53 PM