Project

General

Profile

Howto

To use after automatically after a recording

Use the provided vdr-auto script, and edit your VDR command line to include

-r /usr/local/bin/vdr-auto

Recordings are automatically converted in "keep" mode when the recording ends, if the programmed "lifetime" is >= value set on the command line with -l option (default 8 days). vdr-auto includes a call to noad

To use as a batch (batch.sh)

Create a todo.txt file as follows at whatever level you like in your VDR recording file hierarchy.

find ./ -type d | grep -i '.rec$' |sort > ./todo.txt

Edit the resulting file to suit, and then run it as a batch in the directory containing the todo.txt.
nohup allows the process to continue when you logout.

nohup batch.sh -k -d &

Convert a recording for single use

A normal conversion for use on a portable player in Matroska format

vdr-convert -f mkv -i "recording.rec" 

Example of output: More or Less-EP-EP013183250028-2016-07-29-Fri_16-30-AUDIO.mkv

In this example, a radio recording is converted to mp3. vdr-convert forces the use of the libmp3lame codec if mp3 format is selected

vdr-convert -f mp3 -i "recording.rec" 

Podcast mode

Here a radio recording is converted to a tagged, high quality, highly compressed HE-AAC v2 audio podcast, with original files deleted after successful conversion. The file is put in a subdirectory using the recording title below

vdr-convert -d --podcast /mnt/lvm/Radio -i "recording.rec" 

Example of output: /mnt/lvm/Radio/More or Less/More or Less-EP-EP013183250031-2018-08-31-Fri_16-30.m4a

See Podcast-mode for more description

A full description of Options is provided.

Logging

The script provides summary information to your syslog (example below), and detailed ffmpeg output to /var/log/vdr-convert

Aug 22 09:05:49 ha-server vdr-convert: Processing recording "./From_Dusk_till_Dawn/2009-01-06.23.40.50.99.rec", (files=2) keep=1, delete=1, combine=1, subs=1, extras=-tune film
Aug 22 09:06:35 ha-server vdr-convert: Processing ./From_Dusk_till_Dawn/2009-01-06.23.40.50.99.rec/recording.orig file 1, size 2669M...
Aug 22 09:07:23 ha-server vdr-convert: ./From_Dusk_till_Dawn/2009-01-06.23.40.50.99.rec/recording.orig genindex extracted 4178 kb subtitle data
Aug 22 09:11:24 ha-server vdr-convert: Executing nice -n 19 /opt/data/develop/FFmpeg/ffmpeg -y ...
Aug 22 10:22:21 ha-server vdr-convert: Executing nice -n 19 /opt/data/develop/FFmpeg/ffmpeg -y ...
Aug 22 10:22:52 ha-server vdr-convert: Created/replaced .ts and indexed From Dusk till Dawn-2009-01-06-Tue_23-40-SD (2669M to 1612M (60.4%))

Usage hints

Batch process files with similar requirements. The --extras (-e) option allows you to pass a set of arbitrary parameters to ffmpeg / codec libraries to optimise certain conversion types.

For example if you have a batch of films, you may want to use the -e "-tune film" flag to use "film" tuning with x264. An example command might be:

nohup batch.sh -k -d -e "-tune film" &

As an example of the use of multiple parameters, the author once used "-tune film -aspect 16:9" where the broadcaster had transmitted the first part of the film incorrectly squashed to 4:3 format - when corrected the elongated logo proved the error!

In practice, with default settings and -e "-tune film", the size of a batch of SD films is usually around 2/3 of the original size and indistinguishable in terms of quality - at least to the author's eyes and ears.

Default settings are quite conservative, aiming for quality over size reduction. If you want to save more space, test by increasing the -q value until you notice a drop in quality, note that the default value for keeping is significantly better than ffmpeg default of 23 (which is the value used in single-use mode)

vdr-convert -k -q 23 -i "recording.rec" 

Deinterlacing

x264 produces slightly more compression on deinterlaced content, but almost all broadcast TV content is still interlaced today, even HD. Deinterlacing loses some temporal information, and however good, will produce some artefacts - there is a trade-off between saving space (a few percent), and visual quality. The particular issue with SD material is that it is likely to be upscaled to a larger screen than it's native resolution, any deinterlacing done during conversion at the recording's native resolution cannot take advantage of the ability of a modern TV or media player to interpolate/smooth several lines at it's higher resolution, producing smoother results. Yadif deinterlacing by vdr-convert/ffmpeg although good, can lead to a few more visible artefacts on moving parts of scenes, particularly where there are sharp near-horizontal or vertical lines, moving foliage or on sports events.

For best quality results, the author therefore recommends that SD content is left interlaced and then deinterlaced by the final TV or player at it's higher resolution. This is also more future-proof as once deinterlaced, there is no viable way back. The TV / player (usually automatic), or end-user can decide what type of deinterlacing to apply for best results.

vdr-convert provides an option to enable the yadif deinterlacer if required. The most efficient conversion occurs with y=0. y=1 makes the yadif deinterlacer create 1 frame per field which can improve perceived quality on noisy or fast moving recordings, with perhaps a 20% size penalty.

H265 (HEVC)

x265 has been developed from x264 and shares many features, and is supported as long as ffmpeg has been built with x265 library support. It can produce files of half the size of H264, and in the author's opinion, the quality has recently improved(2018) and nearly matches H264's output. It is constantly improving, so worth trying with each new release, meanwhile it may be better just for HD content with a fixed aspect ratio.

Some conservative settings have been provided for x265, and it's probably best used in the single-use mode if maximum compression is needed.

Important notes:

  • Conversion with the slow preset is extremely slow, esp. on HD content. It could be less than 10% of real time, even with a reasonable CPU.
  • H265 is currently not useful for any content that has aspect ratio changes, because of ffmpeg/libx265 bug https://trac.ffmpeg.org/ticket/5959. The aspect ratio remains the same from the beginning of the recording, unlike H264 which changes aspect as content aspect changes. This is particularly noticeable on recordings of 4:3 material with 16:9 commercials inserted.

VDR1.x recordings

VDR1.x recordings are automatically detected and can usually best be transcoded using the -c option to combine/concatenate files before conversion at the cost of an additional temporary copy of the whole recording during conversion where multiple files are present. This produces a seamless transition at any joins, avoids ffmpeg potentially recognising streams differently between multiple files 001.vdr, 002.vdr..., and just occasionally where a transition between files coincides with a broadcast program change (obviously a rare occurrence). This is a highly recommended option for these recordings, and can safely be used on VDR2.x .ts recordings also. The reason -c is not switched on by default is that broken recordings (interference etc) can fail when combined with -c

Note that VDR1.x conversions are disk intensive when they start. Several copy/conversion/test stages were found essential to achieve reliable conversions with synchronised streams, as documented in Workflow. Reducing the processing stages during development by, for example, having ffmpeg create main and subs files simultaneously lead to failures. Most processes are nice'd to the lowest priority, but ffmpeg in particular will consume all spare CPU resource. In practice this does not interfere noticeably with normal operation of a server - the author runs the script 24x7 on a home automation server that has many real-time processes running, such as analog CCTV capture, and VDR itself of course.

Remember to patch and rebuild your own copy of ffmpeg per the readme if you are having issues with conversions exiting due to noise or subtitle issues. You can test using the -n option to disable subtitles to verify if the cause is subs.

Keep streams as MPEG2 / move to VDR2.x ts format and directory structure

If you just want to recover subtitles and/or AC3 audio and/or move to the current VDR2.x directory structure with .ts files that can also be played on a wide range of players, Kodi, MPC-HC and VLC etc, you can use this command which will take only a minute or two to convert each recording. It (losslessly) copies the MPEG2 video and audio streams and muxes in the recovered DVB subtitles if available. In this case the recording will be about 7% larger than the original due to mpegts container overhead.

vdr-convert -c -k -m -d -i "recording.rec" 

Troublesome recordings

vdr-convert does a number of tests on the transcoded recording; including errors reported by ffmpeg and other tools, file sizes, duration, and VDR reindex when using the -k option. If any fail, it exits, leaving the transcoded file in the recording directory, as well as originals. However, you can normally expect a 98%+ success rate.

Subtitles

Subtitles can cause problems for ffmpeg, often seen as early exit and shortened files (vdr-convert usually detects and warns you of errors or size issues). Check your log for the word "fail" and/or configure email notification. vdr-convert tests subs before conversion, and uses several ffmpeg timestamp options and other strategies to workaround common issues but cannot always manage broken files, which are almost always caused by interference, breaks in the recording, or very occasionally the broadcaster outside spec. In the author's experience significantly fewer than 1% of recordings might suffer this problem. If you have recordings with major DTS timestamp problems, you have a few options:

  1. use the -n option and do not attempt to transcode subtitles at all.
  2. use -m option to copy MPEG2 streams, but still extract subtitles, albeit with size penalty. This may or may not work depending on how bad the problems are.
  3. Finally it is usually possible to manually recover and recombine subs from a broken recording, the conversion order is different for VDR1 and 2, as follows...
    VDR1
    First, transcode using the -c and -n options (do not use -d option), which almost always works OK, leaving you with 00001.ts containing just complete transcoded AV streams. Then use the -j option to create just a subs.ts in the temp directory below the recording.
    VDR2
    First, use the -j option to quickly create just a subs.ts in the recording directory. Then transcode using the -c and -n options (no -d option), which almost always works OK, leaving you with 00001.ts containing just complete transcoded AV streams.
    VDR1 and VDR2
    Now manually combine the AV file 00001.ts and subs.ts (quick), keeping originals, remember to use your patched ffmpeg, otherwise it's likely to fail again:
    /path-to-your-ffmpeg -y -probesize 250M -analyzeduration 600M -copytb 1 -i 00001.ts -i subs.ts -c:s copy -c:v copy -c:a copy -map 0 -map 1 temp.ts
    

    Watch temp.ts (MPC-HC or SMPlayer with mpv engine recommended), subs timing is likely to be way out of sync with the audio/video, several seconds to several minutes, so manually time the difference between subs and audio/video (e.g. xx seconds). Apply this time difference in a quick remux:
    /path-to-your-ffmpeg -y -probesize 250M -analyzeduration 600M -copytb 1 -i 00001.ts -itsoffset 00:00:xx -i subs.ts -c:s copy -c:v copy -c:a copy -map 0 -map 1 temp.ts
    

    Re-check temp.ts for subs sync, replace as 00001.ts when correct.
    The author has successfully used this method on a handful of files which failed during normal conversion.

Other issues

If you have breaks in recordings leaving short fragments, perhaps less than 1MB, the -c (combine) option is useful to join them up (ffmpeg doesn't like very short fragments because its stream probing routines need a significant amount of material to correctly identify all stream types). If the fragments are at the start or end of the recording and not part of the programme, it's safer to just delete them before processing.

If a recording fails, you should first view the output file using one of the listed compatible players, as it may in fact be OK, having failed tests due to, for example, interference effects or breaks during the original recording which can alter apparent duration. If OK, then you can rename the file and delete the originals. Otherwise use one of the strategies above.

Very occasionally the transcoded recording can be larger than the original - this is notified as a fail, and originals retained. Extremely grainy films or film-based programmes can suffer this problem. In this case you can increase the compression with the -q option, revert to the original, or if VDR1.x recording, perhaps best to use the -m option to recover the subtitles but retain the original MPEG2 A/V streams.

ffmpeg has bugs! The author has logged several, https://trac.ffmpeg.org/ticket/5617 and https://trac.ffmpeg.org/ticket/5796 are most relevant at the time of writing. A very small percentage (<0.5%) of tested recordings simply can't be converted with subs. The ffmpeg patch in this project relates to #5617 and helps with poor recordings