Ardour3 Videotimeline

Draft. Work in progress.

This article is about Video-monitoring in the ardour3 Digital Audio Workstation.

About

Synchronous video playback has been available for use with JACK and thus Ardour since ~5 years. Even though it is being used in profession environments there was little to no integration into the Ardour GUI.

The goal of the “video-timeline patch” is to integrate an easily accessible video-track and video-monitor into ardour3 Digital Audio Workstation.

Features

  • video import: save file with session, optionally: extract & import audio from video, scale or transcode video.
  • frame-accurate video-timeline
    • display continuous timeline of video-frames according to current zoom-level
    • read-only: no video editing
    • currently: one video-track only
  • video-monitor window: full-screen or windowed mode, letterbox, timecode OSD
  • video-export dialog: allow to master movies directly muxing A/V from ardour.
  • session-management: remember monitor window settings, retain timeline preferences.
  • customizable timeline-height
  • tested with 23.976, 24, 24.976, 25, 29.97df, 30, 59.94 and 60 fps video-files.
  • video-monitor syncs either to JACK-transport, MTC or to internal ardour-clock (which can slave to MTC), switch happens automatically when changing ardour's clock-source. With internal ardour-transport: moving the playhead directly moves the video along.

History

ardour1: xjadeo only ; CMT-version

ardour2: aicsd - first video-track using CMT-patches.

ardour3: icsd - full integration

ToDo:

  • to be merged upstream
  • –videotimeline to become default.. and use–novideotimeline to explicitly disable it.

see the /tools/videotimline/README file.

future: ardour-NLE ?!

Installation & Setup

nab_07_staging_24.jpg

In order to decode the video, Ardour connects to an external video server. Video-monitoring is provided by [X]Jadeo and transcoding video is done with ffmpeg. The latter two are optional but recommended. The external software is available for OSX and GNU/Linux.

Overview

  • get Ardour3 with video-timeline patch.
  • install icsd (>=alpha-11) and optionally ffmpeg, ffprobe and xjadeo (>= 0.4.12).
  • make sure that there is no firewall on localhost TCP-port 1554 (required for communication between ardour and the video-server).

Ardour will launch the video-server, ffmpeg and [X]Jadeo for you.

Binaries

ardour3 is not yet available as binary. You must compile it from source.

GNU/Linux OSX Win32
ardour3 w/vtl-patch N/A N/A N/A
video-server: icsd here1) here2) here
video-monitor: xjadeo get it from your distributor sourceforge sourceforge
transcoder: ffmpeg get it from your distributor included in icsd 3) ? maybe here?

Source & Build Instructions

Ardour 3

Until the patch gets merged upstream, build-instructions are as follows:

git clone git://rg42.org/ardour3
cd ardour3/
./waf configure --videotimeline
./waf

..OR..

svn co http://subversion.ardour.org/svn/ardour2/branches/3.0 ardour3
cd ardour3/
curl "http://rg42.org/gitweb/?p=ardour3.git;a=commitdiff_plain;hp=master;h=videotl" | patch -p1
./waf configure --videotimeline
./waf

you can either install ardour3 after building it, or launch it directly from the source-folder:

./gtk2_ardour/ardev

see the ardour build instructions for more information.

icsd

The video-server is a bit tricky to compile due to its dependencies (note that there's a static binary available - see above):

git clone git://rg42.org/sodankyla
cd sodankyla
autoreconf 
automake --add-missing
./configure
make src/icsd

If you do not install icsd on your system using sudo make install or by copying the icsd binary into $PATH, Ardour will ask for its location.

Note: on OSX configure it with

CFLAGS="-fnested-functions" ./configure

and don't forget to set PKG_CONFIG_PATH to where the dependent libs can be found.

xjadeo, ffmpeg & ffprobe

The source-code is available from http://xjadeo.sf.net and http://ffmpeg.org.

Please refer to the xjadeo manual and ffmpeg's build-instructions.

Setup

If you run the video-server on the same host as ardour, no further configuration is required. Just make sure there's no firewall on localhost port 1554 (NOTE: on OSX icsd fails to bind to localhost/127.0.0.1 only → you need to choose 0.0.0.0.)

When opening the first video, ardour3 will ask for the path of the video-server binary (unless it is found in $PATH) and also allows you to change the TCP port number.

Transcoding is only available if ffmpeg and ffprobe are found in $PATH (fallback on OSX: /usr/local/bin/ffmpeg_sodankyla - provided with icsd.pkg). The video-monitor likewise requires xjremote in $PATH or /Applications/Jadeo.app/Contents/MacOS/xjremote to be present. xjremote comes with xjadeo and Jadeo respectively.

The video-server can be run on a remote-machine with ardour connecting to it. In this case you need to start-up icsd on the server-machine and configure ardour to access using Menu→Edit→Preferences. You should have a fast network connection (>=100Mbit/s, low latency (switch is better than router) and preferably a dedicated network) between the machine running ardour and the video-server.

While not required for operation, it is handy to share the video-server's document-root filesystem with the machine running ardour. It is however necessary to make use of

  • browsing to a video-file to open
  • displaying a local video-monitor window
  • importing/transcoding video-file to session folder
  • extracting audio from a video-file
  • exporting to video-file

Sharing can be done using any network-file-system (video-files reside on the server, not the ardour workstation) or using NAS. Alternatively a remote replica of the ardour-project-tree that only contains the video-files is an option. The local project folder only needs a copy of the video-file for displaying a video-monitor monitor.

The document root configured in ardour is removed from the local absolute-path to the selected file when making a request to the video-server.

run ./icsd –help

Usage

Getting Started

It is be pretty much self-explanatory.

  • Menu→Session→Open Video
  • Menu→Session→Export→Video
  • Menu→View→Rulers→Video (or right-click the ruler/marker bar → Video)
  • Menu→View→Video Monitor (xjadeo)
  • Menu→Edit→Preferences→Video
  • Menu→Session→Video maintenance→ … (manual video server interaction)

Configuration

Procedures

Beta Testing

API and Internal Details

Video Server Interface

Prototype video-server PHP script

Contact


OLD README file..

 +++ INSTALLATION

When Ardour3 is configured with
   ./waf --videotimeline
it can display a (surprise) video-timeline. It looks like:
   http://rg42.org/_media/wiki/a3vtl3.png

In order to decode the video, Ardour connects to an external video server,
which is currently in the making: git://rg42.org/sodankyla
A statically linked binary is available from
   http://rg42.org/_media/oss/sodankyla/icsd-static-alpha11.tgz
and a prototype-test server PHP-script is described below.

Make sure you don't have a firewall on 'localhost:1554' (default IP/port for
'icsd'). You don't need to install or start 'icsd'. Ardour3 will ask where it
is located (unless it's in $PATH) and start it for you.

If xjadeo (http://xjadeo.sf.net/) is found in the PATH it can be launched
from within Ardour to provide a full-screen video monitor. The option is
presented when opening a video file for the timeline and is later available
from the 'View' Menu. (Note: it works with xjadeo > 0.4.0 but you need a
recent version of xjadeo >= 0.4.12 to get full session-save/restore, also
the non-JACK video-sync mode requires >= 0.4.12 otherwise the reaction-speed
is quite slow.)

The video-server can also be installed on a different machine. In this case
you need to start it yourself and configure its URL in Ardour's Preferences
If you want to use file-selection dialog and xjadeo integration, you should
export the remote-filesystem (eg via NFS ) to the machine where ardour runs on.


 +++ SOURCE AND BUILD INSTRUCTIONS

until it is merged upstream, the build-instructions are as follows:

  git clone git://rg42.org/ardour3
  cd ardour3/
  git checkout -b videotl origin/videotl
  ./waf configure --videotimeline
  ./waf

..OR..

  svn co http://subversion.ardour.org/svn/ardour2/branches/3.0 ardour3
  cd ardour3/
  curl "http://rg42.org/gitweb/?p=ardour3.git;a=commitdiff_plain;hp=master;h=videotl" | patch -p1
  ./waf configure --videotimeline
  ./waf

start Ardour3:
  cd gtk2_ardour/
  ./ardev

or `./wav install` it and launch it the usual way.


 +++ USAGE

It is be pretty much self-explanatory.

Menu->Session->Open Video

Menu->View->Rulers->Video (or right-click the ruler/marker bar -> Video)
Menu->View->Video Monitor (xjadeo)
Menu->Session->Properties->Video (video-server settings)
Menu->Session->Video maintenance-> ... (manual video server interaction)


 +++ PROTOTYPE VIDEO SERVER

There is a prototype PHP script can act as video-server in this folder.

Copy (or link) 'vseq.php' to your document-root (eg /var/www/ on GNU/Linux;
 /Library/Webserver/Documents/ on OSX) so that it can be reached by
   http://localhost/vseq.php
(this is currently the default URL, you can change it in the a3 preferences)

The script requires a recent version of 'ffmpeg' & 'ffprobe'. (ffmpeg.org)
and 'convert' (imagemagick.org). Paths to those executables can be set at
the top of the PHP script.

You can test with a web-browser, fi.

   http://localhost/vseq.php?file=/tmp/tsmm-25.avi&mode=info
   http://localhost/vseq.php?file=/tmp/tsmm-29.97df.avi&frame=10&format=png

where file=<absolute-path-to-a-video-file>

NOTE that this PHP script-solution is veeeeerrrryy slooooow!  It does not
caches frames, and even opens the video-file twice for each decoded frame.
Furthermore the PHP/Web user who runs the script must be have read-access
to the video-file(s).


 +++ DETAILS

The video-server understands the following URL request parameters:

  file=<FILE-NAME> # needs to be readable by the PHP script.
  frame=<frame-number>
  w=<width in pixels>     # default 80
  h=<height in pixels>    # default 60
  format=<output-format>  # default "rgb" ; possible: "png", "jp[e]g", "rgb[a]"

  /vseq.php/info?..
              # special mode which uses file=XXX and
              # returns information about the file separated by a newline '\n'
              #
              # Protocol Version number (currently 1)
              # FPS (float)
              # duration (in frames)
              # start-offset (in seconds)
              # aspect-ratio

   /vseq.php/status?..
              # special mode which returns server status info.


For testing, files with the time-code rendered on each frame
eg. video-frame number 0 displays the text "Frame: 0" and "00:00:00:00"
in various formats/codecs are available from:

  http://tux.gareus.org/torrents/tsmm_mpeg4-mov_10mins.torrent
  http://tux.gareus.org/torrents/tsmm_mjpeg-avi_10mins.torrent
  http://tux.gareus.org/torrents/tsmm_h264-avi_10mins.torrent
  http://tux.gareus.org/torrents/tsmm_h264-mov_10mins.torrent

each torrent contains files with framerates
 23.976, 24, 24.976, 25, 29.97df, 30, 59.94 and 60 fps.


The video-server already works but is changing rapidly and
somewhat tricky to compile due to dependencies:
 - libjpeg, libpng, libav*/libswscale, libltcsmpte (http://ltcsmpte.sf.net/)
 - optionally: libsqlite3, liblo, libjack, SDL, openGL, gavl/gmerlin

  git clone git://rg42.org/sodankyla
  cd sodankyla
  autoreconf
  automake --add-missing
  ./configure
  make src/icsd
  ./src/icsd [-C 512] <docroot>

on OSX configure it with
  CFLAGS="-fnested-functions" ./configure
and don't forget to set the PKG_CONFIG_PATH :)

Use '/' as docroot or configure it in Ardour3: Edit->Preferences->Video
The server URL is 'http://localhost:1554/' and the docroot '/' (unless you
override these defaults with icsd command line args).

NOTE: Make sure to add the trailing slash for a folder
   eg. use '/tmp/' - not '/tmp'
as document-root in both Ardour preferences and the setting for icsd.


 +++ CONTACT

ardour-dev@lists.ardour.org
robin@gareus.org
1) 32-bit LSB, i386
2) Universal binary i386, ppc x86_64
3) Note: The package can co-exist with other ffmpeg installations: The ffmpeg binaries are called ffprobe_sodankyla and ffmpeg_sodankyla and the libs installed under /usr/local/lib/sodankyla/
 
wiki/a3vtl.txt · Last modified: 21.08.2012 17:04 by rgareus