Archive for the ‘Code’ Category

Network speed test

Tuesday, July 14th, 2009

I currently have the need to test the connection speed between two remote machines, so that I know when it is safe to transfer a large file from one to the other without significantly affecting service by causing long periods of downtime. I couldn’t find any software that did this easily, ran as a single executable (no installation) and was free, so I made one.

This currently only tests speed in one direction (upload from the client to the server) because I have symmetric connections on both machines, so either way should be exactly the same, and so that I can just write a client program and use netcat as a server to accept the packets and dump them to /dev/null. It is simple enough to convert this client into a server so that netcat becomes a client or so that they can be used together. Using netcat as a client probably requires quite a large pre-generated random file and may introduce a bottleneck of the hard drive read speed.

This could also be used in the home for something like testing the effect of wifi strength.

The client takes the command line arguments, connects to the server, generates some random data and sends that data repeatedly until the timer expires. It then prints out the results.

The command to use netcat to listen and ignore any data it receives is nc -lp 9000 > /dev/null though the -p isn’t always needed (it was needed on my Debian box but not my ESX 4 box).

An example of use of this program as a client is:

C:\>SpeedTestClient.exe 192.168.1.157 9000
Uploaded 1181614080 bytes in 10.015 seconds.
943875.45 kbps (943.88 mbps)
115219.17 kB/s (112.52 MB/s)

The results are given in kilobits per second and megabits per second (standard measurements for network speed), kilobytes per second and megabytes per second (more useful for knowing how long a file will take to transfer). In this example, a gigabit ethernet connection is getting roughly 944 mbps.

A few seconds later, VLC 1.0 is opened and paused, so nothing is playing and the test is run again. The results are quite interesting:

Uploaded 225083392 bytes in 10.015 seconds.
179797.02 kbps (179.80 mbps)
 21947.88 kB/s ( 21.43 MB/s)

It says it is using next to no system resources when paused but it slows data transfer dramatically and this is with 8GiB of RAM, an otherwise idle CPU and no hard drives being used. When hard drives enter the equation this drops down to about 8MB/s (while VLC is still paused – I have particularly fast hard drives so I still get about 110MB/s when hard drives are used and VLC is turned off).

The code listing is as follows:

using System;
using System.IO;
using System.Net;
using System.Net.Sockets;
using System.Security.Cryptography;
 
namespace SpeedTest
{
    class SpeedTestClient
    {
        static int Main(string[] args)
        {
            if (args.Length < 2)
            {
                Console.WriteLine("Usage: SpeedTestClient <ip> <port> [<seconds>]");
                return 1;
            }
 
            try
            {
                ulong bytes = 0;
                int startTime = Environment.TickCount;
                bool started = false;
                int timeout = 10;
 
                // Get the optional number of seconds argument
                if (args.Length > 2) timeout = int.Parse(args[2]);
 
                // Connect to the server via TCP
                TcpClient client = new TcpClient();
                client.Connect(IPAddress.Parse(args[0]), int.Parse(args[1]));
                NetworkStream stream = client.GetStream();
 
                // Generate some random data. Buffers too small will bottleneck.
                RandomNumberGenerator rng = RandomNumberGenerator.Create();
                byte[] buffer = new byte[32768];
                rng.GetBytes(buffer);
 
                while (true)
                {
                    // Write the data
                    stream.Write(buffer, 0, buffer.Length);
 
                    // Start the timer only after some data has been sent
                    if (!started)
                    {
                        startTime = Environment.TickCount;
                        started = true;
                    }
 
                    // Add to the bytes counter
                    bytes += (ulong)buffer.LongLength;
 
                    if (Environment.TickCount - startTime >= timeout * 1000) break;
                }
 
                PrintFinalStats(startTime, Environment.TickCount, bytes);
 
                try { client.Close(); }
                catch (IOException) { /* Do nothing */ }
            }
            catch (Exception ex)
            {
                Console.Error.WriteLine(ex);
                return 1;
            }
 
            return 0;
        }
 
        static void PrintFinalStats(int start, int end, ulong bytes)
        {
            double seconds = (end - start) / 1000.0;
            ulong bits = bytes * 8;
 
            double bitsPerSecond = bits / seconds;
            double bytesPerSecond = bytes / seconds;
 
            string kbps = string.Format("{0:0.00}", bitsPerSecond / 1000);
            string mbps = string.Format("{0:0.00}", bitsPerSecond / 1000 / 1000);
            string kibs = string.Format("{0:0.00}", bytesPerSecond / 1024);
            string mibs = string.Format("{0:0.00}", bytesPerSecond / 1024 / 1024);
 
            int kPad = Math.Max(kbps.Length, kibs.Length);
            int mPad = Math.Max(mbps.Length, mibs.Length);
 
            Console.WriteLine("Uploaded {0} bytes in {1} seconds.", bytes, seconds);
            Console.WriteLine("{0} kbps ({1} mbps)", kbps.PadLeft(kPad), mbps.PadLeft(mPad));
            Console.WriteLine("{0} kB/s ({1} MB/s)", kibs.PadLeft(kPad), mibs.PadLeft(mPad));
        }
    }
}

A compiled .net executable is available here.

P.S. always remember to turn off VLC before transferring files over the network.

How to set target file sizes in ffmpeg

Tuesday, June 16th, 2009

This script is now available as an ongoing project that can be downloaded. Please see the Video Encoding Project. This post describes the methods and mathematics behind the script.


After much searching, I found almost nothing regarding how to target a specific file size with ffmpeg, such as 700MiB to fit on a CD. Almost everything I found suggested to use a negative bitrate for mencoder, such as -700000. However, this did not work for me at all (even by copying and pasting verbatim). Because of this, I set about doing it manually with some mathematics.

What I am going to do here assumes a single audio stream, a single video stream and nothing else in the output file at all (i.e. no subtitles, no alternate languages, no director’s commentaries, etc).

When encoding media, audio uses a constant bit rate. This means that you can calculate how much space it will take if you know which bit rate you are using and how long the stream is in seconds. This also means that to know how big the video stream should be, we can calculate how big the audio stream will be and subtract it from the target file size. For example, if the target file size is 350MiB and the audio will take up 30MiB, we know the video should aim to be 320MiB (ignoring overheads in the multiplexing, container headers, etc). We can use the same calculations in reverse to work out an average or constant bit rate from a file size and a length in seconds.

In this example, I will be using Dad’s Army Series 6 Episode 1 – The Deadly Attachment. I have already ripped this from the DVD with SlySoft AnyDVD and PgcDemux on Windows. Automating PgcDemux rips is worth a whole entry in itself. Note that I have bought this legally and I do not intend to sell or share it. I just want a copy that I can store on my network and play from any computer. Storing 14 DVD images on hard drives takes quite a lot of space, which is where encoding comes in handy. In fact, some of my box sets contain over 60 DVDs (roughly 270GiB).

———-
To install ffmpeg on Debian, it is best to get it from the Debian Multimedia repository. Add

deb http://www.debian-multimedia.org lenny main
deb-src http://www.debian-multimedia.org lenny main

to your /etc/apt/sources.list and then run

# apt-get update
# apt-get install debian-multimedia-keyring
# apt-get update
# apt-get install ffmpeg

———-

The first thing we need to do to put this into practice is to get the length of the stream. If you supply the file to ffmpeg without telling it to do anything, it will give you an error. However, this error already contains all of the information we need:

$ ffmpeg -i da_6_01.VOB
FFmpeg version SVN-r13582, Copyright (c) 2000-2008 Fabrice Bellard, et al.
  configuration: --prefix=/usr ...
  built on May  3 2009 12:07:18, gcc: 4.3.2
Input #0, mpeg, from 'da_6_01.VOB':
  Duration: 00:29:23.90, start: 0.287267, bitrate: 5987 kb/s
    Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 720x576 [PAR 16:15 DAR 4:3], 9800 kb/s, 25.00 tb(r)
    Stream #0.1[0x80]: Audio: ac3, 48000 Hz, stereo, 192 kb/s
Must supply at least one output file

We can see that the duration is 00:29:23.90.

As this is an error, we need to redirect stderr to stdout. This can be done with 2>&1. We then need the line that contains the “Duration”, which we can get with grep. We can cut this line up to take the time out.

$ ffmpeg -i da_6_01.VOB 2>&1 | grep Duration | cut -d ' ' -f 4 | cut -d '.' -f 1
00:29:23

This can be converted into seconds with the formula s' = (h * 60 + m) * 60 + s.

I used a python script to do this and to perform the necessary calculations mentioned earlier. Note that most of this python script is just for the convenience of getopts and that, if needed, it can be cut down significantly at the cost of flexibility. If you always want the same output size and audio bitrate, this script can be simplified to just a couple of lines.

calc_bitrate.py:

#!/usr/bin/python
 
from getopt import getopt, GetoptError
from sys import argv, exit
 
def secs(h, m, s):
  return (h * 60 + m) * 60 + s
 
def calc_video_bitrate(target_size, abr, time):
  target_bytes = target_size * 1024 * 1024
  audio_bytes = ((abr * 1000) / 8) * time
  return (target_bytes - audio_bytes) / time * 8
 
try:
  opts, args = getopt(argv[1:], "s:a:t:", ["size=", "abr=", "time="])
except GetoptError, err:
  print str(err)
  exit(2)
 
target_size = 350 #MiB
abr = 128 #kbps
time = 0
 
for o, a in opts:
  if o in ("-s", "--size"):
    target_size = int(a)
  elif o in ("-a", "--abr"):
    abr = int(a)
 
if len(args) < 1:
  print "Usage: " + argv[0] + " [-s <size (M)>] [-a <audio bit rate (k)>] [[hh:]mm:]ss"
  exit(2)
 
time_part = args[0].split(":")
if len(time_part) < 2: time_part = [0] + time_part
if len(time_part) < 3: time_part = [0] + time_part
time = secs(int(time_part[0]), int(time_part[1]), int(time_part[2]))
 
print "%d" % calc_video_bitrate(target_size, abr, time)

A typical usage for this would be ./calc_python.py -s 350 -a 128 29:23.

We can then use a combination of these inside a shell script easily. I’m using bash syntax just to nest the commands into a single line but for compatibility with other shells, backticks (`) can be used and this is what I normally prefer. This script loops through all of the .VOB files in the directory and gives their target video bitrate for a 350MiB output and 128kbps audio.

#!/bin/bash
 
for vob in *.VOB
do
  BR=$(./calc_bitrate.py -s 350 -a 128 $(ffmpeg -i $vob 2>&1 | grep Duration | cut -d ' ' -f 4 | cut -d '.' -f 1))
  echo $vob $BR
done

I use this in 2-pass VBR mode with the libx264 video encoder and libfaac audio encoder in an mp4 container just by replacing the “echo” line of the above script with the following 2 lines:

ffmpeg -i $vob -an -pass 1 -vcodec libx264 -vpre fastfirstpass -b $BR -bt $BR -deinterlace -threads 0 -y pass1.mp4
ffmpeg -i $vob -acodec libfaac -ab 128k -ac 2 -pass 2 -vcodec libx264 -vpre hq -b $BR -bt $BR -deinterlace -threads 0 -y `echo $vob | sed 's/VOB/mp4/'`

I have 2 versions of this script, one with “-deinterlace” and one without. PAL video (576i – used in Europe), such as Dad’s Army needs deinterlacing. NTSC video (480 – used in North America) usually does not.

For ripping DVDs to H.264 and maintaining the quality, I wouldn’t use a bitrate lower than 1000k or an audio bitrate lower than 128k stereo. This means that for getting media onto a CD, you probably want no smaller than 350MiB for 30 to 40 minutes or 700MiB for 1 hour or more, depending on your resolution. Also note that I only intend to play these in computers and fitting them onto CDs is just a convenience – standalone DVD players will not be able to play H.264 or AAC (Blu-ray and HDDVD players may be able to – mp4 containers and reading CDs would be more of an issue).

Update:

I have successfully tested this in a LG BD370 Blu-ray player.

I have not tried actually burning these onto CDs yet so if 2 won’t fit on a 700MiB CD, even with overburn (because of overheads that were ignored, as stated earlier) then the simplest solution is to just target a smaller file size with something like “-s 348″.

The result

Dad’s Army episodes are not all of the same length. Some are several minutes longer than others. The following shows how no matter the size of the input file, they all produce the same size output file:

$ ll da_5_*.VOB
-r--r--r-- 1 mythtv mythtv 1364660224 2009-06-04 00:01 da_5_01.VOB
-r--r--r-- 1 mythtv mythtv 1360699392 2009-06-04 00:01 da_5_02.VOB
-r--r--r-- 1 mythtv mythtv  956084224 2009-06-04 00:02 da_5_03.VOB
-r--r--r-- 1 mythtv mythtv 1355888640 2009-06-04 00:02 da_5_04.VOB
-r--r--r-- 1 mythtv mythtv 1114351616 2009-06-03 23:59 da_5_05.VOB
-r--r--r-- 1 mythtv mythtv 1034289152 2009-06-04 00:00 da_5_06.VOB
-r--r--r-- 1 mythtv mythtv 1045202944 2009-06-04 00:00 da_5_07.VOB
-r--r--r-- 1 mythtv mythtv 1345318912 2009-06-04 00:00 da_5_08.VOB
-r--r--r-- 1 mythtv mythtv 1391529984 2009-06-04 00:02 da_5_09.VOB
-r--r--r-- 1 mythtv mythtv 1342748672 2009-06-04 00:03 da_5_10.VOB
-r--r--r-- 1 mythtv mythtv 1158017024 2009-06-04 00:03 da_5_11.VOB
-r--r--r-- 1 mythtv mythtv 1157097472 2009-06-04 00:03 da_5_12.VOB
-r--r--r-- 1 mythtv mythtv 1180098560 2009-06-04 00:04 da_5_13.VOB
$ ll -h da_5_*.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 06:18 da_5_01.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 06:55 da_5_02.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 07:30 da_5_03.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 08:08 da_5_04.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 08:44 da_5_05.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 09:21 da_5_06.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 09:58 da_5_07.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 10:36 da_5_08.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 11:15 da_5_09.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 11:54 da_5_10.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 12:29 da_5_11.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 13:05 da_5_12.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 13:41 da_5_13.mp4

Update:

A pure python replacement script is available here that simplifies use.

Pixel theme

Saturday, June 13th, 2009

I’ve just switched to the Pixel WordPress theme. It’s quite nice but it does have some flaws. Images no longer auto-size so some are too large. I have had to manually resize the widest offenders.

Another problem is that it forces you to have a crappy redundant “Welcome” message. I have removed this in the style.css and, while I was at it, I made the post content justified. If anyone else wants to do the same thing (or I want to repeat this modification after a WordPress update), the following CSS is simply appended to the theme’s style.css:

#welcome { display: none; }
.topContent { text-align: justify; }

There is no logical equivalent to conditional statements!

Monday, May 18th, 2009

For a long time, I have seen chunks of code in languages such as Lua and Python that claim that they can reproduce the C conditional operator just by using two logical operators, “and” and “or”.

The conditional statement in C uses the following format:

v = c ? t : f;

This is equivalent to saying “if the condition c (a boolean expression) is met assign t to v, otherwise assign f to v”. It’s a shorthand way of writing

if (c)
  v = t;
else
  v = f;

Some example outputs:

v = 1 ? "foo" : "bar"; /* v = "foo" */
v = 0 ? "foo" : "bar"; /* v = "bar" */
v = 1 ? 0 : 2;         /* v = 0 */

In the first example, the condition is true (C has no boolean type, anything that isn’t zero equates to true) so “t” is assigned. In the second example, the condition is false (zero) so “f” is assigned. The important thing to note here is that in the third example, the condition is true so “t” is still assigned.

As proof of this final value, the following was taken from a cgwin shell using the gcc compiler

~$ gcc --version
gcc (GCC) 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)
Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 
~$ cat cond.c
#include <stdio.h>
 
int main(void)
{
  printf("The result is %d!\n", 1 ? 0 : 2);
 
  return 0;
}
~$ gcc -Wall -o cond cond.c
~$ ./cond
The result is 0!
~$

The following is an implementation in Python that always gives the same behaviour as the conditional operator in C.

def cond_if(c, t, f):
  if c: return t
  else: return f

We can show the same examples again:

>>> cond_if(True, "foo", "bar")
'foo'
>>> cond_if(False, "foo", "bar")
'bar'
>>> cond_if(True, 0, 2)
0

However, many people claim that the same code can be written more concisely and, more importantly, inline by using logical operators to mimic the behaviour. The following is such a function:

def cond_logic(c, t, f):
  return c and t or f

When we try to actually run this, with the same examples again, we see where it all falls down.

>>> cond_logic(True, "foo", "bar")
'foo'
>>> cond_logic(False, "foo", "bar")
'bar'
>>> cond_logic(True, 0, 2)
2

While it works for the first two examples as it should, any pair of values which can be equated to a boolean expression will corrupt the logic used. In this example, simply passing zero as the “t” makes the result of “c and t” false (because false and _ = false), which reduces it to “false or f”, which returns “f” (because false or _ = _). No combination of parenthesis or variables will correct this problem.

Here’s the final example again but this time in Lua:

> function cond_logic(c, t, f)
>>   return c and t or f;
>> end
 
> = cond_logic(1, false, 2);
2

So, the next time someone tells you that your code can be improved in this way or that logical operators can be used to mimic the conditional operator, tell them that they are wrong and give them an example to prove it, then stick to using if statements if the language does not provide a conditional operator.

libvlc media player in C# (part 2)

Friday, May 8th, 2009

I gave some simplified VLC media player code in part 1 to show how easy it was to do and how most wrapper libraries make a mountain out of a mole hill. In that entry, I briefly touched on using some classes to make it easier and safer to implement actual programs with this.

The first thing to do is write a wrapper for the exceptions, so that they are handled nicely in C#. For a program using the library, exceptions should be completely transparent and should be handled in the normal try/catch blocks without having to do anything like initialise them or check them.

Another thing to do is to move all of the initialisation functions into constructors and all of the release functions into destuctors or use the System.IDisposable interface.

Here is the code listing for the 4 classes used (VlcInstance, VlcMedia, VlcMediaPlayer and VlcException). Note that the first 3 of these are very similar and that the main difference is that the media player class has some extra functions for doing things like playing and pausing the content.

class VlcInstance : IDisposable
{
    internal IntPtr Handle;
 
    public VlcInstance(string[] args)
    {
        VlcException ex = new VlcException();
        Handle = LibVlc.libvlc_new(args.Length, args, ref ex.Ex);
        if (ex.IsRaised) throw ex;
    }
 
    public void Dispose()
    {
        LibVlc.libvlc_release(Handle);
    }
}
 
class VlcMedia : IDisposable
{
    internal IntPtr Handle;
 
    public VlcMedia(VlcInstance instance, string url)
    {
        VlcException ex = new VlcException();
        Handle = LibVlc.libvlc_media_new(instance.Handle, url, ref ex.Ex);
        if (ex.IsRaised) throw ex;
    }
 
    public void Dispose()
    {
        LibVlc.libvlc_media_release(Handle);
    }
}
 
class VlcMediaPlayer : IDisposable
{
    internal IntPtr Handle;
    private IntPtr drawable;
    private bool playing, paused;
 
    public VlcMediaPlayer(VlcMedia media)
    {
        VlcException ex = new VlcException();
        Handle = LibVlc.libvlc_media_player_new_from_media(media.Handle, ref ex.Ex);
        if (ex.IsRaised) throw ex;
    }
 
    public void Dispose()
    {
        LibVlc.libvlc_media_player_release(Handle);
    }
 
    public IntPtr Drawable
    {
        get
        {
            return drawable;
        }
        set
        {
            VlcException ex = new VlcException();
            LibVlc.libvlc_media_player_set_drawable(Handle, value, ref ex.Ex);
            if (ex.IsRaised) throw ex;
            drawable = value;
        }
    }
 
    public bool IsPlaying { get { return playing && !paused; } }
 
    public bool IsPaused { get { return playing && paused; } }
 
    public bool IsStopped { get { return !playing; } }
 
    public void Play()
    {
        VlcException ex = new VlcException();
        LibVlc.libvlc_media_player_play(Handle, ref ex.Ex);
        if (ex.IsRaised) throw ex;
 
        playing = true;
        paused = false;
    }
 
    public void Pause()
    {
        VlcException ex = new VlcException();
        LibVlc.libvlc_media_player_pause(Handle, ref ex.Ex);
        if (ex.IsRaised) throw ex;
 
        if (playing)
            paused ^= true;
    }
 
    public void Stop()
    {
        VlcException ex = new VlcException();
        LibVlc.libvlc_media_player_stop(Handle, ref ex.Ex);
        if (ex.IsRaised) throw ex;
 
        playing = false;
        paused = false;
    }
}
 
class VlcException : Exception
{
    internal libvlc_exception_t Ex;
 
    public VlcException() : base()
    {
        Ex = new libvlc_exception_t();
        LibVlc.libvlc_exception_init(ref Ex);
    }
 
    public bool IsRaised { get { return LibVlc.libvlc_exception_raised(ref Ex) != 0; } }
 
    public override string Message { get { return LibVlc.libvlc_exception_get_message(ref Ex); } }
}

Using these classes is even easier than before, can use proper exception handling (removed for brevity) and cleans up better at the end. In this example, I have added an OpenFileDialog, which is where the file is loaded.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
 
namespace MyLibVLC
{
    public partial class Form1 : Form
    {
        VlcInstance instance;
        VlcMediaPlayer player;
 
        public Form1()
        {
            InitializeComponent();
 
            openFileDialog1.FileName = "";
            openFileDialog1.Filter = "MPEG|*.mpg|AVI|*.avi|All|*.*";
 
            string[] args = new string[] {
                "-I", "dummy", "--ignore-config",
                @"--plugin-path=C:\Program Files (x86)\VideoLAN\VLC\plugins",
                "--vout-filter=deinterlace", "--deinterlace-mode=blend"
            };
 
            instance = new VlcInstance(args);
            player = null;
        }
 
        private void Form1_FormClosed(object sender, FormClosedEventArgs e)
        {
            if(player != null) player.Dispose();
            instance.Dispose();
        }
 
        private void Open_Click(object sender, EventArgs e)
        {
            if (openFileDialog1.ShowDialog() != DialogResult.OK)
                return;
 
            using (VlcMedia media = new VlcMedia(instance, openFileDialog1.FileName))
            {
                if (player != null) player.Dispose();
                player = new VlcMediaPlayer(media);
            }
 
            player.Drawable = panel1.Handle;
        }
 
        private void Play_Click(object sender, EventArgs e)
        {
            player.Play();
        }
 
        private void Pause_Click(object sender, EventArgs e)
        {
            player.Pause();
        }
 
        private void Stop_Click(object sender, EventArgs e)
        {
            player.Stop();
        }
    }
}

Update:

I have just corrected a minor bug (the wrong release function being called on the player handle) and uploaded the full Visual Studio 2005 project. You can download the full project here (or see 1.1.2 version below). It comes with the libvlc.dll and libvlccore.dll for VLC 1.0.1 in the bin\x86\Debug directory so if you have a version other than this, just overwrite those files.

Update for VLC 1.1.2:

You can now download the VLC 1.1.2 compatible version. There were some changes to the way libvlc handles exceptions that needed to be corrected. Other than that, there were a couple of minor function name changes.

Please use these posts as a starting point to use your own code though. These posts are intended to stop people from being reliant on the already existing, large, overcomplicated and quickly outdated libraries. They are not intended to be just another library for people to blindly use without understanding how it works. You can use this to learn how to write your own native interop code on a well designed library then adapt it for your own changes and keep it up to date with whichever version of VLC you want. This also means you never have to use the terrible code on pinvoke.net for other libraries, as you can write your own from the original documentation and it will almost always be better.

Bugfix: VlcException should use Marshal.PtrToStringAnsi not Marshal.PtrToStringAuto

libvlc media player in C# (part 1)

Wednesday, May 6th, 2009

There seems to be a massive misconception about using VLC inside an application and many, many large wrapper libraries have been written. These are often harder to use than libvlc itself, buggy or just downright don’t work (at least not in what will be “the latest” version of VLC at the time you want to write anything).

Using the libvlc documentation directly and the libvlc example I wrote a simple wrapper class that performs the basics needed to play, pause and stop media. Because it is libvlc, things like resizing the video, toggling full screen by double clicking the video output or streaming media from a source device or network are handled automatically.

This code was all written and tested with VLC 0.98a but because it is taken from the documentation and example, it should work for all versions 0.9x and later with only minor changes. Because it is so simple, these changes should be easy to make. Most of the time, these changes will just be slight function name changes and no new re-structuring is needed.

The first thing to note is that there is no version of libvlc for Windows x64. All developers should set their CPU type to x86, even if they have a 32bit machine. If you set it to “Any CPU” then 64bit users will not be able to load libvlc.dll and will crash out. If you are compiling from the command line, this should look something like csc /platform:x86 foobar.cs

The second thing to note, which trips up a lot of users, is that you must specify VLC’s plugin directory. This may make distribution a nightmare, as the plugin directory is a large directory full of DLLs. It may be possible to narrow down these DLLs to just the ones your application actually needs but I don’t know if videolan have any advice about or licensing for redistribution of these.

libvlc is made up of several modules. For the sake of simplicity in this example, I will use 1 static class to contain every exported C function and split them up visually by module with #region.

The nicest thing about VLC, as far as interop with C# goes, is that all memory management is handled internally by libvlc and functions are provided for doing anything that you would need to do to their members. This means that using an IntPtr is suitable for almost everything. You just need to make sure that you pass the correct IntPtr into each function but another layer of C# encapsulating this would easily be able to make sure of that, as discussed in part 2. The only structure that you need to define is an exception, which is very simple. You then simply always pass in references to these structs with ref ex.

The code listing for the wrapper class is as follows:

using System;
using System.Runtime.InteropServices;
 
namespace MyLibVLC
{
  // http://www.videolan.org/developers/vlc/doc/doxygen/html/group__libvlc.html
 
  [StructLayout(LayoutKind.Sequential, Pack = 1)]
  struct libvlc_exception_t
  {
    public int b_raised;
    public int i_code;
    [MarshalAs(UnmanagedType.LPStr)]
    public string psz_message;
  }
 
  static class LibVlc
  {
    #region core
    [DllImport("libvlc")]
    public static extern IntPtr libvlc_new(int argc, [MarshalAs(UnmanagedType.LPArray,
      ArraySubType = UnmanagedType.LPStr)] string[] argv, ref libvlc_exception_t ex);
 
    [DllImport("libvlc")]
    public static extern void libvlc_release(IntPtr instance);
    #endregion
 
    #region media
    [DllImport("libvlc")]
    public static extern IntPtr libvlc_media_new(IntPtr p_instance,
      [MarshalAs(UnmanagedType.LPStr)] string psz_mrl, ref libvlc_exception_t p_e);
 
    [DllImport("libvlc")]
    public static extern void libvlc_media_release(IntPtr p_meta_desc);
    #endregion
 
    #region media player
    [DllImport("libvlc")]
    public static extern IntPtr libvlc_media_player_new_from_media(IntPtr media,
      ref libvlc_exception_t ex);
 
    [DllImport("libvlc")]
    public static extern void libvlc_media_player_release(IntPtr player);
 
    [DllImport("libvlc")]
    public static extern void libvlc_media_player_set_drawable(IntPtr player, IntPtr drawable,
      ref libvlc_exception_t p_e);
 
    [DllImport("libvlc")]
    public static extern void libvlc_media_player_play(IntPtr player, ref libvlc_exception_t ex);
 
    [DllImport("libvlc")]
    public static extern void libvlc_media_player_pause(IntPtr player, ref libvlc_exception_t ex);
 
    [DllImport("libvlc")]
    public static extern void libvlc_media_player_stop(IntPtr player, ref libvlc_exception_t ex);
    #endregion
 
    #region exception
    [DllImport("libvlc")]
    public static extern void libvlc_exception_init(ref libvlc_exception_t p_exception);
 
    [DllImport("libvlc")]
    public static extern int libvlc_exception_raised(ref libvlc_exception_t p_exception);
 
    [DllImport("libvlc")]
    public static extern string libvlc_exception_get_message(ref libvlc_exception_t p_exception);
    #endregion
  }
}

For a sample application to use this simple wrapper, I just created a new Windows form and added a play button, stop button and a panel for viewing the video. In this example, the stop button also cleans everything up so you should make sure to press it before closing the form.

At one point during this code, libvlc can optionally be given a HWND to draw to. If you don’t give it one, it pops up a new player. However, people seem to be confused over how simple this is to do in C# and have been making large amounts of interop calls to the Win32 API to get handles. This is not necessary, as System.Windows.Forms.Control.Handle allows you go get the window handle (HWND) to any component that inherits from the Control class. This includes the Form class and the Panel class (and even the Button class) so all you actually need to pass it is this.Handle (for the handle to the form itself) or panel.Handle (for a Panel called panel). If you want it to start fullscreen, add the command line argument “-f” rather than using the Win32 function GetDesktopWindow().

Because I will be using this to display PAL video, which is interlaced at 576i, I have added some deinterlacing options to the command line. These are --vout-filter=deinterlace and --deinterlace-mode=blend.

Without further ado, here is the code listing for the partial windows form class:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
 
using System.Runtime.InteropServices;
 
namespace MyLibVLC
{
  public partial class Form1 : Form
  {
    IntPtr instance, player;
 
    public Form1()
    {
      InitializeComponent();
    }
 
    private void Play_Click(object sender, EventArgs e)
    {
      libvlc_exception_t ex = new libvlc_exception_t();
      LibVlc.libvlc_exception_init(ref ex);
 
      string[] args = new string[] {
        "-I", "dummy", "--ignore-config",
        @"--plugin-path=C:\Program Files (x86)\VideoLAN\VLC\plugins",
        "--vout-filter=deinterlace", "--deinterlace-mode=blend"
      };
 
      instance = LibVlc.libvlc_new(args.Length, args, ref ex);
      Raise(ref ex);
 
      IntPtr media = LibVlc.libvlc_media_new(instance, @"C:\foobar.mpg", ref ex);
      Raise(ref ex);
 
      player = LibVlc.libvlc_media_player_new_from_media(media, ref ex);
      Raise(ref ex);
 
      LibVlc.libvlc_media_release(media);
 
      // panel1 may be any component including a System.Windows.Forms.Form but
      // this example uses a System.Windows.Forms.Panel
      LibVlc.libvlc_media_player_set_drawable(player, panel1.Handle, ref ex);
      Raise(ref ex);
 
      LibVlc.libvlc_media_player_play(player, ref ex);
      Raise(ref ex);
    }
 
    private void Stop_Click(object sender, EventArgs e)
    {
      libvlc_exception_t ex = new libvlc_exception_t();
      LibVlc.libvlc_exception_init(ref ex);
 
      LibVlc.libvlc_media_player_stop(player, ref ex);
      Raise(ref ex);
 
      LibVlc.libvlc_media_player_release(player);
      LibVlc.libvlc_release(instance);
    }
 
    static void Raise(ref libvlc_exception_t ex)
    {
      if (LibVlc.libvlc_exception_raised(ref ex) != 0)
        MessageBox.Show(LibVlc.libvlc_exception_get_message(ref ex));
    }
  }
}

Note that this section of code is deprecated and the code from part 2 should be used instead.

Adding a pause button is similar to the stop button but without the cleanup.

Here is an example slightly further on down the line but using the same code:
Example of LibVLC

See part 2 for more.

3 colour gradient

Wednesday, March 18th, 2009

Recently I noticed a green-red gradient that I was using wasn’t really what I wanted.
green-red gradient

I wanted it to go through yellow. I have made a Vista sidebar gadget in the past that shows different colours of the horizontal percentage bars for CPU and memory usage which faded from green at 0% to yellow at 50% and then faded from yellow at 50% to red at 100%. I had thought this solved the problem until I tried to use that same formula for a gradient, which turned out to be a triangular gradient.
green-yellow-red triangle gradient

The problem here is that there is only yellow at the very peak of the triangle so it looks pinched. From here it is obvious that a curve is needed. I first looked into Bezier curves, as you can join two of them easily by using the same points on both. However, this seemed a bit complicated. I next used a bell curve, which is actually a Gaussian function. This function is e-x2. This worked well and I used it throughout the development of this gradient but after I was finished I realised that a simple Sine wave from 0 to PI would have sufficed (and produces almost exactly the same result as a Gaussian function). A better function would be a Cosine wave from -PI to PI, as this gives a smooth gradient at either end that repeats perfectly. However, this would need to be normalised so that it takes a percent from 0.0 to 1.0 and outputs a value from 0.0 to 1.0 y = (cos((x*2-1)*pi)+1)/2, which is easy to do in a simple Sine 0 to PI because it is done just by multiplying the input by PI.

The key to this is that when it hits the peak, at 50%, it changes from a green-yellow gradient to a red-yellow gradient. The sine function is not used directly to determine the colour but rather to determine where on a simple colour gradient to choose the colour from, which allows it to be used with any combination of colours. The end result is this:
green-yellow-red sine gradient (sine)
green-yellow-red cosine gradient (cosine)

C# code listing (for ASPX) is as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
using System;
using System.Drawing;
using System.Drawing.Imaging;
 
public partial class PercentBar : System.Web.UI.Page
{
    protected void Page_Load(object sender, EventArgs e)
    {
        using (Bitmap bmp = new Bitmap(100, 20, PixelFormat.Format24bppRgb))
        {
            double w = (double)bmp.Width;
            for (int x = 0; x < bmp.Width; x++)
            {
                Color c = GetTriColour(x / w, Color.Lime, Color.Yellow, Color.Red);
                for (int y = 0; y < bmp.Height; y++)
                    bmp.SetPixel(x, y, c);
            }
 
            using (System.IO.MemoryStream ms = new System.IO.MemoryStream())
            {
                bmp.Save(ms, ImageFormat.Png);
                ms.WriteTo(Response.OutputStream);
            }
            Response.ContentType = "image/png";
        }
    }
 
    public static Color GetTriColour(double percent, Color left, Color centre, Color right)
    {
        if (percent < 0 || percent > 1)
            throw new Exception("Percent must be between 0 and 1");
 
        //double weight = Math.Sin(percent * Math.PI);
        double weight = (Math.Cos((percent * 2 - 1) * Math.PI) + 1) / 2;
 
        return GetColourFromLinearGradient(weight,
           percent < 0.5 ? left : right, centre);
    }
 
    public static Color GetColourFromLinearGradient(double percent, Color start, Color end)
    {
        double a, r, g, b;
 
        if (percent < 0 || percent > 1)
            throw new Exception("Percent must be between 0 and 1");
 
        double npercent = 1.0 - percent;
 
        a = Math.Min(start.A, end.A) + Math.Abs(start.A - end.A) * (start.A > end.A ? npercent : percent);
        r = Math.Min(start.R, end.R) + Math.Abs(start.R - end.R) * (start.R > end.R ? npercent : percent);
        g = Math.Min(start.G, end.G) + Math.Abs(start.G - end.G) * (start.G > end.G ? npercent : percent);
        b = Math.Min(start.B, end.B) + Math.Abs(start.B - end.B) * (start.B > end.B ? npercent : percent);
 
        return Color.FromArgb((int)a, (int)r, (int)g, (int)b);
    }
}

WordPress theme: Kubrick (wide)

Tuesday, February 17th, 2009

The default WordPress 2.7 theme is Kubrick. It’s nice but it is optimised for 800×600 and most people use more than that, so I decided to modify it slightly to optimise for 1024×768 or a similar width. Because of things like window borders and scrollbars, we do not want the width to be exactly 1024 pixels wide so all we do is use the existing widths and add the difference between 1024 and 800 to them. This is a 224 pixel increase in width so if something was 760 it would become 984, if it was 740 it would become 964, etc.

Several files need to be changed for this (make backups beforehand):

The 3 jpeg files are quite straightforward. I just opened them in Microsoft Paint, increased the width by 224 (image -> attributes) making them 984 wide, then dragged the right side of the old image to the right side of the new image and stretched out the middle section to fill the space.

The php file is a bit more complicated. This file reads kubrickheader.jpg, modifies it with your chosen colours and adds the white rounded corners. Again, the only changes here are adding 224 to some of the numbers in this file. We don’t even need to work these out, we can just append “+224″ to the existing numbers where appropriate. First we change the $corners array from
0 => array ( 25, 734 ),
to
0 => array ( 25, 734+224 ),
and do this for every entry in the array.

Slightly lower down in the “Blank out the blue thing” for loop, we change
$x2 = 740;
to
$x2 = 740+224;

and again in the “Draw a new color thing” for loop, we do the same thing, changing
$x2 = 739;
to
$x2 = 739+224;

Lastly, we change the style.css. This is the most complicated part, just because the widths to change are spread out everywhere and missing one will mess the whole page up. This is a bit long to explain, so here’s the diff (modified slightly for reading clarity – don’t try to patch with it). Remember that you want to change the lines with the – in front of them into the lines with the + in front. We can’t just use +224 any more so we actually have to work them out.

As an example, in the first one (#headerimg) you just change 740px to 964px.

@@ -41,9 +41,9 @@

 #headerimg     {
        margin: 7px 9px 0;
        height: 192px;
-       width: 740px;
+       width: 964px;
        }

@@ -236,18 +236,18 @@
 #page {
        background-color: white;
        margin: 20px auto;
        padding: 0;
-       width: 760px;
+       width: 984px;
        border: 1px solid #959596;
        }

 #header {
        background-color: #73a0c5;
        margin: 0 0 0 1px;
        padding: 0;
        height: 200px;
-       width: 758px;
+       width: 982px;
        }

@@ -258,15 +258,15 @@
 .narrowcolumn {
        float: left;
        padding: 0 0 20px 45px;
        margin: 0px 0 0;
-       width: 450px;
+       width: 674px;
        }

 .widecolumn {
        padding: 10px 0 20px 0;
        margin: 5px 0 0 150px;
-       width: 450px;
+       width: 674px;
        }

@@ -311,9 +311,9 @@

 #footer {
        padding: 0;
        margin: 0 auto;
-       width: 760px;
+       width: 984px;
        clear: both;
        }

@@ -570,9 +570,9 @@
 /* Begin Sidebar */
 #sidebar
 {
        padding: 20px 0 10px 0;
-       margin-left: 545px;
+       margin-left: 769px;
        width: 190px;
        }

If you get lost here, you can use my style.css but if you use a different base version to me (I am on the one that comes with WordPress 2.7), my modified style.css may not work for you.

Update 2009-06-13:
With WordPress 2.8′s release, this theme has changed slightly. The right-hand navigation bar is now a different colour from the rest of the page. This is done with the kubrickbg-ltr.jpg image. I have widened mine but I am still using the old images for the rest of the site (header, footer, etc) so it does not fit in perfectly, as you can see from the top and bottom of the page.

Die thumbs.db, die!

Saturday, January 31st, 2009

Someone just told me that they were going to download a program that would clean their windows hard drive of “thumbs.db” files, so I gave them this command line instead:

for /f "tokens=*" %a in ('dir /b /s /aSH thumbs.db') do @(
  echo %a
  del /f /aSH "%a"
)

This very quickly (under 1 minute for my whole C: drive) scans directories recursively for thumbs.db files, removes the “hidden” and “system” attributes and then deletes them, forcing deletion even if the files are read-only.

Of course, you would “cd” to the correct directory first (e.g. cd \ to do the whole drive), if you want to put it in a batch file you need to double up the percent symbols (%a becomes %%a) and you will need appropriate permissions to delete the files it finds (and maybe to change their attributes), so you may need to run this from an elevated command prompt if you are not running it in a directory that you own.

You should also note that the “/aSH” part of the “dir” command (there is no space, as this may also match files and directories called “sh”) assumes that the files are hidden and system files (as they are by default but probably not if you have extracted them from something like a zip file where someone has left them in). If the files are just hidden, just system or neither (just normal files) they will not appear in the list and will not be deleted. An alternative implementation could either run it once with this set and once with just “dir /b /s thumbs.db” or could run it once to remove the S and H attributes (or separately for each of these) and then after they are all “normal” files, run it through again to delete them.

The “echo” line is optional and just shows a verbose output as it deletes things. If you remove it you can also remove the parenthesis and put the whole thing on 1 line if you so wish.

Update:
Better:

for /f "tokens=*" %a in ('dir /b /s /aS thumbs.db') do @attrib -S "%a"
for /f "tokens=*" %a in ('dir /b /s /aH thumbs.db') do @attrib -H "%a"
for /f "tokens=*" %a in ('dir /b /s /aSH thumbs.db') do @attrib -S -H "%a"
 
for /f "tokens=*" %a in ('dir /b /s thumbs.db') do @(
  echo %a
  del /f "%a"
)

Diffpex

Tuesday, January 13th, 2009

I was messing around with the well known Unix command line utility diff and thought to myself “If we just want to see the differences easily (rather than making a patch file), wouldn’t it be better to get rid of all of these garbage characters and use colours for displaying the output instead?”.

So that’s what I did. Here’s the classic “computer” vs “boathouse” example, letter by letter (you would normally do line by line or word by word, which is far easier to read).

computer-boathouse

Not only is “cbompuathouser” an awesome word but if you cover up all of the green letters, you get “computer” while if you cover up all of the red letters, you get “boathouse”. Black letters are common to both so they are always visible. When this really gets interesting is when you change the red or green to the same colour as the text. In this case, we change the green background to a grey background and end up with a clearly legible word “computer” but now it looks like we have “Tipexed” (liquid paper) out all of the unneeded letters from “boathouse”, just as we would if we used a pen and paper.

no-boathouse

This looks amazing when you do it on some big paragraphs that have a few differences in them.