Archive for the ‘Linux’ Category

How to compile ffmpeg from source

Wednesday, June 9th, 2010

ffmpeg is a free and very powerful video encoding tool that is the basis of many other popular video encoding tools, with a command line interface.

Because of the frequent updates to ffmpeg and the codecs it uses, it is often best to download the latest source and compile it yourself. These updates usually include new features, bug fixes, support for new codecs and noticeable speed improvements. Most Linux distributions have outdated packages in their repositories so being able to compile the latest code from source is very useful

I use ffmpeg to output H.264 video with AAC audio in a MP4 container. I also occasionally convert audio to mp3. This means I will be compiling the following:

Preparing to compile on Linux

(If you already compile things on Linux, skip to the next section.)
First make sure that you have all of the tools and headers required to compile on Linux. On Debian based systems (including Ubuntu) you can usually run the following as root to get everything you need. First update your system. It is important that you get the latest kernel and the correct sources for it.

apt-get update
apt-get dist-upgrade

A reboot may be required here if your kernel updated.

Now get the compilers, tools and headers needed to compile software on Linux:

apt-get install build-essential linux-headers-`uname -r` git subversion yasm

NB: your “git” package may be called something like “git-core”

Compile ffmpeg (empty)

Because some libraries require ffmpeg to be installed, we will first install a blank copy with no additional codecs specified.

When I update, I often find that I need to recompile a blank version of ffmpeg before I can compile x264.

Download the latest ffmpeg source with subversion, configure, compile:

svn checkout svn://svn.ffmpeg.org/ffmpeg/trunk ffmpeg
cd ffmpeg
./configure
make

If this all worked with no errors (you can check with echo $?), run

make install

as root. Use su or sudo if you need to. I use su.

Compile faac and faad

Download and extract faac and faad from audiocoding.com.

As above, run

./configure
make
su -c 'make install'

on both libraries.

You can optionally do the same thing with LAME if you want to be able to encode mp3 audio.

NB: if you get a strcasestr error compiling faac, just comment it out in faac_1.28/common/mp4v2/mpeg4ip.h

Compile x264

Download and compile the latest x264 source with git:

git clone git://git.videolan.org/x264.git
cd x264
./configure --enable-static --enable-shared
make
su -c 'make install'

Compile ffmpeg with the above codecs

Now that we have all of the codecs (x264, aac, mp3) successfully installed as libraries on the system, we can reinstall ffmpeg to incorporate these libraries. First go back to the ffmpeg source directory and clean off the previous compile.

cd ffmpeg
make clean

Now configure to enable all of the installed libraries and recompile:

./configure --enable-gpl --enable-nonfree --enable-pthreads --enable-libx264 --enable-libfaac --enable-libfaad --enable-libmp3lame
make

(if it tells you that –enable-libfaad is not an option, just remove it. It’s probably either been rolled into faac or ffmpeg has it built in)

If it all worked, run make install as root.

su -c 'make install'

Testing

As long as /usr/local/bin/ is on your $PATH environment variable, you should be able to just type ffmpeg to check that it is all installed. It should look something like this:

$ ffmpeg
FFmpeg version SVN-r23548, Copyright (c) 2000-2010 the FFmpeg developers
  built on Jun  9 2010 13:35:17 with gcc 4.4.4
  configuration: --enable-gpl --enable-nonfree --enable-pthreads --enable-libx264 --enable-libfaac --enable-libfaad --enable-libmp3lame
  libavutil     50.18. 0 / 50.18. 0
  libavcodec    52.75. 1 / 52.75. 1
  libavformat   52.68. 0 / 52.68. 0
  libavdevice   52. 2. 0 / 52. 2. 0
  libavfilter    1.20. 0 /  1.20. 0
  libswscale     0.11. 0 /  0.11. 0
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...
 
Use -h to get full help or, even better, run 'man ffmpeg'

Using ffmpeg

I have written a small Python video encoding script to make encoding as easy as

enc -t cd file.vob

If anyone has any suggestions to improve this, please feel free to post them in the comments.

Tags: , ,
Posted in Linux | 3 Comments »

Remote recursive sha1sum with php

Wednesday, June 2nd, 2010

To calculate the SHA-1 sums, display them and make them available for download in a sums.gz file:

<?php echo(`find ./some_directory/ -type f | grep -v sums.gz | xargs sha1sum | gzip -c | tee sums.gz | zcat`); ?>

To check the sums:

<?php echo(`zcat sums.gz | sha1sum -c -`); ?>

Tags: , ,
Posted in Code, Linux | 1 Comment »

Improve your SSH with GNU Screen

Friday, September 11th, 2009

SSH is a wonderful thing. You can log in to a terminal remotely and securely as well as tunnel traffic through that connection. However, one of the biggest problems with SSH is that as soon as you close the session (or are disconnected), any processes running are killed, which can lead to things like loss of work or inconsistent files. It also means that you can’t leave things encoding overnight because as soon as you disconnect, it stops encoding. This is where GNU Screen is useful.

When screen is open, you can disconnect and reconnect to get the same screen back again. You can also have multiple screens open at once and tab between them. Ctrl+a then c creates a new screen, Ctrl+a then n tabs to the next one and Ctrl+a then k kills the selected window.

This also means that you can resize windows and even the previous text is also resized if it was truncated or wrapped before.

Just add

screen -R

to your ~/.profile to have it re-open the same screen every time you connect to SSH or, if there isn’t one running, start one.

You can also add a line to the bottom of your SSH session that tells you which tab you are currently looking at, the date, time and other similar details. This is called the hardstatus line.

I use the following hardstatus line in my ~/.screenrc

hardstatus alwayslastline
hardstatus string "%{= kw}[%{= kg}%H%{= kw}]%-=%-w%{= kr}{%{= kw}%n %t%{= kr}}%{= kw}%+w%-=[%{= kg}%Y-%m-%d %c%{= kw}]"

To explain what each part does, I will first give the uncoloured version. %{= __} sets a colour. For example, “kg” is black background and green foreground.

If we remove all of these we get

"[%H]%-=%-w{%n %t}%+w%-=[%Y-%m-%d %c]"

%H is a host name
%-= is padding
%-w is all of the windows before the current one
%n %t is the current window number and name
%+w is all of the windows after the current one
%Y-%m-%d is the date
%c is the time

Here’s how it looks (as always, click to enlarge):
screen

The downside is that scrolling doesn’t seem to work at all. I don’t know if it has just been replaced by something internal to screen but I generally use something like shift+page up to scroll up or the scroll bar in PuTTY.

Tags: ,
Posted in Linux | 1 Comment »

wget in Windows

Monday, June 29th, 2009

wget is a command line utility found on many UNIX like operating systems, particularly Linux, used to download files from the command line.

It is available from gnuwin32 but I had major problems trying to get all of its dependencies and in the end gave up with that. As I already have cygwin installed and wget installed on that, I decided to just take it from there as these files will run as long as cygwin1.dll and any other dll dependencies are present.

I zipped these files up here (2 MB) for those who do not have Cygwin. All you have to do is extract them to somewhere on your system path environment variable (e.g. C:\Windows\System32) and you should be able to use wget from the command prompt as follows:

C:\Users\George\Desktop>wget --version
GNU Wget 1.11.4

Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Originally written by Hrvoje Niksic <hniksic@xemacs.org>.
Currently maintained by Micah Cowan <micah@cowan.name>.

Tags: ,
Posted in Linux | No Comments »

How to set target file sizes in ffmpeg

Tuesday, June 16th, 2009

This script is now available as an ongoing project that can be downloaded. Please see the Video Encoding Project. This post describes the methods and mathematics behind the script.

After much searching, I found almost nothing regarding how to target a specific file size with ffmpeg, such as 700MiB to fit on a CD. Almost everything I found suggested to use a negative bitrate for mencoder, such as -700000. However, this did not work for me at all (even by copying and pasting verbatim). Because of this, I set about doing it manually with some mathematics.

What I am going to do here assumes a single audio stream, a single video stream and nothing else in the output file at all (i.e. no subtitles, no alternate languages, no director’s commentaries, etc).

When encoding media, audio uses a constant bit rate. This means that you can calculate how much space it will take if you know which bit rate you are using and how long the stream is in seconds. This also means that to know how big the video stream should be, we can calculate how big the audio stream will be and subtract it from the target file size. For example, if the target file size is 350MiB and the audio will take up 30MiB, we know the video should aim to be 320MiB (ignoring overheads in the multiplexing, container headers, etc). We can use the same calculations in reverse to work out an average or constant bit rate from a file size and a length in seconds.

In this example, I will be using Dad’s Army Series 6 Episode 1 – The Deadly Attachment. I have already ripped this from the DVD with SlySoft AnyDVD and PgcDemux on Windows. Automating PgcDemux rips is worth a whole entry in itself. Note that I have bought this legally and I do not intend to sell or share it. I just want a copy that I can store on my network and play from any computer. Storing 14 DVD images on hard drives takes quite a lot of space, which is where encoding comes in handy. In fact, some of my box sets contain over 60 DVDs (roughly 270GiB).

———-
To install ffmpeg on Debian, it is best to get it from the Debian Multimedia repository. Add

deb http://www.debian-multimedia.org lenny main
deb-src http://www.debian-multimedia.org lenny main

to your /etc/apt/sources.list and then run

# apt-get update
# apt-get install debian-multimedia-keyring
# apt-get update
# apt-get install ffmpeg

———-

The first thing we need to do to put this into practice is to get the length of the stream. If you supply the file to ffmpeg without telling it to do anything, it will give you an error. However, this error already contains all of the information we need:

$ ffmpeg -i da_6_01.VOB
FFmpeg version SVN-r13582, Copyright (c) 2000-2008 Fabrice Bellard, et al.
  configuration: --prefix=/usr ...
  built on May  3 2009 12:07:18, gcc: 4.3.2
Input #0, mpeg, from 'da_6_01.VOB':
  Duration: 00:29:23.90, start: 0.287267, bitrate: 5987 kb/s
    Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 720x576 [PAR 16:15 DAR 4:3], 9800 kb/s, 25.00 tb(r)
    Stream #0.1[0x80]: Audio: ac3, 48000 Hz, stereo, 192 kb/s
Must supply at least one output file

We can see that the duration is 00:29:23.90.

As this is an error, we need to redirect stderr to stdout. This can be done with 2>&1. We then need the line that contains the “Duration”, which we can get with grep. We can cut this line up to take the time out.

$ ffmpeg -i da_6_01.VOB 2>&1 | grep Duration | cut -d ' ' -f 4 | cut -d '.' -f 1
00:29:23

This can be converted into seconds with the formula s' = (h * 60 + m) * 60 + s.

I used a python script to do this and to perform the necessary calculations mentioned earlier. Note that most of this python script is just for the convenience of getopts and that, if needed, it can be cut down significantly at the cost of flexibility. If you always want the same output size and audio bitrate, this script can be simplified to just a couple of lines.

calc_bitrate.py:

#!/usr/bin/python
 
from getopt import getopt, GetoptError
from sys import argv, exit
 
def secs(h, m, s):
  return (h * 60 + m) * 60 + s
 
def calc_video_bitrate(target_size, abr, time):
  target_bytes = target_size * 1024 * 1024
  audio_bytes = ((abr * 1000) / 8) * time
  return (target_bytes - audio_bytes) / time * 8
 
try:
  opts, args = getopt(argv[1:], "s:a:t:", ["size=", "abr=", "time="])
except GetoptError, err:
  print str(err)
  exit(2)
 
target_size = 350 #MiB
abr = 128 #kbps
time = 0
 
for o, a in opts:
  if o in ("-s", "--size"):
    target_size = int(a)
  elif o in ("-a", "--abr"):
    abr = int(a)
 
if len(args) < 1:
  print "Usage: " + argv[0] + " [-s <size (M)>] [-a <audio bit rate (k)>] [[hh:]mm:]ss"
  exit(2)
 
time_part = args[0].split(":")
if len(time_part) < 2: time_part = [0] + time_part
if len(time_part) < 3: time_part = [0] + time_part
time = secs(int(time_part[0]), int(time_part[1]), int(time_part[2]))
 
print "%d" % calc_video_bitrate(target_size, abr, time)

A typical usage for this would be ./calc_python.py -s 350 -a 128 29:23.

We can then use a combination of these inside a shell script easily. I’m using bash syntax just to nest the commands into a single line but for compatibility with other shells, backticks (`) can be used and this is what I normally prefer. This script loops through all of the .VOB files in the directory and gives their target video bitrate for a 350MiB output and 128kbps audio.

#!/bin/bash
 
for vob in *.VOB
do
  BR=$(./calc_bitrate.py -s 350 -a 128 $(ffmpeg -i $vob 2>&1 | grep Duration | cut -d ' ' -f 4 | cut -d '.' -f 1))
  echo $vob $BR
done

I use this in 2-pass VBR mode with the libx264 video encoder and libfaac audio encoder in an mp4 container just by replacing the “echo” line of the above script with the following 2 lines:

ffmpeg -i $vob -an -pass 1 -vcodec libx264 -vpre fastfirstpass -b $BR -bt $BR -deinterlace -threads 0 -y pass1.mp4
ffmpeg -i $vob -acodec libfaac -ab 128k -ac 2 -pass 2 -vcodec libx264 -vpre hq -b $BR -bt $BR -deinterlace -threads 0 -y `echo $vob | sed 's/VOB/mp4/'`

I have 2 versions of this script, one with “-deinterlace” and one without. PAL video (576i – used in Europe), such as Dad’s Army needs deinterlacing. NTSC video (480 – used in North America) usually does not.

For ripping DVDs to H.264 and maintaining the quality, I wouldn’t use a bitrate lower than 1000k or an audio bitrate lower than 128k stereo. This means that for getting media onto a CD, you probably want no smaller than 350MiB for 30 to 40 minutes or 700MiB for 1 hour or more, depending on your resolution. Also note that I only intend to play these in computers and fitting them onto CDs is just a convenience – standalone DVD players will not be able to play H.264 or AAC (Blu-ray and HDDVD players may be able to – mp4 containers and reading CDs would be more of an issue).

Update:

I have successfully tested this in a LG BD370 Blu-ray player.

I have not tried actually burning these onto CDs yet so if 2 won’t fit on a 700MiB CD, even with overburn (because of overheads that were ignored, as stated earlier) then the simplest solution is to just target a smaller file size with something like “-s 348″.

The result

Dad’s Army episodes are not all of the same length. Some are several minutes longer than others. The following shows how no matter the size of the input file, they all produce the same size output file: 

$ ll da_5_*.VOB
-r--r--r-- 1 mythtv mythtv 1364660224 2009-06-04 00:01 da_5_01.VOB
-r--r--r-- 1 mythtv mythtv 1360699392 2009-06-04 00:01 da_5_02.VOB
-r--r--r-- 1 mythtv mythtv  956084224 2009-06-04 00:02 da_5_03.VOB
-r--r--r-- 1 mythtv mythtv 1355888640 2009-06-04 00:02 da_5_04.VOB
-r--r--r-- 1 mythtv mythtv 1114351616 2009-06-03 23:59 da_5_05.VOB
-r--r--r-- 1 mythtv mythtv 1034289152 2009-06-04 00:00 da_5_06.VOB
-r--r--r-- 1 mythtv mythtv 1045202944 2009-06-04 00:00 da_5_07.VOB
-r--r--r-- 1 mythtv mythtv 1345318912 2009-06-04 00:00 da_5_08.VOB
-r--r--r-- 1 mythtv mythtv 1391529984 2009-06-04 00:02 da_5_09.VOB
-r--r--r-- 1 mythtv mythtv 1342748672 2009-06-04 00:03 da_5_10.VOB
-r--r--r-- 1 mythtv mythtv 1158017024 2009-06-04 00:03 da_5_11.VOB
-r--r--r-- 1 mythtv mythtv 1157097472 2009-06-04 00:03 da_5_12.VOB
-r--r--r-- 1 mythtv mythtv 1180098560 2009-06-04 00:04 da_5_13.VOB
$ ll -h da_5_*.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 06:18 da_5_01.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 06:55 da_5_02.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 07:30 da_5_03.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 08:08 da_5_04.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 08:44 da_5_05.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 09:21 da_5_06.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 09:58 da_5_07.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 10:36 da_5_08.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 11:15 da_5_09.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 11:54 da_5_10.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 12:29 da_5_11.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 13:05 da_5_12.mp4
-rw-r--r-- 1 mythtv mythtv 352M 2009-06-17 13:41 da_5_13.mp4

Update:

A pure python replacement script is available here that simplifies use.

Tags: , , ,
Posted in Code, Linux | 3 Comments »

How to install VMware Tools on a Linux guest

Saturday, June 13th, 2009

Installing VMware tools on Windows guests is easy. You just select to install it from your VMware client and it appears as a CD. You can then install it as you would any other software.

On Linux, it is a bit more difficult. Again, selecting to install it from your VMware client puts it in the CD drive. However, depending on your distribution, it may not be automatically mounted.

For an IDE virtual CD drive this mount, copy and then unmount would be something like

# mount -t iso9660 -o ro /dev/hda /media/cdrom
# cp /media/cdrom/VMwareTools-4.0.0-164009.tar.gz /root
# umount /media/cdrom

For a SCSI, SATA or SAS drive, this would be /dev/sda rather than /dev/hda. The “a” here assumes that it is the first drive. The 3rd SATA drive would be /dev/sdc, for example. In addition to this, partitions are labelled with numbers (such as /dev/sdc2 for the 2nd partition on the 3rd SATA drive) but this is not an issue for optical media such as CDs.

Now that the VMware tools archive has been copied off, it can be extracted.

# tar -xzf VMwareTools-4.0.0-164009.tar.gz

Most people add a “v” to the options for verbose output but not only do I find it pointless, it also slows down the extraction (particularly over SSH, particularly for archives with a large number of files). You do not need it to tell if an error occurred during the extraction, as it tells you anyway. The “x” option is to extract the archive (“c” would be to create one), the “z” means use gzip (“j” would be bzip2) and the “f” specifies the archive’s file name.

Before we install vmware tools, there are some dependencies that we must meet. I am doing this with Debian 5.0.1 AMD64 on a VMWare vSphere (ESX 4.0) host. If you do not meet these dependencies beforehand, you will probably have to break out of the installer with ctrl+c and re-run the whole thing several times, which is not good.

The “killall” tool must be installed. You can tell if it is by running which killall. If a path appears, it is installed. If nothing appears, it is not. In Debian, this is not installed by default and is in the “psmisc” package.

In addition to this, we will need to compile kernel modules during the configure stage of the installation. This is apparently not always required but I have not met a distribution yet on which it was not required to compile them from source (possibly RHEL?). This means that we will need gcc, make and the Linux kernel headers. To take care of these dependencies run

# apt-get update
# apt-get install psmisc gcc make linux-headers-`uname -r`

You may need to update your running kernel if it can no longer find the headers for your running kernel in the repository. You can do this with apt-get dist-upgrade.

Now that you have the dependencies installed, run ./vmware-install.pl. You probably just want to say “yes” to everything. (man yes)

Tags: , ,
Posted in Linux | No Comments »

Installing Debian

Saturday, February 28th, 2009

A couple of posts back, I touched briefly on how I personally install Debian Linux. With a little know how (this post), Debian becomes a far better advertisement for Linux than Ubuntu is, as Ubuntu is no faster than Windows. In fact, Ubuntu out of the box on my computer (Intel E8500, 4GiB of 800MHz RAM, etc) is actually a lot slower than Vista out of the box and as long as you can maintain Windows (i.e. not filling it up with crap), Windows stays in the lead for performance.

For the sake of this post, I will be installing Debian onto a 64bit VMware Server 2.0 virtual machine. If you do this, make sure that if you want sound, you add virtual sound hardware when creating the virtual machine, as I forgot (I’m sure previous versions of VMware added them by default).

Stage 1

Installing Debian is a 2 stage setup, as far as we are concerned. The first part is the actual installer, which takes us up to a working command line install of Linux, perfect for use as a server. The next part installs the applications we want to use as a desktop/laptop computer from the command line and ends in a working graphical desktop. Before we can start, we need to grab the netinst disk image and burn it to a CD (in the case of vmware, simply mount the iso file inside vmware).

To start the install, we turn the machine on with the disk in the drive and boot off it. This takes us to a menu, asking us if we want to do a text install, a graphical install (this menu option is new to Debian 5, though it was available from an “expert” mode before) or other options. I will choose the text install as that is what I am used to, though the graphical install should look quite similar.

Install menu

The Debian installer will prompt us for our language, keyboard layout, locale and so on. This is fairly self explanatory so I will leave it to the reader.

After this, we are asked how we want to partition the hard drive. The easiest thing to do is make sure that you have a block of unallocated space and just use the guided install for unallocated space. You can always go back and change things later if you want to, such as adding a /boot partition or removing a /swap partition. As I am using a new virtual machine with a completely blank disk, I am using the option of “Guided – use entire disk”. You don’t need to wipe your hard drive to install Debian and it is a good idea to set it up before installing with GParted if it is a disk that you already have data on. Just leave a hole in the partition table with GParted and install Debian to that unallocated space in guided mode.

If you are dual booting with another operating system such as Windows, I recommend that you add a 100MB ext2 /boot partition to the start of the drive. This is where GRUB will be installed to and having its own partition means that you can change to a different distribution of Linux or even remove Linux entirely and still keep the ever-useful GRUB on there, with tools like memtest86+.

Remember that you can probably only have 4 primary partitions on 1 drive and that you may want to use an extended partition containing 1 or more logical partitions for the 4th. This is a limit of IDE hard drives but most partitioning software and most SATA hard drives respect this limit.

Partitioning menu

When you are done, it will ask you twice to confirm that you want to write these changes to the disk:

Partitioning confirmation

It will then install the “base system” to the hard disk. This includes things like the Linux kernel and other packages essential for the operating system. If you get the business card CD rather than the netinst CD, you will need to set up your networking before this in the install because they are not on the business card CD and must be downloaded during the installation.

Installing base

After a few minutes, the base system will be installed and you will be asked to set up user accounts. First, you need to set a root password and confirm it. After this, you enter your full name, user name and user password. You will usually be using this user name and password for Linux, switching to root only when you need to make changes like install new software, usually by use of the “su -” command.

passwords

Assuming that we are on the netinst CD, we now set up networking so that we can install any additional packages. I suggest that you install using an Ethernet cable and DHCP, as wireless generally doesn’t work on Linux until you have installed some firmware such as ipw2200 (for Intel 2200 wireless adapters), b43 (for Broadcom wireless adapters), madwifi (for Atheros wireless adapters), etc.

Next, we pick a mirror to download the packages from. If ftp.uk.debian.org does not work for you for some reason or is slow, try using mirrorservice.org. If your ISP is Virgin Media, you may want to try debian.virginmedia.com for an even faster download.

mirror

This is where my guide gets different from any other guide you may see for installing Debian. At the task selection menu (tasksel), we deselect everything. These tasks tend to be bloated with random things that don’t even match the name of the task and even if you think one sounds like you want it (such as “Laptop”), you are probably better off installing the individual packages later on in stage 2 of this guide.

task selection

Because we have removed everything from the list, only a few small packages need to be downloaded and installed. To be honest, I don’t know why any have to be, as we didn’t select any tasks but that just goes to show how tasksel always gives you far more than you want.

task download

All that is left to do is to install the boot loader, GRUB, to the master boot record of the hard drive. This is what will give us a choice of operating system to run when we turn the computer on each time, assuming that we boot off this hard disk by default.

grub install

That’s it for stage 1 and it probably took under 5 minutes to install Debian Linux to your hard drive. Now we can take the CD out of the drive and reboot the machine to the hard drive that we just installed GRUB to.

Stage 2

When we reboot the machine, we see GRUB giving us a choice of Debian or Debian (single-user mode). Single user mode is similar to safe mode in Windows. You only really need to use it if the standard mode does not work. If you want, you can remove it from your /boot/grub/menu.lst later and just press “e” from GRUB to edit the normal mode and add “single” to the kernel line if you ever need single user mode. If you have Windows installed on your computer, it should also have an entry to let you use that and you can change it to the default option inside the menu.lst file if you want, as well as setting the timeout.

GRUB menu

A few seconds after selecting Debian from GRUB, the operating system has loaded entirely and we are at a login prompt. Because we did not install anything, it is all text based at the moment. This would be useful for a server but we probably want a graphical desktop. I’ve tried using a text-only desktop before and it isn’t pretty.

text login

You should log in with your main user account and password then, once logged in, run the command “su -” and enter the root password to gain super user access. In a Linux command prompt, password fields do not display anything when you are typing them, not even asterisks (*). When you are in a root shell, the prompt should change from a $ to a #.

The first thing to do is add the non-free and contrib repositories to apt. You can use an editor such as vim or nano to edit /etc/apt/sources.list to remove any CDROM lines and add “contrib non-free” to the main Debian repository. I don’t add it to the main deb-src or to updates and as of yet I haven’t worked out what “volatile” is, as it is new as far as I know. Here’s what the file should look like in vim, with the new text highlighted in red:

vim /etc/apt/sources.list

You should then run
# apt-get update
to download the latest package lists, followed by
# apt-get dist-upgrade
to upgrade all existing packages, such as the kernel to their latest versions. A reboot may be required here, which you can do easily with # reboot (you can shut down from the command line with # halt).

install list

Now we can begin installing the software to transform this command line into a graphical desktop, using # apt-get install <list of packages>. We want to install xorg, the graphics backend and frontend; gnome-core, the simple package for GNOME that comes with no large applications; gdm, the desktop manager for GNOME that lets us log in from a graphical interface among other things; alsa and alsa-oss for the sound and iceweasel for a web browser (this is firefox really). Other options are gnome-themes for some pretty window borders and gnome-backgrounds for some nice desktop wallpapers.

This will start to download and then install all of these packages, which may take a while depending on your internet speed but you are downloading them now rather than downloading the massive list of CDs or DVDs that you would have downloaded if you hadn’t have used the netinst or businesscard CDs, which will usually contain old packages that need updating anyway.

install progress

Once everything has successfully downloaded and installed, you can start gdm for the first time to turn on the graphical interface. This usually happens automatically on startup so you shouldn’t have to do it again. The command to start the gdm daemon is:
# invoke-rc.d gdm start

start gdm

As soon as this starts, you should see a graphical login prompt:

graphical login

From here you can choose alternative languages, switch between GNOME, KDE, Xfce and so on.

After you log in you should see a working graphical desktop and the installation is complete (ignore my broken sound, I forgot to add a sound card in vmware):

graphical desktop

This is what you would expect from a fresh install of ubuntu but you should notice that it is very snappy and probably takes 30 seconds or less between turning your computer on and having a completely loaded working desktop, without even having to wait for a busy cursor before you can start doing things.

You can watch videos with totem-gstreamer, browse the internet with iceweasel, check your emails with icedove (thunderbird), write code or text files with gEdit, view PDF documents with evince, edit Microsoft office documents with OpenOffice.org, etc.

Tags: ,
Posted in Linux | 1 Comment »

Debian 5

Tuesday, February 17th, 2009

Well, my favourite Linux distribution, Debian has just released version 5. If you don’t know what Debian is, think of it as Ubuntu but easier to customise and much faster, with the added bonus that the Linux community doesn’t laugh at you when they find out you are using it.

I generally do a completely minimum 64bit install from the 150 (ish) megabyte netinst CD and remove every option it gives me, even “standard system”. Rebooting then takes you to a command line login prompt where you can log in as root and installing a GNOME graphical desktop is as easy as plugging in a network cable and typing “apt-get update” (to download the latest package lists) and then “apt-get install xorg gnome-core gnome-themes gdm”. You probably also want to install the “iceweasel” web browser, which is Debian’s name for Firefox to avoid copyright infringement. From there, you have all of the basic features that you would expect to find in Ubuntu but it will run at least 2 or 3 times faster and because you have installed everything yourself, you know exactly what is on there.

See the follow up entry in this blog for full details and step by step instructions:
Installing Debian

I didn’t even notice that the testing branch had been renamed from Lenny to Squeeze (both characters from Toy Story, if you are wondering) months ago, though I did notice that Lenny was gradually losing essential packages such as Iceweasel and the Adobe Flash Player plugin which was making it rather hard to use. In future I should probably use the “testing” synonym for my repositories rather than a specific release name such as “Lenny” or “Squeeze”, as I now know that they get quite bad towards the end of their life.

Now I have the choice to either switch to Debian 5.0 or Debian Squeeze (testing). I used the testing branch (Lenny) before because the stable branch (Etch) stays so far out of date, which is fine for servers but not so good for desktops and terrible for laptops, as laptops tend to take a while to get decent hardware support for devices such as WiFi adapters and waiting a year before you can use your laptop is just ridiculous.

Aside from a lack of working drivers for my ATI Radeon HD4870 X2 graphics card, forcing me to use 1600×1200 rather than 1920×1200 and making scrolling down in a window impossibly slow (the page down key still works quickly), I have no hardware problems with Lenny at all, which should mean that I will have no hardware problems with Debian 5.0 out of the box either. I don’t necessarily want the latest software, just software that works. In fact, a lot of changes made to Linux software frustrate me as they can make things worse.

Because Debian’s website is a nightmare to navigate unless all you want is the “base model” (stable, i386, 31 CDs), here are some links:

Debian 5.0.1 AMD64 netinst can be found here (136 MiB)
Debian Squeeze (testing) AMD64 netinst can be found here (131 MiB)

When these links break in a few months (and they will) post a comment and I will update them. Alternatively you can browse around on http://cdimage.debian.org/cdimage/, which is how I find them in the first place and is far easier than using the main Debian website.

Posted in Linux | No Comments »