Posts tagged: howto

Installing Debian with root on iSCSI

You know, if a normal person wanted a diskless Debian install, they’d use NFS – a widely supported way to have a root filesystem on a network share and therefore reasonably easy to set-up. I however, wanted to have the root filesystem on iSCSI since it’s a nice, fast way of achieving the same thing. I’m now going to document how I achieved this for those who wish to do the same, but first: if you want an easy solution, use NFS. Seriously. You need to have some experience as a Linux sysadmin, developer and Debian user to make this work.

HERE BE DRAGONS

You’re still reading? OK then, here goes.

The first step is to configure your iSCSI target, which I won’t go into here. You may find my previous post, Playing with iSCSI, helpful. Now grab a copy of the Debian netinst CD image, burn it to CD and boot from it on the machine you want to install on. Note: it’s essential that you have another machine already running the version of Debian that you’re installing.

Using another machine, you now need to get a copy of the open-iscsi sources. You should do this on a Debian system running the same version of Debian, with the same kernel on the same architecture as you are installing. Simply run:

apt-get source open-iscsi

You now need to make some modifications to the open-iscsi source to make it work in the Debian Installer environment (and in initramfs, which we’ll come to later). The changes are minimal, but for convenience I’ve produced a patch:

open-iscsi-initrdfix.patch

Change into the directory containing the open-iscsi source, and apply the patch as follows:

patch -p1 < /path/to/open-iscsi-initrdfix.patch

Now simply type make to build open-iscsi. This will build the open-iscsi userspace applications and the necessary kernel modules. If you find that open-iscsi refuses to build kernel modules with a complaint like the following:

make[1]: *** No rule to make target `linux_2_6_26′, needed by `kernel_check’. Stop.

Then don’t worry – this simply means that you can avoid compiling the modules, and will need to copy them from the running kernel instead.

You now need to copy some files, plus a few modules from the running kernel, to a USB pen-drive. From the open-iscsi directory you need the following (ignore the .ko files if the kernel modules didn’t build):

usr/iscsid
usr/iscsiadm
kernel/libiscsi.ko
kernel/iscsi_tcp.ko
kernel/scsi_transport_iscsi.ko

Plus the following files from your running kernel:

/lib/modules/`uname -r`/kernel/crypto/crc32c.ko
/lib/modules/`uname -r`/kernel/lib/libcrc32c.ko

If the open-iscsi kernel modules didn’t build before, copy the following too:

/lib/modules/`uname -r`/kernel/kernel/drivers/scsi/iscsi_tcp.ko
/lib/modules/`uname -r`/kernel/drivers/scsi/libiscsi.ko
/lib/modules/`uname -r`/kernel/drivers/scsi/scsi_transport_iscsi.ko

Now on the machine you’re installing on, begin the Debian installation and continue normally until after the stage where the network is configured. Now switch to a console (Alt+F2), insert your USB pen-drive and mount it. Copy the files off it and place them in /tmp.

You now need to create /etc/iscsi/initiatorname.iscsi with content like the following:

InitiatorName=iqn.1993-08.org.debian:01.665f5fd03076

The actual name isn’t important, just make something up (as long as it’s in the above format).
Now remove your USB pen-drive (important!) and run the following commands to get iSCSI going:

insmod /tmp/libiscsi.ko
insmod /tmp/scsi_transport_iscsi.ko
insmod /tmp/iscsi-tcp.ko
insmod /tmp/libcrc32c.ko
insmod /tmp/crc32c.ko
/tmp/iscsid
/tmp/iscsiadm -m discovery -t sendtargets -p <iscsi target ip address>:<port>
/tmp/iscsiadm -m node -L all

That will load all the required modules, discover your target and then connect to it. All being well, it will appear as a SCSI disk (check dmesg). Now you can continue the install normally. At the final stage of the install DON’T allow the system to reboot – the installed system can’t boot yet.

I decided that I wanted to boot using BOOTP and TFTP. You might want to boot from USB, a flash card or something else, in which case you’re on your own – I’m going to document how to boot using BOOP and TFTP. On your iSCSI target machine (or any machine, actually) configure the required daemons and drop in the Debian netboot files – you may wish to use Preparing Files for TFTP Net Booting as a guide.

Now that your installation is finished, you need to copy some files off the installed system to your pen drive. You need the initramfs and the kernel from /mnt/target/boot/ directory, named initrd.img-<kernelversion> and ‘vmlinuz-<kernelversion>. You also need the entire /etc/iscsi directory.

Now copy the files onto the machine you configured as your TFTP server, and put the kernel image into your tftp directory (probably /tftpboot). You now need to modify the initramfs image to add the iSCSI bits.

First, decompress the initramfs image into an empty directory:

gzip -dc <initramfs image> | cpio -id

Now, you need to copy in the contents of the pen drive to the right places. Copy the /etc/iscsi directory from the pen drive to etc/ in the directory containing the decompressed initramfs image. Also copy the iscsid binary to sbin/, and the iscsiadm binary to bin/.
Now you need to add the following to the end of scripts/init-premount/udev (we need to add it to one of the scripts run at the start of the boot process, and this is as good a place as any):

/bin/ipconfig -c dhcp -d eth0
/sbin/iscsid -f &
/bin/iscsiadm -m node -L all

You also need to copy in the kernel modules, into the correct paths:

lib/modules/<kernelversion>/kernel/crypto/crc32c.ko
lib/modules/<kernelversion>/kernel/lib/libcrc32c.ko
lib/modules/<kernelversion>/kernel/drivers/scsi/iscsi_tcp.ko
lib/modules/<kernelversion>/kernel/drivers/scsi/libiscsi.ko
lib/modules/<kernelversion>/kernel/drivers/scsi/scsi_transport_iscsi.ko

And finally, you need to edit the init script to reference the root partition, e.g:

export ROOT=/dev/sda1

Now re-create the initramfs image by running the following in the root of the directory in which you decompressed the image, and save it to your tftp directory:

find . | cpio -H newc -o > /tftpboot/initrd.img-<kernelversion>

That’s it. Ensure your TFTP server is fully configured, modify the PXE configuration to point to your kernel and initramfs image, then you can restart your system and hopefully you’ll get root on iSCSI…

Related Posts:

Streaming with flumotion

After attending Thomas Vander Stichele’s Flumotion talk at LUGRadio Live 2007 and being suitably impressed, I figured I’d give it a go – partly because it looked cool and partly because I didn’t believe for a second that I could set up a server to compress and stream video and audio data with only a few clicks in the real-world. I am also well aware of how buggy GStreamer (on which it is based) has been when I’ve tried to use it for real-world projects in the past and was not convinced all the features flumotion requires would actually function as expected.

Well, it seems that actually you can get streaming going with a few clicks. After installing the flumotion package in Debian Lenny it was simply a case of running ‘flumotion-admin’ and working through the simple wizard, selecting my super-cheap webcam as the video source and OGG Theora as the codec. After that was done, I simply pointed VLC at the stream and off it went (you can of course use any media player you want, as long as it can handle HTTP streams and the codec you’re streaming in). It literally took about two minutes to get going. I did have to add the ‘flumotion’ user to the ‘video’ group so that it could access the webcam device, but this is a packaging issue rather than a flumotion one.

With my crappy webcam I can get a good enough picture using only 64Kbit/s of bandwidth. Possibly some tweaking could get it even lower, but it depends on the codec. It would be interesting to see how the BBC’s Dirac codec would perform in this environment. Capturing from my webcam (at low resolution) and encoding as Theora uses about 27% of CPU at 1GHz (no, I don’t really have such an old CPU – that’s reduced with frequency scaling in order to save power).

One irritating problem is that you can set up a stream, but as soon as the flumotion server is restarted it completely forgets about it, so all streams are forgotten each time you reboot. Probably I am missing something, but I can’t see an obvious way around this at the moment. Anyway, flumotion is pretty cool and it looks like finally, thanks to GStreamer, Linux’s lack of decent easy-to-use multimedia applications is starting to get fixed. Well done Fluendo for making this happen.

Related Posts:

More Video Goodness with Python and GStreamer

Further to my previous post, Video goodness with Python and GStreamer, I’ve now mastered playback of video along with audio using Python and GStreamer. This is my code to play back an MPEG2 transport stream:

def gstInit(self):
  # declare our pipeline and GST elements
  self.pipeline = gst.Pipeline(“mypipeline”)
  self.src = gst.element_factory_make(“filesrc”, “src”);
  self.src.set_property(“location”, “file.mpeg”)
  self.demux = gst.element_factory_make(“ffdemux_mpegts”, “demux”)
  self.queue1 = gst.element_factory_make(“queue”, “queue1″)
  self.queue2 = gst.element_factory_make(“queue”, “queue2″)
  self.deinterlace = gst.element_factory_make(“ffdeinterlace”, “deinterlace”)
  self.vdecode = gst.element_factory_make(“mpeg2dec”, “vdecode”)
  self.adecode = gst.element_factory_make(“mad”, “adecode”)
  self.vsink = gst.element_factory_make(“xvimagesink”, “vsink”)
  self.asink = gst.element_factory_make(“alsasink”, “asink”)
  
  # add elements to the pipeline
  self.pipeline.add(self.src)
  self.pipeline.add(self.demux)
  self.pipeline.add(self.queue1)
  self.pipeline.add(self.queue2)
  self.pipeline.add(self.vdecode)
  self.pipeline.add(self.deinterlace)
  self.pipeline.add(self.adecode)
  self.pipeline.add(self.vsink)
  self.pipeline.add(self.asink)

  # we can’t link demux until the audio and video pads are added
  # we need to listen for ‘pad-added’ signals
  self.demux.connect(‘pad-added’, self.on_pad_added)

  # link all elements apart from demux
  gst.element_link_many(self.src, self.demux)
  gst.element_link_many(self.queue1, self.vdecode, self.deinterlace, self.vsink)
  gst.element_link_many(self.queue2, self.adecode, self.asink)

def on_pad_added(self, element, src_pad):
  caps = src_pad.get_caps()
  name = caps[0].get_name()
  # link demux to vdecode when video/mpeg pad added to demux
  if name == ‘video/mpeg':
    sink_pad = self.queue1.get_pad(‘sink’)
  elif name == ‘audio/mpeg':
    sink_pad = self.queue2.get_pad(‘sink’)
  else:
    return
  if not sink_pad.is_linked():
    src_pad.link(sink_pad)

def startPlayback(self):
  # start playback
  self.pipeline.set_state(gst.STATE_PLAYING)

Naturally you begin by declaring your pipeline and elements (for an introduction to GStreamer, see my previous post linked above and the link to Jono Bacon’s post included in that post).
Then you link your ‘filesrc’ to your demux element (variable depending on the type of content you’re playing).

Now, you cannot link anything to your demux element until you’ve started playback – this is because your demux element hasn’t yet looked at the data its being given, so it doesn’t know whether it should have audio and/or video pads. So you need a local function to handle the ‘pad-added’ signal which is sent when, you guessed it, a pad is added to an element – we only care about this signal when it relates to the demux element, so we connect that signal from that element to a local function. In this function we first need to determine what type of pad has been added – audio, video, or something else. In my case, because I’m playing MPEG, I know that I’m waiting for audio/mpeg and video/mpeg pads. The precise pads you need to wait for will depend on the content you’re playing.

When each pad is added, you link both the video and audio pads to separate ‘queue’ elements. Queue elements are essentially buffers and you need these for video and audio playback to work at the same time. You can link the audio queue element to your audio decoder and the decoder to your audio sink during initialisation – you don’t need to wait until you can link in the demux element. Likewise for the video queue element.

The equivalent if you want to launch playback from the command-line would be:

gst-launch filesrc location=file.mpeg ! ffdemux_mpegts name=demux ! { queue ! mpeg2dec ! xvimagesink } { demux. ! queue ! mad ! alsasink }

Things become simpler if you’re using a ‘decodebin’ or ‘playbin’ element rather than specifying your elements individually (which is what you’ll do if you want to play more than one type of content). In this case with a playbin you can simply add that one element to your pipeline, after giving it a URI to play, without worrying about what the content is.

def gstInit(self):
  # declare our pipeline and GST elements
  self.pipeline = gst.Pipeline(“mypipeline”)
  self.pbin = gst.element_factory_make(“playbin”, “pbin”);
  self.pbin.set_property(“uri”, “file:///home/user/file.mpeg”)
  
  # add elements to the pipeline
  self.pipeline.add(self.pbin)

def startPlayback(self):
  # start playback
  self.pipeline.set_state(gst.STATE_PLAYING)

And on the command line:

gst-launch playbin uri=file:///home/user/file.mpeg

You’re probably wondering why I went through all the complicated code at the beginning of this post, when you can just use a playbin as above? Well, if you are only going to be playing one type of content you can make initialisation of GStreamer quicker and more efficient if you specify your elements manually rather than asking GStreamer to detect which elements it needs and then set them up. If you are in the position of being able to specify your elements manually, you should do so. If not, then you should aim to use a decodebin element to automatically detect the decoders needed for the content and specify the src and sink element(s) manually.

I haven’t given an example of using a decodebin, because I couldn’t make it work (GStreamer continues to be pretty buggy – you may well find that you need to use development versions of plugins or even core libraries in order to make things work as they should).

Edit: I’ve put together a quick, fairly simplified PlayBin example for those wanting something downloadable they can play with. playbin-example.py.

Related Posts:

Installing Debian with Attansic L1 Gigabit Ethernet

I’ve just bought myself a new motherboard and was initially quite concerned to find that the latest daily-built Debian-Installer image didn’t support the on-board Ethernet, especially when lspci couldn’t even identify it. But luckily I was able to track down the PCI ID and find a driver and equally as lucky I had another AMD64 system with the exact same kernel used on the Debian-Installer CD on which I could compile it.

Since most people won’t be so lucky, I figured I should share my compiled kernel module so people with the same problem can simply drop it in and load it. You’ll need this module if you want to install Debian Testing (Etch) using the netinst CD on an Asus M2V, P5B-E, P5L-MX or P5L-VM 1394 motherboard. It will work with the current daily built AMD64 images of the Debian Installer which use kernel 2.6.18-4.

atl1.ko

To use it, copy it to a floppy or USB drive as preferred, then boot the Debian installer. Switch to a console (Alt+F2), mount your floppy or USB device and copy the file to /tmp. Then simply ‘insmod /tmp/atl1.ko’. Now switch back to the installer (Alt+F1) and continue the install normally.

The module won’t get installed by the installer, so once your system is installed and running you’ll need to copy it into /lib/modules/2.6.18-4-amd64/kernel/drivers/net/. With any luck newer releases of the Debian kernel will include the driver, so this process won’t be necessary.

Related Posts:

Playing with iSCSI

I recently found myself needing more disk space on a system, but unfortunately the system concerned can only take one hard disk and I didn’t want to buy a bigger hard disk because I’ve got a load of smaller drives spare. So I figured I’d whack a hard disk into a USB enclosure and connect it to my server (rather than shut the server down to add the hard disk properly). As I’d been waiting for an excuse to play with iSCSI, I figured I’d give it a go rather than share the drive by NFS as I usually would.

There are basically two halves to iSCSI: a target, which is the device you want to share and an initiator, a system that wants to access the shared device. So on one system you need to setup and configure iSCSI target software and on the other initiator software.

I started by downloading, compiling and installing iSCSI Enterprise Target onto my server. Once installed, you need to set-up a configuration file (/etc/ietd.conf) to share the device which looks hideous, but actually it’s not too hard. This is mine:

Target iqn.2007-03.uk.co.demon.lug:myth
  Lun 0 Path=/dev/sdb,Type=fileio
  Alias myth
  MaxConnections 1

The first line has to be in a particular format:

iqn.yyyy-mm.<reversed domain name>[:identifier]

So it’s basically the date, the machine’s domain name reversed and a made-up identifier (I used ‘myth’ as the disk is for use by MythTV).
Then I give it the device node (sdb), give it an alias and specify that only one initiator may be connected at any one time. You can define security (a username and password, allowed/disallowed hosts etc.) but I’m on a secure network so I didn’t bother. That’s all the required configuration, now start the daemon:

/etc/init.d/iscsi-target start

On the initiator machine I downloaded, compiled and installed Open-iSCSI. The configuration of this is a bit more of a pain, but not too bad. Once it’s installed, you’ll need to run ‘depmod -a’ to allow the kernel to find the newly-installed modules.
You need to start by creating a configuration file called /etc/iscsi/initiatorname.iscsi. Mine is:

InitiatorName=mythtvbox
InitiatorAlias=mythtv

Basically you just have to make up a name and an alias, which the machine will use to identify itself to the target.
Now you need to start the iscsi daemon:

/etc/init.d/open-iscsi start

Now you need to configure the daemon to automatically connect to the target when started. First you need to ‘discover’ the target using the iscsi_discovery command:

iscsi_discovery <ip address/domain name>

In my case, this returns:

starting discovery to 10.0.0.254
Testing iser-login to target iqn.2007-03.uk.co.demon.lug:myth portal 10.0.0.254:3260
starting to test tcp-login to target iqn.2007-03.uk.co.demon.lug:myth portal 10.0.0.254:3260
discovered 1 targets at 10.0.0.254, connected to 0

That demonstrates that the connection is working. Now you can configure the daemon to connect automatically at start:

iscsiadm -m node -p <ip address/domain name> -T <target name as above> –op update -n node.conn[0].startup -v automatic

In my case, I did:

iscsiadm -m node -p 10.0.0.254 -T iqn.2007-03.uk.co.demon.lug:myth –op update -n node.conn[0].startup -v automatic

Now you need to restart the daemon. There’s a slight bug in the iscsi daemon which means that it doesn’t shut down properly (it isn’t really considered ‘stable’ yet), so in addition to stopping it via the init script (/etc/init.d/open-iscsi stop) you’ll need to kill it off manually then restart it:

/etc/init.d/open-iscsi start

All being well, the hard disk should have appeared, so you can access it just like a physical disk connected by SCSI or USB – check in dmesg:

Loading iSCSI transport class v2.0-754.
iscsi: registered transport (tcp)
scsi0 : iSCSI Initiator over TCP/IP
scsi 0:0:0:0: Direct-Access IET VIRTUAL-DISK 0 PQ: 0 ANSI: 4
SCSI device sda: 160086528 512-byte hdwr sectors (81964 MB)
sda: Write Protect is off
sda: Mode Sense: 77 00 00 08
SCSI device sda: drive cache: write through
SCSI device sda: 160086528 512-byte hdwr sectors (81964 MB)
sda: Write Protect is off
sda: Mode Sense: 77 00 00 08
SCSI device sda: drive cache: write through
sda: sda1 sda2 < sda5 >
sd 0:0:0:0: Attached scsi disk sda
sd 0:0:0:0: Attached scsi generic sg0 type 0

Now you can enjoy the iSCSI-goodness. Sure sharing with NFS is easier, but apparently iSCSI is faster (actually, it probably doesn’t help very much at all when you’re talking over USB using emulated-SCSI to an IDE disk…).

Related Posts:

WordPress Themes

Bear