Posts Tagged ‘development’

« Older Entries

It’s MythTV, but lighter!

Monday, March 24th, 2008

Over the last two years I’ve been intermittently working on a light-weight frontend application to work with MythTV. Basically I was impatient with the slow loading-time and somewhat resource hungry nature of the proper MythTV frontend application. This is OK when I want to make use of the various PVR features, but sometimes I just want to tune to a digital radio channel and minimise the application or simply watch live TV, so I don’t need the advanced functionality MythTV offers.

And so LightMyth was born. LightMyth is still very much in development, but today I’ve got it working sufficiently to make a release – version 0.5. At the moment I’ve got it playing back audio-only channels perfectly, but TV playback is still somewhat broken (you can get audio or video, but not both at the same time) – this seems to be a threading issue.

LightMyth

LightMyth is written in Python (yes, maybe Python isn’t a great choice for a lightweight application, but writing the same thing in C or C++ would have taken a lot longer) and uses the very good GStreamer framework for the audio/video playback. It has a (currently very unpolished and feature-incomplete) GTK frontend. It is quite happy to work with a remote MythTV backend over the network and should be equally happy with a local backend running on the same machine.

There are three main reasons why such a simple-sounding application has taken me nearly two years to get functional. The first is that I’m lazy. The second is that when I started, GStreamer was somewhat immature and I was frequently coming up against bugs or missing features that were preventing me from doing what I needed – especially when it came to demultiplexing the MPEG2 transport streams from digital TV. These issues now seem to have all been resolved, and GStreamer is doing everything I need.
The third reason is that I was reverse-engineering the Myth protocol. You may well ask why, considering MythTV is Free/Open Source. Well trying to read someone else’s source is hard enough, but trying to decipher it and figure out the inner workings of a network protocol is even harder. I found it much easier to simply run Wireshark to capture the commands being sent and received by the MythTV frontend, then figure out what they’re doing and implement them into my application.

So I’ve made a fairly wild claim that my application is ‘lighter’. In a very, very non-scientific test I started the MythTV frontend, tuned to a radio channel (BBC 6 Music, in case you’re interested) and minimised it (note there’s no MHEG/interactive stuff enabled, so all its doing is receiving the MPEG stream over the network and playing it back). This consumed 33% of my CPU and 12% of my memory.
By comparison, my LightMyth-radio frontend playing back the same radio channel consumed just 1% of my CPU and 2.7% of my memory. I was pretty surprised by these figures – my main desire was to get a frontend that started-up quicker and I hadn’t really been concerned with CPU or memory usage, but the difference between the two is fairly substantial.

It’s fairly basic at the moment, but it has a nice configuration wizard to assist in setting it up. I will add basic support for the digital program guide at some point (maybe just now & next) but don’t expect any of the PVR functionality (scheduling or playing back recordings etc.) as this is not what LightMyth is for (this is what the main MythTV frontend is for).

If you’re feeling brave, you can go and grab a copy. The license is GPLv3. Any feedback or bug reports would be gratefully received, as would simple success/failure reports – there’s only so much I can test, due to lack of hardware. Digital satellite and cable playback is completely untested, for example…

Tags: , , , , , ,
Posted in Programming, Technology, free software | No Comments »

GPL++

Friday, June 29th, 2007

Today sees the release of the GPL version 3, a license that will either ensure software freedom in the future or split and ultimately destroy the free software community (depending on who you speak to).

As yet, I’m undecided as to how I’ll be using the GPLv3 for my own projects. On the one hand, I do want the extra freedom-ensuring clauses v3 adds, but on the other hand I don’t want to make my code unusable by the many, many applications out there using v2. I suspect for the moment I shall leave things as they are: licensed under v2 with the ‘any later version’ clause so anyone wanting to use my code under v3 code can do so. At some point however, I will have to switch to v3 properly else my code won’t benefit from the extra protections in v3. Before I can do this, I’ll need to thoroughly investigate compatibility issues – e.g. is something it depends on v2 licensed (generally most core libraries under Linux seem to be under a BSD or similar license, leaving the developer free to choose their own license for their work).

One of the main things people seem to dislike about the new license is the anti-tivoisation clauses. Personally, I find tivoisation very irritating and often there is no excuse for it. My company bought an expensive NAS device from Thecus – it’s essentially a PC running Linux. The firmware was/is fairly buggy and there were initially some very serious issues (e.g. a 2GB file size limit imposed by the versions of NFS and Samba it was running). I wanted to fix these problems, which would have been easy to do, but was unable to do so. Sure I could get the GPL-licensed source code, but the box wouldn’t run my modified binaries without them having been encrypted by Thecus. That’s not fair, and it removes my freedom to use free software.

While I’m talking about the GPL v3, I’d like to publicly state that all the contributions I’ve made to GPL-licensed projects may be considered licensed with the ‘any later version’ clause – so if I’ve contributed to your project where the license stated GPL v2-only, you do not need my permission to change to v3 or add the ‘any later version’ clause; you already have it.

Tags: , ,
Posted in digital rights, free software | No Comments »

Packaging for Debian

Wednesday, April 25th, 2007

I’m now the proud maintainer of a package in the main Debian repository. This package contains isomaster, a graphical ISO image editor for Linux. I’ve been using it for a while and originally started producing unofficial Debian packages. I thought it would be impossible to actually get the package included in Debian, since the process seemed to involve a lot of bureaucracy, but actually it was more time-consuming than difficult. I just had to properly Debianise the source code (can mostly be done automatically thanks to dh-make) to allow for automatic package building then jump through various hoops to make sure the resulting package conformed to Debian Policy. Once this was done and the various automatic package-checking tools were happy I used the Debian Mentors system to get the package into Debian, which involves a Debian Developer manually checking the package, then when happy uploading it to Debian on my behalf. Once this has happened there is a final manual check by the Debian ftpteam before it moves into the main repository.

It was a long process, which is why I was so happy to see this the other day:

Accepted:
isomaster_0.8.1-1.diff.gz
to pool/main/i/isomaster/isomaster_0.8.1-1.diff.gz
isomaster_0.8.1-1.dsc
to pool/main/i/isomaster/isomaster_0.8.1-1.dsc
isomaster_0.8.1-1_i386.deb
to pool/main/i/isomaster/isomaster_0.8.1-1_i386.deb
isomaster_0.8.1.orig.tar.gz
to pool/main/i/isomaster/isomaster_0.8.1.orig.tar.gz

Override entries for your package:
isomaster_0.8.1-1.dsc – optional otherosfs
isomaster_0.8.1-1_i386.deb – optional otherosfs

Announcing to debian-devel-changes

Thank you for your contribution to Debian.

If you’re running Debian Unstable you can now apt-get install isomaster (or use your apt frontend du-jour). It should move into Testing (Lenny) in the not too distant future. Before anyone asks, no this does not mean it will appear in Ubuntu. If Ubuntu want to pick up the package and include it in one of their repositories they are free to do so, but I have no interest in submitting or maintaining it there.

Tags: , ,
Posted in linux | No Comments »

Setting lots of permissions

Friday, January 5th, 2007

I thought I’d share a quick script I’ve written to set the permissions for a directory tree. Of course it’s easy to do in a single command if you want the same permissions for directories and files, but that’s almost never the case. In home directories, I need the permissions for files to be 0600 (so the owner only has permission to read and write, but not execute). However for directories I want 0700 (same as before, but with the execute bit set) otherwise the owner won’t be able to enter the directory. I could find no quick or easy way to do this.

I initially tried to write this script in BASH, by essentially tying together ‘find’ and ‘chmod’ but it just wasn’t having it. I needed to read the output of find into an array, but it was interpreting spaces in filenames as separators and so was messing things up – nothing I tried could make it only interpret newlines as separators. So I gave up and wrote it in Python – here it is:

#!/usr/bin/python

from os import chmod
from sys import argv
from os.path import walk
from os import access
from os import W_OK

# get the target directory from the command line
targetdir = argv[1]

def setperms(arg, dirname, names):
  writable = access(dirname, W_OK)
  if not writable:
    print “Ignoring directory “+dirname+”; not writable.”
  else:
    # set the perms of the directory
    chmod(dirname, 0700)
    
    # set the perms of files in the directory
    for item in names:
      # generate the full relative path
      item = dirname+”/”+item
      # can we write the individual file?
      writable = access(item, W_OK)
      if not writable:
        print “Ignoring file “+item+”; not writable.”
      else:
        chmod(item, 0600)

# walk the target directory, calling our function as we go
walk(targetdir, setperms, 0)

Download this code

Simply edit the file to set the permissions you’d like setting – the first (0700) is for directories and the second (0600) is for files. Then call it with the starting path as the first (and only) argument, e.g.:
./setperms.py ./home

The first person to give me the 5-line BASH equivalent gets a punch in the face ;-)

Tags: , , ,
Posted in linux | 2 Comments »

Kernel hacking for fun but not profit, part I: Tasklets

Thursday, December 14th, 2006

I’ve been doing quite a bit of kernel hacking recently and am learning a lot, but for some reason it only just occurred to me that I should be documenting some of the things I’m learning. Learning to hack the kernel is not easy and often there isn’t much documentation (I have Linux Kernel Development by Robert Love [Amazon link] which is really useful – it’s by Novell press, so please buy it second-hand rather than give Novell your money).

I should really start at the beginning, but I’m not going to, since I can’t remember that far back and I’m not sure that there is a sensible place to start in any case. I’m currently working with tasklets, so that’s where I’m going to start.

When writing an interrupt handler in the kernel, there’s one key rule: you need your interrupt handler function to return as quickly as possible. To make that possible, we split the interrupt handler in half: a top-half and a bottom-half. The top half is our main interrupt handler function which does the bare minimum: work out what we need to do in response to the interrupt. The most-used way to implement a bottom-half is a tasklet. A tasklet is a function much like any other in C, except that you don’t call it, you schedule it. When a tasklet is scheduled, it is not run immediately but scheduled to run at some point in the future. This allows your interrupt handler top-half to return quickly without having to wait for the bottom-half to finish.
In a tasklet, we do the actual work that we need to do in response to the interrupt we received – this could be manipulating data structures or reading/writing to hardware.

Here’s a simplified example of an interrupt handler and a tasklet bottom-half based roughly on the code I’m currently working on:

// tasklets are in interrupt.h
#include <linux/interrupt.h>
…
// declare interrupt handler function
static irqreturn_t interrupt_handler(int irq, void *dev_id);
// declare tasklet function
static void handle_disk_removal(unsigned long data);
// declare tasklet
// prototype:
// DECLARE_TASKLET(tasklet_name, function_name, unsigned long data)
DECLARE_TASKLET(disk_removed, handle_disk_removal, 0);

…

static irqreturn_t interrupt_handler(int irq, void *dev_id)
{
  // read interrupt source
  u8 interruptregister = i2c_read_8574(CTRL_ADDR);

  // acknowledge the interrupts to the interrupt controller
  i2c_write_8574(CTRL_ADDR, 0xFF);

  // determine the source of the interrupt
  // NOTE: this is not the right way to determine the source; this is a simplified example
  switch (interruptregister) {
    case 0xFE:
    // we know that when the interrupt register is 0xFE it means that a hard
    // disk has been hot-swap removed
    // schedule our disk_removed tasklet to run.
    tasklet_schedule(&disk_removed);
    break;
  }
   
   …
   
}

static void handle_disk_removal(unsigned long data) {
  // manipulate our disks data structure
  // print a kernel message
  // whatever else we need to do
  …
}

You don’t need to worry about most of the code in the interrupt handler (since every one is different), it is the tasklet_schedule function that is important.

When we declare our tasklet, we give it a name (disk_removed) and give it a function to call (handle_disk_removal). We also have the option to pass it some data in the form of an unsigned long but we don’t need to, so we just pass 0. Incidentally, I don’t think there is one place in the kernel where someone actually passes a value to a tasklet – most often you’ll need to access something that isn’t an unsigned long, so you’ll use a (properly locked) global variable or structure instead.
Now we’ve declared the tasklet, getting it to run is a simple case of calling tasklet_schedule and passing the tasklet name. This will cause the tasklet to run in the future – we don’t know (or care) when, but we can be sure that it will be run. If it gets scheduled more than one before it gets run, it will only be run once.

So tasklets are actually very simple to use. The hard part comes when you need to share data between regular functions, tasklets and your interrupt handler. You have to use proper locking to make sure nothing nasty happens, but locking deserves a post of it’s own, I think.

Tags: , , , ,
Posted in linux | No Comments »

« Older Entries