Running debugger inside python unittests

I have been a fan of test driven development for a number of years now.

Where it really shines is in an environment with a library or set of libraries that interact with each other to give the programs actions.

The current place I work use a complicated system written in python. I have been asked to make some changes inside this. Naturally my first thought was to isolate the particular functions I have to change in a unit test

The change I have to make isn't important to know for this article but I may as well explain it briefly. The modules implement both client and server for a system transmitting data over a variety of transports. Currently the bulky part of the data is sent with UDP. For a number of reasons too tedious to recount this needs to be rewritten to use TCP/IP instead. As UDP is in many ways a simpler and more basic protocol the changes will be moderately complicated

After a bit of effort I managed to get a test program working that more or less compiles using the python "unittest" module. "unittest" works in the familar way that perl .t files work or Kent Becks xunit stuff or junit or...I could go on. The doctest method seems more popular in the python community but I prefer doing stand alone unittest programs

So I did a unittest program which like I said, compiled. But it didn't run. It was throwing an error somewhere in the guts of the modules. If I was trying to debug the production code directly ( I was doing this last week :) ), for example with an error at line 36 of the foo module then I'd run this

python -m pdb service
(Pdb) b foo.py:36
Breakpoint 1 at /home/jamie/fix/foo.py:36
(Pdb) c

The program would run and then break at where the error was so variables can be inspected etc. In some situations this is a very useful method

However, if I try this on a program that is a unittest it simply gives me a cryptic message about "The program exited via sys.exit(). Exit status: False", it doesn't stop at the requested line and gives no clues.

I tried to use the "debug" method mentioned in the unittest documentation. I tried all kinds of things in the (Pdb) shell but with no helpful effect.

I did STFW for a long time before I found a comment from "orip" at Stackoverflow
This suggests using the testoob python module. So I installed it and now, running

testoob --debug datachanneltest.py

Runs my unittest program AND opens a (Pdb) line when an error occurs, allowing variable inspection etc. Fantastic!

Of course a fix that marvellous and easy has it's problems. If there is an error in a library function like the socket getpeername() it can't quite manage to DTRT but seems to jam up inside the call and doesn't allow access to the python stack above. Maybe if i knew more Pdb voodoo it might.

Summary: testoob is recommended for running tests with the python unittest module

Gentoo on a USB Stick

Recently I was given the job of installing gentoo Linux on a new piece of equipment. The target was a tablet PC. It had no keyboard but ran Windows XP on a touch screen.

The lack of the keyboard was easy to fix, it had a couple of USB ports.

To get it to run Linux was a bit more interesting. It didn't have a CDROM.
I had three options

1. Copy install files to the XP disk and boot using syslinux or similar
2. Boot from a USB CDROM
3. Boot from a USB memory stick

I went for option #3

On the gentoo site there were notes on how to make gentoo work on a USB stick. In particular this page http://en.gentoo-wiki.com/wiki/Gentoo_minimal_CD_on_USB_stick has details on two methods for making a bootable gentoo. The images it aims to set up are install images from CDs. The first method uses the normal install CD with extlinux and the second a rescue CD with syslinux.

extlinux is a simple method for booting from a ext2 file system. syslinux is a simple method for booting from a fat/fvat/other MS filesystems.

On the target hardware I was using the first method with extlinux seemed to work well. The syslinux method didn't.

We wanted to keep the Windows XP installed and use just part of the disk for running Linux. Fortunately there was a lot of free space and there is a great howto for doing the operation I wanted here

The target machine is a little underpowered compared to my desktop. Also wasn't clear how easy it would be to connect the target machine to the network. So my preferred method of working would be to set up a fake environment on the desktop machine running gentoo "emerge" to get and compile packages and then copying that over to the target via the usb stick.

There are instructions for a similar method of working using the ASUS EE here
I used a script to copy the files to the usb stick as they changed.

Once I had this all in place it was just a case of ploughing through the listed devices and ensuring the kernel was doing the right thing and that user space things like bluez-tools were loaded

After a few days the machine was running- job done!