Coloured call graphs!

Juha Jeronen has added some features to the Python call-graph generator (pyan) I’ve previously blogged about. With a single command line, I can now get pictures like this:

pyan.py -c backup.py journal.py journalcmd.py links.py --dot | dot -Tpng > backup-use-and-def.png

A bit of colour always shows a program’s structure in a better light. Green for the backup program, orange and blue for high-level and low-level journal manipulation, and yellow for link creation.

As well as automatic colouring by namespace, there are options to control whether use- and define- edges are shown, and whether nodes in each namespaces are grouped together. For instance, instead of drawing define-edges between namespaces and their members as above, we could omit those edges and instead group them. The “fdp” layout algorithm seems to render best for this graph:

pyan.py -n -g -c backup.py journal.py journalcmd.py links.py --dot | fdp -Tpng > backup-use-and-group.png

Juha’s colour and grouping enhancements are implemented for the GraphViz output. pyan also outputs .tgf files for yEd. But “trivial graph format” is just too trivial to support these options, so I may soon extend the program to write the more advanced .graphml format.

Simple but efficient backups

Backups are perennially on my To-do list. Just having them would be good, but more than that I want to automate their creation and maintenance.

My problem, of course, is I prefer to make my own system for doing this rather than use someone else’s. This doesn’t always work out. One of my early attempts was coming along great, and I decided to test it — on its own source directory. It was soon apparent that I had transposed the from and to arguments to the copy function, as the backup program’s source files had all been truncated to zero length. Karmic neatness aside, there is a lesson there: always make a good backup before testing a backup program for the first time.

New SSD

A couple of Fridays ago, a solid state drive — the 256GB Crucial M4 — on which I’d been keeping my eye became available again at a reasonable price, so within a few minutes I placed an order. It arrived on Monday, swaddled in bubblewrap (a useful material to keep around for future eventualities — which inevitably turn out to be the fun of popping bubbles).

It came in a small box, sealed with a sticker but with no accompanying material beyond the packing foam. This seemed to indicate either that the drive was so simple and idiot-proof that no instructions were needed, or that it would be so complicated that prior experience was necessary and a mere few pages of instructions just weren’t going to suffice. It transpires that the drive was not quite idiot-proof after all, as demonstrated in the following account, which I will punctuate with some of fundamental lessons I’ve learned in the last two weeks.

Curious line-endings in FTP

Whilst hurriedly implementing basic FTP support in a program that’s due in a couple of days, I ran into a strange phenomenon:

• Retrieving ftp://login:password@server/data.csv, a multiline text file, will return the file intact.
• Retrieving ftp://login:password@server/data.dat, another multiline text file, won’t: all the data will be on one line.

Is software “engineering” ?

(TL;DR – Yes, but it isn’t architecture.)

This question, and the subject of vocational nomenclature in general, has been grinding my gears for ages, and deserves a rant of its own.

Sometimes it is difficult to know what to call myself. Internally, I’m a programmer. I like writing programs, I can do it reasonably well, and I can envisage useful occupations where programming is the bulk of my work. I read (but don’t necessarily do all the exercises in) books such as The Art of Computer Programming.

The problem is that mere programmers are frowned upon by the business world, which generally looks more favourably on developers, systems analysts, and most of all, software engineers. Programmers do little more than type the code that their more expensive betters tell them to, after it has been filtered down through layers of actual design.

Sleeping in loops considered harmful

NetBeans gave me an interesting code hint today: Thread.sleep called in loop.

DOS filenames are still useful

TL;DR – If you end up with an invalid filename on Windows, you can manipulate the file using the secret DOS name.

Using the NTFS journal for backups

This post (in draft for almost 18 months) describes my amateur understanding of an interesting and useful NTFS feature, the USN Journal, and shows how I’m using it as part of a simple backup program in Python. It also gives some examples of how low-level Windows system calls can be made in Python, using the pywin32 modules.

Sequent calculus fun with Logitext

(This post is overdue; I’ve been more than usually busy at work on a new project, and have not had the energy to do anything interesting when I get home. Well, I saw this a while ago and today out of the blue remembered something interesting in relation to it. So I’ll start by posting about Logitext since it’s a pretty nicely made tool, and because the sequent calculus always deserves a bit of publicity.)

On Reddit some time ago, I saw Logitext. It’s a web-based sequent calculus explorer. It’s like a short text-book on the sequent calculus, with one special feature: the examples in it will change in response to input.

A small contribution to PostgreSQL beta testing

PostgreSQL 9.2 has been in beta for a few weeks, and (since they asked for volunteers) I thought I’d do a bit of informal testing. I’m also eager to see the effects of Index-only scans and other performance improvements (which I’ll report on when I get round to investigating them properly).