Excellent. After ten years of waiting, I can legitimately use that pun for something.
This is a graph from a Liberal Democrat election leaflet recently posted through my door. 32% is two thirds of 48%, no matter what their graph tries to imply. Proportional misrepresentation, if ever I saw it.
Jeremy has noticed that Labour are to a lesser extent guilty of the same trick.
The Ordnance Survey have made a long-awaited announcement that most of their mapping data is now released for free public use. They announced this on April the 1st.
Pull the other one... oh wait, they really have.
This means anyone can perform analyses that would previously only have been possible by paying to license the data. For example, I've taken the OS Locator data, which is a gazetteer of street names in Great Britain, and used a Python script to produce a list of street names in Great Britain ordered by frequency.
The Ordnance Survey CodePoint database has also been released. It's a cut-down version that doesn't include as much detail as the one you get if you pay, but it still includes what until Thursday had been elusive to non-paying users: a grid reference point for every postcode in Britain. You don't get the correspondence between postcodes and addresses/street names; you still have to license that from the Royal Mail.
Still, it's a huge leap in the right direction. The uses to which this data can be put are innumerable; they even include, but are not limited to, trawling through the street name gazetteer looking for naughty words. Not that I would put a respectful wealth of information to such a frivolous purpose, of course.
On a completely unrelated note, there's a Twatling Road near Bromsgrove.
Somehow, I have managed to remain completely oblivious to the BBC4 game show Only Connect until very recently.
Axel first brought it to my attention in this comment, but this week Jeremy showed me the Only Connect flash game. This is based on the third round in the programme. You're given sixteen words, and you have to spot the connections and put them in four groups of four. Your task is hampered by the subtlety and obscurity of some of the connections and further hindered by the tendency for some words to fall into more than one category. The red herrings don't make it any easier. The grid is, however, crafted such that there is only one solution.
Having tried the flash game with limited success, and with the words "No, that's not a group" ringing repeatedly in my head, I then watched all the previous episodes in the current series, helpfully retained by the BBC's iPlayer for people like me who get addicted to it after one episode and spend the weekend yelping answers at the screen.
It's what you might expect if you combine "BBC4" with "game show". The questions range from merely difficult to effectively impossible, hardly anyone has heard of it, and there aren't any prizes. It's further evidence of the law of game show difficulty; in general, the lower the value of the prize given to the eventual winner, the higher the quality and difficulty the game show. Classic game shows such as Mastermind, University Challenge, Countdown and Fifteen to One don't/didn't need to give away huge prizes, as the possibility of becoming the champion of such a competition is enough incentive for the contestants1. Only Connect is in the same category as these shows, except that its position on BBC4 ensures that most people won't even know about it.
That's not to say it should necessarily move from there, though. If it were transferred to a more mainstream channel, the BBC might be tempted to effect some ill-advised tinkering with the format in fear of the Daily Mail headline "licence fee spent on baffling your children with impossible quiz", so perhaps BBC4 is its natural habitat.
Like many geeky programmer types, I can't own a games console for too long without becoming curious about how to write my own software for it. Console manufacturers don't exactly go out of their way to make it easy for you to do this, though, because in general any method of playing your own home-made program can also be used to play pirated games.
Luckily, my Nintendo DSi is one of the easier consoles to develop for. You can buy a special cartridge that accepts a memory card, which is how you trick the console into running your code. There are even Software Development Kits and Tutorials that show you how to make simple programs. Unfortunately the libraries are written mainly for the DS rather than the DSi, so there isn't support for any DSi-specific features like the cameras.
Still, this is a much better situation than with other consoles like the Sony "you will have what we give you and you will like it" PlayStation 3. This xkcd comic appears to hold true for developer accessibility as well.
Anyway, having received the necessary equipment earlier this week, and studied the various tutorials, today I've finished a simple game.
Yes. It's Snake. Not the most difficult game to write, but complex enough to learn the basics of how to draw things on the screen. At the simplest level you can put the graphics engine in a mode that gives you a frame buffer, which is a 256 by 192 array of two-byte pixel values — the dimensions of one of the two screens — and your program can scribble all over that how it likes.
My Snake implementation, however, uses a slightly more involved means of drawing the graphics: a map of tiles. You define all the 8x8-pixel tiles that could be used (corner of snake, horizontal bit of tail, vertical bit of tail, head at various angles, apple, and so on), put them in a certain area of memory, put the map of which tiles you want where in another area of memory, set a few registers to tell the DS where these places are, and all the tiling and drawing is done for you in hardware. All you have to do is update the "map" area of memory to reflect the current state of the game. The set of hardware-implemented graphics features is much richer than that, of course, as with any modern games console, but that's the extent to which they get used in Snake.
There are a great many homebrew applications for the DS. Someone has even written a ZX Spectrum emulator for it, which helpfully serves as a reminder of a long-departed age when getting your machine to a point where you could run your own programs was literally a matter of turning it on.
My latest foray into the world of bodging together an agglomeration of electronics into something resembling a computer has reached that point where I take pictures of it and put them on the internet. I started work on it on October, and finished most of it by Christmas. This week I did Reversi, and today (yesterday now), this unnecessarily long post about the whole thing.
And yes, I've implemented Tetris on it.
For want of a better name, it's called Albert. It's based on a variant of the Z80 microprocessor, which was used by the Sinclair Spectrum and other home computers of its time, including my first computer, the Memotech MTX512. Building the Albert is a step up from my last attempt at a home-made computer, the hand-held Pickle, so called because it used PIC microprocessors.
A PIC, being a microcontroller, has all its code and memory as well as a number of I/O devices on the chip. However, a microprocessor such as the Z80 needs to be provided with memory and I/O devices that connect to it via a data bus, address bus and control lines. So the block-diagram of my Z80 machine looks like this:
I've shown all the basic connections between modules at a general level, rather than getting bogged down in all the individual components and wires. All the capacitors, resistors, and anything to do with power or ground, aren't shown - just assume every device has a 5V input. Particularly, I haven't shown the Z80's control lines that tell the I/O devices and memory chips whether it wants to read or write, and whether this read or write is supposed to go to an I/O device or to the memory. This makes the block diagram above considerably neater than the circuit inside the box actually is.
When the Z80 starts up, it fetches instructions from memory and executes them sequentially, starting with the instruction at memory location 0. The EEPROM is mapped to this region of memory (0000-0FFF hex) using address decoding logic that basically puts the most significant address bus lines through a series of NOR gates.
I've programmed the EEPROM with the bare minimum code to set up the serial port, load a program into memory from the serial port if necessary (very useful in development, but not really so any more, now it can boot from the SD card) and jump into the program it's just loaded. I've set it up so it loads the first sector from the SD card (luckily the display/SD card module provides commands to manipulate the SD card at the sector level as well as the file level). This sector, known as the Master Boot Record, contains code to find the first FAT16 partition, load the code contained in the first sector of that and run it, and that in turn contains code to load into memory at 1000h (which is where the RAM starts) a file on the SD card called SYSTEM.BIN, which is the program that accepts commands from the user and runs them. Especially in the first two of those three stages, this is pretty much the same as how an ordinary computer boots up. This is a very basic shell, whose job is only to ask the user what .BIN file they want to run, load it into memory at location 8000h, and jump into it.
How did I get the code onto the EEPROM in the first place? By building a separate EEPROM programmer that connects to a real computer's serial port, so I can plug the EEPROM chip into it and send the code to it. That should really be the subject of a different article, otherwise this could go on forever.
SYSTEM.BIN also contains, at fixed memory locations, code to jump to certain low-level routines that applications might find useful, for example, code to draw a window on the screen, or accept a keypress from the user, or draw a shape on the display. This is the system call table, and applications that run at 8000h interact with the hardware using these system calls.
The screen above is after I've run "ls" which is a program that lists the contents of the SD card's root directory, "bat" which is another program that uses the system call to get the battery level from the PIC, and the built-in command "time" which (and here's roughly the same path in a bit more detail) causes the shell program to call the system call that tells the UART to send a message to the PIC to ask it to ask the RTC what the time is, so the RTC can tell the PIC what the time is so it can send it back to the UART so it can interrupt the CPU and give it this information so the system call can return and you can get the date and time on the screen. It's a wonder anything works properly when you consider all the hoops a piece of information has to jump through to get to the user.
Perhaps the most important, and certainly the most expensive, device on the computer is the display. It's a 4D Systems uOLED-32028-P1T display, which is not only a 2.8" quarter-VGA display but also an SD card reader. The processor talks to it, via a UART as shown on the diagram, by a simple serial protocol that is well-documented. This means you can send commands to it like "draw a circle here" or "draw this text there" or "send me the file on the SD card called foo.txt".
The display module also has on it a tiny 8 ohm speaker, which you use by putting a WAV file on the SD card and then sending the module the appropriate command to play that WAV file. You might have spotted "TETRIS.WAV" on the directory listing in one of the images above. Yes, when playing Tetris you can press S to make it play the original Tetris music at a volume so pathetically low you have to press your ear against the little holes next to the display to hear it properly. This tends to affect one's success at the game, so the music is mostly left off.
The computer also has a PIC which controls most of the other devices. The CPU talks to the PIC via the UART. The PIC is connected to a number of things:
The PS/2 port. The PIC is programmed to accept keypresses and send them to the CPU. The UART interrupts the CPU whenever it gets data, so the computer gets told of any keypresses. The computer accepts input from any off-the-shelf PS/2 keyboard. The real-time clock, which is a battery-backed module I bought from Cool Components. The PIC talks to this using I2C, which was the protocol the Pickle's PICs used to talk to each other. The green and red LEDs. The green LED is normally left on and the red one off, but they can be modified if the CPU sends the PIC the appropriate commands. The PIC samples the battery voltage as well (actually, half the battery voltage, for reasons marginally less interesting than it would have to be to make me explain it here), and when it drops below 7.1V it flashes the red light. When it drops below 7.0V, the voltage regulator which supplies power to everything in the system isn't specified to work properly (it's supposed to have an input voltage at least 2V higher than the output voltage, which is 5V), so it drops the non-maskable interrupt pin of the CPU. This causes the CPU to immediately branch to a particular address in the EEPROM, where I've put code to tell the display to switch off and the CPU to halt.
All applications interact with the hardware through a (mostly) well-defined interface. They're all written in Z80 assembler and assembled using the assembler that comes in the GNU z80-binutils package.
I've implemented Pong before - it was for a first-year project at university. The computer was a breadboard-wired thing with most of the components provided, and the output device was an oscilloscope.
This Z80 version has three modes: 1-player, 2-player and 0-player. I ran it in 0-player mode, where the AI plays against itself, and it managed to run for long enough without crashing that I could take the picture below. I thought I'd solved the suddenly-locking-up-and-ignoring-all-user-input bug, but it appears to have come back. All I can tell is that it's sat waiting for the display to acknowledge a command. Anyway, here's Pong running properly for a brief instant, but obviously not too brief as the camera managed to capture four frames in the time its shutter was open.
The Albert version of Reversi is basically more of the same. It's even got a scrolling input/output text window for you to enter your move and read messages. The text window code is all in SYSTEM.BIN and text windows are manipulated using system calls. For the pictures above and below, I've left the computer playing against itself. It looks two moves ahead and uses the ordinary algorithm: What's the best move to a depth of 2? Well, try each possible move, work out how many counters you get, and subtract the score of the best move the opponent can make in reply. How do you work that out? Work out the best reply to a depth of 1, which is trying each possible move, working out how many counters you get, and subtracting the score of the best possible move the other player can reply with. And how do you work that out? Work out the best reply to a depth of 0, by trying each possible move, and find which gives the most counters. Don't recurse any further down, as depth is now zero.
In complex mid-game positions it can take the computer up to about ten seconds to decide on its move, so I decided increasing the lookahead depth to 3 would slow things down too far. In the pictures, the number in brackets after the computer's move indicates the "score" it gave to that move, taking into account the likely reply move and the likely reply move to that. The numbers of large magnitude towards the end are because the AI values a winning and losing position as +50 and -50 points respectively.
Everyone knows what this game is like. Everyone who has ever played the game knows that all Tetris implementations, no matter how fair the programmer intended to be, have a mysterious and difficult-to-pin-down bug that causes it never to give you the long thin piece until after it's no longer useful. Indeed there are implementations which actively promote this as a feature.
My implementation (to my knowledge) has no such feature, and nor does the one I did for the Pickle, but it didn't stop me getting a less-than-average score when playing it to take pictures today.
Note the red light has come on. That's the "your six AA batteries are kicking out less than 7.1 volts across them, you probably want to do something about this" warning.
All computers have an analogue clock somewhere on them. In Z80 assembly, it's the programmer's way of smugly showing that they know how to build a lookup table that vaguely approximates the sine function, so the hands point to the right places, even though this is quite a lot easier than it first sounds. Look, the minute markers are only slightly wonky!
Here's a modified version of the clock application, the now-traditional random clock-based drink-recipe generator "Clocktails".
Even when switched off, your face appears in the display if you look at it from the appropriate angle.
There's still scope for improvement, though; the Pickle has a GPS module in it, but this doesn't. It does, however, have the ability to read the GPS output from the Pickle's serial port. I just haven't written the program to do that.
I also need to look into getting an appropriate C compiler so I can write applications easier. Z80 assembly is reasonably logical and flexible, but time-consuming to write. That's not helped by my habit of giving labels incredibly long names to explain exactly what they do and why they're there.
So, what next? Yes, other than "a life"? Don't know. GPS, probably.
As best as I can make out, this is the format of a driver number in the UK, as it appears on a driving licence.
First five letters of your surname, padded on the right with nines if your surname is shorter than five letters. Date of birth, in the format YMMDDY. 50 is added to the month if you're a woman. Initials of first two forenames, or your first initial and a 9 if you only have one forename. Another 9. Two verification characters, computed from the rest of the number.
For example, John James Jenkins, born on the 1st of June 1975, might have a driver number of JENKI706015JJ9AB.
I get the surname and initials parts. At least that's vaguely logical, even if the choice of a 9 as a padding character is a bit arbitrary. But what the steaming monkey fluid were they thinking when they decided on the date of birth bit? What kind of calendrical malefactor puts the decade digit at one end and the year digit at the other, sandwiching the month and date in between? It's like writing twelve o'clock as 10:00:02.
The British do like to criticise the Americans' horribly misordered date format of month-day-year, but our friends from across the pond need only point out our driving licences to secure instant victory in any date format illogicality-based argument.
I can't find any information online as to the reason behind such a ridiculous way of writing a date, so if anyone reading this has any idea, theory, or even wild speculation about why it's done like this, do speak up.
This article is only going to be useful to a small number of people. If you don't have a Vodafone 3G USB stick, or you do but have no interest in using it on Linux, or you do but you've already got it working on Linux, anything below this paragraph is going to be of limited utility to you.
I own one of these.
It's a pay-as-you-go Vodafone 3G USB dongle modem thing. The theory is you put it in your laptop anywhere in the UK and you've got internet access at a certain amount of money per gigabyte.
On the web there's a lot of information, and even more misinformation, on how to get this thing working in Linux. After appropriate wheat/chaff identification, I've found it's perfectly possible with ordinary Linux drivers and tools, and you don't have to use any proprietary software. I've got it working on my EeePC, which now runs Arch Linux after I got annoyed with Xandros' incessant bubble-mongering ("Your system has been plugged in." I know! Who do you think did it?). To save anyone else bashing their head against a wall, here are the settings that worked for me.
First, I'll identify exactly what model of 3G USB stick I'm talking about, so that anyone who happens upon this information doesn't use it if it's not applicable. It's marketed as "Vodafone Mobile Connect". It's manufactured by Huawei, and lsusb identifies it as a "Huawei E220 HSDPA modem".
Also, the wvdial settings below apply only to the Vodafone network, in the UK, on pay as you go. Any other combination requires different settings.
If you want, you can use the Betavine Vodafone Mobile Connect Card (VMCC) Driver application instead of reading the rest of this article, if your distribution has the appropriate libraries available for it to work. For some people this isn't an option. For example, my EeePC has Arch Linux on it, whose Python implementation isn't compiled with UCS4 support, which the Betavine driver requires. Luckily, it's possible to get the USB dongle to connect to the internet manually.
I'm going to assume you've managed to activate the dongle as the instructions say. It's a while since I did that, so I don't remember the exact details, but I think you have to put the SIM card in a phone and make a call with it.
Getting Linux to recognise your USB modem as something that provides internet access is just a matter of sending the right commands to it. When I insert the USB modem, a device /dev/ttyUSB0 gets created. This acts like any other modem on a serial port. If you want you can fire up a terminal program such as minicom and spam AT commands to it. If you don't know what any of that means, just ignore it. /dev/ttyUSB1 might get created as well. The numbers might differ depending on what other USB serial devices you already have attached to your system.
If you don't get the /dev/ttyUSBn devices, maybe you don't have the usbserial module installed. Install it and try again.
The package you need to talk to the modem is wvdial. Install that if it isn't already installed. You'll also want pppd - it's likely your package manager will insist on installing it along with wvdial if it's not already installed. After wvdial has sent your modem the appropriate incantations, pppd lets your system know that it can use the modem as a network interface.
Your /etc/wvdial.conf file should look something like this:
[Dialer Defaults] Init1 = ATZ Modem Type = Analog Modem New PPPD = yes Modem = /dev/ttyUSB0 ISDN = 0 Phone = *99***1# [Dialer vodafone] Username = web Password = web Baud = 460800 Stupid Mode = 1 Init2 = ATV1 Init3 = at+cops=3,2 Init4 = AT+CGDCONT=1,"IP","pp.internet" Phone = *99# ISDN = 0 Modem Type = Analog Modem
These settings worked for me. Once again, this is for Vodafone 3G pay-as-you-go in the UK; it's unlikely to work for anything else. The Init2 line, where it says ATV1, tells the modem to send verbose messages back. The Init3 line tells it to use HSDPA (otherwise known as 3G) rather than GSM or GPRS. Finally, the Init4 line with the AT+CGDCONT command on it tells the modem to make an internet connection using the access point pp.internet.
I've read many conflicting reports about what the Vodafone 3G access point name is for pay as you go customers in the UK, and most of them seem to be wrong. It's pp.internet. Not "internet", not anything dot vodafone dot anything, just pp.internet.
The username and password are web and web. It could be that you can put anything in here, I don't know. Either way, you don't need a username and password. Authentication and billing are accomplished by the fact that the mobile network knows what SIM card the call is coming from.
The dial string *99# tells the modem to make a data call. *99***1# seems to work as well.
The wvdial.conf above describes a profile called vodafone. To initiate a connection, make sure the dongle is plugged in and run wvdial vodafone. I find I have to do this as root. It might be possible to allow a non-root user to run it after suitable manipulation of permissions and groups.
When you run wvdial vodafone, you should see output like this:
[root@scone graeme]# wvdial vodafone --> WvDial: Internet dialer version 1.61 --> Cannot get information for serial port. --> Initializing modem. --> Sending: ATZ ATZ OK --> Sending: ATV1 ATV1 OK --> Sending: at+cops=3,2 at+cops=3,2 OK --> Sending: AT+CGDCONT=1,"IP","pp.internet" AT+CGDCONT=1,"IP","pp.internet" OK --> Modem initialized. --> Sending: ATDT*99# --> Waiting for carrier. ATDT*99# CONNECT --> Carrier detected. Starting PPP immediately. --> Starting pppd at Thu Jan 14 21:11:18 2010 --> Pid of pppd: 1762 --> Using interface ppp0 --> local IP address 10.97.67.181 --> remote IP address 10.64.64.64 --> primary DNS address 10.205.65.68 --> secondary DNS address 10.205.65.68
Once you see it giving you DNS details as above, you should be connected. Run ifconfig and you should see a PPP interface:
ppp0 Link encap:Point-to-Point Protocol inet addr:10.91.91.86 P-t-P:10.64.64.64 Mask:255.255.255.255 UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 RX packets:7 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:3 RX bytes:118 (118.0 b) TX bytes:157 (157.0 b)
You are now connected to the internet, without having to use the Betavine VMCC driver. To disconnect, I find giving the wvdial process a Ctrl-C works. There might be a cleaner way to do this.
According to my telephone, anyway. I wasn't aware of BT's plan to abolish December, but here it is.
Here's the link to the AVI file in case the video-embedding stuff above doesn't work properly.
I've been watching some of the clips from the recently released Tomorrow's World Archive. Predictably, the ones of particular interest to me are the computing-related items. There's a report about Europe's first home computer terminal from 1967, which the narrator informs us is connected by the telephone system to a central "brain". You can send and receive correspondence, read your bank balance and keep informed of stock prices, all for £30 per month! Further to those parallels with today, the four-year-old child uses the terminal with more confidence than some of today's adults.
Even better, though, is the "school computer" report from 1969. At 1:10 we see a group of eleven-year-olds simulating a processor adding two binary numbers - obviously the real computer was in use by another class.
But seriously, the physics teacher in charge of that school's computer project deserves a lot of credit. It's difficult to imagine school IT teachers of today allowing the children to experiment with the computer for themselves and learn how to make their own programs, letting them write games, and even tasking the pupils with fixing the computer when it develops a hardware fault. I'd expect something more like "right, nobody touch the... 'comm'... 'putter'... it's a delicate piece of electronic equipment which youngsters can't possibly hope to understand. Here, take this pen and draw a spider diagram about how IT benefits society."
Anyway. At some point in the future not disclosed by the BBC, parish records from 1539 onwards will be made available online. These include the victims of the 1665 London plague. The deaths were helpfully summarised in "bills of mortality" issued weekly. This happens to be the one from 344 years ago next week:
(Click for bigger. Clearer version here.)
Note how all different deaths are printed on their own line, from Plague: 7165, to Killed by a fall from the Belfrey at Allhallows the Great: 1. It's amusing to think of the person compiling the bill of mortality spending that week gloating to his peers "I told you! You've been making fun of me writing 'Burnt in his Bed by a Candle at St Giles Cripplegate: 0' on these things every week for the last twenty years, but this week, it's finally happened! And no, I was nowhere near St Giles Cripplegate on Wednesday evening, your honour." Unfortunately, it seems they added in the more unusual deaths only when they actually happened. Either way, when this database of parish records goes online, one of the first queries I want answered is something like the following:
select cause_of_death, count(cause_of_death) c from deaths where c < 10 group by cause_of_death order by c
In other words, give me all the causes of death that happened fewer than ten times over the entire length of time covered by the database, with the most unusual first. It'd read like one of those nethack "stupid deaths" lists, or perhaps Wikipedia's list of unusual deaths (which feels the need to point out that it isn't an exhaustive list). Someone might even manage to use SQL injection to insert bogus death records into the database - bonus points for the perfect blend of creativity and plausibility.
More carefully itemised directories of misery and death can be found here - search for "bill of mortality".