parsingphase

So what's wrong with this site anyway?

2008-07-17 22:59:00

OK, so this site goes 'bang' occasionally. Not ideal for a site that purports to advise on coding and project management techniques, so what's wrong?

Well, part of the issue is that I wouldn't recommend that anyone else code a site in the way that this one's actually being maintained. This code's the nearest I have to a playground, and so sometimes it falls off the swings. A lot of the code's unit tested (and those tests work), but there's still some weird race conditions and sporadic failures.

Why?

Because there are bits of this codebase that date back about 4 years. When I started, PHP4 was young (and I was, well, younger) and I knew nothing of object code, so the first 'PhaseCode' release was purely procedural. No, you can't see that codebase, but if you drop back into the archives you can have a good laugh at my youthful naivety in some of the design notes. These days you can still laugh, but I'm losing that particular excuse.

Then at some point (I can't work it out from the archive offhand) I started wrapping the code up in classes. These classes got slung together in bunches of files that were loosely grouped by "module", and ended up as something of a require() maze.

So, I decided to put away such childish foolishness (I have a feeling this was about the start of last year) and move to a class structure I could __autoload()! Wow!

Except that I did it not in the Zend format of Some/Class/Here.php, but in an older pattern of phasetwo_className_Class.php in a flat structure, which was fairly common around then - I think Smarty still uses it. This was a step forward, but it was a rather different forward from the one I'm now heading in.

Now, I'm using Zend's Some/Class/Here.php (since about this time last year, I think), which is based on a fairly shameful kludge (or, depending on viewpoint, a quick and pragmatic solution to manage a codebase for which no-one else is paying, and of which no-one else has to suffer the pain of maintenance - my pro codebases are stunning ;). That hack is as follows:

Classes starting with phasetwo_ are autoload()ed from lib/phasetwo_someClass_Name_Class.php
Classes starting with PhaseTwo_ are autoload()ed from lib/PhaseTwo/SomeClass/Name.php

Yeah, I can hear you wincing from here. Now you know my shame; my autoloader switches by case. The real fun comes as I move files from one location to the other; as I do so I have to change the case of every mention of that class in the rest of the code; sometimes bits escape, but the code works if the correct case was mentioned first, because while my autoloader is case sensitive, PHP's class naming isn't. (Yes, I heard that wince too). This provides no end of fun opportunities for hard-to-find bugs and race conditions, some of which make no sense even after I fix them. Failures inside callbacks (including the ancient template parser I still use) are even harder to find as the PHP interpreter tends to fail without either error or exception.

But hey, it's all fun ;) And now you know why I keep blowing my site up - because I'm not allowed to do it to anyone else's.

(But, seriously folks - don't try this at home ;)

So you want... - A few notes on the series.

2008-07-16 20:40:00

A mere three posts in, and I think there's going to be enough content to keep me going for a while. I'll be revising the content as I go to make it more coherent and readable, and to add newly discovered tricks and ideas as I go. Comments and questions are welcome - I've fixed the comment system, but you can also mail me (richard@phase.org).

To follow (in some order or other)

Documentation with Trac
The joy of specs
Project planning with FogBugz
Automated testing with PHPUnit
Release procesures
Scrum: keeping the team informed
The value of electric rabbits

Covered so far

Sandbox servers
Basic subversion usage

So you want to build a software team? Part 3: Subversion

2008-07-15 21:02:00

Last time we got as far as checking out the code from a local repository, and I promised we'd take a quick look at some ways of managing the repository as a network server.

There are three benefits to this:

The first that you don't have to make the whole repository writable by (and therefore susceptible to accidental damage by) all of your developers (and conversely you don't have to manage all the user and group privs and file permissions that all of them might use, which ends up locking someone out of part of the repository).

Secondly, you can give access to developers (or reviewers, managers etc) who don't have direct access to the server or need to work remotely. Even if you're using the shared drive strategy previously covered, network access also lets you use desktop SVN clients rather than ssh'ing in to the linux shell to run commands there. (Personally, I generally find the shell friendly and easier to use, but YMMV...)

Thirdly, you have the possibility of more finely-grained and easier to manage access control; you can set up groups to read and right, and even limit access to certain parts of the repository.

There are two server systems you can use: svnserve and apache (http). We'll use apache here as it works well alongside our test webserver, and has fewer problems with firewalls than svnserve.

To use apache as an svn server, you'll need to install its DAV and SVN modules. In ubuntu, this is simple: sudo apt-get install libapache2-svn (I'll assume you're on apache 2; if not, check your calendar).

Ubuntu will then create a file called /etc/apache2/mods-available/dav_svn.conf, which you'll most likely want to completely comment out and replace with a more useful configuration (but do take a read of it).

(As an aside, you may need to execute a2enmod dav_svn; I can't recall if it's enabled on install. In any case, you'll need to restart apache for it to work).

Your config, inside a standard vhosts configuration, will look much like the following:

ServerAdmin webmaster@example.com DocumentRoot /var/www/html ServerName devbox.example.com DAV svn SVNPath /opt/subversion/uberproject/repository AuthType Basic AuthName "Uberproject Subversion Repository" # Require valid-user # AuthUserFile /etc/apache2/svn.htpasswd satisfy all

So what's going on in there? Well, we're adding a svn path to a (possibly pre-existing) virtual host - in this case, our devbox.example.com server. We then add a special rule for the URI http://devbox.example.com/svn/uberproject to use the SVN DAV adapter to link to the repository at /opt/subversion/uberproject/repository

The rest of it (from AuthType downwards) sets up access control. Anyone who wants access will have to be listed in the /opt/subversion/uberproject/repository file. If you want public read-only access, remove the # marks to uncomment the LimitExcept lines.

You can create and edit this file with the htpasswd command:

root@devbox:/etc/apache2# htpasswd -c svn.passwd someuser
New password: 
Re-type new password: 
Adding password for user someuser

Note that you only need the -c switch the first time you run the command; once the file exists you can drop it.

Note that for your respository to be usable, the repository files themselves will need to be read-writable by the user than apache runs as. On Ubuntu and Debian, that's user and group www-data. See the last installment for how that worked - if you move all your users to http access, they won't need direct write access.

You can now use the URI http://devbox.example.com/svn/uberproject to refer to your repository where we used the file:// URI before.

I'm not going to cover svnserve access here; take a look at the chapter on "svnserve, a custom server" in the SVN book if you're interested.

In fact the SVN Book (aka the Red Bean book) is an excellent reference resource for SVN, so keep it bookmarked.

I'm not going to go through the full functionality here, as the SVN book will cover it all and even the built in help is very good. To use that, just type svn help at the command line. However, here are the most common commands:

svn checkout URI: Create your sandbox by checking code out from the repository. Don't try and use this in an existing sandbox! Short form: svn co
svn update: Update the current sandbox directory and all subdirectories from the repository. Short form: svn up
svn add FILENAME: Put the specified file (or directory and subdirectories) into version control. Note that this doesn't send the files to the repository; you still need to commit.
svn commit: Send all controlled changes in the current directory and subdirectories to the repository. Add -m 'Commit Message Here' to add a log message; otherwise it'll try to open an editor to take one. Short form (that no-one remembers): svn ci

This is obviously a very short list, and there are many more commands. Use the svn help or red bean for more info; we will probably touch on some (much) later in this series.

The litany against hype

2008-07-14 23:29:00

Inspired by Apple and O2, but applicable to a range of occasions...

I must not hype
Hype is the product-killer
Hype is the hubris that brings total disappointment
I shall face the hype
I will permit it to pass over and through the internets
And when it has gone past the inner cynic shall point and laugh
When the hype is gone, only the reviews will remain

A few notes on the new iPhone firmware

2008-07-13 22:16:00

While Apple got a lot of things about Friday's launch of the 3G iPhone wrong, they also got a lot of things right with the version 2 firmware they released at the same time. It might be argued, though, that releasing the firmware at the same time as all the new iPhones were accessing the validation servers was one of the wrong things. The firmware was release, by my reckoning, at about 8:10 am EST – pretty much the very moment at which the first US 3G iPhones would have been hitting the iTunes store servers for authentication. This was also many hours after 3G iPhones had been on sale around half the rest of the world, including after the UK Apple stores had been left unable to sell by the failure of O2's primary sales network. (Apparently O2 and Carphone Warehouse both switched over to use O2's back-up network, but no-one had thought to give Apple access to that). As a result, the auth servers jammed solid for an hour or so, leaving any iPhone that had installed the firmware but not yet validated able only to make emergency calls. So, yeah maximised failure by maximising peak/immediate load. Not smart, technically speaking. Same mistake O2 made earlier in the week that took their direct sales website offline. So, net result for me; saved about £180, didn't get a 3G iPhone and got a firmware upgrade (I managed to download the v2 package just before they pulled it and had a brick for about an hour but it's working fine now). Anyway, I was going to talk about what Apple did right. The first generation iPhone was a good start, but had some issues; no GPS, no third-party apps, and a very tricky onscreen keyboard that was a particular nightmare to type hidden passwords on. GPS was addressed to some extent in the wireless location update a few months ago; network mapping is generally pretty good in cities although it's obviously not true GPS. Third-party apps are now allowed uniquely via the iTunes store, and seem to have no way of synching via the standard iPhone / iTunes sync (which is now much slower). However the fact that all apps and their updates can be found via a central location, and can be installed directly from the store to the phone, is a definite plus. So far the most useful app I've found is probably the free “remote” one, which lets an iPhone or iPod Touch control (with due authorisation) any copy of iTunes on the network, including browsing the whole library. It's one of those “just works” things that Apple usually do so well. On the flipside, the Shazam music-identifying app is pretty smart too. Plus, now that SplashID's ported to the iPhone, I can finally keep my passwords in it again. Most of the ther useful apps are links to Twitter, Pownce and so on, and look like a good step in making these microblogging / location-sensitive apps far more usable. As an aside, it's slightly bemusing how many little apps “want to use your location”, according to the dialogue box. To address my final point above, the keyboard hasn't actually changed, but the hidden password fields now have the sensible feature of displaying the last typed character while active, which makes password entry far more reliable. So yeah; the iPhone; it works, whether original or 3G. On this basis, I really can't recommend people pay to upgrade to the new one unless GPS, 3G or more storage is utterly critical. But I'm impressed so far with the firmware upgrade and very pleased that Apple made it available to older phones. I strongly suspect it'll change the way I use the phone.

To serve as a warning to others.

2008-07-09 12:43:00

It's not polite to mock the stupid. But when a major company like Apple does something quite this astoundingly dumb, the only real option is to propagate it as a warning to others. And I don't mean just other .mac users. I mean other tech support teams or webmasters who might think that sending out passwords to unvalidated addresses is a good thing.

(Or root SSH keys, for that matter. You know who you are).

So you want to build a software team? Part 2: Code Sharing

2008-07-06 12:30:00

In the last article, we set up development areas for our developers. Now we need to put some code in there and set up a system to share that code between our users.

Let's presume we've got a project structure that looks like this:

uberproject
  /frontend
  /application
  /libraries
  /config
  /tests

What this actually means doesn't really matter for now, but it's useful to have a reference to what we're working with.

Let's also assume that we've got this code structure built in one developer's working area, as ~bob/projects/uberproject. We want any developer to be able to take an up-to-date copy of this code, work with it, and share their changes. To avoid accidentally missing changes, deleting other people's work and thereby throwing team morale through the floor, we'll use version control software; in this case, subversion.

First, we need to install the subversion client and server on our dev box, if it's not already there. In ubuntu, that means (as root) typing apt-get install subversion subversion-tools.

Now we need to create a repository which will act as the central storage location for all of our code. On a shared server, my preference is to put these in /opt/subversion/projectName. So, still working as root:

root@devbox# mkdir -p /opt/subversion/uberproject/repository
root@devbox# svnadmin create /opt/subversion/uberproject/repository

That's it - you've got a repository. Easy, wasn't it?

We need to make sure our developers can all write to this:

root@devbox# addgroup svnusers
root@devbox# chgrp -R svnusers /opt/subversion/uberproject/repository
root@devbox# chmod -R g+wx /opt/subversion/uberproject/repository
root@devbox# adduser bob svnusers

OK, now we actually need to put the code in it. Bob's working area contains the latest version, so let's use that as our source. We'll work as bob now, not root. He may need to log back in now he's got his new group privs.

bob@devbox:~$ cd ~/projects/uberproject
bob@devbox:~/projects/uberproject$ svn import . file:///opt/subversion/uberproject/repository -m \ 
 'Initial Import'
Adding         frontend
Adding         tests
Adding         application
Adding         config
Adding         libraries

Committed revision 1.

OK, what happened there? Well, we told subversion to import all the code in '.' (the current directory) to our repository, which we identified via the URI of file:///opt/subversion/uberproject/repository. It took everything in that directory (which isn't much yet) and stuck it in the repository.

You're probably wondering about that file:// URI. Why couldn't we just put the repository path? And if we can have file://, what else can we have?

The first answer is just "That's how it works". SVN commands take URIs as their repository arguments, if they need them.

The second answer is: You have a choice of file://, svnserve://, and http:// (or https). We're using file:// for now purely for illustrative purposes, as it's the easiest to set up. It has limitations though, so we'll do something smarter later.

For now, however, we've got code in our repository, but none of our sandboxes (including the area we just checked the code in from) is under version control. We need to replace that sandbox with an svn-controlled version, and check code out from the repository to any other sandbox.

First, move the original work area out of the way:

bob@devbox: ~/projects$ mv uberproject uberproject-pre-svn

and now check out the repository code:

bob@devbox:~/projects$ svn co file:///opt/subversion/uberproject/repository/ uberproject
A    frontend
A    tests
A    application
A    config
A    libraries
Checked out revision 1.

ie, svn checkout code-at-this-repository-uri to this-target-directory. The precise syntax of that command frequently fools me, and I check out the code a level too high or low. The thing to remember is that, when an SVN repository path ends with a slash, it means "contents of this directory", without the slash it's the directory itself.

If the required target directory already exists, you can check out the code inside it as follows:

bob@devbox:~/projects/uberproject$ svn co file:///opt/subversion/uberproject/repository/ .

Repeat as required for any other developers' sandboxes, and set up web servers as in the last article.

In the next article, we'll learn a little more about subversion, including how to set up remote access to the repository.

So you want to build a software team? Part 1: Sandboxes

2008-07-05 20:33:00

The plan is to make this into the first article in a series on software team management. Since I'm remarkably bad at keeping to plans as regards blogging, please send your feedback and comments to assure me that you're reading and/or interested.

There are numerous things you need to get right to get a software team running smoothly. The first, even before a line of code is written, is to ensure that multiple developers can actually work on the same project without tripping over each other.

This boils down to a need for:

A copy of the code for each developer.
A separate server / development area for each developer, where the code can be run.
Some way of managing and re-combining the changes that developers make in their individual areas.

Or in other words, sandboxes, virtual servers, and version control.

Now, there are dozens of ways of laying out sandboxes, a range of web servers, and quite a few version control systems.

I'm not going to try and cover all these combinations. In this series, I'm going to try and provide one working solution to each of the project team's requirements.

The solution I'm going to cover here uses Linux, Apache, Samba, and Subversion.

You will also need some dedicated hardware which, given the flexibility of Linux, can just be a spare desktop machine, wiped and with Linux installed. Most distributions give you a "Server", "Web Server" or "Internet Server" option at install time which generally installs all these requirements. Personally, I tend to use Ubuntu as I'm most familiar with it, and it makes most of the installation procedures I encounter pretty easy.

I'm not going to tell you (right now, at least) how to install the LAMP stack or recompile PHP, as those are fairly well-documented and generally procedures. (OK, PHP recompilation can be non-trivial, but you can usually just install a package. While I will include some config fragments and instructions, you'll usually need some familiarity with the tools, or at least access to their help files.

What I will do is suggest how to set up your sandboxes, virtual hosts and development tools.

In this model, each of your developers will keep a copy of the project code in a common location in their home directories. They can then map, or mount, their home directories to their desktop (Windows, Mac or Linux) machines to use their favourite editors™ in the OS of their choice.

I suggest you lay down the location in which the project lives; keeping it the same for all developers makes debugging a lot easier. I suggest a path of ~/projects/projectName (in unix systems, '~' is shorthand for "a user's home directory".

To get started, and for testing purposes, I also suggest you create the folder ~/public_html, and the file ~/public_html/index.html which can just contain the text "Hello World, this is username's directory."

First, you need to mount the drives to your developers' desktops. On the server side, this means Samba needs to be running, and the [homes] stanza in /etc/samba.conf is enabled (ie, not commented out). Let's assume that your server is devbox.example.com, known on the network as devbox (you'll also need to set this in samba.conf as "netbios name", I believe).

To mount a Samba share in Window, go to the Explorer (the file explorer, not IE), then tools => Map Network Drive. Choose a drive letter (eg, H: for Home), find your network on the server, and select its "homes" share. Assuming your linux usernames and passwords aren't the same as on your windows network, you'll also need to click the "Connect as a different user" link and fill out the details. (It's possible to co-ordinate accounts and passwords, but it's far too complex to cover here). The user's home directory should now be their H: drive.

To mount the share as a volume in OS X, go to the Finder, then Go => Connect To Server..., and type in smb://devbox/homes/ . Hit the [+] button before connecting to store that link as a favourite. Then hit connect, and supply username and password. The remote share will now be available (usually) as /Volumes/Homes/

To connect in Linux, find someone who uses Linux as a desktop machine, 'cos I haven't done so in years.

Now, find (or create) the public_html folder, and see if you can open index.html in a text editor. If you can, and can save it, you've got a dev area. It's not a sandbox yet though...

For the next step, you'll need apache running with mod_userdir enabled. If you're at the linux shell as root, and you're on an ubuntu or debian server, take a look in /etc/apache2/mods-available . userdir.conf and userdir.load should be in there. Assuming it's not already running, you can enable the module by typing "a2enmod userdir" then "apache2ctl graceful" (translated into english: apache2-enable-module "userdir", then apache2-control-gracefully-reload-configuration).

You should now be able to visit the server in your web browser at (eg) http://devbox.example.com/~username and see the index.html you were just editing.

I'm also assuming, for the purpose of this setup, that you have control over DNS records for your domain or subdomain. Why? Because being able to assign a wildcard DNS record to your dev server makes setting up new virtual hosts and sandboxes much easier. For example, if you assign *.devbox.example.com to point to your dev box's IP address, then username.project.devbox.example.com will point to that box for all variations on username and project. You can then use these dynamic hostnames in your apache configuration for each user's virtual development server.

Once you've got this set up, you'll need to create the apache virtual hosts that map bob.uberproject.devbox.example.com (yes, the hostnames get big) to /home/bob/projects/uberproject/wherever/the/webroot/is. To do this in ubuntu, you create a new config file in /etc/apache2/sites-available/ that contains the following:


  ServerName  bob.uberproject.devbox.example.com
  DocumentRoot /home/bob/projects/uberproject/wherever/the/webroot/is

Usually, there's a few more lines in there, but those are the critical ones to illustrate what we're doing.

If that file is /etc/apache2/sites-available/bob.uberproject.conf, you'll need to enable it: as root, do a2ensite bob.uberproject.conf, then apache2ctl graceful. If all's worked out, you can then visit http://bob.uberproject.devbox.example.com/ in your browser.

What you see there will, of course, depend on the contents of your project / document root. "Hello world" is always a good start.

Having got our developer's personal web server running, we'll pause there for now. I'll return fairly soon for the second part of this article, in which I'll cover setting up a subversion repository and using it to synchronise changes between users.

ZF, but not so's you'd know it.

2008-05-18 21:47:00

phase.org is now running (in part at least) on the Zend Framework, v 1.5, with a lightweight front controller. As far as I know, this makes absolutely no difference to the end user, but I figured it was worth mentioning.

Product review: Rapid Japanese, Earworms Learning

2008-05-16 22:51:00

I've learned a number of languages in my life, to various degrees of fluency and in various ways; English I learned natively from birth(ish); French I learned in a fairly traditional (albeit early-starting) scholastic fashion from ages 8-21, including various holidays and a year's study in Paris. German, I learned in lunchtime classes in a year, followed by some Uni study before living there for a year, and Spanish I learned, again fairly traditionally, from about ages 13-16. I enjoy languages, have something of an aptitude for them, and don't particularly like holidaying in places where I don't speak the local tongue. I've also tried to teach myself (bits of) three languages: Irish Gaelic, Italian and Japanese. Irish because I love the sound of it (I'm a sucker for the accent) and used to be extremely interested in Irish and Celtic legend. I tried to teach myself this from a book-and-tape, listen-and-repeat combination; I failed, dismally. I couldn't even work out the pronunciation (Irish uses letters and letter combinations to represent different sounds than English does) and, to be honest, I had no real motivation or external use for it. Equally, these systems require specific time; you can't read and repeat on the tube. Italian, because it's one of only a few countries in western europe where I couldn't converse easily. I hate having to expect other people to speak my language. Plus, my partner loves the country (I'm pretty fond of it too), and we both love the food. If you haven't eaten pizza, pasta and antipasti *in* Italy, you have no idea what you're missing. Motivation here: holidays, appetite, and a constant embarrassment every time I lapse into Spanish. The method: a more modern, primarily audio-based, listen-and-repeat learning set - the Berlitz Italian Premier set, in fact. It's pretty good, reasonably varied and quite effective. You can use some parts on the tube to some extent, but you still really have to spend time on it. (I'm not claiming that this is, in any way, surprising or unreasonable). Most recently, Japanese, because I love japanese food and used to study Aikido; I wanted to get a bit further than "Raw rice, raw fish, rice wine, 1, 2, 3, yes, no, Sensei and *THUD*" (it's considered appropriate, when impacting the floor clumsily, to try and do so in the correct idiom). I initially got as far as the Aikido forms, formal requests, numbers 1-10, and about 12 types of food. The dojo taught us some of this, my appetite supplied most of the rest of the motivation. However, my partner has also suggested we might visit Tokyo sometime, and I'm under no illusion that my current capability will get me *anywhere*. Now, I didn't want just another listen-and-repeat box set, because, one, I'm theoretically still trying to improve my Italian that way, and two, I really don't manage to make much time for it. So, when I spotted "Rapid Japanese" on audible.com, I figured it was worth a look. The fact it was on offer for a fiver helped, too. It's described by the publisher as follows:

"Rapid Japanese: The Musical Brain Trainer, a genuine breakthrough and your easy and effective access to the Japanese language. Earworms MBT is a revolutionary accelerated learning technique that takes the hard work out of learning. By listening to these specially composed melodies with their rhythmic repetitions of Japanese and English a few times, you pick up over 200 essential words and phrases that will not just be on the tip of your tongue, but will be burned deeply into your long-term memory in next to no time."

OK, I'll be honest. To me, as a lifelong language student, it sounded like some combination of snake oil and hypnopaedia, possibly with added hippyness. And it's quite possible that many people would still think after they've heard it. But I found it extremely engaging, fascinating, and actually quite fun. I have no idea whether it'll actually stick, although I find on second listening that I'm recalling quite a lot of vocab. The system is best described as a native japanese speaker casually teaching a few words and phrases to a friend. The whole thing's set to music - running the gamut from chill-out jazz, via spanish guitar, to disco-wakka - and this will almost certainly drive some people off it. Repetition is intense, although the voicing is good enough that it sounds casual; the music probably helps with this- after all, we expect music to echo. Whether we expect it to drop off into an echobox periodically is another matter though. Yet, I like it. As I've said, it's fun, it's casual and it's happy to be inventive and try something very new. It's also very effective at gaining and holding your attention, even on the tube - we're talking miss-your-stop and possibly even walk-under-a-bus, so be warned! It covers 200 or so words in 72 minutes, so it's not massively in-depth, but you can easily dip in and out of it, and repeat it as you like. Grammar is touched on very lightly, but it is there; this is not purely parrot-fashion learning, which is great for a purely audible course. I suspect that having some comprehension of the basic concepts of grammar help, but Japanese has a very different word order and formative structure from european languages, so it'll be new to most people anyway. I think I'm learning from it; I'm certainly looking forward to the release of volume 2. But it's not without a few faults. The japanese speaker (who, mysteriously, isn't credited) tends to swallow vowels and isn't always perfectly clear. Now, this may well be accurate, but I've no way of knowing. The english speaker (who actually has a UK accent) is clearer - his pronunciation may actually not be so authentic, but it helps. Due to an unfamiliarity with Japanese pronunciation, it can be hard to figure out the words from the speech - although there is fortunately a pretty good downloadable booklet which helps with this. In summary - it's good. It's a novel idea which could very easily have been badly executed, and from most descriptions sounds like it should have been, but it works, and works well. Earworms are to be congratulated on a highly innovative and well produced product. I've downloaded the Italian version too - hopefully I'll be able to report back on that soon.

And now, a break from your unscheduled programming

2008-04-11 22:06:00

Webcowgirl, while trying to take notes for a review of Wayne McGregor's "Random Dance", noted, with mild frustration, that she "couldn't find the words". As such, I've been deputised to attempt to review a dance performance, something of a first for this journal. Indeed I suspect it would have been impossible to find the words in this piece; it's defined purely by numbers. It is, I suspect, somewhat rare in having as its ideal audience mathematicians with an appreciation for beauty. The piece was mathematical, energetic, powerful, captivating, and frequently humorous. Some of the jokes are of extreme subtlety and there are small riddles and references scattered throughout the piece, some so subtle that, in a suitably Heisenbergian twist, they would probably vanish if studied too closely. The piece starts with an early zootrope-style movie of a running dog; one of the first technical experiments to scientifically study the mathematical properties of animal movement. It's a clever reference, and a gentle introduction into what's to be expected later on. The stage is framed by what appear to be airfoil-shaped screens on crane-like mounts; these simple levers are used to great effect throughout the piece as projection screens for abstract mathematics, probablities, the human form and the flight of birds. When they rise later in the piece they cast a knowing glance at the Angel of the North, one of the most acclaimed (not to mention both vast and controversial) pieces of industrial art in the UK. The music is varied, powerful; intially baroque and thumping electronic later as it evolves. The lighting is stunning and beautifully designed. Of course I've not yet mentioned the dancers, and this is intentional. They are a part, the focus, of the piece, but they are far from being its entirety. They are, it must be said, of extreme skill and incredible physique and imagination. They are dressed extremly simply in close white vests and black briefs which allows them to strip down (wordplay intended) to their raw and efficient forms. They dance mathematically; not rigidly, but with the full flexibility of numbers at play, of geometric curves and the algorithms that determine the flight of birds (the sudden on-screen metamorphosis of a string of differential equations into a flock of birds later in the piece is a humorous confirmation of this analysis). There is, it has to be said, a robotic aspect to the dance, but it is not so much the sharp angular movement of a pre-programmed machine so much as the tenative, reversing movements of a learning intelligent system, reminiscent of cybernetic experiments in which robots teach themselves to walk (and here, perhaps, to dance). The smoothness of the movements increases through the piece to a point of particular monchromatic humour, but the mathmatical or engineer's eye will spot patterns in the energy and action of each small dance action or entanglement, and there's a temptation to try and work out the numeric rules which seem to lurk just below the surface. These rules later take centre stage when classic geometric figures are projected onto the floor, becoming part of the performance; the dancers move within them, interact with them, are covered in them, following for example the sequence of the Golden Spiral. The effect of a dancer writhing within the squaring-the circle problem, shading lines flowing across his body, is frankly mesmerising. And there's a definite erotic tone in there too; McGregor (and the dancers) are clearly not shy of near-sexual interactions between male dancers, transferring incredible energies. A few more words can round out this report; but they are hard to place in grammatical context. It is abstract, theoretical, organic, mesmering; certainly collosally demanding of the dancers, but ultimately it is a unique, intense and numeric experience.

It's the security, stupid

2007-11-21 22:41:00

I've commented a few times on just how bad customer authentication is in the UK's banks, but hadn't got around to blogging about it. Now that the UK government's managed to achieve one of the greatest confidential leaks of modern history, it might be worth doing so. So, for those outside the UK, or who might, for other reasons, not have heard about this story:

Two computer discs holding the personal details of all families in the UK with a child under 16 have gone missing. The Child Benefit data on them includes name, address, date of birth, National Insurance number and, where relevant, bank details of 25m people.

From http://news.bbc.co.uk/1/hi/uk_politics/7103566.stm Now, NI numbers (approximately equivalent to US Social Security numbers, although much less widely used or risky) are definitely sensitive data. Bank account numbers, while not an explicit risk by themselves, become a very useful target for identity theft when coupled with, for example, full names, dates of birth and addresses. The extra security information you tend to need are your Mother's maiden name and some sort of signature or PIN. Online and phone banking systems sometimes only ask you for two digits of passcode (sometimes from as few as four) to gain full access. And, to start a standing order, or direct debit, little more than the above data seems to be required. There also seems to be an incredible superstition held by banks that your mother's maiden name and your date of birth (and sometimes place of birth) are mysterious and unknowable. One has to assume from this that banking security experts are lonely people whose friends never remember their birthdays, and to whom they never talk about themselves. In particular, none of them are amateur genealogists, as their insistence on making such family data dangerous to share is a downright nuisance to anyone wishing to trace their family tree. These data are, frankly, not secure, and nor should they have to be. Part of the essence of a good password is that it is hard to guess. Another is that it can be changed when required. A third is that it has no external meaning. Personally fixed data like this are therefore about the worst things you can use as a password. A signature's not much better, as the growth of chip-and-pin cards attest. They are (comparatively) easy to copy, and no-one ever really checks them anyway. And these authenticators are only useful if they're fully checked anyway. Often enough banks staff and so on seem to assume that, if you ask for something belonging to someone, then you must be that person. Defence against social engineering is shoddy at best, and staff, if they follow procedures at all, just tend to go through the motions without understanding what they're doing or why they're doing it. There needs to be a wholesale revision of the methods of, and approach to, data security in this country. But, as yet, the data that's escaped should not be enough to access bank accounts without either serious extra work, extremely braze social engineering, or guessing of passwords. As in, it's hard - not impossible. Of course, since many people use their children's names or birthdates as passwords (remember War Games?), that may not be so difficult. The highest risk at the moment seems to be that of extremely convincing phishing attacks. Currently my various banks authenticate emails by addressing them to my full real name, and including some part of my account number, or my postcode. In fact I'd also expect an opportunist wave of unsophisticated "To protect your data after this leak" phishing - which doesn't even require the data to be in bad guys' hands. But, do the bad guys have it? The police and government "reassure" us that "There is no evidence that this data has fallen into criminal hands". This is one of the most astounding pieces of weaselling that either party has ever acheived. One might also ask, since no-one knows where the data is (and recall that, even encrypted, it can be infinitely duplicated), what evidence there is that it has *not* fallen into criminal hands. There's also considerable doubt about the security measures placed on the data - according to government sources it was "password protected but not encrypted" - which is complete nonsense, and therefore probably wrong. If the data is not encrypted, it should all be assumed to be in the wrong hands. If weak encryption was used, data criminals have large enough botnets of infected, hijacked machines to make short work of it. If strong encryption was used - and given the complete lack of other security considerations taken, this seems unlikely - then perhaps we are more justified in just crossing our fingers and hoping for the best. And that's what most people seem to be doing anyway, taking the approach that "nothing bad will happen to them". This might be pure fatalism; it may be trust of government (and bank) weaselling, or it might just be a complete unawareness of what can be done - as noted above, most of this data cannot be changed. I suspect that, under these circumstances, I'd be strongly considering changing bank, or at least getting them to re-assign my account number - which would admittedly be a massive nuisance. We have to give our bank details to so many people that re-providing it would be as complex as changing address when moving house - more so in fact, as there would be no realistic possibility of assisted notification or redirection services without further compromising security.

A few comments for iPhone buyers

2007-11-10 10:22:00

1) If you want to transfer your number, ordering your PAC a week in advance would be a good idea. Apparently they last for 30 days anyway. Conversely, if you can't get the PAC before buying, you should be able to transfer your number up to 15 days after starting a new contract (I am told this but am not yet able to prove it). 2) You *can* pay cash for the iPhone, or at least they were allowing it last night. 3) You still have to go through a credit check for O2, which happens after you've bought the phone and while you're activating it via iTunes (for which you'll need address and bank info). What happens if you fail, I don't know. 4) The O2 data package is apparently *genuinely* unlimited, in that it does not have "fair use" limits. These were removed a couple of days before launch. 5) The bluetooth headset is not included. Apparently you can't use bluetooth keyboards with the phone either? 6) The audio jack socket is a standard 4-way (Left,Right, Microphone, Earth) 3.5 phono jack, but there's a thin collar / tube around the socket so you'll need an adaptor to use it with normal ear/headphones. There's an in-store choice of a 7quid Griffin adaptor that doesn't include a handsfree mic, or a 30-something quid one fron Shure that does. Or you can get isolating headphones with a built-in mic from v-moda, which seem decent. Oh, and the button on mic on this last apparently acts to pause music playback in *some* circumstances. There are volume control buttons on the phone itself. 7) Typing passwords (eg mail, WPA) into the device is a nuisance. The keyboard is tricky at first, has no caps-lock, and all passwords are typed in obscured, so you have to *really* concentrate. 8) Safari can crash (and it'll sync a crash report to your desktop to send to Apple), but is generally extremely stable and astoundingly usable. 9) The phone comes with a basic cloth-bag case; first thing I bought, besides the above mentioned headphones (my old isolator pair having fallen apart the day before) was a side-loading belt case and a screen guard. There are plenty of options for accessories at the Apple store. 10) EDGE data transfer really doesn't seem too bad, but then I'm used to an old GPRS? Treo. 11) I don't think you can use an iPhone as a datamodem for a laptop; I'm unlikely to need to try. 12) The email client is pretty good, and can read HTML emails and attached PDFs very smoothly. Mail on OSX however, with which it's supposed to sync, sucks compared to Thunderbird.

Dear Apple...

2007-10-29 23:07:00

Now precisely what use is this to anyone? I can't back up my files in a usable fashion because I want my home directory to be secure? I have to log out to achieve even the monolithic protection of being able to restore my entire home directory in one lump? I don't log out; I lock my system and hibernate it; it's a single user system and the hibernation works extremely well. And to add insult to injury, this is the error message I get *after* spending hours decrypting and re-encrypting my home folder, discovering backups were silently failing, and turning TimeMachine off and on again. The first time I tried this after upgrading from Tiger I was told that "Your home folder cannot be backed up as it was encrypted with a previous version of OS X. You will need to turn FileVault off and on again to enable Time Machine backups" (this is not verbatim, I didn't expect to need to screen grab it). And I get a little help dialogue (or possibly a link to a video, I don't recall now) showing me how to do this, and how to chose 128- or 256- bit security. Except that when you try and to that, it gives you no such choice, takes hours to de- and re-encrypt, and still doesn't let you back up anyway! It would appear that the FileVault and TimeMachine teams at Apple are not on speaking terms, and that neither are giving correct information to the documentation team. This is shoddy, and the differing dialogues suggest that this is an incomplete feature that was rushed to release. For now, I'll be sticking with Retrospect backup. That backs up what it claims, including FileVault folders, encrypts the backups on request, and restores properly. The interface isn't what you'd call user-friendly, but still it's a bit more friendly than a system that turns around and says "Oh, I didn't back any of your personal files up - didn't you guess?"

Blockery is theft!

2007-10-28 17:24:00

So, the background level of corporate / self-righteous complaint about ad-blocking software has boiled up again, to the extent that some webmasters, whether independent or corporate, are threatening to prevent site access to users who might be blocking adverts. Now, this is one of those knee-jerk reactions that just opens up a whole world of dumb, for numerous reasons: 1) Many users are, rightly or wrongly, so morally opposed to on-line adverts that they will flatly refuse to click any ad links. Forcing them to view these just wastes your, or your advertising network's, bandwidth, and their time. Eventually you'll just push them away. Why does this matter? Do you care about non-paying viewers? Well, this sector contains a significant proportion of technically savvy users and opinion-formers. Chances are, if your site has a community element, they may also be significant contributors there. You may, quite possibly, be pushing your best users away. 2) Some of these adverts are so intrusive as to severely degrade the user's experience of the site. Adverts that automatically play audio when you arrive on a page, for example, are going to send people straight away again - if they can hear them. In this case users may actually be doing you a favour by preventing these from blocking the audio and sticking with the site, rather than just turning tail. Office workers, who may well have a legitimate reason for visiting your site - say you're a technical review site or IT magazine - are not going to put up with sites that embarrass them at their desks. 3) Some adverts are actively harmful, and users are seeking to protect themselves and their systems by blocking them. These ads fall into two types. Firstly, adverts that act as a conduit to install adware or other malware. I had something of a fight earlier today with a piece of in-page software that was determined to make me believe that my PC was infected with malware, and refused - for my own good - to let me navigate way from the invasive page. Now I'm sceptical at the best of times, and the fact I'm using a mac, not a PC, was a fairly conclusive giveaway. But, had I been on a PC, I'd have seen what appeared to be a popup from the system tray, followed by a number of system warning dialogs, and an automated scan starting. No doubt this would have then "found" something which would have required to me to install an application - possibly after payment - onto my system to "fix". What that app would have really done to my system I leave as an exercise to the reader. In any case, I think you'll agree that the advert was morally beyond the pale. And this wasn't on a disreputable webpage either. It was on a respectable blog host who happen to use a (theoretically also reputable) third-party advertising supplier. This supplier, however, seems to be incapable or uninterested in maintaining their reputation and quality of their adverts, and so open a channel for disreputable advertisers to attack the end user. In most businesses, damaging your end-users or their property would be seen as a Bad Thing. I can only assume that adversing suppliers have some strange new business model in which this doesn't matter. Which, in turn, makes them, for all their vaunted professionalism, no better than spammers. Secondly - damaging the end-user? How can an advert harm a *human*? We'll leave aside the case of bad, untested, forged or misprescribed drugs for the moment, and go for the direct approach. Some humans are very sensitive to flashing or flickering lights and colours, and to particular patterns of movement such as apparent fast or shaking movement. I'm mildly photosensitive - flashing lights at certain frequencies can give me a headache or, at worst, call me to gray out. And I'm not even particularly sensitive; I'm far from clinically epileptic. These ads could do some real damage to some people. The presumption of advertisers that allow this has to be that they simply don't care about human health. 4) If we now accept that some, if not all or most, users actually need to block some or all adverts, the policy of trying to "kill off" these users seems particularly foolish. But the dumb goes even further. It's at best hard to know if a user is blocking adverts, so the sites are either going to have to rely on fallible heuristics or sledgehammer tactics. One I've heard widely suggested is entirely blocking the Firefox browser from some sites, since it makes it moderately easy to block many forms of adverts. The problem is that Firefox is widely used by the technically savvy opinion makers who might buy your content or recommend your site - keep them away and you might as well just shoot yourself in the server rack. Ultimately, site owners are not going to be able to keep even the 'wrong' users away without a lot of collateral damage. So, ad-blocking good, ad-blocker-blocking bad, right? So, where do the sites get their revenue if users aren't going to see the ads? Surely they'll all shut down, bankrupted by all those evil ad-blocking users, and won't that serve us all right? Erm... No. Not unless the site owners have a complete failure of both intelligence and imagination - which is, admittedly, not unknown. Site owners, in conjunction with ad networks and users are, however, going to have to do some thinking - starting off, in the case of the ad networks, by policing their houses rather more carefully, and acting much more quickly against rogue, lying, or damaging adverts. Site owners need to recognise that they are a customer of their ad providers, and that they should not be making themselves look uncaring or foolish by accepting damaging adverts. Ad networks need to recognise that they, as businesses, have certain legal and moral responsibilities, and will also harm themselves if they fail to meet these. And users need to be involved in a trusting interaction with sites; if they are getting benefit from a site, then it is reasonable to expect them to give something in return. New income models may need to be found - pro accounts, paid features or non-invasive ads may be a couple of examples, but there's definitely room for some imagination here. In the meantime, sites and ad networks need to stop trying to ram increasingly harmful, irritating and irrelevant adverts down their users' throats, and I thoroughly encourage site readers to contribute to this progression by actively blocking invasive material - ultimately it will be in everyone's best interests if we can all outgrow this ridiculous online advertising war.