It could be worse than Terminators

June 2009

Dear Dave,

Never build an intelligent computer.

It's bound to end badly. The exact details of the disaster are hard to predict - the thing might shove you out an airlock, start World War III, imprison you in the Matrix or send an unstoppable robot back in time to kill you - but whatever form catastrophe takes, it's unlikely to be a happy experience. I know this from watching movies, so it must be true...

Thankfully, judging by the utter stupidity of most of the computers I have to deal with in daily life, the rise of the machines is still a long way off. It's certainly far enough in the future to make the possibility of a zombie outbreak a more pressing concern in the short to medium-term than the evolution of Skynet. (Remember: Stock up on chainsaws not armour-piercing bullets.) I mean, if the military really had had a computer capable of playing WarGames with Matthew Broderick in 1983, I'm sure most of the technology would have filtered out into civilian life by now. We should have been having philosophical conversations on the futility of nuclear war with our laptops long ago.

Instead, we have automated, voice-recognition call centres. These give us four options for our customer enquiry, none of which is right, and then pick one at random whatever we say anyway. I encountered a system recently which didn't even have an option to go back to the previous question. When it misheard 'Report a Problem' as 'Billing', I was stuck in a sub-menu dealing with statements, direct debits and Visa cards with no way to escape. I had to hang up and start again.

If that computer ever becomes self-aware and tries to take over the world, I can't see it getting very far.

The armed forces probably do have better artificial intelligence than that but I doubt it's great. Even if they're fifteen years ahead of everything else, the Gulf War must have been full of incidents like this:

Skynet (Beta): Thank you for calling Central Command. Please state your name, rank and serial number.
Soldier: Pulse R., General, 5559781.
Skynet: Good day, Sergeant Pul. Please state the second and fourth numbers of your PIN.
Soldier: No, that's not right.
Skynet: I'm sorry, your PIN has not been recognised. Please repeat your name, rank and serial number, remembering to speak slowly and clearly. If you are in an area of high background noise, you may need to change location.
Soldier (over gunfire): Erm... I've what?
Skynet: Good day, Private Erm. Please state the first and third numbers of your PIN.
Soldier: Just put me through to an operator.
Skynet: I'm sorry, I didn't understand that. Would you like me to put you through to an operator?
Soldier: Yes.
Skynet: In order to ensure you are sent directly to the most appropriate member of our highly-trained staff, please state the nature of your request. Do you wish to make an enquiry about your most recent pay slip, log a fault pertaining to essential equipment, report a sighting of a Weapon of Mass Destruction or request an air strike?
Soldier: I need an air strike right now!
Skynet: All our operatives in the accounts department are currently busy. Please hold for assistance. (There's a click and classical music starts to play.) Thank you for your patience. Your call is important to us...

Things are doubtless more sophisticated than this these days, but if the army has Terminator robots then there's a good chance they could be out-foxed by hiding behind a newspaper, wearing a false nose and moustache or speaking in a very bad French accent. A mildly rutted field or a little light foliage would also be liable to make them stumble over and lie on the ground waving their limbs in the air (for the five or six minutes it took for their batteries to run out).

Nope, I don't think we have anything to worry about from anyone attempting to build a brilliantly clever super computer for a while yet. It's simply too hard.

I did find a far more worrying line of research in the technology section of a local museum, however. It was from a project in the Seventies where programmers worked on developing a computer as intelligent as a five-year-old child. There wasn't much to see other than a robotic arm which played Draughts clumsily but the very idea was almost as frightening as turning a corner to encounter a group of suspiciously grey-skinned people groaning about brains.

The coders were either hugely ambitious or had never met a five-year-old child. Five-year-olds are pretty smart. They may not be able to play Draughts well but they can be cunning, observant and empathetic, not to mention sly, scheming and manipulative. They can outwit adults with ease.

The lack of realism in the project's goal was scary. The truly terrifying part, though, was the thought of what would have happened if, after thirty years, they'd finally succeeded. With that level of cluelessness, the researchers might well have hooked the thing up to the Disney website for a little play while they went out for a couple of drinks to celebrate. Unfortunately, by the time they stumbled back to the lab wearing sparkly, cone-shaped hats and drunkenly blowing tooters, we'd all have been doomed.

Imagine a five-year-old connected directly into the world's networks. Rather than being hunted down by metallic warriors with glowing red eyes, the human race would starve to death as global production was switched from food to bubble mixture, balloons and all the plastic tat you find in the bottom of party bags. Adults would be forced to work all day as slaves, pushing small children on swings and roundabouts. There would be nothing on TV which didn't involve talking animals.


Humanity clearly had a lucky escape when the project ran out of funding decades ago. Nonetheless, just to make sure, I snuck back to the museum after dark and dropped the whole exhibit into a bubbling vat of liquid steel.

You can't be too careful, after all.

Yours in a woman's world,


PS Never mind five-year-olds, even a four-year-old can be ruthlessly logical and devastatingly astute:

Marie was being awkward on Saturday. She wanted to go to the shops with Sarah but wasn't willing to put any effort into getting dressed. She made a big fuss about the effort involved with pulling on each piece of clothing, and had to stop for a rest every few seconds. Then, when I went to rub on her eczema cream, she started dancing. Everything was a struggle.

"Could you be good for Mummy today?" I said, not wanting Sarah to have to put up with this kind of behaviour for hours at the shopping centre. "Do what she says and don't argue with her the whole time."

I immediately realised the chances of Marie going along with this were small, so I decided to give her something easy to agree to, so we could work up from there. "Be nice, and reply properly when she asks you a question. Don't just stamp your foot or say, 'Bah!'"

Marie hung her head and pouted.

"Come on," I said. "How hard can it be to not say, 'Bah!'?"

She screwed up her face and seemed on the point of tears.

"What's the matter?"

"But," sobbed Marie, "she might ask me what noise a sheep makes!"

Somehow, I went from telling her what to do, to having to give her a mint to cheer her up. She was all smiles by the time she left the house - not because she'd acquiesced to be good but more because she knew she'd won yet another victory.


At least the robots would be quick...

No comments:

Post a Comment