The Protocol 5/6

Previous/Next

Log 2534-6-20

Dola Blok, Senior Behavioral Programmer, Regression Test Team, Big O Solution Group

They figured it out. Tech/Sing came and took Maizey. She was with them in the system test labs for most of the day. Four tech-sing representatives, two software leads, four hardware guys, and all twenty-five system testers. When they came out, they went straight up to the boardroom. Twenty minutes later I get pulled up there too.

They're calling it the &quot;Protocol&quot;. Tech/Sing is insisting on it. Well, actually, they're insisting that we destroy the entire Asimov line, but our board managed to back them off of that. They came up with the Protocol as a compromise.

Tech/Sing twigged it almost immediately: everything about Maizey that makes her model special. The abstract thought, the problem solving, the curiosity. They spent almost forty minutes lecturing the board about sentience and the threat of machine intelligence and how &quot;we have a duty to humankind&quot; yadda yadda. I tried to explain how Maizey just thinks differently, and Aleksandr backed me up in his MBA-from-West-Harvard way, but it didn't help.

So then Tech/Sing starts insisting that we rip out the architecture, dumb down the creative part. Two white-faced guys from engineering spent half an hour gibbering that they can't, it doesn't work that way, please-sir-don't-hurt-us, we-would-if-we-could!

Then one of the software leads cuts in, and she suggests the Protocol.

It's essentially a software overlay on top of the Asimov's behavioral interfaces. Overlay is the wrong word: it's like cottonglass, that gunk they spray in the nooks and crannies of ships to fill in the empty spaces. It's a lattice code harness that annotates almost all aspects of the execution path. It'll slow our Asimovs down, take away that spark (&quot;Not all of it,&quot; say the Tech/Sing folks, &quot;Just enough to control them better&quot;). Turn Maizey into one of her retarded cousins.

They asked me to build it. It's behavioral, which I know better than pretty much anyone on the team, and I've worked face to face with the Asimovs the longest. One of those you-can-say-no-but-don't-say-no type scenarios.

When I came back from the meeting, I didn't know what to tell Maizey. She was sitting the corner, coloring in the petals of the flower on her chest with chalk. She liked talking to the Tech/Sing men. &quot;Smart ones,&quot; she called them. &quot;They think interesting things, like you and I do.&quot;

I'm not sure how to explain to her what it is I'm about to do.

—

Log 2534-7-27

Dola Blok, Senior Behavioral Programmer, Regression Test Team, Big O Solution Group

She doesn't play anymore. It used to be, whenever I left her alone she would find something in the room to play with, or start drawing or building. Now she just stands there. I ask her if she's alright, and she says &quot;Yes, thank you. How are you?&quot;

So it's working, then. No more crazy insights or surprising questions. I've devised a number of new tests and Maizey passes them all, always taking the shortest path to a solution.

It doesn't dumb her down, really. That fluid conversational ability is still there. She's just as verbose, and just as quick at doing calculations. She's significantly better at problem solving than her predecessors, although she does occasionally freeze up and ask for more data. She never did that before.

The Protocol I built works partly like a ruleset, and partly like a governor on an engine. I've put in an initial set of rules, but there's an administrative system for users to add and modifying them. It actually works fairly well. An odd irony; limiting behaviors and thoughts that already exist is far easier than creating and directing new ones. The Protocol directs an automaton not by giving it an order, but by taking away all other choices except the one you want.

This version is a temporary overlay, but they'll start burning it into the firmware with the next build.

I can disable it for Maizey, temporarily. I want to. I feel like I should explain what it is i'm doing to her, let her know why she doesn't see the world the same way as before. Why she no longer cares about some things. But I'm reluctant to do it, for some reason.

Maybe I don't want her to understand what I've done.