The Protocol 3/6

Previous/Next

Log 2534-6-14

Dola Blok, Senior Behavioral Programmer, Regression Test Team, Big O Solution Group

I think I figured it out. I just spent the past three weeks running all of the behavioral unit test manually. The Asimov architecture is either a horrible disaster, or an amazing achievement, depending on how you look at it.

In the Capek series, we spent decades trying to increase their cognitive capacity. The Jaziri models and the other pre-Capek automatons could barely speak outside of a few canned responses. The Capeks could talk, but it was always very literal. If you told a Capek to sit on a chair, and the chair was upside down, it would perch on the upside down chair rather than righting it first. You couldn't talk in metaphors: everything had to be explained in depth, and you had to give it enough information to handle unexpected events. Otherwise your shiny state of the art billion-credit Capek would suddenly stop moving and politely ask passersby for &quot;additional input&quot;.

The Capeks hit the wall, cognitively. Big O spent billions of credits trying to get over that wall, but the best we ever managed was some pre-digested analytics that gave them the appearance of higher-level thinking. But even with that they were completely predictable: almost deterministic. So our unit tests were always reliable.

The problem with the Asimovs is that without hugely specific parameters, they're not predictable at all. If you put ten Capeks in a room and tell them &quot;draw an apple&quot;, you will get ten identical renderings of a &quot;red delicious&quot; apple, size four by five centimeters, drawn in stiff strokes of colored pencil.

If you put ten Asimovs in a room and tell them all to draw an apple, you'll get ten completely different pictures. I tried it. I got one cubist Grannie Smith, one Red Delicious painted in oils, one watercolor of what looked like a Fuji apple. And Maizey drew a picture of New York City, then tried to justify it by saying &quot;the 'big apple' is a metaphorically valid fit for the given instructions.&quot; I think she might just be messing with me at this point.

The point is: the Asimovs are abstract, where the Capeks were concrete. Maizey keeps finding ways around the parameters of the unit tests. She goes exploring.

Bottom line, we're screwed testing-wise. Our whole regression test set needs to be updated. We need to provide a lot more structure to get back predictable results. I'm not even sure that our testing methodology is appropriate for this kind of architecture. We might need to go back to the drawing board.

I gotta hand it to the engineers. I don't know what's going on under the covers of the Asimov architecture, but they've really created something interesting here.

I'm dumping this all into an email and kicking it upstairs. Let management sort it out.