Why equipment software cannot run the right-hand side of the V-model

There is probably no equipment engineer who does not know the V-model. Down the left-hand side, from requirements through detailed design to implementation; back up the right-hand side, through unit test, integration test and system verification. It is in the textbooks, and it is in your company’s quality manual.

And yet almost nowhere in automation-equipment development does the work actually follow the diagram. The left-hand side mostly gets written. On the right-hand side, the bottom two rungs are empty.

This is not because the field is lazy.

The left-hand side can be written from experience

The descent down the left of the V — requirements definition, risk assessment, safety requirements specification, system design, detailed design, implementation — can, in fact, just about be carried out.

There are documents from past machines, there is experience of building similar mechanisms, and there is someone who remembers where the last project got stuck. Not perfectly, but between legacy documents and the experience of veterans, the left-hand side gets filled in.

The problem is the right-hand side.

The bottom two rungs of the right-hand side do not physically exist

The right-hand side of the V climbs from the bottom: unit test → integration test → system verification → acceptance verification. In the software industry, the bottom of that stack — the unit test — is the densest layer. You isolate a single function or a single class, feed it inputs, and check the outputs.

Try doing the same thing on an automation machine.

You want to isolate the Z-axis lift logic and exercise it on its own. But the Z axis is wired to a motor driver, wired to a home sensor, wired to the supervisory controller, and bound by interference conditions with the neighbouring axis. None of it moves until it has been assembled into a single piece of hardware. The structure you would need in order to take it apart and test it unit by unit simply does not exist.

Equipment software has no unit tests not because nobody wrote them, but because there is physically nothing to write them against.

The result is that verification rests entirely on integration test → system verification → commissioning on site. The right-hand side of the V is operated with its bottom two rungs missing.

And the equipment industry has run this way, with those rungs missing, for decades. The method is simple — if the machine moves, it’s fine.

When do you pay for “if it moves, it’s fine”?

This empirical form of verification has actually worked. Or at least, it has looked as though it were working. That is why it continues.

The bill arrives when the machine does not move. And it arrives in the most expensive place possible.

Software engineering has an old rule of thumb: the later a defect is found, the more it costs to fix. If fixing it at the design stage costs 1, then integration test costs 10, and the field or volume production costs 100 to 1000. In equipment development, you add rework on site, schedule slip, and resubmission for safety certification on top of that.

But what really hurts is not the figure itself. It is that you cannot isolate the fault.

With unit tests and module-level verification, you can narrow down where the error lives, at least to some degree. Without them, when a problem appears on site, you have to begin like this —

“Is this a design problem or an implementation problem?”

Having to start from that judgement is, in itself, what the verification cost of equipment software really consists of. Think back to how many people and how many weeks a debugging effort took. Most of it was not time spent fixing. It was time spent working out what was wrong.

The industry has made pragmatic compromises

It is not as though nothing has been tried.

• Make design reviews more thorough
• Keep design changes to a minimum
• Use only modules that have already been verified
• Strengthen integration testing

All of these are right. And all of them can reduce errors but cannot eliminate them. A review is read by humans, so humans miss things; and “use only verified modules” also means “do nothing new”.

Try to push one step further, and here is what happens.

Introduce static analysis. The moment you run it over legacy code, several thousand warnings come out. You have no idea which of them are real problems, so the first job is triage. The heaviest day of the whole exercise is day one.

Introduce unit tests. To do that, you first need an implementation that can be decomposed into testable units. Which means paying the entire cost of implementation and build up front.

There is a reason both of these stop at “we know we ought to”.

What is missing is not tests, but a way to check the design

It is worth reframing the question here.

The bottom two rungs of the right-hand side stay empty because equipment offers no way to exercise what has been implemented. So stop trying to exercise what has been implemented — there is another move available: check the design itself.

This is not word games. Look at what we actually write on the left-hand side of the V. The detailed design states what states the machine has, in what order it moves, and under what conditions the interlocks engage. There are a number of questions that can be settled from that information alone.

• Under this design, does a sequence exist in which both axes enter the hazardous zone at the same time?
• Does the recovery sequence from emergency stop contain a combination of sensor states from which there is no way out?
• Does this interlock condition really close off every hazard scenario it was meant to?

Every one of these can be answered without the physical machine. And every one of them is too expensive to answer once the physical machine exists.

What to read next

“Check the design itself” may be an unfamiliar idea in the equipment industry. But it is not a new one. The world has already moved towards designing with models; there is just one rung missing above it.

What that rung is, is the subject of What comes after “design with models”.

How checking a design differs, concretely, from testing it is covered in What is the difference between testing and formal methods?.