Reducing the Cost Test and Verification with Integrated Testing

Test and verification raise some significant questions about defining an approach to commercial spacecraft equipment becuase this a major source of cost, especially if you lump test and verification together with inspection and receiving the way it's done today in the U.S. government space program.

These cost issues have their origins in the Apollo era. They wanted a connector to hook up to those oxygen lines on the front of the suit. The suit side of the connector was already designed, so they needed a new custom design for the connectors on the lines. The connectors had to be lock-lock, meaning it took three separate actions to disconnect them.

They went to a vendor who made excellent mechanical hardware for the new connectors. The vendor could do it, and came up with a connector that cost about $600 each. (That sounds like a lot for a little fluid connector, but remember this lock-lock gizmo had 58 precision-machined parts, each one made by hand.)

The trouble was, the vendor didn't have a way to "flight qualify" the parts. He wasn't set to do the paper trail or cleaning or inspection required by the specifications. And he wasn't interested in getting set up to do all that stuff just to provide a handful of connectors for space suits.

So when NASA received the parts, they sent them to Air Research, where folks built the paper trail. They also disassembled each oxygen connector, cleaned it (with freon and stuff), and inspected it under a microscope. They carefully documented these procedures and had the requisite number of inspectors carefully observing each procedure. Result: the cost of the $600 connector grew to $7,000.

Now, we're not talking about greedy contractors soaking the government here. Every man-hour invested was required by the specifications and government procurement regulations; it was people doing hard work for a reasonable salary, but every man-hour adds up.

All this might have seemed like a good idea at the time, but in retrospect, we cannot offer any valid technical arguments for spending that much money on those connectors.

Questions abound about just this one scenario:

1. Why the lock-lock mechanism? (In the unlikely event the connector did come off, a ball detent would seal the valve, the suit would overpressurize, and the oxygen flow would shut off.)

2. Why not use off-the-shelf parts from deep-sea diving equipment? (Yes, they needed a connector to fit the existing suit connector, but why not change the suit connector? Nobody here knows; and EVA engineering is my day job.)

3. Why not trust the lab techs to do their job without two or three quality inspectors looking over their shoulders?

4. Why measure and test each connector, when you could get a better test by mating it to its counterpart connector and doing an integrated test of the whole suit all at once?

5. Why not write the spec for matching pairs of connectors? (Each Apollo spacecraft and the space suits that went with it was one-of-a-kind. So what if the Apollo 14 suits couldn't mate to Apollo 11 hookups? These systems never came together.)

The problem goes way beyond just the Apollo space suit gas connectors. Apollo provided the genesis of these expensive, inexplicable practices, but they are still with us today. Today, every part that goes into a spacecraft has to be manufactured to exacting tolerances, and then tested to verify that it matches those tolerances.

Mating parts are never match-drilled; each side has to be built to be interchanged with similar parts throughout the spacecraft, even when nobody can think of a plausible scenario for those parts ever being interchanged. Interchangeability requires orders of magnitude tighter tolerances, which in turn requires orders of magnitude more inspection and testing, and orders of magnitude more occurences when good parts are rejected because they don't meet an unreasonably exact specification, and of course this whole mess means orders of magnitude more cost.

Our challenge in the Artemis Project is to write requirements and specifications that makes sense. This includes requirements for testing and verifying spacecraft parts.