Sunday, January 4, 2009

Root Cause Analysis

In a previous career, I worked as a systems administrator for twelve years. When working with computer problems, one of the first things you need to learn to do is to see past the symptoms, and treat the underlying condition. It's something called "root cause analysis", and it's the backbone of computer troubleshooting.

In my present career, I do instrumentation at an international astronomical observatory. When diagnosing and maintaining scientific instrumentation, one of the first things you need to learn to do is see past the symptoms and treat the underlying condition. Root cause analysis again.

Recently, my KAP rig has suffered from an oddball problem: Each time I try to rotate the pan axis, there's a lag, and then it starts moving. The first time this happened in the field, I swapped batteries and went back into the air. When the symptoms didn't go away, I assumed both sets of batteries were low.

In a later flight I realized the small pinion in my pan axis gear reduction was loose. DOH! Without thinking about it, I tightened the screw that holds the pinion in place, and put it back in the air. Problem solved, right?

As it turns out, wrong.

About the third time I tightened that screw, I realized something else was wrong. I hadn't done proper root cause analysis; I'd just treated the symptoms of a loose screw. A little more poking around revealed the culprit: The pan axis itself was binding. This put additional torque on the pinion gear, which put extra force on the screw, which pulled it loose, which made the pan axis have backlash, which made for the lag when I tried to rotate it.

ARGH!

The cause of the tight pan axis was a bushing that was too tight in its hole. I made a new one with 0.005" clearance, and everything ran like a well-oiled clock. I also used thread lock compound on the pinion gear screw, so it shouldn't get loose as easily. For good measure I also used a synthetic grease on the two bushings on the pan axis, so hopefully this will be the last time I deal with this problem.

Which is the real benefit of root cause analysis: never having to see a problem come back.

Time to go fly a kite!

Tom