A tale of a database-turned-detective

In this tech tale, a data entry error and the 'overkill' fix demanded by top brass unearths a surprising secret

Twenty-some years ago, I worked as a programmer for a large public utility.

One day, a customer with a $42 monthly "budget" bill was going out of town for a couple months and knew that her overall use would be lower than the prescribed amount. Before leaving, she went to her local office and asked if the clerk could override the preset payment. Our system allowed local clerks latitude to make adjustments, so the clerk tabbed through the screen fields to the bill amount and typed in "35."

[ Also on InfoWorld: Read memorable Off the Record stories from 2009 in "Tall tales of tech -- that happen to be true." | Send your IT Off the Record story to offtherecord@infoworld.com -- if we publish it, we'll send you a $50 American Express gift cheque. ]

Back in those pre-Windows days, the field size on the screen was much wider than the two digits needed for most bills, and instead of clearing out the old amount, the clerk typed "35" on the left side of the field and accidently left "42" on the right side. Instead of replacing 42 with 35, the computer filled in 0s between them, and the customer was sent a bill for $3,500,042.

Off the Record submissions

The customer raised a ruckus, even though we quickly corrected the problem. This $3.5 million residential utility bill even made the front page of the local paper.

Perhaps we should have all gotten a good laugh out of it, but the "top floor" guys didn't find it amusing. Our department was given orders to fix the program and make sure it never happened again.

Fortunately for me, I wasn't in charge of the billing system, but I was in charge of the bill printing subsystem -- basically a 10,000-line Cobol report program. The solution assigned to me was to code the program to write and send a new daily report that listed the five largest bills for each revenue class that had been printed overnight. I got a copy, my manager got a copy, and several copies went "somewhere else." If we saw any suspicious bills, we were to go to the mailroom and pull the physical document before it was mailed.

The "high bill" report seemed like overkill for such an unusual quirk of a problem, especially when the screen programmers fixed the data entry code. However, paranoia kept the daily report coming -- and we never did have to run to the mailroom to yank a bill out of the mail.

A year or so later, someone noticed that a lot of the high residential bills seemed to be in the same region served by one local office. This prompted an internal investigation, and we found out that a clerk had been embezzling. When people paid their bills in cash, she'd pocket the money. When cut-off notices came for nonpayment, she'd intercept them. She'd also intercept the growing bills and manually retype bills for smaller amounts so that customers never knew that the public utility wasn't getting paid. It had been going on for years, and no one noticed anything before the daily high bill report.

So an overreaction to a small embarrassment turned into a fraud detection tool. That was 21 years ago, and I wouldn't be surprised if that report is still being printed every day.

Aside from the surprising embezzlement twist, what I remember from this incident is a reminder that you should never count on employee training to take care of a problem you can solve with the software. In this case, the clerks had been trained to clear out old entries before adding new ones, but this time the clerk was probably working too fast or was distracted. The fact that it was fixable by our screen programmers means that it could have been prevented by better coding. Even with all the easy error-checking tools built into modern tools like Microsoft Access, the developer has to actually use them to prevent stupid inputs as much as possible.

This story, "A tale of a database-turned-detective," was originally published at InfoWorld.com. Read more crazy-but-true stories in the anonymous Off the Record blog at InfoWorld.com.


Copyright © 2010 IDG Communications, Inc.