Set my data free

Last weekend I helped a friend categorize her Schedule C expenses. All of her business income is in QuickBooks, but the expenses aren’t. I would have to reconstruct those from bank and credit card records. Although this friend has online accounts at both institutions, my Spidey sense was tingling: I knew there was going to be trouble.

Free IT resource Hear how top CIOs turn change into a competitive advantage. Sponsored by HP Free IT resource Attend the SOA Executive Forum: Breaking SOA Bottlenecks SOAExecForum.com/may2007 Sponsored by InfoWorld

As it turned out, the bank was a knock-over. It doesn’t export data in QuickBooks’ IIF (information interchange format) but does offer CSV (comma-separated variable). I had long ago written a CSV-to-IIF translator. So with a minimum of fuss I was able to suck the 2005 expenses into QuickBooks, where my friend could begin tagging them.

The credit card company’s defenses, though, were more formidable. Its site had a CSV dumper, too, but when I asked for 2005 transactions, all I got back was fourth-quarter records. The 12 statements from 2005 are available as PDFs, but that wasn’t what I had in mind.

Agent: I’m sorry, sir, there’s nothing else we can do.

Me: Or rather, nothing else you will do.

Agent: Would you like me to fax you those statements?

Me: (Grumble.)

Here’s a little secret I didn’t tell them. I have a superpower that enables me to do battle with the evil of data lock-in. I can’t leap tall buildings or crush lumps of coal into diamonds, but when I look at the barriers that divide one data format from another, they seem hardly to exist. For me, data transformation is almost an autonomic reflex, like breathing.

But PDF? Please, oh please, don’t make me dig the data out of those PDF files. I cajoled, I begged, I threatened. But sadly, convincing organizations to make exceptions is not a superpower I possess. So I found a PDF-to-Excel translator and went to work.

The results weren’t pretty. Entropy runs only one way, after all. It takes work to convert a less orderly system into a more orderly one. So naturally the output had to be massaged.

As I ran through a series of regular-expression search-and-replace operations in my programmer’s text editor, I was dimly aware of the fact that I was exercising a freak talent. What do normal people do? Transcribe the numbers by hand, I guess. Or, perhaps equally likely in the case of Schedule C, just invent them.

It doesn’t have to be this way. PayPal, for example, will happily disgorge all my transactions as far back as 2000. It even offers IIF, on the assumption that not everyone can easily convert to it from CSV.

I know what you’re thinking: That’s a security risk. And you’re right, it is. But am I any safer with all my data sitting in PDFs? I’m tempted to joke that, if we regard PDF as a mode of encryption, my statement history actually is safer than a raw transaction history would be.

[ Talkback: right or security risk? ]

But seriously, I should be able to encrypt my historical data in any format, so long as it’s not necessary to the operation of the service and I’m willing to be responsible for the key.

If I don’t make that choice, though, let’s get real. If I want to turn my data into HTML, or IIF, or PDF for that matter, I will. If you can do those transformations for me, great. But first things first. Just give me my data when I ask you for it. Not ink on paper, not a bitmapped image of ink on paper, and not even a vector representation of ink on paper. Just the data.