The next generation of spreadsheets will be a database

Flexibility and ease of use have made the spreadsheet the popular but unfortunate choice for makeshift databases; we can do better

For a long stretch of early personal computing history, three software products dominated the software market: WordPerfect for word processing, Lotus 1–2–3 for spreadsheets, and dBase for databases. Why, then, did the word processor and spreadsheet remain as the stalwart core of the knowledge worker’s toolkit, while the database became relegated to the domain of technical experts?

In the early days of personal computing, spreadsheets and databases happily coexisted because there was a clear separation of responsibility between the two paradigms. At the time, spreadsheets were intended primarily for financial modeling -- they were numerical calculation engines. Databases, on the other hand, were for everything organization-related -- a pet fish distributor’s shipment list or a ballet troupe’s subscriber mailings, for example.

Unfortunately, databases were never quite as easy to use as word processors or spreadsheets. The latter two offered the advantages of remarkable simplicity and direct manipulability, while database products tended to leak much more of their technical implementation and associated complexity to the user. For instance, dBase required the user to design a database in code with a domain-specific language; even later programs such as Access, FileMaker, FoxPro, and Lotus Approach required users to learn a confusing schema designer interface.

Spreadsheet pushmi-pullyu

For the sake of convenience, but at the cost of real data structure, ordinary users began using spreadsheets as makeshift databases. Joel Spolsky recounts an epiphanic moment as a Microsoft Excel program manager in 1991 when, after visiting dozens of Excel customers, he realized almost none of them were actually using the product for its imagined purpose of financial modeling, but instead as “a convenient way to make a table.”

During this time, the office productivity landscape had also shifted from a multiplayer market to one dominated by Microsoft, which crushed Lotus 1–2–3 and dBase with its own Excel and Access products. The growing gray area between spreadsheets and databases presented an internal challenge to the product and marketing teams on those two products under the same Redmond roof. At what point in the spectrum of use cases should Excel-as-database be replaced by a full-on Access install?

Microsoft drew a somewhat arbitrary line between the two and, in doing so, created a self-fulfilling prophecy. Databases could have become easier to use and more spreadsheetlike, rather than vice versa. Instead, Excel became increasingly cannibalistic of its older sibling; as Excel 2003’s documentation proclaimed, “with Excel lists, you get database-like functionality in the spreadsheet” (emphasis mine).

The trouble was, the spreadsheet was simultaneously pulled in a completely different direction by a vocal minority of number-crunching power users. The resulting product increasingly suffered from split personality, which is painfully apparent to the casual, “simple database”-type user: The inability to properly link different tables together (contacts and companies), lack of file attachment support, and loosey-goosey (read: error-prone) field types. Simple tasks like entering long notes into a cell requires obscure key combinations to insert new lines and results in bloated rows that make the spreadsheet unnavigable.

The solution: A real database with spreadsheet simplicity

What users really need is a product that combines the fast and flexible aspects of a spreadsheet’s interface with the structure of a true relational database. By focusing on organizational use cases, this lightweight database would not compromise design elegance for the sake of number-crunching capabilities. It would be a real database with a quasi-spreadsheet interface, rather than a spreadsheet with quasi-database capabilities.

This is easier said than done, however. Core database concepts such as multiple views, many-to-many relationships, and forms need to be made accessible in a reimagined design that feels immediately intuitive to users.

This database-with-spreadsheet-interface would fill the needs of small teams whose departmental, highly specific use cases are not important, complex, or sensitive enough to justify a formal IT initiative. It would abstract away the technical complexities of traditional databases: setting up a hosting server, creating and maintaining indexes for performance, and so on. However, these databases for the citizen developer should also provide an API, so if a solution grows and eventually justifies IT resources, it can be programmatically accessed and shared with other teams and services rather than remain a departmental silo.

To deliver maximum value to users, the product must -- in addition to its novel interface design -- be built upon modern collaboration paradigms that reflect new work styles and consumer behaviors. In short, it must live in the cloud, support real-time collaboration, and be usable from mobile devices.

Cloud-based and collaborative

A cloud-based product can support instantaneous sharing and syncing of data with collaborators, while eliminating much of the complexity of database creation. There’s no installation or configuration, and no deployment or maintenance of a server to host the database. Declarative-style databases in the cloud exist in the form of products such as Intuit Quickbase, Zoho Creator, and the Salesforce1 platform, but like their desktop software equivalents (such as Access or FileMaker), their complexity still puts them beyond the reach of users.

No traditional database products come close to the standard of collaboration set originally by Google Docs. Google’s word processor and spreadsheet products offer not only a cloud-based experience, but also synchronous, real-time collaboration. Seeing teammates’ changes appear alongside your own is a far different experience than working in virtually isolated chambers and occasionally emailing updated files. With Google Docs, it’s as if you and your colleagues are simultaneously typing into the same computer.

Google Docs was also pivotal in changing the way we think about our work deliverables. Documents are no longer static artifacts to be printed and forgotten, but alive and evolving, with revision histories and contextual conversations that have as much value as the document itself. Using modern version control systems, programmers have long thought of their codebases in this way -- now it’s coming to everyone else as well.

This type of collaboration cannot simply be bolted onto an existing product. From a design standpoint, every possible interaction in the product must consider the collaborative ramifications, rather than focus strictly on the solo-user scenario. The product must present real-time changes, surface revision history, and enable contextually specific conversations -- all in an intuitive manner for the user.

Real-time synchronization also must be engineered into the product’s technical architecture from the ground up; it brings a host of hidden complexities, such as handling merge conflicts or handling undo in a multiuser context. Consider the scenario where user A does something, user B does something else in the same document, then user A wants to undo their last action without losing user B’s action. These problems are more complex in the context of a relational database than in either a word document or a spreadsheet, where the merge conflicts can be handled in a well-understood manner.

Mobile too

The incredible growth of mobile and tablet devices has required a complete rethinking of traditional interaction designs for the touch paradigm. For productivity tools, however, it’s important to remember that the desktop will remain equally as important a paradigm for the foreseeable future, with all its advantages in data entry and manipulation. The ideal productivity application is not mobile-first or mobile-only, but an app that adapts fluidly to each device as a distinct medium.

Traditional spreadsheets were ideal for the keyboard and mouse era, but the infinite 2D grid was never intended for touchscreens. The result is a painful user experience punctuated by squinting, pinching, mistapping, and formatting headaches.

Our user database concept must seamlessly translate to a touch device -- transforming each record from a spreadsheet row on desktop into a tappable and swipeable card metaphor for mobile, enabling field-type-specific input with specialized date-picking and numeric keyboards, camera input, and so on.

Over the past few decades, the spreadsheet has increasingly become the go-to workhorse for users to create a makeshift database, not because it was the right tool for the job, but because it was the only tool within reach. A lightweight database for the nontechnical user deserves a central place in the knowledge worker’s toolkit today, but in order to claim this position, it needs to be elegantly reimagined.

My company, Airtable, was founded to address this need. More than two years in the making, Airtable is the first real-time synchronized relational database and the first to offer collaborative undo. Our design and engineering team iterated upon its interface design until it was as intuitive as possible for the user. We are the originators of this emerging category, and we hope others will join us in bringing more powerful but intuitive tools to the mainstream user. Spreadsheet or database? It doesn’t have to be a choice, and it certainly shouldn’t feel like a battle.

Howie Liu is the founder and CEO of Airtable, a user database designed around an intuitive spreadsheet interface, real-time collaboration, mobile access, and cloud access. He previously led social products at Salesforce, via the acquisition of the relationship intelligence CRM Etacts.

New Tech Forum provides a venue to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to newtechforum@infoworld.com.

From CIO: 8 Free Online Courses to Grow Your Tech Skills
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.