Steer Your Product Clear of the Ponzi Scheme

I had the great pleasure of spending last week in New Zealand while speaking at Webstock 2009. An amazing country with some great people and an equally great web conference, there were great conversations, perspectives, and talks given over the course of the week, and I’m hopeful I’ll be able to make it back again in the future. I gave a workshop on a practical view of Agile, and I think it was generally well-received, but this isn’t about my session.

One noteworthy moment came with the penultimate talk of the conference, Bruce Sterling’s The Short but Glorious Life of Web 2.0, And What Comes Afterward. Bruce delivered a speech, rather than a presentation, recounting some of the origins of the Web 2.0 (I used to cringe when saying that, but I guess I’m over it) movement, and how they’ve panned out.

While Bruce was in the weeds a fair bit, and may even have come across as a troll to some, he did surface an apt comparison between the financial crisis and how most of us think about Web 2.0.

The Comparison

I wish I had a transcript of the talk to quote the relevant section for the sake of accuracy, but I’ll do my best to paraphrase it accurately.

As most know by now, one of the core catalysts in the financial crisis was the practice of taking hundreds of risky mortgages and reselling them as ostensibly-less-risky packages called CDOs. Credit rating agencies would review these CDOs, fail to recognize the fact that underwriting standards on the individual mortgages were quite poor, and award the CDO a high credit rating as a whole, opening the flood gates for millions to invest with confidence in what turned out to be a very poor investment built on toothpicks.

Bruce likened these CDOs to the “web-as-a-platform” notion that has been a major part of the web over the last several years. Many successful startups have been based on the notion of mashing up data from elsewhere on the web in various ways, at least in part, and Bruce warned that this practice, too, could really be akin to building our businesses on a foundation of toothpicks.

I think there’s a point there. Especially as web companies (often still startups) begin to feel the crunch of the economic situation, we need to be questioning how our products are positioned — that is, how much we’re invested in other products and what the effects would be if these other products suddenly vanished. If you had a business based on Ma.gnolia, or perhaps more likely, web application platform Coghead, you’re probably hurting right now.

And remember, before you laugh at the notion that Amazon might decide to close up shop on things like S3 or EC2, widely-respected investors were laughing at the notion that the value of real estate could possibly decrease, too. If Amazon were facing losses (or even less-than-stellar gains) on any of these cloud technologies, tightened pursestrings and investors could very feasibly force the hand of even great companies like Amazon to do things they wouldn’t otherwise do.

Positioning Your Product

What I’m not suggesting is that we all flee from the notion that we can build our own work based on the work and APIs of others. What I do suggest is that we bring a critical eye to all external dependencies, and build our products in ways that may gracefully handle the failure of any of these. Prima facie, there appear to be two significant principles worth considering:

1. If the dependency is minor, ensure that the system functions reasonably in the absence of the service in question. Consider that Google Maps went down, and your app relies on Google to display a map alongside listings on your website, but in a purely informative way. It may not be an enormously huge deal to use your site without the map, but it’s important that the front-end (the markup, the CSS, the layout, etc) is able to handle the absence of the map item. In fact, make sure that the page looks as good as possible without it, and make sure the backend isn’t going to produce a 500 error if Last.fm doesn’t respond to your API request.

2. If the dependency is major, have a fallback available. If, on the other hand, you are highly dependent on the ability to geocode incoming locations, it’s important that you have some sort of backup in place should Google Maps go down. The easiest way to do this would be to build your geocoding mechanism on top of an abstraction that can handle multiple APIs. So if Google Maps goes down, you’re able to switch to Yahoo! Maps without having to completely rewrite your code. If you’re dependent on Ma.gnolia, you’re using an abstraction that allows you to easily switch to Delicious or Diigo. If you’re using Amazon S3, you have a backup of your data and your storage code allows you to switch to a different datastore in a reasonable amount of time.

Of course, there’s always a Third Way for every two ways. If you’re able to do so, you can enter into a formal agreement with these external dependencies, and contractually obligate them to give you enough time to make any changes you need to make before they go away. This is probably often the toughest route to go, and only worth considering if the external dependency is approachable in that way.

Don’t Go Nuts, but Be Aware

Look, if you’re a hobbyist, and you’re just looking to make something cool and useful, then you don’t really need to worry about any of this. Just do what you do, stay smart, and build the best app you can, but think of it like making a really complex design without ever saving it — the journey is the reward, and it could vanish even faster than it was created.

But if you have aspirations for funding, or if you expect to feed your family (or, more importantly, someone else’s family) on a product you’re developing, hedging your position on external data sources and web apps should be a legitimate concern for you. Your app doesn’t need to have 99.99% uptime, or have automated fallback measures, or seven levels of defense, but you shouldn’t be in the position where you need to be down for multiple days or weeks when one service fails you.

As any financial guru will tell you, when things look like they might get tight, get conservative in your investments and diversify. Every service you use to build your own product represents an investment you’ve made, so ensure you’re positioned well.