Craigslist clusterfuck

Posted by under Business, Internet

Craigslist is one of those ubiquitous, old school internet properties. Started back in 1995, it’s safe to say it has gotten a little stagnant. It still looks exactly like it did when it launched and has failed to keep up on more recent innovations. Yet somehow it still dominates the market. A series of events over the past few weeks have brought up some questions about the site’s future relevance and questionable business dealings. I’ll get back to this in a bit, but first I need to explain how we got here in the first place.

Padmapper was a site run by a single web developer, trying to make apartment searching just a little less painful. This especially resonated with me because at the time I was looking for a new place in New York (which is an absolute nightmare). What he was doing was scrapping data from a variety of sources (mainly Craigslist) and then aggregating all those apartment listings and plotting them on a map. The map then had these great filters for important things like location, rent, bedrooms, bathrooms and type of lease. For a place like New York this was especially important because location is very important. A tool to sort through all the terrible listings and just focus on the good ones saves so much time. (New Jersey does not count as being five minutes from midtown).

At some point his little site got too big and caught the attention of Craig Newmark and the gang at Craigslist. The normal Silicon Valley response would have been to offer to buy Padmapper and bring the developer on board at Craigslist to build innovative features like this into the site. Instead they sent a cease and desist letter to Padmapper and threatened to sue if the site continued to use their data. The developer claims he tried to get in touch with the Craigslist legal team to no avail. Unable to even have a conversation with Craigslist, Padmapper was forced to stop using that data, but only temporarily. It’s important to point out that, while Padmapper was using Craiglist’s data, they were bringing users back to the original posting on Craigslist. They were serving ads on the map mashup, but they argued, were not taking anything away from Craigslist or stealing their users per say.

Enter 3Taps, a company that makes access to data on the web easier and more convenient to use. They take major sites like Craigslist and scrape the data from Google’s indexed cache and then turn it into a well-documented API for web developers to build upon. The distinction was key for Padmapper, 3Taps turns Google into a middle man in this entire situation. No one is dumb enough to sue Google, right? Satisfied that this would be a legal workaround, Padmapper turned back on the Craiglist listings, but this time through 3Tap as opposed to directly. The result? Craigslist sued them both.

All while this is going on, Craig Newmark is abdicating any responsibility for the situation, constantly tweeting things along the lines of him being the customer service rep and Jim Buckmaster, being the CEO, is the person these questions should be directed to. Newmark is estimated by some sources to be worth more than a billion dollars as a result of the empire he has built around Craigslist. To this day he still goes through and manually responds to posts flagged for abuse, answers the phones and responds to customer complaint emails. For him it has always been more about the community and bringing people together, rather than anything more material, like making money. Despite this hippie mentality, the company is for some reason notoriously litigious.

Around the same time Craigslist makes some moves internally to update their terms of service to say that users are giving exclusive rights over their listings. Seems like a clear move to bolster their legal case against Padmapper, 3Taps or any future targets.

To me, this entire thing is very bizarre. Why would you even sue these people? They aren’t doing anything that hurts your core business, if anything they are improving on and adding to it. You’re generating tons of negative press and provoking the ire of the internet masses, which is never a good idea.

You would think Newmark would be worried about some new innovative take on the classifieds coming along and wiping him out. He hasn’t improved the site since the very beginning. And yet, no one has come along and destroyed him like he destroyed the newspaper classifieds before him. There’s a saying that when an industry ceases to innovate, they start to hire lobbyists and lawyers. I wonder when he hired his first lawyer.

All of this got me thinking about a bit of news I saw a few months back. TechCrunch had discovered that Craigslist had posted an ad (on Craigslist of course), seeking a user experience expert to make the site, “faster, friendlier and easier.” It also mentions a need for mobile development skills. To me this indicates that they might finally be working on an internal project to roll out some changes and get with the future. It would explain why they didn’t just buy Padmapper and instead want to snuff them out as even a potential competitor. If they have been working on an internal project for months now, they might be close to rolling something out. And now there are reports that in certain markets, they are testing new embedded maps in ads. Nothing too revolutionary, but at least it is a step in the right direction.

My prediction is that within the next year, Craigslist is going to slowly roll out a lot of small iterations to the site, never making it seem like a major redesign, but twelve months from now we will be looking at a very different experience. Newmark himself has admitted that he doesn’t really know why Craigslist is so successful and as a result he has been very hesitant to change it.

The legal implications of data scraping could have large ramifications and even set some legal precedent that might impact innovation on the web. But all indications are that this thing is going to drag out and won’t be revolved anytime soon.

In my mind, Craigslist is dominant because of network externalities. Each additional user makes the service even more valuable and entrenched. They have now captured such a large market share, it seems nearly impossible for a new company to come along and disrupt them. However, time and time again we’ve seen companies in this seemingly unassailable position be beaten into irrelevance (MySpace, Yahoo search, etc.).

I’ll use this case to further support the thesis I started to explain in my Best of All Worlds Review.

In my mind Facebook’s death knell will be a proliferation of niche social networking sites eating away at Facebook’s giant user base. Best of All Worlds is merely the first of many to come. Facebook’s death will be one of a thousand tiny paper cuts.

Although I was talking about Facebook, the argument is the same. The only way Craigslist gets disrupted and replaced is by more niche services eating away at key verticals. There are a lot of verticals within the site (furniture, real estate, etc.) that are large enough to have a dedicated site built around them. These things are certainly cyclical in nature, progress is inevitable.

Additional interesting reading on the topic: