Businessweek Archives

Maybe Some Gardens Have Walls for a Good Reason?


Skype's Celebrity Encounter |

Main

| HP's CSC Play: A Milestone Deal?

January 04, 2006

Maybe Some Gardens Have Walls for a Good Reason?

Rob Hof

According to Search Engine Roundtable, the popular online classified-ad site Craigslist may be blocking search engines such as Google from crawling its site. Search experts like John Battelle make a good case that this is not a great idea, since it makes it harder for people to find the listings. Don't yet know why Craiglist is apparently doing this, though CEO Jim Buckmaster recently listed several reasons the site earlier blocked Oodle from scraping it. But Greg Yardley wonders if maybe Craigslist has a point: "As Google morphs into a portal, why should Craigslist allow it to rip-off and monetize their content?" He also wonders:

This does raise interesting questions, though - since Craiglist’s voluntary absence from the search engines makes them all a little bit less useful. What if Craigslist is starting a trend? How many key sites have to block search engine traffic before the engines’ relevance starts to sink, and what could that do to paid search? In an atmosphere of site defection and declining search engine relevance, could a site like Craigslist suddenly allow one and only one search engine to spider its site - for a fee?

I tend to side with John on this, but it will be interesting to hear what Craig Newmark himself has to say.

UPDATE: Danny Sullivan at SearchEngineWatch says Craigslist in fact is not blocking search engines, just blocking duplicate pathways to the listings. As Emily Litella would say....

01:06 AM

Google, craigslist

TrackBack URL for this entry:

http://blogs.businessweek.com/mt/mt-tb.cgi/

I also wonder the same thing. Especially when Yahoo! Local results are so much more relevant than Google search results, but Google searches turn up Yahoo! data as its first hit... many times including the phone number you need, without ever having to go to Yahoo!. I wouldn't blame Yahoo! for being upset.

Posted by: Eddie at January 4, 2006 09:21 AM

In the past, I've noticed that Craigslist postings turn up in searches on Feedster.

Posted by: AHCB at January 4, 2006 11:00 AM

Well, it begs the question, does Google let other companies crawl its data?

Posted by: Heather Green at January 4, 2006 03:34 PM

>>Well, it begs the question, does Google let

>>other companies crawl its data?

In a sense, yes, there is a way to grab the keywords queries and the frequency of them. As for the index data, why? Anyone can start a crawler, all it takes is a simple script, a huge pipe and lots of disk space and horsepower(if you are a success).

Posted by: saleem moonbeemer at January 4, 2006 05:29 PM

interesting to see if this will affect google's stock.

Posted by: The Guide to Getting More Out of Travel at January 4, 2006 05:29 PM

So in a sense all Microsoft has to do to win the search war is to hand out enough monetary compensation to top site in order to get them to be only indexed by MSN.

However I doubt even Microsoft would have enough resources for such a strategy, and I doubt the Search industry would remain profitable if such tactics where employed especially if Google had to fight back in the same way.

Posted by: Michael Zarb at January 5, 2006 06:38 AM


Race, Class, and the Future of Ferguson
LIMITED-TIME OFFER SUBSCRIBE NOW

(enter your email)
(enter up to 5 email addresses, separated by commas)

Max 250 characters

 
blog comments powered by Disqus