Bringing Craigslist Back

I’ve been wrestling with whether to bring back Craigslist listings in the search results. I’ve found a way to include them that I’m told is legally kosher since it doesn’t touch their servers at all, but it still seems somewhat dickish to go against their wishes in this, and I’ve always had a lot of respect for what they’ve done for the world. Also, court seems like it’d be no fun.

But then I did some back of the envelope estimates of how much of people’s time and effort it would waste if I didn’t, and it became clear how much less nice it is to waste the time of millions of apartment hunters out of stubbornness or some clearly inaccurate assumption about the will of the community. If it takes half of PadMapper’s millions of monthly users 3 hours longer now to find an apartment, that’s over 350 man-years wasted per month, or 5 lifetimes. That really pisses me off. Apartment hunting is an activity which, unless you enjoy it or it teaches you something, adds no value to the world, and should be over as quickly as humanly possible so that you can get back to doing other, more productive things.

So, effective as soon as I can bring the new code up (almost certainly by the time you read this), I’m bringing the Craigslist search results back. This will be courtesy of the fine folks at 3taps, who provide an API for data about Craigslist postings that they gather via indirect means and so aren’t subject to CL’s Terms of Use. It’s not nice of me to do from Craigslist’s perspective, and I’d really love to just work with them to solve these problems, but my conscience is clear. I’m still going to send all traffic back to them and all that jazz.

Thanks to everyone who sent them letters, there were a surprisingly large number of you. I was a bit depressed that there didn’t seem to be much of a response from them, but it was pretty incredible to see the outpouring of support (still ongoing), and I really appreciate you all taking the time out to help out with this.

Happy hunting, everyone.
Eric

Update: Edited for language, it was immature of me to curse, sorry.

Update2: There seems to be a lot of confusion, so I should clarify that CL’s legal claim was about their TOU, not about copyright, since PadMapper isn’t reposting the listings, only summarizing bedrooms, bathrooms, price, etc. and linking to the original to read more. Using 3taps just makes it so it’s not a TOU issue.

This entry was posted in Uncategorized. Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.
  • Vote on HN

125 Comments

  1. Alex
    Posted August 6, 2012 at 5:55 pm | Permalink

    I just wanted to be another person to say thanks. Padmapper was totally invaluable to me during my latest move. You’re making software that makes a difference in peoples’ lives.

    • Posted August 6, 2012 at 5:58 pm | Permalink

      Thanks Alex, I appreciate the support. And that’s really good to hear that it was helpful to you, that’s what makes working on it great :-)

  2. Twelveth Dragon
    Posted August 7, 2012 at 11:34 am | Permalink

    Build your own CL?

  3. Alex P
    Posted August 7, 2012 at 9:15 pm | Permalink

    It’s a fucking mess man — and I hope the whole scraping issue gets handled in courts properly. CL has been holding that data hostage and throwing its weight around for years! Very un-hippie of them, from such a hippie CEO :)

    They are taking the wrong approach to this, there is no way they will win long term no matter what the outcome of the lawsuit will be. And hopefully there will be some clear guidelines for scraping & syndication after this.

    The way it is now .. you can’t create a complementary service to CL — no reason for that. If they don’t provide an API to access their data then scraping should be OK as long as you’re not clearly and deliberately stealing content (search engine food or otherwise).

    Anyway, good luck to 3taps and yourself. Hopefully this will hit critical mass and you’ll start getting good attorneys and media coverage.

  4. Jason
    Posted August 8, 2012 at 10:22 am | Permalink

    Thank you for this.. I also agree with your assessment of the waste of human time and resources. You’re doing the right thing.

  5. Ursula
    Posted August 8, 2012 at 8:38 pm | Permalink

    We love padmapper- we’ve been looking in the Bay area for the last month and a half- and every time you’ve gone offline it has become incredibly evident just how much time you save! We catch listings that we missed in the long dreary trance of scanning the days postings- many of them repeats, putting them on a spreadsheet, mapping them, and we never even get to the walkscore.

    Padmapper is by far the most useful thing that has happened to craigslist postings in 6 years. What a great app, the need is so great, and craigslist has been becoming so polluted with bogus listings as to be unusable- maybe there’s an opportunity to replace craigslist by getting enough folks in the Bay area to use a different posting board.

  6. xilex
    Posted August 9, 2012 at 1:58 pm | Permalink

    whoever’s running craigslists must be knuckleheads. it’s somewhat easy to search for things to buy, but definitely not when it comes to apartments. padmapper has helped me locate apartments twice in the past 4 years (i’ve only had to move twice). i tried craigslist before discovering padmapper, and it was too difficult to find anything. hope craigslist can come to its senses and you guys can work something out. i have to move again next year and i’m worried about how i’ll find housing now.

  7. Sean
    Posted August 10, 2012 at 4:57 pm | Permalink

    Hello -

    I’m in the process of moving to Seattle, and I just wanted to say that I’ve found your site invaluable in the past, and I would be extremely grateful for anything that can be done to keep craigslist postings up on your site.

    There just isn’t a viable alternative out there when searching for apartments…

    Thanks much,

    Sean

  8. Chris
    Posted August 17, 2012 at 5:40 pm | Permalink

    Dear Eric,

    Please keep us updated on the latest (legal) developments. I think we have all noticed the CL postings coming and going on Padmapper during the past few weeks, so I’m guessing something is up.

    Personally, I was lucky to find my dream apartment with the help of Padmapper just before the CL seize and desist incident, but I can only imagine what it is going to be like next time I need to move if Padmapper is no longer around.

    I wish you the best, and once again, THANK YOU for helping me and so many others find our new homes in a simple, efficient and time-saving way!!!

    Padmapper rules! :)

  9. Posted August 22, 2012 at 3:29 am | Permalink

    Padmapper/Padlister is a near perfect application for my business as a small landlord.

    The big landlords with those big, nice, overpriced complexes don’t need it quite as much. They also seem to stay out of Craigslist for the most part since they have so many units to advertise.

    So if all small landlords migrate to Padlister, then CL eventually goes obsolete.

    Here’s how I workaround CL banning PM: In my CL ad, I always add this sentence:

    For a showing, please click through this link to pad mapper and RSVP to the Open House:
    https://sites.google.com/site/mydenverrentals/home

    It gets the prospect to the padmapper listing where he can also apply online. Once a Craigslist searcher starts seeing a few of these padmapper listings, he will start going to padmapper first, and other CL posters will notice and start doing it.

    And this is how we get CL out of the apartment business. Of course, they are welcome to provide a padmapper-style interface themselves, and we’ll use it.

    • Paul
      Posted September 24, 2012 at 4:42 pm | Permalink

      Kevin in Denver, watchout.

      I did this sort of thing with other sites like Ridejoy and CL just resorts to flagging and blocking your posts! I even wrote them an email about it and they never responded.

      So much for the CL we all enjoyed for years. I sure hope there is plenty of backlash on this issue.

      –Paul in PDX

  10. Prevost
    Posted August 22, 2012 at 9:21 am | Permalink

    CL is becoming evil, be the runner up and remove to all of us the pain to ever go back on that ugly pos.

  11. Jim Worrall
    Posted August 27, 2012 at 10:12 pm | Permalink

    Eric,

    I love your service but regret Craigslist is being so selfish. If CL cared about their customers they would welcome your service which assists their customers. As a long term advertiser on CL I am grateful for the additional exposure your site offered and the new tenants you brought me. If you need someone to testify at your trial that your use of CL data was welcomed and beneficial please let me know.

    James.p.Worrall (at) gmail

    • Posted August 27, 2012 at 10:27 pm | Permalink

      Thanks Jim! I really appreciate the offer. Hopefully it won’t be necessary, though… it could take two years to get to trial!

  12. Matt
    Posted August 30, 2012 at 4:51 pm | Permalink

    What happened to WalkScore? I can’t see it for any listing…

  13. Anon
    Posted September 8, 2012 at 11:26 am | Permalink

    Thanks for taking the time to make such a great tool, and then for finding a way to leverage the data from CL! You are awesome dude!

  14. Tony
    Posted September 12, 2012 at 3:44 pm | Permalink

    So, did you lose craigslist again? I’ve used padmapper and haven’t come across any hits from CL but when I go to CL directly there are hundreds. What’s up with that?

    • Posted September 12, 2012 at 5:05 pm | Permalink

      It did for a few days. It should be coming back, though, and by next week, I think we should be back to solidly searching CL.

      • Tony
        Posted September 16, 2012 at 12:07 am | Permalink

        Cool. I was getting worried there for a sec. Padmapper’s still #1

  15. Jordan Staiger
    Posted September 16, 2012 at 6:22 am | Permalink

    I love the concept of this website. What happens if two people post the same apartment with the same exact address? What happens when you pull data from CL for the same apartment posted by different people?

    Jordan

    • Posted September 17, 2012 at 12:54 pm | Permalink

      Thanks Jordan. It depends, if it thinks it’s a duplicate, it will remove it, but unless it’s sure, it typically just puts them both on the map.

  16. Jen
    Posted September 20, 2012 at 7:28 pm | Permalink

    Padmapper is absolutely amazing. The most useful tool I have ever seen for housing hunting. Now it just needs the data. I’m sorry Craig’s List has been so horrible. I hope you are able to bring it back soon.

  17. Sean
    Posted September 24, 2012 at 11:11 pm | Permalink

    Thank you for this great website. It’s saved me a ton of time searching for rentals in my area. However, the Craigslist listing doesn’t seem to show on the map in Vancouver, BC where I am, but other listings show up OK. Is that something to do with Craigslist blocking or did you just forget about us Canadians?

    Thanks Again,
    Sean.

    • Posted September 25, 2012 at 12:26 am | Permalink

      How could I forget about you Canadians? I love Canadians.

      It looks like you’re right, and I’m not entirely sure what’s going on on the backend, I don’t have direct control over the CL importing operations – I get that from a third party now. I can inquire about it with them, though. Thanks for letting me know!

      Eric

  18. Robert
    Posted October 4, 2012 at 11:37 am | Permalink

    Craigslist now has a Map View option in San Diego.. How original, sigh..

    http://sandiego.craigslist.org/search/apa?query=&srchType=A&useMap=1&minAsk=&maxAsk=&bedrooms=

  19. Daniel
    Posted October 21, 2012 at 1:44 pm | Permalink

    Can I suggest another site to get postings from? It’s a site from Québéc, I think, and there is a lot of good apartments there. It’s http://www.duproprio.com.
    I think this could be a good addition to all Québec users :)
    Please, don’t let this mess with CL put you down. Padmapper is fantastic!

  20. Jennifer
    Posted November 8, 2012 at 7:43 am | Permalink

    Can I ask a question? I have asked Craigslist this several times and only get a generic
    e-mail response that does not answer the question.

    For four year plus I have been posting ads on Craigslist and want to remove them
    from where they have saved everyone of them. (probably on their servers somewhere) Is there a way I can delete all the saved ads from Craigslist. Or ask them to delete them.

    Hope someone knows the answer to this problem.

    Thanks,

    Jennifer

    • Posted November 8, 2012 at 6:19 pm | Permalink

      Hi Jennifer, I’m not sure about that – oftentimes it’s technically trickier to delete things than it is to create them, so sometimes sites don’t enable it. Are you sure there’s no “Delete” link on those saved listings?

  21. morgan
    Posted November 19, 2012 at 11:19 am | Permalink

    hi, just saying thanks. getting craigslist back would be super crucial. You’ve seriously saved my life multiple times

  22. Alex
    Posted November 20, 2012 at 7:54 pm | Permalink

    What’t the update on this? How come CL is back in search? Are they backing down? Did u find a work around?

    Congratz either way.

    • Posted November 21, 2012 at 1:58 pm | Permalink

      Hey Alex, Craigslist search results should be up on PadMapper, is this not the case for you? Maybe try resetting your filters?

  23. Scott
    Posted December 6, 2012 at 11:04 am | Permalink

    Are the Craigslist listings being blocked again? They have not shown up on Padmapper for the past couple of weeks for me.

    • Posted December 6, 2012 at 3:31 pm | Permalink

      Hi Scott, nope, they’re on there. Perhaps hit “Reset Filters” on the filter box if it appears to make sure you’re not filtering them out somehow.

      • Scott
        Posted December 8, 2012 at 11:01 am | Permalink

        Eric,

        Okay, I figured it out. With Craigslist, the apartment/studio rentals are still showing up , but the the roommate listings are not showing up anymore. Which is why when I filter only to show rooms/shares nothing shows up.

        Scott

  24. AB
    Posted January 3, 2013 at 6:17 am | Permalink

    As a landlord, I appreciate what padmapper does; someone just came to view a rental because they saw it on padmapper. Thanks padmapper! Now if some site would give craiglist some real competition for listings!

  25. D. Hubbard
    Posted January 12, 2013 at 6:20 pm | Permalink

    Eric
    Thanks much for the software. I am very particular about where I am willing to live. Padmapper affords me the ability to refine that search area without having to spend hours upon hours doing it… I hate moving and cant stand looking for a new place! So, thank you for cutting tha time a minimum.
    On behalf of your site I was glad to send an email to the boys at Craigslist. Good luck. And again, thanks.
    d.

  26. Ariol
    Posted January 13, 2013 at 11:58 am | Permalink

    I’m happy you guys went through with this. Turned out great…

  27. golly
    Posted January 24, 2013 at 2:02 pm | Permalink

    Maybe Craigslist is not what I thought it was? Craigslist has always been a model of
    what was great about the internet–unselfish?

  28. Katie
    Posted March 26, 2013 at 5:26 am | Permalink

    I’m having trouble with CL listings – every time I click on one, it says “Has pics, but not shown here for now” which I get, but then when I try to follow the link, it goes to a dummy CL page with no info – “Page Not Found.” Is this my browser, or some CL witchery, or something else?

    • Posted March 26, 2013 at 6:08 am | Permalink

      Hm, they seem to be working for me. What city, and could you send me some examples?

  29. Christy
    Posted March 29, 2013 at 8:33 am | Permalink

    We have our garage-apt (studio) on Craiglist, but it doesn’t show in your results. I tried tweaking the filters but I sitll don’t see it. Can you help me make sure it’s “findable” as opposed to just PEBCAK? ;) Thanks!

  30. Posted October 26, 2013 at 5:33 am | Permalink

    CL doesn’t have copyright rights on any ads cause it just not CL’s ad, but user. User holding any copyright right. IMHO.

5 Trackbacks

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>