Hacker News new | past | comments | ask | show | jobs | submit login

I'm genuinely confused about the legality about building a business whose sole purpose is to scrape the web, repackage and sell that information.

One side of the fence, you have all this publicly accessible information, so it's like.. why not? Anyone can obtain it. Why should you be restricted?

Then on the other side you have companies who will place all sorts of legal restrictions, TOS/EULAs, etc to prevent against web scraping; sending out cease & desists and even sometimes going as far as taking someone to court.

What's the best + safest approach here for a company like Priceonomics?




> "I'm genuinely confused about the legality about building a business whose sole purpose is to scrape the web, repackage and sell that information"

Like Google?


Exactly. I am very much on the side of "If it's publicly available info, it's free to access and repackage" but a lot of companies and judges feel otherwise.


Can you list some specific examples of companies to whom this pertains?

By that I mean companies who think their data shouldn't be publicly available and also fail to place a no-crawl flag in their robots.txt.


Well, a robots.txt (much like an implicit TOS/EULA that nobody ever sees nor agrees to) isn't exactly legally binding.

Yelp, LinkedIn, and Craigslist are examples that have taken legal action recently against other entities scraping and re-purposing the data.


I think part of the problem is that some services' TOS do give users the option to restrict access to their information so if you go and scrap it that might cause problems to the startup where that content was posted to.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: