Well, a robots.txt (much like an implicit TOS/EULA that nobody ever sees nor agrees to) isn't exactly legally binding.

Yelp, LinkedIn, and Craigslist are examples that have taken legal action recently against other entities scraping and re-purposing the data.