Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The Wayback Machine has ignored robots.txt for a few years at this point. The only way to get them to stop scraping or remove content is by asking them directly.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: