Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

They do have their own web crawler. I don't have any idea what percentage of their results come from it, but presumably as they grow their own search results will take an ever-increasing slice of the pie.


There doesn't seem to be any evidence that it's used for indexing at all; it seems more like it's just used to scrape additional info from sites that they index via real search engines like Bing, Yandex, and Yahoo.

https://news.ycombinator.com/item?id=4851680


Their own crawler is not used for any organic search results, only widget stuff like Instant Answers (and spam detection apparently as per sibling comment).




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: