18 Jul 2012, 00:00

Common Crawl – Open Spider Crawl

Common Crawl is a non-profit foundation dedicated to building and maintaining an open crawl of the web, thereby enabling a new wave of innovation, education and research.

via Common Crawl.