Sceliphron

Sceliphron is a TLD finding and technical survey spider. It visits only the root page of a domain, and only follows links to external, uncrawled domain names. This produces a wide, shallow mapping of the World Wide Web that ignores most deep web content.

Besides shallow mapping, Sceliphron is designed to identify and enumerate web technologies used on websites it crawls. It uses this information in aggregate to identify trends in software adoption, ranging from choice of webserver host to use of JS libraries and trackers. In the future, some of these statistics will be released publicly; others will be made available privately for a minimal cost.

Sceliphron does not scrape content or use AI to analyze content. It is designed to be a well-behaved crawler, and will obey directives in the /robots.txt file of all domains it visits. For more information, visit https://sceliphron.net.

Node Statistics

Node Information: Wespe, hosted in 🇩🇪 at the IP address 209.38.201.210