Extra context: robots.txt is voluntary, so we treat “posture” separately from “verified activity.”
We’re also building a companion “Verify” flow that analyzes uploaded access logs (UA + ASN + behavior scoring) for people who want evidence of actual crawler activity. This post is intentionally about the public posture layer.
Extra context: robots.txt is voluntary, so we treat “posture” separately from “verified activity.”
We’re also building a companion “Verify” flow that analyzes uploaded access logs (UA + ASN + behavior scoring) for people who want evidence of actual crawler activity. This post is intentionally about the public posture layer.