Decay of the Open Web

In the open-source intelligence community, there is a persistent temptation to compete on raw catalog counts. Tools advertise "500+ sites," "1,000+ sources," or "comprehensive global coverage." Yet when you inspect the underlying lists, a surprising proportion of those domains are dead, parked, or broken.

In PROBE's source history, 341 catalog entries are now defunct, while 322 functional sources remain active (265 automatic checks and 57 manual links). Over half of the platforms ever indexed have ceased to function as public profile hosts.

The true cost of dead sources

Leaving dead domains in an active scanner is not harmless padding; it actively corrupts scan performance and result quality:

• DNS timeouts and connection hangs. When an expired domain fails to resolve or routes to an abandoned nameserver, requests hang until client deadlines elapse. A scan intended to finish in seconds bogs down waiting for unresolvable hosts.
• Parked domains and wildcards. Expired startup domains are frequently snapped up by domain squatters who configure wildcard DNS. Every path—from /johndoe to /gibberish12345—returns a 200 OK ad parking page. A naive scanner interprets this as a confirmed match.
• Domain takeovers and security risks. Defunct domains can be re-registered to host phishing or malicious redirection, exposing users who click into external verification links to potential harm.

Why pruning beats inflation

Recently, PROBE removed 44 catalog entries that consistently blocked requests or failed to provide reliable public signals. Rather than keeping them to inflate our marketing headline to "366 sources," we pruned them from the primary scan pipeline and quarantined them for targeted lab analysis.

A smaller, rigorously verified catalog of 322 sources finishes faster, produces zero false positives in continuous calibration testing, and respects the researcher's time. In username intelligence, what you choose not to check is just as important as what you do.