Methodology

Measuring Username Search Accuracy Honestly

One accuracy percentage cannot tell the full story when some sites do not provide a clear answer.

6 min read

Update, 28 September 2026: A new full-catalog improvement cycle raised decision accuracy to 96.38% and coverage to 75.92% while preserving zero observed false matches. Read the latest results.

Checking a username is not like looking it up in one database. Every site responds differently: some return clear errors, some redirect to login, and some block automated requests. We therefore measure both how often PROBE is right and how often it can give a clear answer at all.

95.34%decision accuracy
72.44%decision coverage
947controls reviewed

The test

We tested the same checking logic used by the live site across all 324 automatic sources. The test used 299 usernames known to exist and 648 deliberately unusual usernames expected not to exist. Another 49 active sources require a manual check, so they were not included in the automatic score.

Blocked and unclear responses were counted separately rather than treated as right or wrong. That distinction matters: quoting 95.34% accuracy without also saying that PROBE reached a clear answer in 72.44% of tests would make it sound more complete than it is.

What changed

The first test produced eight false matches across Kick, Scratch, Brickset, and Greasy Fork. Fixing the URLs and site-specific checks removed all of the false matches we had observed. We also reviewed twenty accounts that PROBE appeared to miss: twelve were genuine detection problems, four test accounts had been deleted or renamed, and four could not be checked reliably without a person opening the site.

After those fixes, PROBE correctly found 85.05% of existing accounts when it reached a clear decision, up from 76.82%. The final test recorded 182 correct finds, 472 correct misses, no false matches, 32 missed accounts, and 261 cases where PROBE could not give a firm answer.

What we can honestly say

When PROBE gave a found-or-not-found answer in this test, it was correct 95.34% of the time. It gave a clear answer in 72.44% of cases. Seeing no false matches across 648 made-up usernames is encouraging, but it does not mean false matches are impossible.

That is why PROBE shows unclear results and asks you to verify matches yourself. Sites change all the time, so checking accuracy is ongoing work.