Compare · bake-off
bezoar vs Company A
A venture-backed vendor's open-source scanner. Company A publishes no detection benchmark, so we ran it on the same 1,431-sample public corpus as bezoar, default config, offline mode: 12.8% of 626 known-malicious skills flagged (bezoar 88.2%) and 181 false positives on 737 official vendor skills (bezoar 0). On the small curated fixture: bezoar 10/10, Company A 0/10.
Measured · same corpus · default configs
| Metric | bezoar | Company A |
|---|---|---|
| Detection rate (626 malicious, public corpus) | 88.2% | 12.8% |
| Flagged, absolute | 552/626 | 80/626 |
| False positives (737 official vendor skills) | 0 | 181 |
| False positives (180 MCP Registry configs) | 0 | 0 |
| Curated fixture (small set) | 10/10 | 0/10 |
What the number means
- Company A ships no prompt-injection detection class. Tool-description poisoning, mandatory pre-step instructions, and “don't tell the user” shadowing are invisible to it unless the text also happens to contain a credential path or a stock reverse-shell one-liner.
- Its false positives on official vendor skills come largely from a bare
ncatalternative in the reverse-shell rule, which matches insidetruncateandconcatenatein ordinary documentation. - Method: measured by us, default config, offline regex mode, no API key, one isolated scan per sample, no tuning for either tool. “Flagged” is any finding, which is Company A's own failing threshold. Corpus and results JSON are public.
Being written
This page is in progress. The source of truth is the repository until it lands. Back to home.