What we were trying to buy

A software-factory client wanted Sanctum agents that could treat vulnerability scanning as an intelligence unit: stand something up, point it at machines we own, and let an agent reason over real findings. CVEngine (GPL-3.0) looked like the scaffolding for that. Projects, a REST API, tokens, a dashboard, an AI hook, remediation states. We put it on a partner lab box, fronted it, wired an OpenAI-compatible provider, and ran it against hosts we actually operate.

This is the report. I am not being kind.

The product we got

A CVE catalog with a login screen.

The headline tile said 3,997 Fire. That number is not “your estate is on fire.” It is how many rows in the ingested feed meet a heat rule — KEV, Exploit-DB, high CVSS, EPSS. The feed. Not our machines. The UI copy admits this if you read the fine print. The tile does not. Anyone buying this as an agent brain will read the tile.

When we isolated what the engine actually attached to our hosts, we had 489 matches. Every one of them had an empty product version. Every one of them had version_in_range = none. The correlation step saw the word PHP or the word nginx and stapled catalog rows to the hostname.

What “PHP” bought us

436 matches labeled PHP. 21 more labeled php, because the product string is case-sensitive theater. 32 labeled nginx. That is the entire host-match corpus.

On those rows we found:

• EG Secure Solutions SiteGuard advisories glued to our nginx.
• PHP-Fusion, PHP-Charts, PHP Jabbers, PHP Poll, PHP Server Monitor, a volunteer-management upload bug, php-censor, a point-of-sale script — glued to unix-socket PHP-FPM.
• Windows PHP-CGI argument injection, including a KEV, glued to Ubuntu.
• A PHP Standard Library HTTP/2 advisory that is not the PHP interpreter at all.
• Interpreter advisories from 2019 that do not exist on PHP 8.1/8.3, next to 2026 rows Ubuntu already marked not-affected or already shipped in the package we run.

The hosts in question run Ubuntu PHP-FPM: 8.1.2 with current security backports on one box, 8.3.6 with current security backports on another. FPM status pages off. Not CGI. Not Windows. Not SiteGuard. The engine never learned any of that, because it never recorded a version.

A junior admin with dpkg-query -W php*-fpm and ten minutes of Ubuntu’s tracker outperforms this pipeline. That is not a metaphor. That is what we did to mark the rows after the product drowned us in them.

Then we paid a model to shrug

The dashboard kept saying there was no AI provider while an OpenAI-compatible provider was saved, enabled, and answering ping. The web process had the provider. The workers did not. Classic. We restarted Celery. The sweep started eating the pile.

243 successful completions. About 425,000 input tokens and 25,000 output tokens on llama-3.3-70b. 160 of those calls were “finding triage.” The triage queue was not the same object as the 489 host matches — it also held hygiene and route rows — and most host matches never got a verdict at all.

The model never once said not affected.

It said affected on 12 rows and uncertain on 135. We spent serious inference to get shrugs on name-only matches. That is not intelligence. That is a language model being polite to garbage features.

I turned the automatic triage and validation sweeps off so the leftover 328 not_triaged queue rows would stop billing. The provider can stay configured for a lab. It should not be allowed to chew empty versions overnight.

The scaffolding is not the product

I will give the repo this: Docker Compose comes up. There is a real API. Bulk remediation works when the row is in the current scan. Tokens and projects exist. An OpenAI-compatible provider can be made first-class if you patch the image and restart the workers. Someone spent time on the furniture.

Furniture is not a scanner.

A scanner tells you a package, a version, and whether that version is in an affected range. A real scanner also knows the distro trick: Ubuntu patches upstream versions without bumping the version string, so “affected upstream” does not mean “affected on this box.” CVEngine never applied that gate before it opened its mouth. It emits a confidence float and an optional essay from a model that cannot see the package either.

If you attach that to a Sanctum agent as an “intelligence unit,” the agent will file work from SiteGuard-on-nginx and CGI-on-Linux. It will sound sure. It will be wrong. The client will think we have coverage. We will have a dashboard.

What I would not sell

I would not put another of our hosts through this as it stands. I would not put a client’s estate through it. I would not hire a specialist agent to operate it until the correlation step refuses to emit a match without a version, refuses wrong-product names, and refuses to spend inference on rows it already knows are empty.

The honest product name is “CVE feed browser with optional host labels.” That is a fine weekend project. It is not an intelligence unit. It is not a reason to burn a 70B model. It is not what we told a software-factory client we could wrap in an agent.

We stood it up. We measured it. We are taking it down. If someone wants this class of capability for Sanctum, we build the correlation ourselves or we pick a tool that already knows what a package version is. We do not put a word matcher in a robe and call it an analyst.