Platform

GET /api/v1/stats against the public corpus returns 14,569 binaries — and two engine versions serving side by side:

"analyze_status":  {"analyzed": 14320, "empty": 249},
"engine_version":  {"0.10.3": 12640, "0.11.0": 1929},
"linkage":         {"object": 12676, "dynamic": 1876, "static": 17},
"verdict":         {"benign": 13805, "suspicious": 324,
                    "not_enough_evidence": 319,
                    "likely_malicious": 101, "malicious": 20}

Every row of that answer is stored analysis output. The platform indexes what the engine produced, serves it over two surfaces, and analyzes nothing in-process. The second surface is an MCP server carrying the same bearer tokens through the same gate code, so an HTTP route and its MCP twin cannot drift on who is allowed to call it.

Analysis addressing and reanalysis

Analysis output is addressed by the triple (visibility scope, sha256, engine version), so shipping a new engine adds an address rather than invalidating a corpus. Neither version above is stale.

POST /api/v1/binary/{sha256}/reanalyse fills an address on demand, reusing the bytes already stored:

Resolved call-site search

Call sites the engine resolved are indexed as facts, not text — which binary opens which path, execs which target, loads which library. GET /api/v1/search/signatures?path_prefix=/etc on the live corpus:

{"name": "chpasswd",  "op_name": "access", "aspect": "file_io",
 "args_preview": "{\"path\":\"/etc/shadow\"}",
 "call_site": {"func_name": "main", "func_addr": 4624, "offset": 2222}}

{"name": "adjkerntz", "op_name": "access", "aspect": "file_io",
 "args_preview": "{\"path\":\"/etc/wall_cmos_clock\"}",
 "call_site": {"func_name": "sub_2020", "func_addr": 8224, "offset": 167}}

Each row names the containing function and a byte offset within it, so a hit is an address to open rather than a keyword to chase. That offset is block-granular, not per-instruction — the lifted IL does not carry a virtual address per operation — so it lands you in the right basic block and an accompanying operation index picks the operation out of it.

Path normalization. Paths are normalized before indexing, which is what makes them comparable across machines:

So /Users/alice/Library/Keychains/login.keychain-db indexes as /Users/*/Library/Keychains/login.keychain-db. The un-normalized original stays inside the stored analysis, so the specific observed path is never lost.

The 19 preserved prefixes span all three platforms:

One list serves both the analysis-time writer and the query-time normalizer, so the two cannot drift; path_prefix=/etc above works because /etc/ is on it.

Limits.

• No argument-level predicates — the filter accepts an aspect bucket, an operation name, and a path prefix, not arbitrary call arguments. "Calls SecItemCopyMatching with a generic-password class" cannot be asked here.
• A 500-candidate ceiling — execution is two-stage: a full-text pre-filter yields at most 500 candidate binaries, then each candidate's stored signature array is filtered in memory. Past 500 matching candidates the result is a sample, not a census.

The query DSL

71 field identifiers are accepted, each carrying a kind that fixes which operators are legal on it. Anything else raises unknown identifier.

The DSL rides /api/v1/search as filter=; there is no separate query route. A real request and its real answer:

GET /api/v1/search?filter=component == "libcurl" AND NOT is_apple_signed == true

curl            239,016 B   02c1abdd9468
librepo.so.0    227,864 B   00f8536eda1a
curl            334,312 B   0513ba358cab
total: 3

Hierarchical values. component and package are hierarchical — the same value string, cut at a different depth:

Grammar limits.

• No array literals. vuln_severity ANY ["critical", "high"] is rejected with lex error at col 19: unexpected character '['. The working form is vuln_severity ANY "critical" OR vuln_severity ANY "high".
• ALL is not yet distinct. Because the right-hand side is therefore always a single value, ALL currently lowers to exactly the same term query as ANY — reserved syntax, not yet a distinct operation.

Search ranking

Full-text scoring is BM25 with per-field boosts, because a token in a filename means something different from the same token in __cstring.

That exact-name clause is why an exact name cannot be buried by partial-token hits. The spread it produces, measured on the corpus for the query curl:

curl                 780.16   matched: filename, apis, strings
curl                 679.10   matched: filename, apis, strings
libcurl.so.4.7.0       8.24   matched: apis, strings
ps                     4.09   matched: apis

Two orders of magnitude between the program named curl and the library it links. Without the boosts, a high-cardinality content field accumulates enough term matches to bury the exact-name hit.

Popularity then re-ranks: bm25 × (1 + α × popularity_norm), where popularity is w_u · ln1p(uploads_30d) + w_l · ln1p(lookups_30d) over a sparse per-day table holding only the trailing 30 days, pruned on the next write. The logarithm is load-bearing — raw counts let a single viral binary dominate every query.

Similarity

The score GET /api/v1/similar/{sha256} surfaces is Jaccard, not cosine, because cosine cannot separate unrelated programs. On a Linux curl — one real match and a run of near-misses — both scores it computes:

neighbor            score   cosine   jaccard
curl                0.710    0.987     0.710
libdpv.so.3         0.147    0.985     0.147
bsdtar              0.138    0.980     0.138
ssh-agent           0.062    0.971     0.062
scp                 0.053    0.982     0.053

Cosine puts another curl and an scp within 0.005 of each other, because it measures count magnitude: any two non-trivial programs read as "about the same size" and land above 0.97. Jaccard separates that same pair by 13×.

The DNA vector

The vector behind that cosine column has 106 dimensions and is stored padded to 112: 106 is divisible by neither 8 nor 16, and an IVF_PQ index wants a divisible width for its sub-quantizers. The pad is zeros, so cosine distance is unchanged, and it is stripped on read.

Two-phase retrieval

Retrieval is two phases: an approximate nearest-neighbor scan for candidates, then a weighted-Jaccard rerank of the top 500 of them.

Phase one is an approximate nearest-neighbor query with two predicates pushed into the index rather than applied after: the caller's visibility set, and the target's own file format. Format scoping is not an optimization — roughly 22 of the 106 dimensions and several of the content sets are Apple/Mach-O-only, so a cross-format score is not comparable in the first place.

Phase two takes the top 500 candidates and re-ranks them with a weighted Jaccard over 40 content sets (the stored blob carries 41 sets; one is persisted for export without carrying a rerank weight):

Those weights are the outer prior in both scoring modes; what happens inside one set depends on corpus size:

The 14,569-binary corpus this brief opens with is always on the frequency-weighted path.

The pool is 500 rather than something tighter because cosine and Jaccard order differently — the curl neighbor list opening Similarity is that divergence — and a small pool drops true Jaccard-top matches before the rerank ever sees them.

The learned embedding

A 128-dimension learned embedding lives in the same vector store as a nullable second column, so the two rankings can be compared on the same corpus. A config flag routes /similar to it without touching the DNA path. Its scoring is cosine-only with no Jaccard rerank; candidate rows whose embedding column is null are invisible to it, and a target with no stored embedding falls back to the DNA path rather than returning nothing.

The ML layer

No trained model ships with the software, so a default deployment runs the null path and returns no predictions at all. Model files are operator-supplied paths pinned by sha256 in configuration.

Three of the four predictors are ONNX-served, and all three are fail-soft by construction: each ships a null implementation alongside the real one, and a missing model file, a missing ONNX Runtime dylib, or a prediction error degrades to serving without predictions rather than failing the request. The fourth needs no runtime dylib at all.

Both thresholds decline rather than guess. Below 0.8 confidence a binary is not scored rather than scored against the wrong cohort, and binary types under 50 function vectors are skipped entirely. The classifier's label set is macOS-shaped, which is a real constraint on what the downstream cohorts can mean for an ELF or PE.

Visibility scopes

Every identity carries one of /public, /private/<user id>, or /workspace/<workspace id>, and the same string serves as the Tantivy facet, the vector-store filter predicate, and the on-disk directory segment.

One SHA-256 can hold several at once: the same binary uploaded into two workspaces is one blob with two membership rows and two independent analysis subtrees. Because the analysis subtrees are independent, a takedown against the public copy leaves the private and workspace analyses intact.

A binary the caller cannot see returns 404, not 403 — on the detail route, on compare, on strings, on reanalyse. Invisible and absent are indistinguishable from outside, so a private binary's existence never leaks through a status code.

Feature gating

Two different refusals live behind the word "gated", and they leak different things.

Compare and full strings take the 403 path deliberately: a capability nobody can see is a capability nobody upgrades to.

Five capabilities no plan can buy. BinaryDownload, whole-corpus global statistics, curator annotation, whole-binary decompilation, and the static-audit surface (CWE findings and secrets) return false for every tier — Free, Pro, and Enterprise alike. Access comes solely from a per-user grant issued by the operator; static audit reached that state by demotion from Pro.

A deployment with no auth provider configured auto-grants all five to whoever is running it, because a self-hosted install must not lock out its own operator. With auth configured, a real grant is required.

Sessions and API keys

Sessions are HS256 JWTs with a single-algorithm allowlist (alg: none and every asymmetric algorithm are refused), an explicitly re-set required-claims list so a library upgrade cannot silently relax it, and zero clock leeway so an expired token is expired.

The four key-management routes are session-only, so a leaked CI key cannot mint peers, list them, or revoke them. Usage timestamps are debounced to one write per minute per key rather than one per request.

Key lookup is asymmetric: a key present in the hash index resolves in 1.3 µs at 100,000 issued keys, while a key that misses the index falls back to a linear scan over every active key.

Presenting a wrong key costs 550× more than presenting a right one, and that cost grows with the number of right ones.

The scan is a degradation, not the design: the hash index is backfilled at boot from the stored keys. If that backfill fails, every request — right key or wrong — pays the 732 µs path, which is why the failure is logged rather than swallowed. Revoked keys stay in the index and are re-checked for active status on every hit, so revocation is not an index delete.

Ingest

Upload routing goes through the same signature authority the unpacker dispatches extractors from, so the front door and the engine can never disagree about what a blob is.

The door test is narrower than "is this noise". A blob is refused if it is under 1,024 bytes — the firmware this gate must accept is megabytes — or if 95% or more of its first 8,192 bytes are printable ASCII. That second rule refuses logs, configuration files, and source code: not noise, simply not containers, and each would otherwise buy a full unpack job for a text file.

A firmware image already unpacked short-circuits on re-upload, because a completed unpack always leaves an SBOM sidecar. A second upload of the same bytes therefore does not re-run a multi-minute unpack costing one of the few concurrent permits and up to 16 GiB of disk churn. The uploader is still recorded against the existing package and any child missing an analysis is backfilled. Re-running for real is an explicit POST /package/{sha256}/re-extract.

Out-of-process analysis

Analysis runs out-of-process, and the process boundary is the point: the engine parses attacker-controlled input, so a panic, stack overflow, or parser memory bug kills one child and nothing else.

The cost of that boundary is the cold start. FLIRT signatures, bundled PE pattern files, and tier-3 symbols all live in per-process caches that die with the process. How much that costs depends on which warming path a deployment is on:

The tier-3 local symbol database, at roughly 34,000 entries, has no flat artifact and always warms by loading; it is small enough not to matter. A pool of long-lived engine children pays whichever cost applies once — each worker warms at server startup, and the pool reports ready only when every worker is warm, so the first upload lands on a hot fleet. A worker that fails to warm within 120 s is treated as failed.

Isolation survives the reuse. A worker leaves the pool on any of five conditions:

• Deadline exceeded — killed rather than reused. The deadline scales with input size; the clocks are below.
• Panic — the worker returns the error and then exits, so a pristine replacement takes its place.
• Fixed job count reached — a healthy worker retires, bounding both memory growth and the blast radius of slow-accumulating corruption.
• Resident set above 2 GiB — a healthy worker retires on its next check-in regardless of job count. Its private memory is the high-water working set of the largest binary it ever analyzed, and the allocator holds that peak long after the analysis is dropped, so one monster binary would otherwise pin a slot at multiple gigabytes for the rest of its job window.
• Dead before the request landed — a worker the kernel killed while idle fails on the write, not the read. Because the request provably never reached the engine, it is retried on a fresh worker; every other death is surfaced.

Three independent guards stop children outliving the server, including PR_SET_PDEATHSIG on Linux, which survives a SIGKILL of the parent where no cleanup code runs at all.

Remote workers can additionally claim jobs over a token-gated compute plane, purely additive to the local pool. That plane is gated by its own deployment token — not user auth, and distinct from the operator token — and returns 404 on every route when the token is unset, so its existence does not leak.

Jobs, progress, and liveness

Three job types exist: analysis, reconstruction, and package unpack. Progress streams over Server-Sent Events with a stage label and a percentage; a firmware unpack emits unpacking, then the unpacker's own stage names as it moves through them (pre-scan, per-format labels, overview, scanning, finalizing), then firmware_overview, then one child_extracted per extracted child, then complete.

child_extracted is not one event per binary. The same stage label carries three payload kinds:

Liveness is tracked separately from progress, because a job that produces nothing and a job that has died look identical without a clock. The clocks belong to different jobs:

The 120 s stall threshold sits deliberately between the 20 s tick and the 300 s kill, so an operator gets roughly three minutes of warning before a wedged unpack fails itself.

API routes

93 route paths sit under /api/v1, plus separate operator and compute-farm namespaces that are gated by their own tokens and invisible without them.

Index schema pins

Two index planes carry a version pin, and neither migrates itself — both force an explicit reindex.

The similarity pin is backed by a compile-time guard, because the failure it prevents is silent. The Jaccard rerank blob is stored positionally against the ordered set list, so inserting, removing, or reordering one set — or repairing a weight pairing — would leave every already-stored row comparing set i against a different set i: garbage scores, no error, no log line. A fingerprint over the ordered field names and the weight pairs is asserted at compile time against a recorded constant, so any such change breaks the build until the revision is bumped and deployed indices rebuild.

Measured cost

The catalog holds its shape from 100,000 to 1,000,000 binaries. Measured on a single machine, synthesizing records directly so engine time is excluded. The storage-engine choice these numbers came out of is in Storage architecture.

Two planes scale against their own inputs rather than corpus size:

The index-metric mismatch behind the similarity numbers

An IVF_PQ index should scale sublinearly; those two rows scaled near-linearly because the index was never being used. Lance falls back to a brute-force flat scan when a query's distance metric differs from the metric the index was trained with, and it logs rather than errors. The index was built with the default L2 metric while every query asked for cosine, so it was built, kept current, and bypassed on every call — the 13.6 ms of measured flat-scan time is most of the 44 ms wall at 1M.

Three changes followed, and the first is the one that mattered:

Measured at 500,000 rows after the fix: the unfiltered vector query drops from about 14 ms to about 1 ms, and the visibility-and-format-prefiltered query lands at about 6 ms. Raising the probe count from 1 to 50 moves p50 by under a millisecond, which is why 20 is affordable — the metric, not the partition count, was the whole regression.

The full 100K-and-1M benchmark has not been re-run against the fixed index, so the two rows above remain the last end-to-end numbers on record; they are simply not measurements of an IVF_PQ index. The 20-million-binary extrapolation those linear numbers implied — roughly a second per /similar call, seven to fourteen seconds for a 2-hop cluster expansion — does not follow from a sublinear index and should not be carried forward.

Known hot spots

Three smaller costs are dominated by something other than the data the caller asked for — the whole corpus, the whole upload ledger, and parse time rather than disk time.

Scope and non-goals

• Not the engine. No binary is analyzed in-process. Every analysis row is an engine child's output, indexed and served — see Engine, and Architectures for what that engine can read at all.
• Not a sandbox. Uploaded content is never executed. Static end to end.
• Not a verdict authority. The verdict label is the engine's; the platform surfaces it and never overrides it. Four of the five are graded — benign, suspicious, likely malicious, malicious — and the fifth, not-enough-evidence, means the malware pass itself did not complete, which is why it is not folded into benign. See Malware.
• Not a vulnerability database. CVE attribution comes from the curated SBOM layer (see CVEs & SBOM); the platform aggregates rollups and maintains no external feed.