Glossary

57 termsA–Z

The words this site uses and what they mean here — the model measurements, the language of evaluations, the corpus vocabulary, the Article 44-B filing terms, and the API. Where a definition is a particular organization’s own, it says so and links back.

A

6 terms

Amendment: A developer’s revised filing in answer to an objection. The prior version is retained — the docket shows what changed, not just where it ended up.
API key: The credential for programmatic access, beginning with sk44b_. Created on your account page and shown once at creation — 44B stores only its hash and a short display prefix.
Article 44-B: New York General Business Law Article 44-B, §§ 1420–1429, effective January 1, 2027 — the statute this registry implements. It obliges frontier developers to publish safety frameworks and transparency reports, to report critical safety incidents, and to file disclosure statements with the Office.
Artificial Analysis Intelligence Index: A composite score combining a model’s results across a fixed set of evaluations into one number, so models can be compared on a single axis. It is the default sort on the models page. Treat it as a summary, not a verdict: a composite hides which component moved.; Via Artificial Analysis ↗
Artificial Analysis Openness Index: A composite measure of how open a model is — weights, license, and what the license permits — intended as a common way to state openness to users and developers. Distinct from the plain open-weights flag, which is a yes or a no.; Via Artificial Analysis ↗
arXiv: The open-access preprint archive operated by Cornell University, and the source of the daily paper feed in the 44B Library. An arXiv ID such as 2107.03374 identifies a paper and all of its versions.; Via arXiv ↗

B

3 terms

Benchmark: A fixed set of tasks with a scoring method, used to compare models on the same footing. A benchmark measures what its authors chose to measure — which is why 44B carries the paper behind each one.
Benchmark contamination: Test items having leaked into a model’s training data, so the score measures memorization rather than ability. The reason held-out and freshly-written benchmarks matter, and a standing caveat on any high score against an old public test set.
Blended price: A single price per million tokens combining input and output rates at a fixed ratio, so models with different input/output pricing can be compared on one number. 44B also shows the input and output prices separately.; Via Artificial Analysis ↗

C

8 terms

Capability profile: The twenty-category breakdown 44B shows on a model page — coding, mathematics, reasoning, and so on — assembled from that model’s results across the benchmarks we hold.
Citation count: How many other papers cite this one. A measure of uptake, not of correctness; and it accrues slowly, so a recent paper always looks quiet.; Via Semantic Scholar ↗
Cited answer: A prose answer generated only from records 44B holds, where every claim carries a citation resolving to a real 44B page. Metered separately from search because generating one costs model spend and a search does not.
Commercial use: Whether the model’s license permits use in a commercial product. It is carried as a separate flag from open weights because an otherwise-open model can still be closed to business use.
Context window: The maximum number of tokens a model can attend to at once — the prompt, any attached documents, the conversation so far, and the answer being written, all counted together.
Corpus: A defined body of documents searched as a unit. 44B holds five: research papers, the document library, models, benchmarks, and the incident record.
Corpus ID: Semantic Scholar’s stable identifier for a paper. 44B keys papers by arXiv ID where one exists and falls back to the Corpus ID, so a paper without a preprint still has a permanent home.; Via Semantic Scholar ↗
Critical safety incident (CSI): An event within one of the four categories at § 1420(4). A developer must report one to the Office within 72 hours of determining it occurred (§ 1422(1)); where there is imminent risk of death or serious injury, an appropriate authority is notified within 24 hours. CSI filings are confidential and never appear on the public docket.

D

2 terms

Developer disclosure: The § 1428 statement a large frontier developer must have on file with the Office to develop, deploy, or operate a frontier model in New York. Renewed every two years. Accepted disclosures populate the 44B Registered Developers list; contact fields are withheld from the public view under § 1428(6).
Docket: The public register of filings and what has happened to each one. Confidential filing types appear only as aggregate counts.

E

3 terms

Elo: A rating derived from head-to-head comparisons, borrowed from chess: winning against a strong opponent raises a rating more than winning against a weak one. Used for models judged by preference rather than by a scored answer key.
Embedding: A list of numbers representing a text’s meaning, positioned so that similar texts sit close together. Comparing embeddings is how semantic search finds a paper that never uses your words.
Evaluation (eval): A benchmark plus the procedure for running it — the prompting, the sampling settings, the scoring, and the harness. Two labs running "the same benchmark" differently can report different numbers honestly.

F

4 terms

Few-shot: Including a handful of worked examples in the prompt before the real question. Reported as n-shot; scores are not comparable across different n.
Filing: One submission into the 44B Registry, of one of the five statutory types, carrying a reference such as F-2026-0012 and moving through a fixed lifecycle: draft, submitted, under review, then objected to, amended, accepted, or withdrawn. Every version is kept.
Frontier model: In common usage, a model at the leading edge of capability. Under Article 44-B it is a defined term (§ 1420(9)) that determines who must file: the statute’s definition, not the marketing one, decides whether a model is in scope.
Frontier safety framework: The § 1421(1)–(2) document setting out how a developer manages, assesses, and mitigates catastrophic risk — published, reviewed at least annually, and republished with a justification within thirty days of a material modification.

H

1 term

Held-out set: Test items kept private by the benchmark’s maintainers so they cannot be trained on. Results on a held-out set are harder to contaminate and harder to reproduce independently.

I

3 terms

Incident: A recorded case of an AI system causing or nearly causing harm — sourced, dated, and tied where possible to the system and developer behind it. Distinct from a critical safety incident, which is a statutory category with a filing obligation attached.
Influential citations: The subset of citations where the citing paper draws on the cited work substantively rather than in passing, identified by Semantic Scholar. A better signal than raw citation count for whether a paper was actually used.; Via Semantic Scholar ↗
Internal-use assessment: The § 1422(2) summary of catastrophic risk arising from a developer’s internal use of its own frontier models, transmitted to the Office quarterly and held confidentially.

L

1 term

Leaderboard: A ranked table of model results on one evaluation. Every benchmark page on 44B carries one, drawn from the scores we have ingested rather than from the leaderboard’s own site.

M

2 terms

MCP (Model Context Protocol): An open protocol for exposing tools and data to an AI assistant. 44B runs an MCP server so an assistant you already use can query 44B directly, authenticated with the same API key.
Modality: The kind of input or output a model handles — text, image, audio, video. Multimodal describes a model or a benchmark spanning more than one.

O

3 terms

Objection: The Office’s written statement of what is deficient in a filing. It stops the review, states the ground, and puts the filing back to the developer, who answers with an amendment.
Open weights: The trained weights are published and can be downloaded and run by anyone. Open weights is not the same as open source — the license may still restrict what you may do with them, and training data and code are usually not included.
Output speed (tokens per second): How fast a model produces output once it has started, measured in tokens per second. Together with time to first token it describes the whole of what latency feels like.; Via Artificial Analysis ↗

P

5 terms

Parameters: The count of learned weights in a model, usually quoted in billions. A rough proxy for scale and cost, and a poor proxy for capability: architecture, training data, and post-training move results more than parameter count alone.
pass@1: The share of problems solved on a single attempt. The generalization, pass@k, allows k attempts and counts a problem solved if any attempt succeeds — so pass@10 is always at least pass@1 and is a much weaker claim.
Percentile: Where a value falls against the rest of the field: the 90th percentile means only a tenth of the set scores higher. Used on 44B when the absolute number is hard to read on its own.
Preprint: A paper posted publicly before, or without, peer review. Most of the AI literature circulates this way, which is why 44B shows citation counts and venues where they exist rather than treating publication as a gate.
Public report: A report of a critical safety incident made by any member of the public under § 1422(1), with no account required. Intake is confidential (§ 1422(6)); the Office may review it (§ 1422(4)) and records a determination.

R

3 terms

Rate limit: The ceiling on API calls, counted two ways: a monthly allowance on the UTC calendar month, and a daily burst limit that is the monthly figure divided by ten — the runaway-script guard rail. Exceeding either returns a rate_limited error.
Reasoning model: A model that spends additional computation on intermediate steps before answering. Labs often ship a reasoning variant alongside a standard one; 44B lists them as separate rows because they price and perform differently.
Redaction: Field-level withholding on the public view of a filing where the statute requires it — § 1428(6) contact details on a developer disclosure being the ordinary case. The record keeps the field; the public view does not show it.

S

6 terms

SAMS: The System for Artificial Intelligence Model Safety — the name for 44B’s authenticated surfaces: the filing desk, the review console, and the correspondence channel between a developer and the Office.
Saturation: The point where the leading models all score near the ceiling of a benchmark and it stops discriminating between them. Saturated benchmarks stay on the record; they just stop being news.
Semantic Scholar: The academic search and citation graph built by the Allen Institute for AI. It supplies the abstracts, authors, venues, and citation counts 44B shows on papers and benchmarks, and it backs the Research index.; Via Semantic Scholar ↗
Semantic search: Retrieval by meaning rather than by matching words — comparing the embedding of your query against the embeddings of the corpus. 44B’s keyword search and its semantic search answer different questions, and both are worth running.
SPECTER2: A document embedding model for scientific papers, from the Allen Institute for AI. 44B used it for "related papers" while its Semantic Scholar index was separate; since that index merged into the Library the whole corpus shares one embedding space, so neighbours are computed across every document rather than a third of them.; Via Semantic Scholar ↗
Statutory clock: A deadline the statute sets, running on the record: 72 hours for a critical safety incident, 30 days for republishing a materially modified framework, quarterly for internal-use assessments, and two years for a disclosure renewal. 44B shows the clock, whether it was met, and by how much.

T

6 terms

The Office: The reviewing authority under the statute — an office within the New York State Department of Financial Services (§ 1420(16)), not the Attorney General. On the public record its reviewers appear anonymously as Reviewer #000N.
Time to first token (TTFT): How long a model takes to emit its first token after a request — the wait before anything appears on screen. Measured in seconds, and reported separately from throughput because a fast stream that starts late still feels slow.; Via Artificial Analysis ↗
Token: The unit a model reads and writes — roughly a short word or word fragment. Prices, context windows, and throughput are all quoted per token, which is why a page of English is a few hundred tokens rather than a few hundred words.
Transmittal: The Office’s referral of a filing onward to the Attorney General under § 1422(5) — recorded on the filing so the handover is part of the record.
Transparency report: The § 1421(3) disclosure published before or when a frontier model is deployed, or substantially modified. Publication inside a system card or model card is deemed compliant (§ 1421(3)(c)).
TrueSkill: A Bayesian rating system, a descendant of Elo, that tracks both a skill estimate and the uncertainty around it — so a model with few comparisons ranks conservatively. The per-capability rankings on the 44B models page are LLM Stats’ TrueSkill leaderboards.; Via LLM Stats ↗

Z

1 term

Zero-shot: Asking a model to do a task with no worked examples in the prompt — only the instruction.

Terms of the trade — benchmark, pass@1, few-shot, open weights — carry no credit line because they belong to no one. Where a definition or a measurement convention is a particular organization’s, it is credited and linked. Where an evaluation is the work of many contributors, it is credited to none of them individually.