AI SDK Change Tracking: Catch Model and API Updates Early

Official SDKs are unusually useful observatories because many are generated from an API description. Stainless says its generators build SDKs from OpenAPI specifications, and Fern documents the same OpenAPI-to-SDK pipeline. A schema change can therefore propagate into typed model unions, request parameters, beta headers, resource classes, examples and changelogs across several languages.

The original EyesTech draft overstated what those artifacts can prove. It claimed a universal 14–28 day lead, 98.5% noise removal and secret-model verification from HTTP timing without a dataset or reproducible records. Those figures have been removed. This guide uses passive collection from public GitHub repositories, npm and PyPI; it does not probe private, undocumented or access-controlled endpoints.

Why public SDKs expose useful change signals

A generated SDK is a materialized view of an API contract. For example, OpenAI’s public openai-node shared types identify the file as generated from an OpenAPI specification by Stainless and expose model names through TypeScript unions. Anthropic’s public TypeScript SDK also marks generated resource files and publishes a versioned changelog. These files are excellent for answering “what changed in the client contract?”

They are weaker at answering “is this feature enabled for my account?” A string may be added for documentation parity, an upcoming release, a private beta, a cloud-specific route or backwards compatibility. Some SDK parameters also accept arbitrary strings, so the absence of a literal does not prove a model is unavailable. Treat generated code as evidence about the SDK, not direct evidence about production routing.

The four-source collection map

1. Git commits and immutable tags

Start with the provider’s official public repository. Compare release tags or commit hashes rather than a moving branch. GitHub’s compare-two-commits API returns changed files and can provide diff or patch media types; unauthenticated requests work for public resources, subject to public rate limits. Preserve the base SHA, head SHA, retrieval time and file path.

2. Package registries

The npm registry’s public package metadata endpoint returns versions, distribution tags, dependencies and tarball integrity information. PyPI’s JSON API returns project metadata plus release files and SHA-256 digests. Registry timestamps establish when an artifact became public, while hashes let another researcher verify that they inspected the same bytes.

3. Changelogs and release notes

Changelogs often classify an otherwise ambiguous diff. Anthropic’s Python SDK changelog, for example, separates features, fixes, documentation and internal code-generation updates. A generated file changing alongside an explicit “api” feature entry is stronger evidence than a literal appearing alone.

4. Official documentation

The provider’s model page, API reference, pricing page and release announcement remain the authority for availability and commercial terms. When SDK code and documentation disagree, report the discrepancy and its timestamps. Do not silently choose the more exciting source.

A passive semantic-diff workflow

1. Define the watch surface. List official repositories, package names and high-signal paths such as shared model types, request schemas, beta headers, resources and changelogs.
2. Snapshot immutable versions. Store release tags, commit SHAs, registry version numbers, timestamps and package hashes.
3. Discard mechanical churn. Ignore formatting-only changes, generated comments, import ordering, documentation wrapping and dependency-lock noise.
4. Extract semantic objects. Compare string literals, enum members, method signatures, endpoint paths, request fields and required/optional status.
5. Corroborate. Search another official language SDK, the package release, changelog and provider documentation for the same concept.
6. Classify the finding. Use observed, corroborated or confirmed. Do not call a model “released” from a code diff alone.
7. Publish the evidence bundle. Include links, hashes, retrieval time, extraction rules, exclusions and a correction path.

This is the same denominator discipline EyesTech applies to reasoning-token cost audits: a headline metric becomes useful only after the measurement boundary is explicit.

Reproducible Python example: compare model-like literals

The script below runs against an already cloned public repository and two explicit Git references. It never contacts an API service. It extracts quoted strings containing common model-family terms, then reports additions and removals. Adapt the allowlist and paths to the repository you are studying; a generic regex is a triage tool, not a semantic parser.

from __future__ import annotations

import re
import subprocess
from pathlib import Path

MODEL_TOKEN = re.compile(
    r"['\"]([^'\"]*(?:gpt|claude|gemini|llama|model)[^'\"]*)['\"]",
    re.IGNORECASE,
)

def git_show(repo: Path, ref: str, file_path: str) -> str:
    result = subprocess.run(
        ["git", "-C", str(repo), "show", f"{ref}:{file_path}"],
        check=True, capture_output=True, text=True,
    )
    return result.stdout

def tokens(source: str) -> set[str]:
    return {match.group(1) for match in MODEL_TOKEN.finditer(source)}

repo = Path("openai-node")
path = "src/resources/shared.ts"
before = tokens(git_show(repo, "v5.0.0", path))
after = tokens(git_show(repo, "v6.0.0", path))

print("added:", sorted(after - before))
print("removed:", sorted(before - after))

Use real tags that contain the chosen path, and pin the repository URL and commit objects in your report. For production monitoring, replace the regex with a TypeScript or Python AST parser and keep snapshots as JSON so reviewers can inspect the exact extracted set.

How to score evidence without inventing precision

A probability such as “85% likely” is unjustified unless it comes from a documented, labeled historical dataset with out-of-sample calibration. A three-level evidence label is less dramatic and more reproducible.

False positives that repeatedly fool SDK watchers

Generated clients can also lag documentation. The correct finding may be “the SDK has not caught up,” not “the provider is hiding a model.” This distinction matters when comparing fast-moving agent products such as those covered in EyesTech’s Claude Code and Codex workflow analysis.

Ethical and operational boundaries

• Collect only intentionally public repositories, release pages, documentation and registry metadata.
• Respect rate limits, robots directives, licenses and repository terms.
• Do not test guessed identifiers against private, staging or undocumented endpoints.
• Never attempt to bypass authentication, geographic controls, allowlists or account permissions.
• Redact credentials or secrets accidentally committed to public history and follow the provider’s security-reporting process.
• Separate product intelligence from vulnerability research; use coordinated disclosure for security impact.

OpenSSF’s package-repository guidance treats machine-readable registry signals as transparency mechanisms rather than guarantees. That is the correct mental model here: observable metadata supports an investigation, but it does not certify the service behind it.

EyesTech Intelligence Desk workflow

Disclosure: This article is a methods guide based on public documentation and repositories. EyesTech did not obtain private provider data, test access-controlled endpoints or measure a universal pre-announcement lead time. Examples were checked on 14 September 2026. Corrections follow the EyesTech editorial policy.

Frequently asked questions

Can an SDK diff confirm an unreleased AI model?

No. It can confirm that an identifier or interface appeared in a public SDK version. Availability requires official documentation, an announcement or authorized account-level evidence.

Should I monitor GitHub, npm or PyPI first?

Monitor all relevant official surfaces. GitHub explains the code change, while registries provide immutable versions, publish timestamps and artifact hashes. Their combination is stronger than either source alone.

Is endpoint probing part of SDK change tracking?

No. A defensible public-intelligence workflow does not need guessed requests to undocumented services. Passive artifact analysis is reproducible and avoids confusing gateway behavior with product evidence.

The publication rule

Publish the artifact before the interpretation: repository, path, before-and-after hashes, package version, retrieval time and exact semantic change. Then apply the narrowest evidence label the record supports. That turns SDK monitoring from rumor production into auditable technical intelligence.