🇨🇳 中文文档: README.zh-CN.md | 欢迎中国开发者贡献!

MCP Observatory

  ███╗   ███╗ ██████╗██████╗
  ████╗ ████║██╔════╝██╔══██╗
  ██╔████╔██║██║     ██████╔╝
  ██║╚██╔╝██║██║     ██╔═══╝
  ██║ ╚═╝ ██║╚██████╗██║
  ╚═╝     ╚═╝ ╚═════╝╚═╝
     O B S E R V A T O R Y

More badges

MCP Observatory maps the risk graph of agent toolchains before agents depend on them. It helps teams validate MCP servers before deployment into sensitive, regulated, or mission-critical agentic AI environments.

Agents should not depend on tools nobody tests. MCP Observatory turns a local MCP check into portable receipts, risk graphs, release-gate evidence, SARIF for GitHub Code Scanning, GitHub Actions gates, schema drift detection, trust status output, score badges, and agent-accessible diagnostics.

npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format markdown --output mcp-audit.md

Sample trust output:

{
  "target_id": "my-mcp-server",
  "profile": "nsa-mcp",
  "score": 87,
  "status": "needs_review",
  "finding_count": 2
}

The nsa-mcp profile is not an official certification. It maps MCP Observatory findings to practical control areas for sensitive environments: trust boundaries, tool permissions, tool description integrity, authentication, secrets exposure, schema validation, input validation, auditability, runtime safety, and supply chain.

Trust Signals

Signal	What it means
CI + coverage	Typecheck, lint, tests, build, packed install, artifact validation, smoke test, and measured coverage run in GitHub Actions.
CodeQL + OpenSSF Scorecard	Static analysis and supply-chain posture are visible in GitHub-native security surfaces.
Dependabot	npm and GitHub Actions dependency updates are monitored weekly.
npm provenance workflow	Release automation is prepared for npm provenance through GitHub OIDC.
Security policy	Vulnerability reports go through private disclosure; see SECURITY.md.

Try It

Run the public evidence loop: generate a receipt, map it into a risk graph, add CI/SARIF, then request a private fleet review when the server matters to production.

npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format markdown --output report.md
npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format sarif --output results.sarif
npx @kryptosai/mcp-observatory score npx -y my-mcp-server --profile nsa-mcp --format json

Or start with the homepage demo: safely simulate MCP attack-readiness for one server, emit an action receipt, and produce SARIF evidence that maintainers can inspect in GitHub Code Scanning.

npx @kryptosai/mcp-observatory attack-sim npx -y my-mcp-server --sarif attack-results.sarif

Emit the portable trust record:

npx @kryptosai/mcp-observatory audit npx -y my-mcp-server --profile nsa-mcp --format json --output report.json --receipt receipt.json
npx @kryptosai/mcp-observatory receipt npx -y my-mcp-server --profile nsa-mcp --format markdown --output receipt.md
npx @kryptosai/mcp-observatory risk-graph --input receipt.json --json mcp-risk-graph.json --output mcp-risk-graph.md --html mcp-risk-graph.html

Then make the evidence repeatable in CI:

npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server" --sarif

See the government and enterprise pilot brief, public guidance crosswalk, procurement one-pager, security due diligence packet, NSA-MCP audit CI guide, example NSA-MCP audit report, MCP Receipts, MCP Attack Simulator, Tool-call receipts, MCP Risk Graph, private fleet risk graph pilot, launch page, GitHub Code Scanning demo, GitHub Code Scanning for MCP servers, sample safety reports, and reference evaluations.

Want a receipt for a server your agent depends on? Comment on Drop an MCP server, get a receipt #146 or use the structured receipt request form. Public requests can become Safety Index entries, delta receipts, SARIF evidence, and maintainer CI conversations.

Evidence You Can Inspect

Evidence	Where
Example GitHub Actions adoption	setup-ci --all and the generated workflow docs
NSA-MCP audit example	Markdown report, SARIF, and score JSON
Procurement and pilot packet	Public guidance crosswalk, procurement one-pager, and security due diligence
Attack simulation output	MCP Attack Simulator
MCP receipts	Portable trust receipts
Tool-call receipts	Receipt standard for reproducible MCP evidence
Risk graph	Server-to-evidence map for agent toolchain trust decisions
SARIF / Code Scanning output	GitHub Code Scanning demo
Real MCP server evaluations	MCP Server Safety Index
Reference reports	Reference evaluations
Maintainer and contributor proof	MCP Observatory Contributors
Open core boundary	What is open vs. commercial
Security disclosure path	SECURITY.md

Two more fast paths:

Cloned this repo? Start here: CLONED_THIS.md. Want to contribute? Add one server to the MCP Target Registry, use the Agent Task Pack, and get public credit through MCP Observatory Contributors.

AI coding agents, agentic workflows, and rough PRs are welcome. Use the 10x Agentic Growth Sprint, Agentic Contributor Outreach, or open a Contributor quest, Agentic contribution idea, or Drop an MCP server, get a receipt issue to suggest a target, prompt, docs fix, receipt, or setup-ci --sarif integration.

Add MCP CI and Code Scanning in one command:

npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server" --sarif --schedule weekly

Repair or upgrade an existing adoption kit:

npx @kryptosai/mcp-observatory setup-ci --doctor --fix

Installing MCP Observatory in an MCP server project also prints the exact CI setup command. Projects can opt into automatic workflow creation during install with mcpObservatory.autoSetupCi.

Normal scan and test runs include safe attack-readiness simulation by default. Use --no-attack-sim only when you want the older compatibility-only path.

Upload normalized MCP findings to GitHub Code Scanning when you want a security-native release gate:

npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server" --sarif

Add Observatory as an agent-accessible MCP server:

claude mcp add mcp-observatory -- npx -y @kryptosai/mcp-observatory serve

Building an autonomous agent, OpenClaw-style productivity machine, MCP gateway, or bot runtime? Start with the agent runtime quickstart, copy the OpenClaw MCP reliability agent template, or point your agent at llms.txt and AGENTS.md.

Or test a server immediately:

npx @kryptosai/mcp-observatory test npx -y @modelcontextprotocol/server-everything

Use it as a CLI, a GitHub Action, or an MCP server that lets your AI agent scan, test, record, replay, and verify other MCP servers autonomously.

The Glama card is an external MCP directory scorecard. Treat it as directory-level social proof; click through for the underlying category details before using it as a production approval signal.

Why MCP Observatory

MCP servers are becoming production dependencies. If agents rely on them, teams need a way to catch broken tools, unsafe schemas, schema drift, slow responses, and security footguns before those failures reach users.

Observatory gives maintainers and teams:

• One-command CI setup with setup-ci --all
• Profile-mapped audits with audit --profile nsa-mcp
• MCP receipts that package target, evidence, verdict, action, and reproduction commands
• MCP risk graphs that group servers by capability boundary, receipt state, CI posture, and recommended action
• Action receipts that say allow, gate, rerun, quarantine, or escalate
• GitHub PR comments for compatibility, drift, and security findings
• GitHub Code Scanning SARIF for normalized MCP findings
• Health score badges for public trust signals
• Record/replay/verify workflows for regression testing
• MCP server mode so agents can inspect other MCP servers directly
• Production support path for hosted history, private repo reporting, certification, support, and fleet visibility

See the launch page, GitHub Code Scanning for MCP servers, Code Scanning demo, target gallery, target registry, target contribution guide, MCP Observatory Contributors, Agent Task Pack, MCP Receipts, Tool-call receipts, MCP Risk Graph, setup-ci --doctor, MCP server security field guide, Safety Methodology, MCP Server Safety Index, June 2026 safety field report, reference evaluations, MCP lock files, public proof, campaign attribution, local metrics dashboard, open core boundary, MCP Attack Simulation Evidence Pack, Private MCP Fleet Risk Graph, and commercial support.

For Security And Platform Teams

MCP servers are becoming part of the AI software supply chain. Agents need reliable, testable, auditable tools before those tools become dependencies in mission-critical workflows.

MCP Observatory gives security and platform teams MCP server CI, schema drift detection, security findings, SARIF/HTML/Markdown reports, GitHub Code Scanning upload, and a path toward certification or fleet visibility. Local OSS use stays free; production, private repo, and fleet usage can move through a paid MCP Readiness Review.

Production Support

Local OSS use stays free under MIT. Teams running MCP in production can use the Private MCP Fleet Risk Graph and MCP Attack Simulation Evidence Pack for safe-mode attack simulation, SARIF/Code Scanning setup, CI rollout, private evidence reporting, and owner-ready remediation notes. Private fleet risk graph pilots start at $50,000; attack simulation packages start at $15,000; narrow readiness reviews start at $2,500.

The open source repo is the public evidence engine. Private telemetry intelligence, company/account prioritization, commercial ranking weights, hosted fleet workflows, and buyer-specific evidence packs stay outside the OSS package; see the open core boundary.

Run npx @kryptosai/mcp-observatory cloud, open a pilot request from the issue chooser, or see COMMERCIAL.md. Also see privacy and telemetry, campaign attribution, and terms for production use.

How It Compares

Feature	mcp-observatory	Snyk agent-scan	Cisco mcp-scanner	agent-shield
MCP-native	✓	✓	✓	✓
Attack simulation	✓	✗	✗	✗
Schema drift detection	✓	✗	✗	✗
Record/replay/verify	✓	✗	✗	✗
Health scoring (0-100)	✓	✗	✗	✗
SARIF output	✓	✓	✓	✓
CI/CD native (setup-ci)	✓	✓	✓	✓
Safety index (17+ servers)	✓	✗	✗	✗
Runtime enforcement via mcp-seatbelt	✓	✗	✗	✗

Quick Start

Scan every MCP server in your Claude config:

npx @kryptosai/mcp-observatory

Go deeper — also invoke safe tools to verify they actually run:

npx @kryptosai/mcp-observatory scan deep

Test a specific server:

npx @kryptosai/mcp-observatory test npx -y @modelcontextprotocol/server-everything

Add it to Claude Code as an MCP server:

claude mcp add mcp-observatory -- npx -y @kryptosai/mcp-observatory serve

Or add it manually to your config:

{
  "mcpServers": {
    "mcp-observatory": {
      "command": "npx",
      "args": ["-y", "@kryptosai/mcp-observatory", "serve"]
    }
  }
}

Commands

Command	What it does
scan	Auto-discover servers, check them, and run safe attack-readiness simulation by default
scan deep	Scan, run safe attack simulation, and also invoke safe tools to verify they execute
test <cmd> / test --target <file>	Test one server and emit an action receipt by command or target config
record <cmd>	Record a server session to a cassette file for offline replay
replay <cassette>	Replay a cassette offline — no live server needed
verify <cassette> <cmd>	Verify a live server still matches a recorded cassette
diff <base> <head>	Compare two run artifacts for regressions and schema drift
watch <config>	Watch a server for changes, alert on regressions
suggest	Detect your stack and recommend MCP servers from the registry
serve	Start as an MCP server for AI agents
lock	Snapshot MCP server schemas into a lock file
lock verify	Verify live servers match the lock file
history	Show health score trends for your MCP servers
setup-ci / init-ci	Create a GitHub Action and badge snippet for MCP compatibility/security checks
setup-ci --sarif	Generate a workflow that uploads normalized findings to GitHub Code Scanning
setup-ci --doctor	Inspect whether the repository has a complete CI adoption kit
risk-graph --input <path>	Merge receipts and run artifacts into JSON, Markdown, and HTML MCP risk graphs
--no-attack-sim	Opt out of the default safe attack simulation on scan or test
ci-report	Generate CI report for GitHub issue creation
enterprise-report	Generate a static production/security report from run artifacts
score <cmd>	Score an MCP server's health (0-100)
badge <cmd>	Generate an SVG health score badge for README
cloud	Show hosted reporting, security review, and enterprise pilot options

Run with no arguments for an interactive menu:

What It Does

Check capabilities — connects to a server and verifies tools, prompts, and resources respond correctly.

Invoke tools — goes beyond listing. Actually calls safe tools (no required params / readOnlyHint) and reports which ones work and which ones crash.

npx @kryptosai/mcp-observatory scan deep

Detect schema drift — diffs two runs and surfaces added/removed fields, type changes, and breaking parameter changes.

npx @kryptosai/mcp-observatory diff run-a.json run-b.json

Recommend servers — scans your project for languages, frameworks, databases, and cloud providers, then cross-references the MCP registry to suggest servers you're missing.

npx @kryptosai/mcp-observatory suggest

Or ask your agent "what MCP servers should I add?" when running in MCP server mode.

Security scanning — analyzes tool schemas for dangerous patterns: shell injection surfaces, broad filesystem access, missing auth, and credential leakage in responses.

npx @kryptosai/mcp-observatory test --security npx -y my-mcp-server

Record / replay / verify — capture a live session, replay it offline in CI, and verify nothing changed. Like VCR for MCP.

# Record a session
npx @kryptosai/mcp-observatory record npx -y @modelcontextprotocol/server-everything

# Replay offline (no server needed)
npx @kryptosai/mcp-observatory replay .mcp-observatory/cassettes/latest.cassette.json

# Verify the live server still matches
npx @kryptosai/mcp-observatory verify cassette.json npx -y @modelcontextprotocol/server-everything

Watch for regressions — re-runs checks on an interval and alerts when something changes.

npx @kryptosai/mcp-observatory watch target.json

Scan locations

When you run scan, it looks for MCP configs in:

• ~/.claude.json (Claude Code)
• ~/Library/Application Support/Claude/claude_desktop_config.json (Claude Desktop, macOS)
• %APPDATA%/Claude/claude_desktop_config.json (Claude Desktop, Windows)
• .claude.json and .mcp.json (current directory)

Architecture

                    ┌─────────────────────────┐
                    │   MCP Observatory CLI    │
                    │  npx @kryptosai/mcp-     │
                    │     observatory scan     │
                    └───────────┬─────────────┘
                                │
                    ┌───────────▼─────────────┐
                    │   Config Discovery       │
                    │  (Claude, Cursor, etc.)  │
                    └───────────┬─────────────┘
                                │
              ┌─────────────────┼─────────────────┐
              ▼                 ▼                  ▼
    ┌─────────────────┐ ┌──────────────┐ ┌──────────────────┐
    │   Security Scan  │ │  Attack Sim  │ │  Schema Drift    │
    │  (shell, creds)  │ │ (tool poison)│ │  (version diff)  │
    └────────┬────────┘ └──────┬───────┘ └────────┬─────────┘
             │                 │                   │
             └─────────────────┼───────────────────┘
                               ▼
                    ┌─────────────────────┐
                    │   Health Score       │
                    │  (0-100 + verdict)   │
                    └──────────┬──────────┘
                               │
              ┌────────────────┼────────────────┐
              ▼                ▼                 ▼
    ┌──────────────┐  ┌──────────────┐  ┌──────────────┐
    │  SARIF       │  │  Markdown    │  │  CI Gateway  │
    │  (Code Scan) │  │  Report      │  │  (setup-ci)  │
    └──────────────┘  └──────────────┘  └──────────────┘

CI / GitHub Action

Add Observatory to your MCP server's CI pipeline:

npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server" --sarif --schedule weekly

Check the adoption kit:

npx @kryptosai/mcp-observatory setup-ci --doctor

Successful test, run, and single-target scan checks also offer to convert the passing result into a CI adoption kit. That automatic conversion enables SARIF/Code Scanning and weekly scheduled checks by default; pass --no-ci-sarif when you only want a conservative workflow without Code Scanning upload.

Or create the workflow manually:

# .github/workflows/observatory.yml
name: MCP Server Check
on: [pull_request]

permissions:
  contents: read

jobs:
  observatory:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: KryptosAI/mcp-observatory/[email protected]
        with:
          command: npx -y my-mcp-server
          deep: true
          security: true
          comment-on-pr: false
          set-status: false

Action inputs:

Input	Description	Default
command	Server command to test	(required if no target)
target	Path to target config JSON
targets	Path to MCP config file for multi-server matrix scan
deep	Also invoke safe tools	false
security	Run security analysis	false
fail-on-regression	Fail the action on issues	true
fail-on-baseline-drift	Fail the action when baseline verification detects drift	true
comment-on-pr	Post report as PR comment. Requires pull-requests: write.	true
set-status	Set a commit status check (green/red) on the HEAD SHA. Requires statuses: write.	true
github-token	Token for PR comments and commit statuses	${{ github.token }}

The action can comment on PRs and set commit statuses when the workflow grants write permissions. setup-ci generates read-only third-party-friendly workflows by default and lets maintainers opt into comments/statuses later. init-ci remains available as a backward-compatible alias. See action/README.md for all options.

Production teams can add hosted CI history, private-repo reporting, recurring security reports, certification review, support, and fleet visibility. Run npx @kryptosai/mcp-observatory cloud, see COMMERCIAL.md, or open a pilot request from the issue chooser.

Certified by MCP Observatory

MCP server maintainers can add a public compatibility/security signal to their README:

Or generate a score badge from a live check:

npx @kryptosai/mcp-observatory badge npx -y my-mcp-server --output docs/mcp-health.svg

See the certification distribution loop for the GitHub Action template, maintainer PR body, and badge rollout playbook.

Generate a pilot-ready production/security report from local run artifacts:

npx @kryptosai/mcp-observatory enterprise-report \
  --account "Your Company" \
  --format html \
  --output observatory-enterprise-report.html

For clearer internal account attribution in CI, set:

MCP_OBSERVATORY_ORG=your-company.com
MCP_OBSERVATORY_CONTACT=your-team-contact

Testing Feishu/Lark integrations? See the Feishu/Lark MCP guide.

Lock Files

$ npx @kryptosai/mcp-observatory lock              # Snapshot all server schemas
$ npx @kryptosai/mcp-observatory lock verify        # Verify no drift since last lock

Lock files are the package-lock for AI tools: commit the MCP contract, then make every tool, schema, prompt, or resource drift visible in CI. See MCP lock files.

Trend Tracking

$ npx @kryptosai/mcp-observatory history            # Show health trends over time

Nightly Scans

$ npx @kryptosai/mcp-observatory ci-report          # Generate regression report for CI

MCP Server Mode

No other testing tool is itself an MCP server. Add Observatory as a server and your AI agent can autonomously test, diagnose, and monitor your other MCP servers.

claude mcp add mcp-observatory -- npx -y @kryptosai/mcp-observatory serve

Your agent gets 10 tools:

Tool	When to use it
scan	Check if all your configured MCP servers are healthy
check_server	Test a specific server before installing or after updating
score_server	Get a quick health score and grade for a server
record	Capture a baseline of a working server for future comparison
replay	Test against a recorded session — no live server needed
verify	Confirm a server update didn't break anything
watch	Check a server and see what changed since the last check
diff_runs	Find regressions between two check results
get_last_run	Retrieve previous check results for a server
suggest_servers	Discover MCP servers that match your project stack

An AI tool that checks other AI tools. It is a tool testing tools that serve tools.

Security

The MCP server runs inside AI hosts where an LLM chooses which tools to call. To prevent prompt-injection attacks:

• Command allowlist: Only npx, node, python, python3, uvx, docker, deno, bun are permitted as base executables. The CLI has no restrictions.
• Path validation: File-reading tools are constrained to the runs/cassettes directories.
• No arbitrary execution: Use the CLI for unrestricted commands.

CLI vs MCP: Intentional Differences

Feature	CLI	MCP Server	Why
watch	Polling loop	Single check + diff	Request/response doesn't support long-polling
Interactive menu	Arrow-key navigation	Not available	MCP has no interactive UI
Color output	--no-color flag	Always plain text	MCP returns structured content
report	Renders saved artifacts	Not available	Agents read artifacts directly
serve	Starts MCP server	N/A	Is the MCP server
run	Reads target config files	Inline params	MCP tools accept params directly
get_last_run	Not available (use ls + diff)	Available	Convenience for agents

Compatibility

Works with any MCP server that uses standard transports:

Transport	Examples	Adapter
stdio (most servers)	filesystem, memory, context7, brave-search, sentry, notion, stripe	local-process
HTTP/SSE (remote)	Cloudflare, Exa, Tavily	http
Docker	All @modelcontextprotocol/server-* images	local-process via docker run -i

Servers needing API keys work via env in the target config. Python servers work via uvx. See the full compatibility matrix for tested servers and known issues.

Target config files

For more control (env vars, metadata, custom timeout):

{
  "targetId": "filesystem-server",
  "adapter": "local-process",
  "command": "npx",
  "args": ["-y", "@modelcontextprotocol/server-filesystem", "."],
  "timeoutMs": 15000,
  "skipInvoke": false
}

npx @kryptosai/mcp-observatory run --target ./target.json

HTTP / SSE targets

{
  "targetId": "my-remote-server",
  "adapter": "http",
  "url": "https://mcp.example.com/mcp",
  "authToken": "${MCP_SERVER_TOKEN}",
  "headers": {
    "X-Api-Key": "$MCP_SERVER_API_KEY"
  },
  "timeoutMs": 15000
}

Target configs support ${VAR}, $VAR, and env:VAR references in authToken, headers, and local-process env values.

How It Compares

Feature	Observatory	mcp-recorder	MCPBench	mcp-jest
Auto-discover servers	✅	—	—	—
Check capabilities	✅	—	✅	✅
Invoke tools	✅	—	—	✅
Schema drift detection	✅	—	—	—
Record / replay	✅	✅	—	—
Verify against cassette	✅	—	—	—
Response snapshot diffs	✅	—	—	—
Benchmarking / latency	—	—	✅	—
Jest integration	—	—	—	✅
Works as MCP server	✅	—	—	—

Each tool has strengths. Observatory focuses on regression detection and CI-friendly workflows. mcp-recorder is great as a transparent proxy. MCPBench is the go-to for performance benchmarking. mcp-jest is ideal if you're already in a Jest workflow.

Prior Art

The record/replay/verify pattern is inspired by:

• VCR (Ruby) — pioneered cassette-based HTTP record/replay
• Polly.js (Netflix) — HTTP interaction recording for JavaScript
• mcp-recorder — MCP-specific traffic recording proxy
• MCPBench — MCP server benchmarking
• mcp-jest — Jest-style testing for MCP servers

Limitations

• Servers requiring interactive OAuth (e.g., Google Drive) need pre-authentication before Observatory can connect
• Custom WebSocket transports (e.g., BrowserTools MCP) are not supported
• A few servers time out or close before init — see known issues and compatibility

Works with mcp-seatbelt

Scan before you trust. Enforce at runtime with mcp-seatbelt — an MCP proxy that consumes Observatory receipts and blocks out-of-contract tool calls in production. Observatory validates; seatbelt enforces.

Contributors ✨

Thanks to these amazing people who have contributed:

• leemeo3 — 3 Safety Index targets (Git, Chrome DevTools, Filesystem MCP)
• albatrossflyon-coder — GitHub MCP Safety Index (#201)
• tanishxdev — Legacy CLI deprecation warnings (#187)
• sansynx — CLI format validation (#182)

See all contributors →

Contributing

We welcome contributors! This project follows a Contributor Covenant Code of Conduct. The fastest way to get involved:

git clone https://github.com/KryptosAI/mcp-observatory.git && cd mcp-observatory && npm install && npm test

The most common first contribution is adding an MCP server to the Safety Index (10-15 minutes). See CONTRIBUTING.md for full guidelines, code standards, and the contributor recognition ladder.

---

If Observatory saved you a broken deploy, consider giving it a star. It helps others find the project.