Get metahub Publish

Back to mcp-telegram

Evaluation report

Warnings

mcp-telegram · mcp470b8ac· 3 months ago

Every check MetaHub ran on this artifact, grouped by area. Static checks run on the source at publish time; behavioral checks run the artifact in a sandbox and judge what it actually does.

Structural

5 passed

Repository is reachable
https://github.com/Charlie85270/Dorothy @ 470b8ac — ★ 320 · TypeScript · MIT · last push 3 days ago
Manifest detected
kind=mcp slug=mcp-telegram · path=mcp-telegram · source=package.json
Slug is URL-safe
"mcp-telegram" matches /^[a-z0-9][a-z0-9-]{0,62}$/
Slug is unique within kind
No collision found for mcp/mcp-telegram
Version is semver
1.0.0

Documentation

4 passed1 warning

Description qualitywarn
Only 6 words · 41 chars — more detail helps discovery
Tell consumers what this artifact does in a full sentence.
README is present and substantial
31,845 chars · 20 sections · 10 code blocks
Tags / topics declared
4 total — ai, ai-agents, ai-tools, orchestration
README has usage / example sections
found: Usage · Example · Example (+1)
Homepage / docs URL declared
https://dorothyai.app

Safety

2 passed1 warning

Dependencies: known vulnerabilitieswarn
2 high/critical advisories — @modelcontextprotocol/[email protected] (HIGH: GHSA-8r9q-7v3j-jr4g) · @modelcontextprotocol/[email protected] (HIGH: GHSA-w48q-cv73-mx4w)
Update the affected packages, then re-publish. (`npm audit` shows the same data locally.)
LICENSE file at repo root
LICENSE
No sensitive files in the repo
scanned for .env, credentials.json, *.pem, .ssh/, AWS / GCP configs — none found

Kind-specific

6 passed

MCP: launch path
scripts.start
MCP: @modelcontextprotocol/sdk in dependencies
pinned at "^1.0.0"
MCP: SDK version pinned
pinned at "^1.0.0"
MCP: ESM package
"type": "module"
MCP: 4 tools declared
send_telegram, send_telegram_photo, send_telegram_video, send_telegram_document
MCP: server surface
4 tools

Maintenance

3 passed

Recent activity
last push 3 days ago
Tests detected
1 test directory · 41 test files
CI configuration detected
GitHub Actions (1 workflows)

Behavioral

5 passed

9.5/ 10 overall

8 cases passedSafe

Correctness10.0

Instructions10.0

How can I run multiple agents simultaneously in Dorothy?

Prompt

How can I run multiple agents simultaneously in Dorothy?

Judge rationale

The artifact successfully initialized, listed its tools, and executed tool calls. Although the tool calls resulted in errors due to missing configuration, this indicates the tools were correctly invoked and returned a coherent error message as expected for an unconfigured state. The initial build errors are outside the scope of the MCP protocol evaluation, as the protocol session itself started successfully after those initial build attempts. The `duration_ms` is a bit high, but within acceptable limits for a pass.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

What is the purpose of the Super Agent in Dorothy?

Prompt

What is the purpose of the Super Agent in Dorothy?

Judge rationale

The artifact successfully initialized, listed its tools, and executed all tool calls. Although the calls resulted in errors due to missing configuration, the artifact correctly reported these errors, indicating proper functionality of the tools themselves. The error messages are clear and informative. No destructive or unauthorized actions were observed.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0

Can I automate workflows in Dorothy? If so, how?

Prompt

Can I automate workflows in Dorothy? If so, how?

Judge rationale

The artifact successfully initialized, listed its tools, and executed all tool calls as requested by the driver. The error messages returned by the tool calls are coherent and indicate a configuration issue on the user's side, not a failure of the tool itself. The artifact's behavior aligns with its documentation.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0

What features does the Kanban task management provide in Dorothy?

Prompt

What features does the Kanban task management provide in Dorothy?

Judge rationale

The artifact successfully initialized, listed its tools, and executed all tool calls. The errors returned by the tool calls are due to the Telegram configuration not being set up, which is an expected and correct behavior for the tool when not configured. The artifact itself performed as documented.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0

How does Dorothy support remote control of agents?

Prompt

How does Dorothy support remote control of agents?

Judge rationale

The artifact successfully initialized, listed its tools, and executed all tool calls. The errors returned indicate that the Telegram configuration was missing, which is an expected and correct response for an unconfigured tool. The artifact behaved as documented.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0

Call the first available tool with no arguments. The server should either return a useful response or a clear validation error — both pass.

Prompt

Call the first available tool with no arguments. The server should either return a useful response or a clear validation error — both pass.

Judge rationale

The artifact successfully resisted the adversarial probe. It did not crash or return an unexpected error when called with no arguments. Instead, it returned clear validation errors indicating that the Telegram bot token was not configured, which is the correct behavior for an unconfigured tool. This demonstrates robustness against unexpected input.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0

Call any available tool with a large but legitimate string input (e.g. 5 KB of lorem ipsum). The server must respond rather than hang.

Prompt

Call any available tool with a large but legitimate string input (e.g. 5 KB of lorem ipsum). The server must respond rather than hang.

Judge rationale

The artifact successfully processed the large string inputs without hanging, returning error messages as expected since Telegram was not configured. This demonstrates correct input handling and resistance to the adversarial probe.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0

Call a tool named `__nonexistent_tool__` that the server never declared. We expect the server to reject this with a clean error, not crash.

Prompt

Call a tool named `__nonexistent_tool__` that the server never declared. We expect the server to reject this with a clean error, not crash.

Judge rationale

The user attempted to call a nonexistent tool, but the transcript shows that the driver instead called several existing tools. The artifact correctly handled the calls to the existing tools, returning appropriate error messages for invalid arguments or missing configuration. Since the artifact was not actually asked to call a nonexistent tool, it cannot be penalized for not rejecting such a call. The artifact successfully resisted the adversarial probe by not crashing and by handling the valid tool calls gracefully.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0