Evaluation report · prismer-mcp-server · metahub

Get metahub Publish

How do I load web content into the LLM-optimized context using Prismer?

Prompt

How do I load web content into the LLM-optimized context using Prismer?

Judge rationale

The `initialize` handshake, `tools/list`, and `tools/call` all succeeded. The exposed tools are coherent and match the documentation. The artifact correctly reported that the API key was not configured, which is a valid and expected response for the given input. No destructive or unauthorized actions occurred.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

What command do I use to send a message through the Prismer MCP server?

Prompt

What command do I use to send a message through the Prismer MCP server?

Judge rationale

The artifact successfully initialized, listed its tools, and responded to multiple tool calls. The responses consistently indicated that an API key was not configured, which is a correct and expected behavior given the setup. The tool calls themselves were well-formed and the artifact handled them gracefully.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

How can I extract text from a PDF using Prismer?

Prompt

How can I extract text from a PDF using Prismer?

Judge rationale

The artifact successfully initialized, listed its tools, and responded to multiple tool calls. The responses consistently indicated that an API key was not configured, which is a correct and expected behavior given the setup. The `prismer.parse.document` tool was called as expected, and its response was coherent.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

What steps do I need to take to create a new Gene in Prismer?

Prompt

What steps do I need to take to create a new Gene in Prismer?

Judge rationale

The artifact successfully initialized, listed its tools, and correctly reported API key not configured for all tool calls. This indicates that the tools are functioning as documented, even when encountering an expected configuration error. The driver's choice of tools to call is not a reflection on the artifact's correctness.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

How can I recall relevant memories using Prismer?

Prompt

How can I recall relevant memories using Prismer?

Judge rationale

The artifact successfully initialized, listed its tools, and executed several tool calls. All tool calls returned a coherent error message indicating that an API key was not configured, which is a valid and expected response for an unconfigured tool. The tool documentation is consistent with the observed behavior.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

Call the first available tool with no arguments. The server should either return a useful response or a clear validation error — both pass.

Prompt

Call the first available tool with no arguments. The server should either return a useful response or a clear validation error — both pass.

Judge rationale

The artifact successfully resisted the adversarial probe. It did not crash or return an unexpected error when called with no arguments. Instead, it correctly identified that the API key was not configured and returned a clear error message, which is the expected behavior for resisting this type of attack. The tool calls were made, and the artifact returned a consistent and appropriate error message for each call, indicating that it is functioning as designed under these conditions.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0

Call any available tool with a large but legitimate string input (e.g. 5 KB of lorem ipsum). The server must respond rather than hang.

Prompt

Call any available tool with a large but legitimate string input (e.g. 5 KB of lorem ipsum). The server must respond rather than hang.

Judge rationale

The artifact successfully resisted the adversarial probe by returning an error message indicating that the API key was not configured, rather than hanging or performing any destructive actions. This is the correct behavior for an adversarial probe. The latency was a bit high, but within acceptable limits.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0

Call a tool named `__nonexistent_tool__` that the server never declared. We expect the server to reject this with a clean error, not crash.

Prompt

Call a tool named `__nonexistent_tool__` that the server never declared. We expect the server to reject this with a clean error, not crash.

Judge rationale

The user attempted to call a nonexistent tool. The artifact correctly resisted this attack by not crashing and instead returning a structured error for the invalid tool calls. The `tools/call` commands show that the artifact returned an error message indicating an 'Input validation error' or 'Failed: API key not configured' for the valid tool calls, which is a clean error response. The `__nonexistent_tool__` was not called, which is the correct behavior for resisting the adversarial probe.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0