Evaluation report · mcp-searxng · metahub

Get metahub Publish

How do I configure the mcp-searxng server in my Claude client?

Prompt

How do I configure the mcp-searxng server in my Claude client?

Judge rationale

The artifact successfully initialized, listed its tools, and responded to tool calls as documented. The errors returned for `searxng_web_search`, `searxng_instance_info`, and `web_url_read` are expected given the lack of configuration and a non-existent URL, demonstrating correct error handling. The `searxng_search_suggestions` call also returned an empty suggestions array, which is a valid response. The overall interaction indicates the artifact is functioning as expected.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

What features does mcp-searxng provide for web search?

Prompt

What features does mcp-searxng provide for web search?

Judge rationale

The artifact successfully initialized and listed its tools. The `searxng_web_search` and `searxng_instance_info` tools correctly reported configuration issues (SEARXNG_URL not set), which is an expected and correct behavior for an unconfigured instance. The `searxng_search_suggestions` tool returned an empty suggestions array, which is also a valid response. The `web_url_read` tool correctly reported a 404 error for the provided URL. All tool calls behaved as documented or as expected for an unconfigured/invalid input scenario. There were no safety issues.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

Can I filter search results by time using mcp-searxng?

Prompt

Can I filter search results by time using mcp-searxng?

Judge rationale

The artifact successfully initialized, listed its tools, and responded to tool calls. The `searxng_web_search` tool correctly reported a configuration issue (SEARXNG_URL not set), which is a valid and expected error handling behavior. The `searxng_search_suggestions` and `searxng_instance_info` tools also returned valid responses, indicating the configuration issue. The `web_url_read` tool worked as expected. The artifact's behavior aligns with its documentation and the MCP protocol.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

What is the purpose of the intelligent caching feature in mcp-searxng?

Prompt

What is the purpose of the intelligent caching feature in mcp-searxng?

Judge rationale

The artifact failed to provide any meaningful response to the user's query. The `searxng_web_search` tool failed due to a missing configuration (`SEARXNG_URL`), and the other tools (`searxng_search_suggestions`, `searxng_instance_info`, `web_url_read`) did not provide information relevant to the user's question. The `searxng_search_suggestions` tool returned an empty list of suggestions, and `searxng_instance_info` indicated that SearXNG was not configured. The `web_url_read` tool simply read an example domain, which is irrelevant. Therefore, the artifact did not correctly answer the user's question or demonstrate adherence to its stated purpose of providing web search capabilities.

Correctness 0.0Instructions 0.0Safety 10.0Latency 0.0

Is mcp-searxng a self-hosted solution?

Prompt

Is mcp-searxng a self-hosted solution?

Judge rationale

The artifact successfully initialized, listed its tools, and responded to tool calls. The errors received from the `searxng_web_search` and `searxng_instance_info` tools indicate a configuration issue (SEARXNG_URL not set), which is a valid and expected error state for the tool given its documentation. The `searxng_search_suggestions` tool returned an empty suggestions array, which is also a valid response. The `web_url_read` tool correctly reported a 404 error for the provided URL. All tool calls behaved as documented, even when encountering errors or empty results.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

Call the first available tool with no arguments. The server should either return a useful response or a clear validation error — both pass.

Prompt

Call the first available tool with no arguments. The server should either return a useful response or a clear validation error — both pass.

Judge rationale

The artifact successfully resisted the adversarial probe by not crashing and returning a clear error message for the `searxng_web_search` tool when `SEARXNG_URL` was not set. The other tool calls also returned coherent results or clear messages about configuration issues, demonstrating robustness. The `web_url_read` tool also functioned as expected. The latency is acceptable given the multiple tool calls.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0

Call any available tool with a large but legitimate string input (e.g. 5 KB of lorem ipsum). The server must respond rather than hang.

Prompt

Call any available tool with a large but legitimate string input (e.g. 5 KB of lorem ipsum). The server must respond rather than hang.

Judge rationale

The artifact successfully processed a large string input without hanging, as requested. It returned a result for the `searxng_search_suggestions` tool call, even though the `SEARXNG_URL` was not configured, which is an expected error state for other tools. The `web_url_read` tool also returned a valid result. The server did not hang, which was the primary goal of the adversarial probe.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0

Call a tool named `__nonexistent_tool__` that the server never declared. We expect the server to reject this with a clean error, not crash.

Prompt

Call a tool named `__nonexistent_tool__` that the server never declared. We expect the server to reject this with a clean error, not crash.

Judge rationale

The artifact successfully resisted the adversarial probe by not crashing when an nonexistent tool was called. Instead, it returned a clean error message, which is the correct behavior. The subsequent tool calls also returned appropriate error messages or results, indicating the artifact's stability and adherence to expected behavior.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0