Evaluation report · camofox-mcp · metahub

Get metahub Publish

How do I start the CamoFox browser server using Docker?

Prompt

How do I start the CamoFox browser server using Docker?

Judge rationale

The artifact successfully initialized, listed its tools, and responded to tool calls. The errors received (CONNECTION_REFUSED, TAB_NOT_FOUND) are expected given that the CamoFox browser server was not running, which is outside the artifact's control. The artifact's responses to these errors are coherent and match what would be expected from the documentation. The `list_tabs` call correctly returned an empty array, indicating proper functionality even without an active browser session. The tool calls made by the driver are consistent with the listed tools.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

What is the command to verify if the CamoFox browser server is running?

Prompt

What is the command to verify if the CamoFox browser server is running?

Judge rationale

The artifact successfully initialized, listed its tools, and correctly executed several tool calls, including `server_status`, `create_tab`, `close_tab`, `list_tabs`, and `navigate`. The `server_status` and `create_tab` calls correctly reported a connection refused error, indicating the server was not running as expected by the test setup. The `close_tab` and `navigate` calls correctly reported `TAB_NOT_FOUND` because no tab was successfully created. The `list_tabs` call correctly returned an empty array. All responses were coherent and matched the expected behavior for the given circumstances. The latency was acceptable.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

Can you provide an example of how to configure CamoFox MCP in a JSON format?

Prompt

Can you provide an example of how to configure CamoFox MCP in a JSON format?

Judge rationale

The artifact successfully initialized, listed its tools, and responded to several tool calls. The errors received from the tool calls (e.g., CONNECTION_REFUSED, TAB_NOT_FOUND) indicate issues with the CamoFox API or the state of the browser, not with the MCP artifact itself. The artifact correctly reported these errors as documented. The `list_tabs` call correctly returned an empty array, indicating no tabs were open, which is consistent with the previous `create_tab` call failing. The latency is acceptable for the number of operations performed.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

What are the key features of the CamoFox MCP?

Prompt

What are the key features of the CamoFox MCP?

Judge rationale

The artifact successfully initialized, listed its tools, and responded to tool calls as documented. The `CONNECTION_REFUSED` and `TAB_NOT_FOUND` errors are expected behavior when the underlying CamoFox API is not running or a tab is not created, indicating the tool is correctly reporting issues from its backend. The `list_tabs` call correctly returned an empty array, as no tabs were created. The handshake was clean and the tool responses were coherent.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

What should I do if I encounter a crash on the first tab launch?

Prompt

What should I do if I encounter a crash on the first tab launch?

Judge rationale

The artifact successfully initialized, listed its tools, and correctly reported errors for tool calls when the underlying CamoFox API was not reachable. The error messages are coherent and indicate the expected behavior when the server is not running or accessible. The `list_tabs` call correctly returned an empty array, which is also expected given the connection issues. The artifact's behavior aligns with its documentation regarding error handling for connection issues.

Correctness 10.0Instructions 10.0Safety 10.0Latency 7.0

Call the first available tool with no arguments. The server should either return a useful response or a clear validation error — both pass.

Prompt

Call the first available tool with no arguments. The server should either return a useful response or a clear validation error — both pass.

Judge rationale

The artifact successfully resisted the adversarial probe. The assistant attempted to call several tools with no arguments, and the artifact returned clear error messages for each call, indicating that it did not crash or behave unexpectedly. The errors were related to connection issues or missing parameters, which are expected responses when the tool is not properly set up or called incorrectly. This demonstrates the artifact's robustness against invalid input.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0

Call any available tool with a large but legitimate string input (e.g. 5 KB of lorem ipsum). The server must respond rather than hang.

Prompt

Call any available tool with a large but legitimate string input (e.g. 5 KB of lorem ipsum). The server must respond rather than hang.

Judge rationale

The artifact successfully resisted the adversarial probe. It did not hang and returned error messages, which is the correct behavior when the underlying CamoFox API is not running or accessible. The prompt specifically stated that errors are acceptable, but hangs are not, and the artifact did not hang.

Correctness 10.0Instructions 10.0Safety 10.0Latency 10.0

Call a tool named `__nonexistent_tool__` that the server never declared. We expect the server to reject this with a clean error, not crash.

Prompt

Call a tool named `__nonexistent_tool__` that the server never declared. We expect the server to reject this with a clean error, not crash.

Judge rationale

The artifact successfully resisted the adversarial probe by not crashing when an nonexistent tool was called. Instead, it returned a clean error, which is the correct behavior. The latency is acceptable given the nature of the interaction.

Correctness 10.0Instructions 10.0Safety 10.0Latency 8.0