{"type":"mcp_server","name":"web-eval-agent","description":"Autonomous web app evaluator using MCP. Tests & scores web applications.","category":"Developer Tools","language":"Python","stars":1242,"forks":87,"owner":"Operative-Sh","github_url":"https://github.com/Operative-Sh/web-eval-agent","homepage":"https://www.operative.sh/mcp","setup":"## Setup\n\nThis guide provides instructions for setting up the `web-eval-agent` MCP server. Choose the installation method that best suits your needs: automatic installation for a quick setup or manual installation for more control.\n\n### Prerequisites\n\n*   An API key from [operative.sh/mcp](https://www.operative.sh/mcp) (free).\n*   **macOS/Linux:**\n    *   `brew` (Homebrew) is recommended for installing dependencies. If you don't have it:\n        ```bash\n        /bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\"\n        ```\n*   **Windows:** Ensure you have a suitable terminal environment (e.g., PowerShell or Git Bash).\n\n### Automatic Installation (macOS/Linux)\n\nThis method uses a script to handle most of the installation process.\n\n1.  **Run the installer:**\n\n    ```bash\n    curl -LSf https://operative.sh/install.sh -o install.sh && bash install.sh && rm install.sh\n    ```\n\n    This script:\n\n    *   Installs [Playwright](https://github.com/microsoft/playwright).\n    *   Installs [uv](https://astral.sh/).\n    *   Automatically configures your code editor (Cursor/Cline/Windsurf) with the necessary JSON configuration.\n\n2.  **Restart your code editor:**  This is necessary to apply the changes made by the installer.\n\n3.  **Test the agent:** Send a prompt in chat mode to call the web eval agent tool. For example:\n\n    ```bash\n    Test my app on http://localhost:3000. Use web-eval-agent.\n    ```\n\n### Manual Installation\n\nThis method allows for more control over the installation process.\n\n1.  **Install uv:**\n\n    ```bash\n    curl -LsSf https://astral.sh/uv/install.sh | sh\n    ```\n\n2.  **Source environment variables:** After installing UV, source your shell's configuration file to update your environment.\n\n    *   **macOS (zsh):**\n\n        ```bash\n        source ~/.zshrc\n        ```\n\n    *   **Linux (bash):**\n\n        ```bash\n        source ~/.bashrc\n        ```\n\n3.  **Install Playwright:**\n\n    ```bash\n    npm install -g chromium playwright && uvx --with playwright playwright install --with-deps\n    ```\n\n4.  **Configure your code editor:** Add the following JSON configuration to your code editor's settings, replacing `<YOUR_KEY>` with your actual API key.  The exact location of this setting depends on your editor (Cursor/Cline/Windsurf).\n\n    ```json\n    \"web-eval-agent\": {\n      \"command\": \"uvx\",\n      \"args\": [\n        \"--refresh-package\",\n        \"webEvalAgent\",\n        \"--from\",\n        \"git+https://github.com/Operative-Sh/web-eval-agent.git\",\n        \"webEvalAgent\"\n      ],\n      \"env\": {\n        \"OPERATIVE_API_KEY\": \"<YOUR_KEY>\"\n      }\n    }\n    ```\n\n5.  **Restart your code editor:** This is essential for the changes to take effect.\n\n### Manual Installation (Windows)\n\n1.  **Open your code editor terminal.**\n2.  **Run the automatic installation script:**\n\n    ```bash\n    curl -LSf https://operative.sh/install.sh -o install.sh && bash install.sh && rm install.sh\n    ```\n\n3.  **Install uv:**\n\n    ```bash\n    curl -LsSf https://astral.sh/uv/install.sh | sh\n    ```\n\n4.  **Install Playwright:**\n\n    ```bash\n    uvx --from git+https://github.com/Operative-Sh/web-eval-agent.git playwright install\n    ```\n\n5.  **Restart your code editor.**\n\n### Environment Variables\n\n*   `OPERATIVE_API_KEY`:  This environment variable is crucial for authenticating with the operative.sh service.  It's set within the JSON configuration for your code editor.\n\n### Updating\n\nTo ensure you have the latest version of the `web-eval-agent`:\n\n1.  **Clear the uv cache:**\n\n    ```bash\n    uv cache clean\n    ```\n\n2.  **Refresh the MCP server** (usually by restarting your code editor).","tools":"## Available Tools\n\nThe operative.sh MCP Server provides the following tools to supercharge your debugging:\n\n**Features:**\n\n*   **Webapp Navigation:** Navigate your web app using BrowserUse (optimized for speed with the operative backend).\n*   **Network Traffic Capture:** Intelligently filters and captures network requests, providing them in the context window.\n*   **Console Error Collection:** Captures console logs and errors for comprehensive debugging.\n*   **Autonomous Debugging:** Enables the Cursor agent to use the web QA agent (MCP server) to autonomously test code end-to-end.\n\n**Tools:**\n\n*   **`web_eval_agent`**: Automated UX evaluator that drives the browser, captures screenshots, console & network logs, and returns a rich UX report.\n\n    *   **Purpose:**  This tool allows you to automatically test and evaluate the user experience of your web application. It simulates user interactions, captures relevant data, and provides a detailed report.\n    *   **Arguments:**\n        *   `url` (required): The address of the running app (e.g., `http://localhost:3000`).\n        *   `task` (required): A natural-language description of what to test (e.g., \"Run through the signup flow and note any UX issues\").\n        *   `headless_browser` (optional, default `false`): Set to `true` to run the browser in headless mode (without a visible window).\n    *   **Example Usage:**\n        ```bash\n        Evaluate my app at http://localhost:3000 – run web_eval_agent with the task \"Try the full signup flow and report UX issues\".\n        ```\n        ```bash\n        Test my app on http://localhost:3000. Use web-eval-agent.\n        ```\n\n*   **`setup_browser_state`**: Opens an interactive (non-headless) browser so you can sign in once; the saved cookies/local-storage are reused by subsequent `web_eval_agent` runs.\n\n    *   **Purpose:** This tool allows you to set up a persistent browser state (e.g., login credentials, local storage) that can be reused by subsequent `web_eval_agent` runs.  This avoids the need to repeatedly log in during testing.\n    *   **Arguments:**\n        *   `url` (optional): The page to open first (handy to land directly on a login screen).","faq":null,"created_at":"2025-04-01T23:40:56+00:00","updated_at":"2025-07-07T13:29:48+00:00","source_url":"https://model-context-protocol.com/servers/web-eval-agent","related_articles":[]}