# web-eval-agent

> MCP Server

Autonomous web app evaluator using MCP. Tests & scores web applications.

## Overview

- **Category:** Developer Tools
- **Language:** Python
- **Stars:** 1240
- **Forks:** 87
- **Owner:** Operative-Sh
- **GitHub:** https://github.com/Operative-Sh/web-eval-agent
- **Homepage:** https://www.operative.sh/mcp
- **Created:** 2025-04-01T23:40:56+00:00
- **Updated:** 2025-07-07T13:29:48+00:00
- **Source:** https://model-context-protocol.com/servers/web-eval-agent

## Setup

## Setup

This guide provides instructions for setting up the `web-eval-agent` MCP server. Choose the installation method that best suits your needs: automatic installation for a quick setup or manual installation for more control.

### Prerequisites

*   An API key from [operative.sh/mcp](https://www.operative.sh/mcp) (free).
*   **macOS/Linux:**
    *   `brew` (Homebrew) is recommended for installing dependencies. If you don't have it:
        ```bash
        /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
        ```
*   **Windows:** Ensure you have a suitable terminal environment (e.g., PowerShell or Git Bash).

### Automatic Installation (macOS/Linux)

This method uses a script to handle most of the installation process.

1.  **Run the installer:**

    ```bash
    curl -LSf https://operative.sh/install.sh -o install.sh && bash install.sh && rm install.sh
    ```

    This script:

    *   Installs [Playwright](https://github.com/microsoft/playwright).
    *   Installs [uv](https://astral.sh/).
    *   Automatically configures your code editor (Cursor/Cline/Windsurf) with the necessary JSON configuration.

2.  **Restart your code editor:**  This is necessary to apply the changes made by the installer.

3.  **Test the agent:** Send a prompt in chat mode to call the web eval agent tool. For example:

    ```bash
    Test my app on http://localhost:3000. Use web-eval-agent.
    ```

### Manual Installation

This method allows for more control over the installation process.

1.  **Install uv:**

    ```bash
    curl -LsSf https://astral.sh/uv/install.sh | sh
    ```

2.  **Source environment variables:** After installing UV, source your shell's configuration file to update your environment.

    *   **macOS (zsh):**

        ```bash
        source ~/.zshrc
        ```

    *   **Linux (bash):**

        ```bash
        source ~/.bashrc
        ```

3.  **Install Playwright:**

    ```bash
    npm install -g chromium playwright && uvx --with playwright playwright install --with-deps
    ```

4.  **Configure your code editor:** Add the following JSON configuration to your code editor's settings, replacing `<YOUR_KEY>` with your actual API key.  The exact location of this setting depends on your editor (Cursor/Cline/Windsurf).

    ```json
    "web-eval-agent": {
      "command": "uvx",
      "args": [
        "--refresh-package",
        "webEvalAgent",
        "--from",
        "git+https://github.com/Operative-Sh/web-eval-agent.git",
        "webEvalAgent"
      ],
      "env": {
        "OPERATIVE_API_KEY": "<YOUR_KEY>"
      }
    }
    ```

5.  **Restart your code editor:** This is essential for the changes to take effect.

### Manual Installation (Windows)

1.  **Open your code editor terminal.**
2.  **Run the automatic installation script:**

    ```bash
    curl -LSf https://operative.sh/install.sh -o install.sh && bash install.sh && rm install.sh
    ```

3.  **Install uv:**

    ```bash
    curl -LsSf https://astral.sh/uv/install.sh | sh
    ```

4.  **Install Playwright:**

    ```bash
    uvx --from git+https://github.com/Operative-Sh/web-eval-agent.git playwright install
    ```

5.  **Restart your code editor.**

### Environment Variables

*   `OPERATIVE_API_KEY`:  This environment variable is crucial for authenticating with the operative.sh service.  It's set within the JSON configuration for your code editor.

### Updating

To ensure you have the latest version of the `web-eval-agent`:

1.  **Clear the uv cache:**

    ```bash
    uv cache clean
    ```

2.  **Refresh the MCP server** (usually by restarting your code editor).

## Tools

## Available Tools

The operative.sh MCP Server provides the following tools to supercharge your debugging:

**Features:**

*   **Webapp Navigation:** Navigate your web app using BrowserUse (optimized for speed with the operative backend).
*   **Network Traffic Capture:** Intelligently filters and captures network requests, providing them in the context window.
*   **Console Error Collection:** Captures console logs and errors for comprehensive debugging.
*   **Autonomous Debugging:** Enables the Cursor agent to use the web QA agent (MCP server) to autonomously test code end-to-end.

**Tools:**

*   **`web_eval_agent`**: Automated UX evaluator that drives the browser, captures screenshots, console & network logs, and returns a rich UX report.

    *   **Purpose:**  This tool allows you to automatically test and evaluate the user experience of your web application. It simulates user interactions, captures relevant data, and provides a detailed report.
    *   **Arguments:**
        *   `url` (required): The address of the running app (e.g., `http://localhost:3000`).
        *   `task` (required): A natural-language description of what to test (e.g., "Run through the signup flow and note any UX issues").
        *   `headless_browser` (optional, default `false`): Set to `true` to run the browser in headless mode (without a visible window).
    *   **Example Usage:**
        ```bash
        Evaluate my app at http://localhost:3000 – run web_eval_agent with the task "Try the full signup flow and report UX issues".
        ```
        ```bash
        Test my app on http://localhost:3000. Use web-eval-agent.
        ```

*   **`setup_browser_state`**: Opens an interactive (non-headless) browser so you can sign in once; the saved cookies/local-storage are reused by subsequent `web_eval_agent` runs.

    *   **Purpose:** This tool allows you to set up a persistent browser state (e.g., login credentials, local storage) that can be reused by subsequent `web_eval_agent` runs.  This avoids the need to repeatedly log in during testing.
    *   **Arguments:**
        *   `url` (optional): The page to open first (handy to land directly on a login screen).