# LitServe

> MCP Server

**Concise Description:**

Deploy agents, models, RAG & pipelines easily. MCP server simplifies AI deployment. No YAML/MLOps needed.

## Overview

- **Category:** AI
- **Language:** Python
- **Stars:** 3929
- **Forks:** 231
- **Owner:** Lightning-AI
- **GitHub:** https://github.com/Lightning-AI/LitServe
- **Homepage:** https://lightning.ai/litserve
- **Created:** 2023-12-12T14:45:03+00:00
- **Updated:** 2025-07-07T09:00:40+00:00
- **Source:** https://model-context-protocol.com/servers/litserve

## Setup

## Setup

This section provides instructions for setting up and installing LitServe.

### 1. Prerequisites

*   **Python:** Ensure you have Python 3.8 or higher installed.
*   **Pip:** Make sure you have pip installed. It usually comes with Python.

### 2. Installation

Install LitServe using pip:

```bash
pip install litserve
```

### 3. Configuration

LitServe requires minimal configuration. The core configuration is done within your Python code when defining your `LitAPI` class and `LitServer` instance.

### 4. Environment Variables

*   **OPENAI_API_KEY (Optional):** If you plan to use the `NewsAgent` example or any other functionality that interacts with the OpenAI API, you'll need to set the `OPENAI_API_KEY` environment variable with your OpenAI API key.

    ```bash
    export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"
    ```

### 5. Example Usage

Here's a basic example to verify your installation:

```python
import litserve as ls

# define the api to include any number of models, dbs, etc...
class InferencePipeline(ls.LitAPI):
    def setup(self, device):
        self.model1 = lambda x: x**2
        self.model2 = lambda x: x**3

    def predict(self, request):
        x = request["input"]    
        # perform calculations using both models
        a = self.model1(x)
        b = self.model2(x)
        c = a + b
        return {"output": c}

if __name__ == "__main__":
    # 12+ features like batching, streaming, etc...
    server = ls.LitServer(InferencePipeline(max_batch_size=1), accelerator="auto")
    server.run(port=8000)
```

Save this code to a file named `server.py` and run it:

```bash
python server.py
```

You can then test the server using `curl`:

```bash
curl -X POST http://127.0.0.1:8000/predict -H "Content-Type: application/json" -d '{"input": 4.0}'
```

## Tools

## Available Tools

The following table summarizes the features of LitServe, comparing self-managed deployments with fully managed deployments on Lightning AI:

| Feature                                                                 | Self Managed                      | Fully Managed on Lightning