# xiaozhi-esp32-server

> MCP Server

ESP32 server backend. Enables quick setup of ESP32 device control.

## Overview

- **Category:** AI
- **Language:** Python
- **Stars:** 10537
- **Forks:** 1875
- **Owner:** xinnan-tech
- **GitHub:** https://github.com/xinnan-tech/xiaozhi-esp32-server
- **Created:** 2025-02-02T14:25:16+00:00
- **Updated:** 2025-07-07T16:18:30+00:00
- **Source:** https://model-context-protocol.com/servers/xiaozhi-esp32-server

## Setup

## Setup

This project offers two deployment options, choose the one that best suits your needs:

#### 🚀 Deployment Options

| Deployment Type | Features | Use Cases | Documentation | Hardware Requirements | Video Tutorial |
|---|---|---|---|---|---|
| **Simplified Installation** | Smart conversation, IOT, MCP, Visual Perception | Low-resource environments, data stored in configuration files, no database required | [① Docker](docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) / [② Source Code](docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver) | 2 cores, 4GB RAM if using `FunASR`; 2 cores, 2GB RAM for API-only | - |
| **Full Module Installation** | Smart conversation, IOT, MCP Access Point, Visual Perception, OTA, Smart Control Panel | Complete feature experience, data stored in a database | [① Docker](docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [② Source Code](docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97) / [③ Source Code Auto-Update](docs/dev-ops-integration.md) | 4 cores, 8GB RAM if using `FunASR`; 2 cores, 4GB RAM for API-only | [Local Source Code Startup Video Tutorial](https://www.bilibili.com/video/BV1wBJhz4Ewe) |

**Detailed instructions for each deployment method can be found in the linked documentation.**  The following is a general outline of the steps involved.

#### Prerequisites

Before you begin, ensure you have the following installed:

*   **For Docker Deployment:**
    *   [Docker](https://www.docker.com/)
    *   [Docker Compose](https://docs.docker.com/compose/install/) (Recommended)
*   **For Source Code Deployment:**
    *   [Python 3.7+](https://www.python.org/downloads/)
    *   [Java JDK](https://www.oracle.com/java/technologies/javase-downloads.html) (For Java components)
    *   [Node.js](https://nodejs.org/en/download/) and [npm](https://www.npmjs.com/get-npm) (For Vue frontend)
    *   [Redis](https://redis.io/docs/getting-started/installation/) (For Full Module Installation)
    *   [MySQL](https://www.mysql.com/downloads/) or other compatible database (For Full Module Installation)

#### Installation Steps

**1. Clone the Repository:**

```bash
git clone https://github.com/xinnan-tech/xiaozhi-esp32-server.git
cd xiaozhi-esp32-server
```

**2. Choose a Deployment Method:**

Follow the instructions in the appropriate documentation file based on your chosen deployment method:

*   **Simplified Installation:**
    *   [Docker](docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)
    *   [Source Code](docs/Deployment.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%8F%AA%E8%BF%90%E8%A1%8Cserver)
*   **Full Module Installation:**
    *   [Docker](docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%B8%80docker%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97)
    *   [Source Code](docs/Deployment_all.md#%E6%96%B9%E5%BC%8F%E4%BA%8C%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E8%BF%90%E8%A1%8C%E5%85%A8%E6%A8%A1%E5%9D%97)

**3. Configuration:**

*   **API Keys:**  You will need to configure API keys for various services (ASR, LLM, TTS, VLLM) in the configuration files.  Refer to the documentation for specific instructions.
*   **Database Configuration:** For Full Module Installation, configure the database connection settings in the appropriate configuration files.
*   **Other Settings:**  Adjust other settings as needed based on your requirements.

**4. Environment Variables:**

The project may require certain environment variables to be set.  These are typically used for API keys, database credentials, and other sensitive information.  Refer to the documentation for a complete list of required environment variables. Example:

```bash
export DATABASE_URL="mysql://user:password@host:port/database"
export OPENAI_API_KEY="your_openai_api_key"
```

**5. Run the Application:**

Follow the instructions in the documentation to start the application.  This may involve running Docker Compose, executing Python scripts, or building and running Java applications.

#### 🚩 Configuration Recommendations

The project supports two configuration schemes:

1.  `Entry-Level Free` Configuration: Suitable for personal/home use. All components use free options, requiring no extra payment.
2.  `Streaming` Configuration: Suitable for demonstrations, training, or scenarios with more than 2 concurrent users. Uses streaming technology for faster response times and a better experience.

| Module Name | Entry-Level Free Setting | Streaming Configuration |
|---|---|---|
| ASR (Speech Recognition) | FunASR (Local) | 👍 FunASRServer or 👍 DoubaoStreamASR |
| LLM (Large Language Model) | ChatGLMLLM (Zhipu glm-4-flash) | 👍 DoubaoLLM (Volcano doubao-1-5-pro-32k-250115) |
| VLLM (Visual Large Model) | ChatGLMVLLM (Zhipu glm-4v-flash) | 👍 QwenVLVLLM (Qianwen qwen2.5-vl-3b-instructh) |
| TTS (Text-to-Speech) | ✅ LinkeraiTTS (Lingxi Streaming) | 👍 HuoshanDoubleStreamTTS (Volcano Dual-Stream Speech Synthesis) |
| Intent (Intent Recognition) | function\_call (Function Call) | function\_call (Function Call) |
| Memory (Memory Function) | mem\_local\_short (Local Short-Term Memory) | mem\_local\_short (Local Short-Term Memory) |

## Tools

## Available Tools

**Deployment:**

*   **Docker Deployment:** Containerized deployment for simplified setup and management.
    *   **Simplest Installation:** Deploys only the server component, suitable for low-resource environments. Stores data in configuration files, eliminating the need for a database.
    *   **Full Module Installation:** Deploys all modules, including the control panel, for a complete feature set. Requires a database for data storage.
*   **Source Code Deployment:** Manual deployment from source code, offering greater customization.
    *   **Simplest Installation:** Deploys only the server component, suitable for low-resource environments. Stores data in configuration files, eliminating the need for a database.
    *   **Full Module Installation:** Deploys all modules, including the control panel, for a complete feature set. Requires a database for data storage.

**Testing:**

*   **Audio Interaction Test Tool:** Located at `main/xiaozhi-server/test/test_page.html`. Open with Google Chrome to test audio playback and reception, verifying Python-side audio processing.
*   **Model Response Test Tool 1:** Located at `main/xiaozhi-server/performance_tester.py`. Run `python performance_tester.py` to test the response speed of ASR (Speech Recognition), LLM (Large Language Model), and TTS (Text-to-Speech) modules.
*   **Model Response Test Tool 2:** Located at `main/xiaozhi-server/performance_tester_vllm.py`. Run `python performance_tester_vllm.py` to test the response speed of the VLLM (Visual Language Model).

**Functionality:**

*   **Core Service Architecture:** WebSocket and HTTP server-based architecture with console management and authentication.
*   **Voice Interaction System:** Supports streaming ASR, streaming TTS, VAD (Voice Activity Detection), multi-language recognition, and voice processing.
*   **Intelligent Dialogue System:** Supports various LLMs for intelligent conversations.
*   **Visual Perception System:** Supports various VLLMs for multimodal interaction.
*   **Intent Recognition System:** Supports LLM-based intent recognition and Function Call, providing a pluggable intent processing mechanism.
*   **Memory System:** Supports local short-term memory and mem0ai interface memory, with memory summarization capabilities.
*   **Tool Calling:** Supports client-side IOT protocol, client-side MCP protocol, server-side MCP protocol, MCP access point protocol, and custom tool functions.
*   **Management Backend:** Web-based management interface for user, system, and device management.
*   **Plugin System:** Supports functional plugin extensions, custom plugin development, and plugin hot-reloading.