{"type":"mcp_server","name":"opik","description":"Opik is an open-source LLM evaluation framework designed to help build better, faster, and cheaper LLM systems through tracing, evaluations, and dashboards for RAG chatbots, code assistants, and agentic pipelines.","category":"AI","language":"Python","stars":20100,"forks":413,"owner":"comet-ml","github_url":"https://github.com/comet-ml/opik","homepage":"https://www.comet.com/docs/opik/","setup":"## Setup\nOpik is available as a fully open source local installation or using Comet.com as a hosted solution.\nThe easiest way to get started with Opik is by creating a free Comet account at [comet.com](https://www.comet.com/signup?from=llm&utm_source=opik&utm_medium=github&utm_content=install&utm_campaign=opik).\n\nIf you'd like to self-host Opik, you can do so by cloning the repository and starting the platform using Docker Compose:\n\nOn Linux or Mac do:\n```bash\n# Clone the Opik repository\ngit clone https://github.com/comet-ml/opik.git\n\n# Navigate to the repository\ncd opik\n\n# Start the Opik platform\n./opik.sh\n```\n\nOn Windows do:\n```powershell\n# Clone the Opik repository\ngit clone https://github.com/comet-ml/opik.git\n\n# Navigate to the repository\ncd opik\n\n# Start the Opik platform\npowershell -ExecutionPolicy ByPass -c \".\\opik.ps1\"\n```\n\nUse the `--help` or `--info` options to troubleshoot issues.\n\nOnce all is up and running, you can now visit [localhost:5173](http://localhost:5173) on your browser!\n\nFor more information about the different deployment options, please see our deployment guides:\n\n| Installation methods | Docs link |\n| ------------------- | --------- |\n| Local instance | [![Local Deployment](https://img.shields.io/badge/Local%20Deployments-%232496ED?style=flat&logo=docker&logoColor=white)](https://www.comet.com/docs/opik/self-host/local_deployment?from=llm&utm_source=opik&utm_medium=github&utm_content=self_host_link&utm_campaign=opik)\n| Kubernetes | [![Kubernetes](https://img.shields.io/badge/Kubernetes-%23326ce5.svg?&logo=kubernetes&logoColor=white)](https://www.comet.com/docs/opik/self-host/kubernetes/#kubernetes-installation?from=llm&utm_source=opik&utm_medium=github&utm_content=kubernetes_link&utm_campaign=opik)\n\n\nTo get started, you will need to first install the Python SDK:\n\n```bash\npip install opik\n```\n\nOnce the SDK is installed, you can configure it by running the `opik configure` command:\n\n```bash\nopik configure\n```\n\nThis will allow you to configure Opik locally by setting the correct local server address or if you're using the Cloud platform by setting the API Key","tools":"## Available Tools\n\n1.  Tracing: Track all LLM calls and traces during development and production.\n2.  Annotations: Annotate your LLM calls by logging feedback scores using the Python SDK or the UI.\n3.  Playground: Try out different prompts and models in the prompt playground.\n4.  Datasets and Experiments: Store test cases and run experiments.\n5.  LLM as a judge metrics: Use Opik's LLM as a judge metric for complex issues like hallucination detection, moderation and RAG evaluation.\n6.  CI/CD integration: Run evaluations as part of your CI/CD pipeline using our PyTest integration.\n7.  Log all your production traces: Opik has been designed to support high volumes of traces, making it easy to monitor your production applications.\n8.  Monitoring dashboards: Review your feedback scores, trace count and tokens over time in the Opik Dashboard.\n9.  Online evaluation metrics: Easily score all your production traces using LLM as a Judge metrics and identify any issues with your production LLM application thanks to Opik's online evaluation metrics.","faq":null,"created_at":"2023-05-10T12:57:13+00:00","updated_at":"2025-03-28T22:31:26+00:00","source_url":"https://model-context-protocol.com/servers/llm-evaluation-tracing-rag-monitoring","related_articles":[{"title":"Mastering opik in 2025","url":"https://model-context-protocol.com/blog/opik-guide"}]}