An open platform for operating large language models (LLMs) in production. Fine-tune, serve, deploy, and monitor any LLMs with ease. With OpenLLM, you can run inference with any open-source large-language models, deploy to the cloud or on-premises, and build powerful AI apps. Built-in supports a wide range of open-source LLMs and model runtime, including Llama 2， StableLM, Falcon, Dolly, Flan-T5, ChatGLM, StarCoder, and more. Serve LLMs over RESTful API or gRPC with one command, query via WebUI, CLI, our Python/Javascript client, or any HTTP client.

Features

  • Fine-tune, serve, deploy, and monitor any LLMs with ease
  • State-of-the-art LLMs
  • Flexible APIs
  • Freedom To Build
  • Streamline Deployment
  • Bring your own LLM

Project Samples

OpenLLM Screenshot 1

Project Activity

See All Activity >