Start Here

Welcome to the OpenArc documentation!

Installation

• Linux
• Windows
• Docker

Commands

OpenArc includes a command line tool for controlling the server.

• openarc list — List models added to the config.
• openarc serve — Start the OpenArc server.
• openarc load — Load a model from the config.
• openarc status — Check loaded models.
• openarc bench — Benchmarking tool for LLMs.
• openarc tool — OpenVINO utilities.

Configuration

OpenArc uses a YAML based config system. See Examples for a config block for each model type.

• LLM
• VLM
• Kokoro
• Qwen3-ASR
• Qwen3-TTS

Performance

Scheduler, runtime properties, and multi-device recipes.

• scheduler_config
• runtime_config
• Multi-Device Inference
• Speculative Decoding
• Model Caching

API Usage

Request-time parameters passed via extra_body on the OpenAI-compatible API.

• Qwen3-TTS
• Qwen3-ASR

Concepts

• Tool and Reasoning Parsing

Models

Models to get you started and where to find more!

OpenArc is deeply integrated with the Huggingface Ecosytem and has been written from the ground up to handle a ton of deployment complexity but still demands some calories to choose what models to use.

We are working on improving this process with experimental GGUF support coming, as well as a new frontend application similar to LM-Studio!

Below are some models to get started which are known to work. My huggingface has many

• Model Sources
• LLMs
• VLMs
• Text to Speech
  • Whisper
  • Qwen3-ASR
• Speech to Text
  • Kokoro
  • Qwen3-TTS
• Embedding
• Rerank