Models
Sources
There are a few sources of preconverted models which can be used with OpenArc
If you need help converting a particular model join Discord and we can help you!
| Models |
|---|
| gemma-3-4b-it-int8_asym-ov |
| Gemma-3-12b-it-qat-int4_asym-ov |
| Qwen2.5-VL-7B-Instruct-int4_sym-ov |
| Nanonets-OCR2-3B-LM-INT4_ASYM-VE-FP16-ov |
| Qwen3-VL-4B-Instruct-int4_asym-ov |
Whisper
| Models |
|---|
| distil-whisper-large-v3-int8-ov |
| distil-whisper-large-v3-fp16-ov |
| whisper-large-v3-int8-ov |
| openai-whisper-large-v3-fp16-ov |
Qwen3-ASR
| Models |
|---|
| Qwen3-ASR-0.6B-INT8_ASYM-OpenVINO |
Kokoro
| Models |
|---|
| Kokoro-82M-FP16-OpenVINO |
Qwen3-TTS
| Models |
|---|
| Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO |
| Qwen3-TTS-12Hz-VoiceDesign-1.7B-INT8-OpenVINO |
| Qwen3-TTS-12Hz-Base-1.7B-INT8-OpenVINO |
| Models |
|---|
| Qwen3-Embedding-0.6B-int8_asym-ov |
| Models |
|---|
| Qwen3-Reranker-0.6B-fp16-ov |
How do I control thinking?
Qwen3.5 utilizes chat instructions for thinking control. You can enable thinking by using the parameter chat_template_kwargs with a value of {"enable_thinking": true} and disable it by setting the value to {"enable_thinking": false}.
Previous reasoning is also retained within a conversation. You can enable or disable it similarily by adding preserve_previous_think to chat_template_kwargs.
For example, to enable thinking and disable previous reasoning, you would pass chat_template_kwargs with a value of {"enable_thinking": true, "preserve_previous_think": false}.