The default model repository of openllm

This repo (on main branch) is already included by openllm by default.

If you want more up-to-date untested models, please add our nightly branch.

openllm repo add nightly https://github.com/bentoml/openllm-models@nightly

Supported Models

Table of Contents

• Llama-3.1
• Llama-3
• Phi-3
• Mistral
• Gemma-2
• Qwen-2
• Qwen-2.5
• Gemma
• Llama-2
• Mixtral
• Mistral-Large
• Codestral

---

Llama-3.1

Model	Version	Huggingface Link
llama3.1	405b-instruct-awq-4bit-54b7	HF Link
llama3.1	70b-instruct-awq-4bit-7c3e	HF Link
llama3.1	70b-instruct-fp16-c283	HF Link
llama3.1	8b-instruct-awq-4bit-c135	HF Link
llama3.1	8b-instruct-fp16-44b5	HF Link

---

Llama-3

Model	Version	Huggingface Link
llama3	70b-instruct-awq-4bit-e96c	HF Link
llama3	70b-instruct-fp16-45fe	HF Link
llama3	8b-instruct-awq-4bit-b159	HF Link
llama3	8b-instruct-fp16-72f8	HF Link

---

Phi-3

Model	Version	Huggingface Link
phi3	3.8b-instruct-fp16-baed	HF Link
phi3	3.8b-instruct-ggml-q4-50c9	HF Link

---

Mistral

Model	Version	Huggingface Link
mistral	24b-instruct-nemo-e7a4	HF Link
mistral	7b-instruct-awq-4bit-4175	HF Link
mistral	7b-instruct-fp16-9926	HF Link

---

Gemma-2

Model	Version	Huggingface Link
gemma2	27b-instruct-fp16-56d1	HF Link
gemma2	9b-instruct-fp16-bf96	HF Link

---

Qwen-2

Model	Version	Huggingface Link
qwen2	0.5b-instruct-fp16-114e	HF Link
qwen2	1.5b-instruct-fp16-743d	HF Link
qwen2	57b-a14b-instruct-fp16-13ca	HF Link
qwen2	72b-instruct-awq-4bit-5384	HF Link
qwen2	72b-instruct-fp16-4755	HF Link
qwen2	7b-instruct-awq-4bit-89de	HF Link
qwen2	7b-instruct-fp16-c17f	HF Link

---

Qwen-2.5

Model	Version	Huggingface Link
qwen2.5	0.5b-instruct-fp16-6a1b	HF Link
qwen2.5	1.5b-instruct-fp16-86f6	HF Link
qwen2.5	3b-instruct-fp16-faad	HF Link

---

Gemma

Model	Version	Huggingface Link
gemma	2b-instruct-fp16-e12f	HF Link
gemma	7b-instruct-awq-4bit-1134	HF Link
gemma	7b-instruct-fp16-e12a	HF Link

---

Llama-2

Model	Version	Huggingface Link
llama2	13b-chat-fp16-15d8	HF Link
llama2	70b-chat-fp16-8365	HF Link
llama2	7b-chat-awq-4bit-8f2f	HF Link
llama2	7b-chat-fp16-5e52	HF Link

---

Mixtral

Model	Version	Huggingface Link
mixtral	8x7b-instruct-v0.1-awq-4bit-2117	HF Link
mixtral	8x7b-instruct-v0.1-fp16-55c3	HF Link

---

Mistral-Large

Model	Version	Huggingface Link
mistral-large	123b-instruct-awq-4bit-e339	HF Link
mistral-large	123b-instruct-fp16-eb4a	HF Link

---

Codestral

Model	Version	Huggingface Link
codestral	22b-v0.1-fp16-0d5b	HF Link

---