mannix/llama-3.3:iq2

mannix/ llama-3.3:iq2_xxs

350 Downloads Updated 1 year ago

New state of the art 70B model. Llama 3.3 70B offers similar performance compared to Llama 3.1 405B model. I-Quants models.

tools

ollama run mannix/llama-3.3:iq2_xxs

curl http://localhost:11434/api/chat \
  -d '{
    "model": "mannix/llama-3.3:iq2_xxs",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='mannix/llama-3.3:iq2_xxs',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'mannix/llama-3.3:iq2_xxs',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 1 year ago

1 year ago

bd719ef45d5c · 19GB ·

model

archllama

parameters70.6B

quantizationIQ2_XXS

19GB

license

LLAMA 3.3 COMMUNITY LICENSE AGREEMENT Llama 3.3 Version Release Date: December 6, 2024 “Agreement�

7.6kB

license

Llama 3.3 Acceptable Use Policy Meta is committed to promoting safe and fair use of its tools and fe

5.6kB

params

{ "stop": [ "<|start_header_id|>", "<|end_header_id|>", "<|eot_id|>"

96B

template

{{- if or .System .Tools }}<|start_header_id|>system<|end_header_id|> {{- if .System }} {{ .System }

1.5kB

Readme

Quantization from fp32
Using i-matrix calibration dataset calibration_datav3.txt
I-Quants models, only if passing tests iq2_xs, iq2_xxs, iq3_xxs, iq4_nl, iq3_s, iq2_s, iq4_xs, iq3_xs
Default quantization iq4_nl

New state of the art 70B model. Llama 3.3 70B offers similar performance compared to Llama 3.1 405B model.

Model reference