Llama 3.1 8B Instruct

Meta's widely used 8B instruct model with 128K native context and tool calling.

Overview

Llama 3.1 8B Instruct is one of the most fine-tuned and benchmarked open models. The weights are gated: accept Meta's licence on Hugging Face and paste your Hugging Face token when you deploy.

Key features

• ✓
  128K native context (32K on this template)
• ✓
  Huge fine-tune ecosystem
• ✓
  Tool calling

Use cases

• →
  Baseline chat model
• →
  Starting point for fine-tuning
• →
  Classification and extraction

On this template

Runs on the vLLM (OpenAI-compatible API) template, exposing its OpenAI-compatible API at a private HTTPS address. You also get root SSH and JupyterLab.

Quick start

1. Click Deploy and pick an available GPU — only offers with enough VRAM are shown.
2. Wait for the status to turn running. The first start downloads the model, which can take a few minutes.
3. Use the connection details on your rental page:

curl https://r12-abcd1234-app.gpu.clearlayer.ca/v1/chat/completions \
  -H "Authorization: Bearer <app-token>" \
  -H "Content-Type: application/json" \
  -d '{"model": "meta-llama/Llama-3.1-8B-Instruct", "messages": [{"role": "user", "content": "Hello!"}]}'

from openai import OpenAI

client = OpenAI(base_url="https://r12-abcd1234-app.gpu.clearlayer.ca/v1", api_key="<app-token>")
resp = client.chat.completions.create(
    model="meta-llama/Llama-3.1-8B-Instruct",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)