Text vLLM (OpenAI-compatible API)

Qwen3 8B

Dense 8B model with switchable thinking / non-thinking modes, strong multilingual chat and tool use.

Overview

Qwen3 8B is a dense language model from Alibaba's Qwen team. It can answer directly or reason step by step ("thinking mode") in the same model, and supports tool calling and over 100 languages. In BF16 it uses about 16 GB, leaving plenty of room on an RTX 5090 for a 32K context and concurrent requests.

Key features

✓
Thinking and non-thinking modes in one model
✓
Tool / function calling
✓
32K context on this template
✓
Apache 2.0 — free for commercial use

Use cases

→
Chat assistants and agents
→
Summarisation and extraction
→
Multilingual support bots

On this template

Runs on the vLLM (OpenAI-compatible API) template, exposing its OpenAI-compatible API at a private HTTPS address. You also get root SSH and JupyterLab.

`MODEL_ID`	`Qwen/Qwen3-8B`
`VLLM_ARGS`	`--max-model-len 32768 --gpu-memory-utilization 0.90`

Quick start

Click Deploy and pick an available GPU — only offers with enough VRAM are shown.
Wait for the status to turn running. The first start downloads the model, which can take a few minutes.
Use the connection details on your rental page:

curl

curl https://r12-abcd1234-app.gpu.clearlayer.ca/v1/chat/completions \
  -H "Authorization: Bearer <app-token>" \
  -H "Content-Type: application/json" \
  -d '{"model": "Qwen/Qwen3-8B", "messages": [{"role": "user", "content": "Hello!"}]}'

Python

from openai import OpenAI

client = OpenAI(base_url="https://r12-abcd1234-app.gpu.clearlayer.ca/v1", api_key="<app-token>")
resp = client.chat.completions.create(
    model="Qwen/Qwen3-8B",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)