Ollama

Run quantised open models with one command. Pulls the models you choose at start-up and serves the Ollama API.

What's inside

• ✓
  Ollama server on port 11434
• ✓
  Models in OLLAMA_PULL are pulled at start-up (space-separated)
• ✓
  Protected by the gateway: user edge / your app token
• ✓
  Logs: /var/log/onstart.log

How you connect

Access	Where	Auth
SSH (root)	ssh -p <port> root@clearlayer.ca	Your SSH key
JupyterLab	https://r<id>-<random>.gpu.clearlayer.ca	Token in the link
Ollama API (port 11434)	https://r<id>-<random>-app.gpu.clearlayer.ca	Login edge / app token

Environment variables

Variable	Default
OLLAMA_PULL	llama3.1:8b
APP_TOKEN	random per rental
HF_TOKEN	optional, for gated Hugging Face models

Models from the model library set these for you.

Quick start

curl -u edge:<app-token> https://r12-abcd1234-app.gpu.clearlayer.ca/api/generate \
  -d '{"model": "llama3.1:8b", "prompt": "Hello!", "stream": false}'

import requests

r = requests.post("https://r12-abcd1234-app.gpu.clearlayer.ca/api/chat", auth=("edge", "<app-token>"), json={
    "model": "llama3.1:8b", "stream": False,
    "messages": [{"role": "user", "content": "Hello!"}],
})
print(r.json()["message"]["content"])