The model in your browser

This page runs the real engine. It is a WebAssembly build of the same Rust crate that the command line program uses. Greedy decoding gives the same tokens as the program on your computer, and a test in the repository proves it.

The page downloads the checkpoint that you choose from Hugging Face. The smallest is 6 MB and the largest is 556 MB. Your browser keeps the download in its cache, so a second visit to the same checkpoint is fast. Start with Nano · 8 bits, which is 11 MB.

The engine has one thread here, and it runs on the thread of the page. Because of this the page cannot draw while the model writes an answer. Keep maximum new tokens small for a quick answer.

Choose a checkpoint

Checkpoint

Or any repository on Hugging Face

Select a checkpoint and use the button.

Ask the model

Mode chat — the chat format is applied generate — the text continues Example Name one ocean. Why is the sky blue? Write three rules for a safe kitchen. The capital of France is

Your text

Maximum new tokens Temperature Top-k Seed

What the page does

1. It downloads four files from the repository that you choose: config.json, tokenizer.json, tokenizer_config.json and model.safetensors. The third one is optional.
2. It gives the four parts to Model.fromParts, which builds the engine.
3. It sends your text through the same code as the command line program.

Nothing goes to a server of this site. The only network requests are the downloads from Hugging Face.

The checkpoints in the list

The list holds the three original checkpoints and the quantized copies of them. A quantized copy is smaller both to download and to hold in memory, because the engine keeps the codes and rebuilds each value inside the multiplication. The table under Small checkpoints gives the measured quality of each one.

The two ternary everywhere entries do not work. They are in the list because the measurement is the point: a model of this size cannot carry ternary weights in every matrix, and the page lets you see what that looks like.

Your own repository works as well. Write owner/name in the field under the list. The repository needs config.json, tokenizer.json and model.safetensors, and the architecture must be one that this engine knows.

A browser gives the module 4 GB of memory, and the engine needs approximately three times the size of the file while it reads a checkpoint. The 556 MB Pro checkpoint of 32-bit floats is therefore near the limit; its 8-bit copy of 150 MB is not.

Temperature 0 is greedy

Temperature 0 always selects the most probable token. The answer is then the same on each attempt, and it is the same as the answer on your computer. A temperature above 0 uses the random numbers, and the seed then controls the result.

Do the same work on your computer

uv pip install bananamendy
bananamendy pull nano
bananamendy chat --prompt "Name one ocean." --temperature 0

Your computer has more threads, so it is faster. It can also use the larger models: bananamendy pull mini and bananamendy pull pro.