ThinEmbed: compressed query encoders

Size, quality and speed in the browser

harrier-0.6b on SciFact, test split, 300 queries; nDCG@10 against the model's own fp32 index. One idle laptop, Intel integrated GPU, headless Chrome; latency per query; peak memory of all Chrome processes. The ThinEmbed rows are the released files.

The Q3_K and Q2_K files are our GPTQ exports onto llama.cpp's formats; llama.cpp's own quantiser reaches the same quality at that size. One device, one model, one corpus: discrete GPUs and phones are not measured. Source: vq_results.md §2.

The smallest files against other approaches

harrier-0.6b on SciFact, test split, 300 queries, fp32 0.7559. Native: llama.cpp on the CPU. Browser: our WebGPU runtime. Simulation: torch simulation of the quantised weights.

Paired against BitNet-270m at 2.12 bits per weight: SciFact +0.007 [−0.014; +0.028] and NFCorpus −0.005 [−0.020; +0.009], both indistinguishable; ArguAna +0.044 [+0.023; +0.065]. Sources: vq_results.md §1, model card of ThinEmbed-harrier-0.6b.

Czech: WebFAQ-cs

Qwen3-Embedding-0.6B on WebFAQ-cs (PaDaS-Lab/webfaq-retrieval, CC BY 4.0): 71 529 passages, 7 231 human-judged test questions; nDCG@10 against the model's own fp32 index.

ThinEmbed row: torch simulation of the file on the full test split. In the browser (Intel integrated GPU) the file reproduces the simulation on the first 50 questions to the fourth decimal (0.6393 against 0.6393); p95 130 ms, load 2.5 s. Calibration on 3 000 WebFAQ-cs training questions, one seed. The released scalar Czech files are measured on a Czech supreme-court index instead (see the released files). Source: model card of ThinEmbed-Qwen3-Embedding-0.6B-vq.

Released files

Each file is a query encoder for exactly one document encoder and its settings; the index does not change. Quality in % of the model's own fp32 nDCG@10, test splits unless stated.

The Czech supreme-court index: 55 071 segments, 1 137 synthetic queries; the data are not public. Each repository's model card carries the compatibility, the recipe and the sha256 of every file. Sources: the model cards in docs/release of the repository.

Quickstart

In the browser with the npm package @moravian/thinembed-vq. Needs WebGPU with shader-f16 (Chrome, Edge); there is no CPU fallback.

import { loadVqwClient } from 'https://moravian.ai/vqweb/vqweb.js';   // or: npm install @moravian/thinembed-vq

const client = await loadVqwClient(
  'https://huggingface.co/moravianai/ThinEmbed-Qwen3-Embedding-0.6B-vq/resolve/main/qwen3-0.6b-vq3.5d2-webfaq_q-vocfull.vqw');
const q = await client.embed('Kolik stojí parkování v centru Prahy?');   // Float32Array(1024), L2-normalised
// score = dot(q, d) against your Qwen/Qwen3-Embedding-0.6B document vectors

Measuring a client on your own index with the PyPI package thinembed; the document vectors you already have are never re-encoded:

pip install "thinembed[torch]"

eq import  --name mycorpus --docs corpus.jsonl --doc-emb index.f32 --doc-ids ids.json --teacher qwen3-0.6b --language cs
eq synth   --dataset mycorpus --lang Czech --n-per-doc 2 --as-calib     # synthetic queries + calibration text
eq cache   --dataset mycorpus --teacher qwen3-0.6b                      # fp32 query vectors of the unchanged model
eq compile --model model-f16.gguf --types Q4_K_M Q3_K --calib data/calib/mycorpus_synth_q.txt --table q4_0
eq eval    --dataset mycorpus --teacher qwen3-0.6b --clients models/*.gguf --bin llama-embedding --rule
eq compare results/perq/mycorpus_<a>.npz results/perq/mycorpus_<b>.npz  # paired bootstrap, same queries

Links

• Code, technical report and verification recipe: github.com/rosecky/embedding-quantization-public (Apache-2.0)
• Technical report: technical_report.md · vector-quantised results: vq_results.md
• npm: @moravian/thinembed-vq · package source and README
• PyPI: thinembed · package README
• Released files: huggingface.co/moravianai
• Browser demo: moravian.ai/demo

Code Apache-2.0. Client files under their base model's licence (MIT or Apache-2.0). Demo corpora SciDocs (BEIR, CC BY 4.0) and WebFAQ-cs (CC BY 4.0).