Post
2457
0.6B reads 27B
A 0.6B model runs in your browser tab and searches 123,287 photographs that a 27B model encoded months ago. The big model is never downloaded and never runs.
Demo: RiverRider/0.6b-reads-27b
Head: RiverRider/srt-browser-head-118k
Two models normally cannot understand each other. Each builds its own private way of representing meaning, so an image encoded by one is gibberish to the other. We fitted a small linear read-out on each frozen model, aimed at the same space. Not a fine-tune: a map outside the weights, reading a hidden state the model already computed. Now the small model's reading of your sentence compares directly against the large model's reading of a photograph.
So the 27B model's work becomes a file. It looked at the gallery once on a datacenter GPU and left about a kilobyte per photograph. Nothing of it runs when you search. The 0.6B half runs on a phone CPU through WebAssembly: no WebGPU, no API key, no server, nothing leaves the tab, and it works offline after the first load.
Privacy here is structural rather than promised, because there is no server to trust. The index is a file you hold instead of an embedding bill you pay, and it can outlive the model that made it.
Receipts. Median rank 33 of 123,287 puts the right photograph in the top 0.027%. Shuffled-pair control 0.0002. The 4 KB runtime anchor is not optional: the same head scores 0.2300 under PyTorch fp16 but 0.0154 under candle Q4, and 0.1952 once anchored.
Honest limit: word order is at chance, 0.468 on minimal pairs, n=141. "A dog chasing a boy" and "a boy chasing a dog" look alike to it. This is a bag of content, not a parser of relations.
Apache-2.0. The negatives sit in the model card next to the positives.
A 0.6B model runs in your browser tab and searches 123,287 photographs that a 27B model encoded months ago. The big model is never downloaded and never runs.
Demo: RiverRider/0.6b-reads-27b
Head: RiverRider/srt-browser-head-118k
Two models normally cannot understand each other. Each builds its own private way of representing meaning, so an image encoded by one is gibberish to the other. We fitted a small linear read-out on each frozen model, aimed at the same space. Not a fine-tune: a map outside the weights, reading a hidden state the model already computed. Now the small model's reading of your sentence compares directly against the large model's reading of a photograph.
So the 27B model's work becomes a file. It looked at the gallery once on a datacenter GPU and left about a kilobyte per photograph. Nothing of it runs when you search. The 0.6B half runs on a phone CPU through WebAssembly: no WebGPU, no API key, no server, nothing leaves the tab, and it works offline after the first load.
Privacy here is structural rather than promised, because there is no server to trust. The index is a file you hold instead of an embedding bill you pay, and it can outlive the model that made it.
Receipts. Median rank 33 of 123,287 puts the right photograph in the top 0.027%. Shuffled-pair control 0.0002. The 4 KB runtime anchor is not optional: the same head scores 0.2300 under PyTorch fp16 but 0.0154 under candle Q4, and 0.1952 once anchored.
Honest limit: word order is at chance, 0.468 on minimal pairs, n=141. "A dog chasing a boy" and "a boy chasing a dog" look alike to it. This is a bag of content, not a parser of relations.
Apache-2.0. The negatives sit in the model card next to the positives.
- 7 replies
