Taste Oracle
Claude already knows what everyone thinks of nearly every film. So the shortest useful description of one person's taste is a description of how they diverge from everyone. This experiment compressed 791 Letterboxd ratings into a 400-word "taste constitution," then measured whether it predicts held-out ratings better than consensus does. Honest result: partly.
The constitution
The deliverable itself — no film titles allowed, only generalizable claims. This is the 400-word point on the compression frontier.
Rejection detection — where the compression pays
A consensus model almost never predicts this rater will pan a film — it caught 1 of 20 test-set rejections. The constitution caught 9, at the same precision. The taste signal that survives compression isn't point accuracy; it's knowing which acclaimed films will fail for him.
But point accuracy tied — and dev gains didn't survive
Mean absolute error on the untouched test set is a statistical tie between the oracle and both baselines (paired t < 0.7). The dev-set improvement was real overfitting: seven scoring runs against the same 100 films selected for dev-specific quirks.
The compression frontier
Word count vs dev-set error across constitution drafts. 124 words already beats every baseline on dev; the last 270 words bought ~0.1 stars. Gray points are dominated drafts.
Where the ratings actually land
Predicted vs actual rating bands on the 100 test films. Consensus piles everything into "solid–high." The oracle spreads mass toward the real distribution — right shape, imperfect aim.
The divergence signature
The films that define this taste: largest gaps between his rating and the community's, across all 791 films.
Consensus tracks him until the year 2000
Mean gap between his rating and consensus, by decade. Pre-2000 he is a half-notch harsh; after 2000 the gap triples. The constitution's single strongest rule is this curve.
Honest findings
- The constitution beats consensus at calibration, not accuracy. Bias fell from +0.49 to +0.13 stars and predictions use the full 0.5–5.0 scale — but MAE tied (0.82 vs 0.83).
- Rejection detection is the real win: 45% recall vs 5%. The single most useful question — "will this acclaimed film fail for me?" — is where 400 words of divergence description carries genuine signal.
- Dev iteration overfit, and the test set caught it. Dev MAE 0.685 → test MAE 0.82. Selecting among 7 drafts on 100 films buys ~0.1 stars of illusory improvement.
- Taste has an entropy floor prose can't cross. The same rater gave Poor Things 4.0 and Kinds of Kindness 1.0; Tenet 5.0 and Interstellar 2.5; The Hateful Eight 1.5 and Once Upon a Time in Hollywood 4.5. Any rule broad enough to generalize misfires on half of these; only memorization would capture them, and memorization isn't taste.
- Even the best cheap trick — "consensus minus half a star" — hits MAE 0.81 on test. The gap between that and a perfect model is mostly irreducible film-level idiosyncrasy.
Method
| run | what it knows | dev MAE | test MAE | test ≤0.5 |
|---|
791 ratings, stratified by rating band × decade into train 591 / dev 100 / test 100. Model: claude-sonnet-4-6, one film per call, title + year only — the true rating never enters the prediction context. Test set touched exactly once, after the constitution was frozen. Predictions cached; full pipeline is ~5 files of plain Python.