Core Insight

MemPalace is a case study in attention arbitrage: launch an imperfect-but-real open-source project with a celebrity face, fabricated benchmark numbers as the hook, and let controversy + community labor do the rest. The technical product is secondary to the narrative architecture.

---

The Playbook (Replicable Pattern)

Inputs:
  - A-list name with thematic resonance to the product
  - Conceptually appealing open-source project (flaws acceptable)
  - Inflated benchmark numbers as transmission vector
  - Pre-existing crypto/community distribution network

Outputs (4 days):
  - 37.6K GitHub stars
  - 3M+ views on launch post
  - Forbes / Kotaku / CyberNews coverage
  - pump.fun meme coin (50% creator reward split)
  - Identity shift: crypto nobody → AI thought leader
  - Free bug fixes via community (40+ merged PRs)

Key mechanic: Controversy generates MORE attention than clean launches. The benchmark fraud triggered audits that triggered media coverage that triggered stars. Ben chose "impressive" first, then let the community make it "correct."

---

Benchmark Fraud Taxonomy (7 Patterns)

Each pattern is independently reusable as a detection heuristic:

| Fraud Type | MemPalace Instance | Detection Method |

|---|---|---|

| Metric substitution | recall_any@5 reported as if E2E QA accuracy | Ask: what is the official eval metric for this benchmark? |

| Teaching to the test | Manual patches for 3 specific LongMemEval questions | Check if eval script has hardcoded question IDs |

| Bypass via over-retrieval | LoCoMo 100% via top_k=50 on 19-32 session datasets | Compare top_k to dataset size |

| Lossy compression claimed lossless | AAAK regex+truncation called "30x lossless" | Require round-trip fidelity test |

| Attribution laundering | 96.6% is raw ChromaDB performance, credited to MemPalace | Ablate the novel component; does score drop? |

| Paid dependency hidden | "No API" but 100% score requires Claude API | Check if top score requires paid external calls |

| Non-standard subset | Used 1,986 questions incl. 446 adversarial vs standard 1,540 | Verify subset matches benchmark paper's eval protocol |