Method
A duplicated term is one word carrying two popular meanings. We call the older one the original and the newer one the duplicate, the meaning that rode in on an established name. Every page sits at deduplication.tech/term/definition1/definition2, with the original first.
Evidence
Wikipedia pageviews. Monthly views of the English Wikipedia article for each meaning, from July 2015 to last month, pulled live from the Wikimedia Pageviews API. Redirects resolve to the canonical article first. We compare the last 12 months to get the duplicate's share of attention, and smooth with a 3-month moving average to find the last month where the lines crossed.
Origin dates. Hand-researched approximate dates of first use for each meaning. "Ancient" means the sense predates modern record-keeping.
Spoken usage: YouTube and Movies & TV (in progress)
Pageviews show what people look up. To see which meaning people actually say, we're labeling real spoken mentions of every term from two sources:
- YouTube, via YODAS. Transcripts of hundreds of thousands of hours of Creative Commons YouTube videos, released by its authors under CC BY 3.0 (paper).
- Movies & TV, via the OpenSubtitles corpus from opensubtitles.org (Lison & Tiedemann, 2016). We publish counts and shares from it, never dialogue.
How a mention gets labeled. We find every line containing the term, keep the sentence around it, and a model decides whether it means the original, the duplicate, or something else. Low-confidence lines go to a human reviewer. Each term page will get YouTube and Movies & TV filters showing the split by source and by year.
Colour system
Every colour on the site answers one question: which meaning owns the word?
Where you see it. On a term page the left half is always grey (original) and the right half black with green (duplicate), whoever is winning. The bar under the hero is the word split by ownership. The "Canonical today" and "Status" boxes take the colour of whichever side wins. In the cloud, word size is popularity (both meanings' Wikipedia views combined) and colour is ownership.
Verdicts
Over 60% duplicate share: duplicate dominant. Under 40%: original holds. In between: contested. The last 12 months of Wikipedia pageviews decide. When neither source separates the meanings (MTS, for example), the page shows no verdict rather than guess.
Limits
Pageviews measure lookups, not usage, and an article can cover more than one sense. Phrase searches miss usages that never spell the meaning out. Treat each page as evidence, not a ruling.