Follow the words.
Question the source.
An investigation desk for shared language. Compare passages, inspect the matching words, and weigh archive evidence. Similarity is a lead, not a verdict.
Compared as word 5-grams after lower-casing and stripping punctuation.
A URL alone works for Wikipedia pages and for sites that allow cross-origin reads. Most sites don't — then paste the page text too, and the URL is still used for archive dating.
use a Tracer server (web search, CDX dates, full-page fetch)
Run it yourself from /server (FastAPI or Docker). Leave empty to run everything in this tab.
How it works
Every candidate goes through the same steps, all in this tab unless you point it at a server.
- Text in. Pasted text is used as is. A Wikipedia URL is fetched through the Wikipedia API, which allows cross-origin reads. Any other URL is tried directly; when the site blocks it (most do), the card says so and asks you to paste the text.
- Lexical match. Both texts are lower-cased, stripped of punctuation and diacritics, and cut into overlapping 5-word shingles. I compute the exact Jaccard of the two shingle sets, a 128-permutation MinHash estimate of it, and containment: the share of the suspect's shingles found in the candidate. Containment is the one that decides, because a copy usually sits inside a bigger page full of navigation and other text.
-
Paraphrase. Lexical overlap can't see a rewrite, so
MiniLM (
all-MiniLM-L6-v2, 8-bit, run by transformers.js in a Web Worker) embeds the suspect in passages of at least 12 words, and every candidate sentence once; a window of 1–3 consecutive candidate sentences is the word-weighted mean of their vectors, because a paraphrase merges and splits sentences. The semantic score is the share of suspect words whose passage has a candidate window at cosine ≥ 0.60. Those passage pairs get a dotted underline in the side-by-side view. - Class. Containment ≥ 0.90 is an exact copy. ≥ 0.40 (or Jaccard ≥ 0.40) is a near-duplicate. Otherwise, semantic coverage ≥ 0.35 is a paraphrase, and containment ≥ 0.05 is a partial quote. Anything below that is unrelated.
- Dates. For URLs I ask the Wayback Machine when it first captured the page. Wikipedia pages also get the timestamp of their first revision. What the page says about itself is shown too, but only as a claim. The earliest dated candidate is a lead, not an established origin. Self-reported dates only count when nothing independent exists.
The server in the repo runs the same pipeline in Python with the same hash functions (a parity test checks the browser and server agree bit for bit). It adds web search (DuckDuckGo HTML, or Brave/SerpAPI with a key), full-page fetch with trafilatura, htmldate for the page's claimed date, and Wayback CDX for exact first-capture timestamps.
Measured results
All numbers are from runs on 2026-10-14 on my laptop (Windows, Chrome 155 headless with a fresh profile, so nothing cached). The example is the opening of Pride and Prejudice, chapter 1 (342 words, 338 distinct 5-grams), against six sources.
| source | containment | jaccard | minhash | semantic | class |
|---|---|---|---|---|---|
| pemberley.com chapter page | 1.000 | 0.377 | 0.336 | 1.00 | exact copy |
| Gutenberg #1342 chapter I | 1.000 | 0.398 | 0.383 | 1.00 | exact copy |
| study-guide copy (mine) | 0.447 | 0.350 | 0.414 | 0.68 | near-duplicate |
| modern retelling (mine) | 0.021 | 0.012 | 0.008 | 0.52 | paraphrase |
| Wikipedia article (live) | 0.065 | 0.002 | 0.016 | 0.18 | partial quote |
| Moby-Dick opening | 0.000 | 0.000 | 0.000 | 0.00 | unrelated |
Dates. The Wayback Machine first captured the pemberley.com chapter page on 1998-12-02 (CDX, 08:54:41 UTC). The page's own metadata, read by htmldate, says 2004-01-01. The Gutenberg HTML file was first captured on 2008-09-20, ten years after the release date Gutenberg prints in the book (June 1, 1998). The Wikipedia article's first revision is 2001-09-26, its first capture 2004-05-04. This makes pemberley.com the earliest dated candidate in this selected set; it does not prove the passage was present in that capture.
The live probes are approximate. The browser can only use the Wayback availability API (see limits). Over five cold runs it returned pemberley's 1998-12-02 on three and 1999-05-04 on two, and Gutenberg's 2008-09-20 every time, in 4–8 requests each. For Wikipedia it returned 2004-05-04 once and 2004-07-05 the other time (8 and 20 requests). The second run waited 60.5 s in total for those probes. The results are drawn after about 26 s and the dates are filled in when the probes finish.
Server, with web search. The FastAPI version with DuckDuckGo and no API keys: 2 quoted queries returned 12 URLs. 7 were fetched and 5 failed (Goodreads refused the connection, and three sites answered 403). Of the 7, 5 were exact copies, 1 a near-duplicate and 1 a partial quote. The earliest independent evidence was authorama.com, first archived 2004-10-09, while its metadata claims 2025-01-01. janeausten.org claims 2026-01-01 and was archived in 2021. 48.4 s cold, 21.8–24.3 s with the Wayback cache warm; the rest is page fetching. 21 pytest tests pass, including a parity test showing that the browser and the server compute identical tokens, hashes, MinHash signatures and scores.
What I learned
Page metadata and archive dates answer different questions. pemberley.com says 2004 and authorama.com says 2025, but the archive saw them in 1998 and 2004. It goes the other way too: Gutenberg's printed release date (1998) is earlier than the archive's first capture of that URL (2008), and it's probably true, since the file just moved. That's why the tool only falls back on claims, and why it calls an archive date an upper bound rather than "the" date.
Containment, not Jaccard, decides "copy". Both real copies score Jaccard below 0.40, because the copied passage sits inside a page with navigation and the rest of the chapter. Jaccard would have labelled two verbatim copies as merely "near". The right question is what share of the suspect is found inside the candidate.
Sentence-by-sentence embeddings miss paraphrases. My first version averaged the best sentence-to-sentence cosine. That scored my retelling 0.55 and the Wikipedia plot summary 0.42, too close to separate. Rewrites merge and split sentences, so now I compare suspect passages against 1–3 sentence windows of the candidate and count how much of the suspect has a match at ≥ 0.60. That gives 0.52 against 0.18. Building each window from averaged sentence vectors gives the same scores and cut the browser embedding time from 58 s to about 10 s.
CORS shapes the product. Browsers can't read the archive's best endpoint (CDX) or almost any source page. A static demo has to be built around the few open doors: Wikipedia's API and the Wayback availability API. The full pipeline lives in the server.
Limits
What the browser version can't do. I checked each endpoint
with curl -H "Origin: https://temirlanzzz.github.io":
-
web.archive.org/cdx/search/cdx: no CORS header, so browsers can't read it. This is the exact first-capture API, and only the server uses it. -
web.archive.org/web/<ts>id_/<url>(raw archived pages): no CORS header, so browsers can't read old page text. -
wayback-api.archive.org/wayback/available: sendsAccess-Control-Allow-Origin: *, so it works. It only answers "the closest capture to T", so the browser probes Jan 1 and Jul 1 of each year from 1996. The answer is the earliest capture those probes hit, which is an upper bound. Sometimes the API returns nothing, or an HTML "Temporarily Offline" page that fails CORS; the UI says so when that happens. (archive.org/wayback/availabletimed out from my network, so I use thewayback-apihost.) - Page text: only Wikipedia (its API sends CORS headers) and the rare site that allows cross-origin reads. For anything else you paste the text. In the example, the pemberley.com and Gutenberg texts were captured by my build script and are labelled that way. The "recorded CDX" dates next to them come from a server run on 2026-10-14.
- No web search. Search engines don't allow cross-origin calls without a key you would have to expose, so finding candidates is a server feature.
Both versions. The thresholds (0.90 / 0.40 / 0.05
containment, 0.35 semantic coverage) are tuned on one example, not
on a labelled benchmark. The paraphrase margin is modest (0.52 vs
0.18), and MiniLM will call a close summary of the same scene a
paraphrase. The two sources marked "written for this demo" are
mine, made to show a near-duplicate and a paraphrase. The server's
embeddings are optional (requirements-embeddings.txt
pulls in PyTorch). Without them it reports
semantic: null and cannot detect paraphrases;
model-enabled inference is not covered by the tests. A first
capture tells you when the archive noticed a page, not when
someone wrote the text: this one was written in 1813.