Home / AI / Near Duplicate Finder
Near Duplicate Finder
Find lines, sentences or paragraphs that say the same thing, even in different words, and remove the copies.
Free, no uploads, no sign-up. Runs 100% in your browser.
How Near Duplicate Finder works
In AI mode each item becomes a MiniLM sentence embedding in a Web Worker and items closer than your threshold are grouped. Word overlap mode compares two-word shingles instead, with no download. The first item of each group is kept.
Input: Text, TXT, CSV. Output: TXT, CSV.
How to use Near Duplicate Finder
- Paste or open your text.
- Choose line, sentence or paragraph, and the similarity level.
- Find the groups.
- Download the cleaned text or the groups as CSV.
Tips
- Nothing is uploaded. Your files stay in this browser tab.
Near Duplicate Finder FAQ
- Are my files uploaded?
- No. This tool runs entirely in your browser tab. Files are read from your disk into memory and the result is generated locally.
- Is it free? Are there limits or watermarks?
- Free and open source (MIT). There are no file-size caps, daily quotas, accounts or watermarks.
- Does it work offline?
- Yes, once the page (and any engine it needs) has loaded, it keeps working without a connection thanks to the service worker cache.
Related tools
- Duplicate Line Remover: Remove repeated lines, keep only repeats, ignore case and spaces.
- Semantic File Search: Search many PDFs, Word files, notes and code by meaning, not just exact words, all offline.
- Zero-shot Text Classifier: Sort messages, tickets or headlines into labels you choose, with no training, on your device.
- AI Summarizer: Summarize PDFs, Word files and long text on your device, as paragraphs or key points.