How to use it
Paste text. Remove duplicates runs as you type and when you press the button. Match case keeps lines that differ only by letter case. Drop empty lines removes blank lines as well as duplicates. Copy and Download use the result. The first time a line appears is the one that stays.
Example
With case matching on, Beta stays because it is not the same line as beta. The second alpha and the second beta are removed. The blank line stays unless you drop empty lines.
Input
alpha
beta
alpha
Beta
betaResult
alpha
beta
Beta
How it works
Lines are split on line breaks. Comparison is exact when Match case is on, and uses Unicode lowercasing when it is off. A line already kept is skipped later. Empty lines are ordinary lines until Drop empty lines is on. The output uses newline separators and does not sort the text.
Common errors
- Beta disappeared Match case is off, so Beta and beta count as the same line. Turn Match case on to keep both.
- The blank line vanished Drop empty lines is on. Turn it off to keep blank lines that are not duplicates.
- The order changed It should not. If a line moved, it was a later duplicate of an earlier line and only the earlier one remains.
Limitations
The tool does not trim spaces, so "alpha" and "alpha " are different lines. It does not remove duplicate words inside a line. Text longer than 500,000 characters is refused.
Privacy
The lines you paste are processed in your browser. PepsTools does not receive them, store them, or send them to another service.
Questions
Which duplicate is kept?
The earliest line. Later copies are removed.