Remove duplicate lines

Lists collected from spreadsheets, logs, exports and copy-paste often contain repeated lines. Removing them by hand is slow and error-prone. This tool deletes duplicates instantly and gives you options for what counts as a duplicate.

How to use it

1. Paste your text with one item per line.
2. Choose options: trim whitespace, ignore case, remove empty lines, sort.
3. Copy the unique lines and check how many duplicates were removed.

Example

Input:

apple
Banana
apple
banana 
cherry

With default settings, only the second apple is removed: Banana and banana differ in case and a trailing space. With trim and ignore case enabled, the result is apple, Banana, cherry, keeping the first occurrence of each.

Options explained

• Trim: ignores leading and trailing spaces, which are common in copied data and invisible on screen.
• Ignore case: treats Apple and apple as the same.
• Sort: outputs lines alphabetically instead of in original order.
• Remove empty lines: drops blank lines.

Command-line equivalents

• sort -u file.txt sorts and removes duplicates.
• awk '!seen[$0]++' file.txt removes duplicates but keeps the original order.
• PowerShell: Get-Content file.txt | Select-Object -Unique.

Tips

• Line endings matter: a file mixing Windows CRLF and Unix LF endings can make identical-looking lines differ. Trim handles this.
• For email lists, also lowercase the addresses or use an email normalizer.

Processing happens in your browser, so lists of customers or emails stay private.