Duplicate line remover

How to remove duplicate lines

1. Paste your list into the left box, one item per line. A column copied from a spreadsheet works as it is.
2. Pick the options you need. The result on the right updates instantly, and the counts show how many lines were removed.
3. Click Copy result, and paste the clean list back where it came from.

Your options are kept in the page address, so a bookmark opens the tool set up the same way. The list itself is never saved or uploaded.

What the options do

Ignore case

Treats "Apple", "apple" and "APPLE" as the same line, keeping whichever copy the "When a line repeats" setting picks. Leave it off when capitals matter, such as for passwords, codes or file names on Linux.

Trim spaces

Removes spaces and tabs at the start and end of each line before comparing, and in the result. Copied lists often carry invisible trailing spaces that make two identical-looking lines different, so this is on by default. Spaces inside a line are always kept and compared.

Remove empty lines

Drops blank lines (including lines with only spaces when trimming is on). With it off, the first blank line is kept like any other line and further blank lines count as duplicates.

Keep the first or last copy

"Keep the first copy" leaves each line where it first appears, which suits most lists. "Keep the last copy" leaves it where it last appears, which is useful for logs and change lists, where the latest entry is the one that counts.

Sort A to Z or Z to A

Sorts the unique lines alphabetically, using your browser's language rules so accented letters sort in their natural place. Numbers sort by value, so "item 2" comes before "item 10".

Common uses

• Email and contact lists. Combine lists from several sources and remove the repeats before importing them. Tick Ignore case, since email addresses are not case sensitive in practice.
• Keywords and tags. Merge keyword exports or tag lists and sort them for a clean master list.
• Spreadsheet columns. Paste a column to see how many distinct values it has: the "Unique lines" count is the answer.
• Logs and data files. Collapse repeated log lines to see each distinct message once.
• Code and config. Clean up import lists, word lists and allow lists.

Tips and troubleshooting

• Two lines look identical but are not removed. They differ somewhere you cannot see, usually in capitals, a non-breaking space or a different kind of dash or quote. Turn on Ignore case, and check the spaces inside the line.
• Everything came out on one line. The list was probably separated by commas, not line breaks. Put each item on its own line first.
• A paragraph broke into separate lines. This tool works line by line. For prose, use remove line breaks instead.
• Mixed capitals? Make every line consistent first with the case converter, then remove duplicates.
• Need a total? The character counter counts lines, words and characters of the result.