Duplicate Lines Remover

About Duplicate lines remover

The duplicate lines remover keeps the first copy of each line and deletes every later repeat, leaving the original order untouched. Two switches decide what counts as a duplicate: whether capital letters matter and whether surrounding spaces are ignored. It is a quick way to clean keyword lists, exported names, log lines and any other one-item-per-line data.

How to use it

1. Paste your lines into the Text with Duplicate Lines box.
2. Set the Case sensitive switch. It is on by default, so Apple and apple are treated as different lines.
3. Set the Trim whitespace switch, which is also on by default, to ignore spaces at the ends of lines.
4. Click Remove Duplicates and read the result, along with the original, removed and remaining line counts.

Case-insensitive cleanup

apple
Banana
Apple
banana
cherry

apple
Banana
cherry  (case sensitive off: 2 duplicates removed)

When to use it

• Deduplicating a pasted list of email subjects, product names or search terms before importing it.
• Shrinking a log excerpt by dropping lines that repeat exactly.
• Merging two lists by pasting one under the other and removing whatever appears twice.
• Tidying a keyword list where the same phrase arrived in different capitalization.

Questions about Duplicate lines remover

Which copy of a duplicate is kept?

The first one. Later matching lines are dropped, so the order of your remaining lines is exactly the order in which they first appeared.

Are blank lines treated as duplicates?

Yes. An empty line is a line like any other, so the first blank line survives and the rest are removed. With Trim whitespace on, lines holding only spaces count as blank too.

What does turning off Trim whitespace change?

Lines are then compared exactly as typed, so a line with a trailing space is different from the same line without one. Leave it on unless the spacing is meaningful.

Does trimming change the lines I get back?

Yes. When trimming is on, the kept lines are output already trimmed, so leading indentation is removed as well. Turn the switch off if you need to keep indentation.

Related tools

• Emojis removerRemove all the emojis from any given text with ease.
• Character counterCount the amount of characters and words of a given text.
• Text repeaterRepeat any text a set number of times with a custom separator.
• Whitespace removerTrim, collapse or strip whitespace and convert between tabs and spaces.
• Text separatorSeparate text back and forth by new lines, commas, dots, etc.
• Email extractorExtract email addresses from any kind of text content.