List deduplicator
Drop duplicate items from a line-based or delimiter-separated list — case and whitespace folding, keep-first or keep-last, optional sorting, plus totals and a duplicate breakdown.
Runs in your browserEvery computation happens in your browser — your data never leaves this device.
Deduplicated list
One item per line, ready to copy.
Paste a list to deduplicate it
Total: 0 · Unique: 0 · Removed: 0 · Duplicate groups: 0
What this tool does
- Deduplicate merged lists, logs or exported spreadsheets in one pass while keeping the original order.
- See which items repeat and how many times: the breakdown lists every occurrence position.
- Switch to keep-last when the newest row should win, e.g. one status row per ID in row order.
Example
Input
apple banana apple cherry banana
Output
apple banana cherry
Duplicate rows are removed and first occurrences keep their order; turn on ignore-case to fold Apple and apple.
Frequently asked questions
Are Apple and apple duplicates?
Not by default. Turn on “Ignore case” and they become the same item. Ignoring case only affects the comparison — the output keeps the spelling of the occurrence you chose to keep.
Are blank lines kept?
No, they are dropped by default. In most exports a blank line is layout residue and would otherwise count as an empty item. Turn off “Drop empty items” when blank lines are meaningful.
How many items can it handle?
200,000. Beyond that it reports an error instead of freezing the tab, because dedupe runs on the browser main thread.
How do I dedupe a comma-separated list?
Switch the split mode to “By delimiter” and type a comma — \t means a tab and | a pipe. The output is one item per line, ready to paste back into a sheet.
Keywords：列表去重去重复重复项dedupeunique去重list unique重复次数