Replace Pioneer Home   All Examples   Free Download

 New request --free  RSS: Replace Pioneer Examples

803.Advanced search and replace -- How to remove all lines that have duplicate contents in CSV files?

User: editor -- 2011-06-21          << 802  804 >>
Hits: 1770
Type: Advanced search and replace   
Search all Advanced search and replace examples
Description:
How to remove all lines that have duplicate contents in CSV files?
There is a csv file, some of lines have same cells but the cells have different order, how to keep only one copy of those duplicated lines?
Input Sample:
A,B,C
C,B,A(duplicate with A,B,C)
A,B,B
D,B,A
B,A,B(duplicate with A,B,B)
Output Sample:
A,B,C
A,B,B
D,B,A
Answer:
Hint: You need to Download and install "Replace Pioneer" on windows platform to finish following steps.
1. ctrl-o open the CSV file
2. ctrl-h open replace dialog
* set 'replace unit' to 'Line'
* set 'replace with pattern' to:

click 'Advanced' page
* set 'run following at the beginning of replace' to:

* set 'run following for each matched unit' to:


3. click 'replace', done!

Screenshot 1:  Replace_Window


Screenshot 2:  Replace_Advanced_Window


Similar Examples:
How to remove all lines with duplicate value in column B in CSV file? (77%)
How to remove all lines that contain ABC but not contain DEF in text file? (67%)
How to remove all the lines that do not contain any of words in a list? (67%)
How to remove all lines that can be found in another file? (65%)
How to remove partially duplicated lines in a text file? (64%)
How to remove duplicate lines from a text file? (63%)
How to delete lines that start with duplicate text? (63%)
How to extract all lines that has more than 4 columns from csv file? (63%)

Check Demo of Advanced search and replace
Keywords:
cell  contents  duplicated  toc  csv files  else  keep  duplicate  duplicat  dupl  copy files different  duplicate lines csv  remove same  remove csv  remove same file  duplicate csv  remove duplicated lines  copy lines