Replace Pioneer Home   All Examples   Free Download

 New request --free  RSS: Replace Pioneer Examples

913.Regular expression replace -- How to remove consecutive duplicate words in multiple files?

User: vlad -- 2012-02-06          << 912  914 >>
Hits: 2584
Type: Regular expression replace   
Search all Regular expression replace examples
How to remove consecutive duplicate words in multiple files?
I would like to scan 30000 pages for duplicate text lets say if i have "online online" i want it to become "online" and so for all the duplicates.
The only problem I do not know how many duplicates I possibly have and which duplicates I have so I can not just put "online online"etc.
Input Sample:
This is an online online page
Output Sample:
This is an online page
Hint: You need to Download and install "Replace Pioneer" on windows platform to finish following steps.
1. open "Tools->Batch Runner" window
2. drag multiple page files from "windows file browser" to "batch runner" window
3. click "fast replace" to open "fast replace" window
4. click "Add" to add a new rule
* set "search" to:

* set "replace" to:

* click "ok"
5. make sure "reg exp" and "cross line" options are selected
6. click "start" and click "ok", done.

Screenshot 1:  Fast_Replace_Window

Similar Examples:
How to remove/delete consecutive duplicate lines in a text file? (77%)
How to remove some specified characters from multiple filenames? (72%)
How to extract and remove common lines from multiple files? (68%)
How to auto remove/delete consecutive duplicate sentences or phrases in a text file? (68%)
How to remove duplicate lines from a text file? (67%)
How to remove additional line breaks from multiple files? (65%)
How to remove partially duplicated lines in a text file? (65%)
How to remove the first occurrance of duplicated words in each line? (64%)

Check Demo of Regular expression replace
remove consecutive duplicate words  how to remove consecutive duplicate words  remove consecutive duplicate  online  duplicates  duplicate words  consecutive  onsecutive  cut  pages  remove duplicate line online  batch remove duplicates in a text file  batch remove duplicates in text file  remove duplicates text file batch file  batch file remove duplicates text  remove words from multiple text files  remove duplicate words online  batch file to remove duplicates in text file