Replace Pioneer Home   All Examples   Free Download

 New request --free  RSS: Replace Pioneer Examples

913.Regular expression replace -- How to remove consecutive duplicate words in multiple files?

User: vlad -- 2012-02-06          << 912  914 >>
Hits: 145
Type: Regular expression replace   
Search all Regular expression replace examples
Description:
How to remove consecutive duplicate words in multiple files?
I would like to scan 30000 pages for duplicate text lets say if i have "online online" i want it to become "online" and so for all the duplicates.
The only problem I do not know how many duplicates I possibly have and which duplicates I have so I can not just put "online online"etc.
Input Sample:
This is an online online page
Output Sample:
This is an online page
Answer:
Hint: You need to Download and install "Replace Pioneer" on windows platform to finish following steps.
1. open "Tools->Batch Runner" window
2. drag multiple page files from "windows file browser" to "batch runner" window
3. click "fast replace" to open "fast replace" window
4. click "Add" to add a new rule
* set "search" to:

* set "replace" to:

* click "ok"
5. make sure "reg exp" and "cross line" options are selected
6. click "start" and click "ok", done.

Screenshot 1:  Fast_Replace_Window


Similar Examples:
How to remove/delete consecutive duplicate lines in a text file? (77%)
How to remove some specified characters from multiple filenames? (72%)
How to extract and remove common lines from multiple files? (68%)
How to auto remove/delete consecutive duplicate sentences or phrases in a text file? (68%)
How to remove additional line breaks from multiple files? (65%)
How to remove partially duplicated lines in a text file? (65%)
How to remove all lines that have duplicate contents in CSV files? (64%)
How to remove duplicate lines in multiple files in different sub-folders? (63%)

Check Demo of Regular expression replace
Keywords:
remove consecutive duplicate words  remove consecutive duplicate  duplicates  duplicate words  online  consecutive  onsecutive  cut  pages  test  remove duplicates  remove duplicate line online  batch remove duplicates in a text file  batch remove duplicates in text file  batch duplicates text  remove duplicates text file batch file  batch file remove duplicates text  consecutive words