The reason why I decided to asks this question is that I've been recently working on a data mining project where I have massive raw data files in plain text format and by the word massive I really mean the size of the files are so big that you can't simply browse them using any text editor.
Sometimes I encounter a badly formatted row somewhere in those file, and let's say I waa remove it. Unquestionably, that would be impossible to keep all the records in the memory, and using simply a StreamReader to read the blocks, and copying the line to another file over a StreamWriter is very inefficient.
The problem is that, in data mining, raw files are normally preserved in plain text or csv format.
I've been thinking would it be possible to just point out to a certain position in the file and then modify the range of bytes associated with that block without having to use the "Temp File" trick?
Of course I have come up with ideas such as breaking the large file down to small pieces and add an indexing function to solve the problem, but asking this question only and only for educational reasons.