advogato: iostreams performance in C++
17th October 2007
salimma,
-
I ran into that performance problem too, with my simple newmail.cc
mbox summary program (see update_mbox_cache()). You can
work around it by replacing
std::getline() with the normal .getline() member of the io
class, and using a non-std::string buffer.
It's still processing things line-by-line, which when compared to something like grep, is inefficient. Grep reads large blocks of the file, I think 16k or more at a time, and does its own very fast processing. But replacing std::string with a char buffer made the performance acceptable for me.
I believe the main cause of performance issues with std::string stem from the way iostreams insert a character at a time, and when using std::string, it is not very smart when growing its buffer.
I also find Boost regex to be rather slow at times, but perhaps I'm not using them correctly.