Revision 79edb41...

Go back to digest for 4th March 2012

Optimization in Development Tools

Milian Wolff committed changes in [kdevelop] /cpp:

optimize: reduce memory consumption of Token class by 50% on 64Bit

By removing the ParseSession pointer from it, we get rid of 8 bytes
and furthermore reduce the alignment size to 4. This way, we now
only require 12 bytes per token compared to 24 bytes previously.

This also allows us to define the Token class as a primitive type,
potentially speeding up the TokenStream even further.

The "cost" is a changed API, to get the string representation of
a token, one must now ask the TokenStream. In practice this is very
rarely a real pita, as before one often did


now you just do


Furthermore I've consolidated the tons of custom "AST* node to QString"
functions into one central ParseSession::stringForNode.

Finally, I've replaced some costly token.symbol() == IndexedChar("somechar")
with the much faster token.kind == Token_xyz comparisons.

All in all, this should a) make our code faster and b) let it use much
less memory while at it. For the big resource file in the bug below,
the difference of 50% in the Token class results in ~250MB less memory

File Changes

Modified 18 files
  • /cpp
  •   languagesduchaineditorintegrator.cpp
  •   languagesduchain/declarationbuilder.cpp
  •   languagesduchain/dumpchain.cpp
  •   languagesduchain/expressionvisitor.cpp
  •   languagesduchain/name_visitor.cpp
  •   languages/parser/codegenerator.cpp
  •   languages/parser/dumptree.cpp
  •   languages/parser/lexer.cpp
  •   languages/parser/lexer.h
  •   languages/parser/name_compiler.cpp
  •   languages/parser/parser.cpp
  •   languages/parser/parsesession.cpp
  •   languages/parser/parsesession.h
  •   languages/tests-parser.cpp
  •   languages/parser/tests/test_generator.cpp
  •   languages/parser/tests/test_parser.cpp
  •   languages/parser/tests/test_parser.h
  •   languages/parser/tests/test_parser_cpp2011.cpp
18 files changed in total