Revision 815881
Go back to digest for 8th June 2008Optimization in Development Tools
Big and fat optimization, both in terms of memory and execution time:
Memory:
- Don't keep the all HTML doc associated to each link as well as all the children nodes. Instead, remove everything unneeded after the parsing process, i.e., its own node and the named items needed for the anchors
- Don't cache all the KHTMLParts used to parse the anchors
Execution:
- Move the parsing work into the helper thread, so practically only KIO is working in the main thread
- Use the already parsed named items in order to check the anchors, instead of initialize a KHTMLPart for each one. Currently, elements with ID attribute are missing
The memory optimization basically makes KLinkStatus viable for checking large sites.
Here it checked Qt 3 html docs (15623 links) using ~200MB.
File Changes
- /trunk/KDE/kdewebdev/klinkstatus/src
- /main.cpp
- /engine/linkchecker.cpp
- /engine/linkstatus.cpp
- /engine/linkstatus.h
- /engine/linkstatus_impl.h
- /engine/searchmanager.cpp
- /parser/htmlparser.cpp
- /parser/htmlparser.h
- /utils/utils.h