Revision ff9edf8...

Go back to digest for 6th April 2014

Bug Fixes in KDE-PIM

Dan Vratil committed changes in [akonadi/1.12] /src:

Fix potential severe data loss during copy and move operations

Move and copy operations on larger sets of items can take some time, because
we need to make sure that all items have full payload stored in Akonadi. This
is necessary especially with local resources, like maildir, which have cache
timeout set to 5 minutes (after that all their payload is removed from Akonadi
and is fetched from HDD on demand, because it's cheap, fast and does not
unnecessarily duplicate emails in maildir and in database). However fetching
large amount of items via ItemRetriever takes a lot of time, sometimes it can
take even more than 5 minutes. And in such case there is a very high chance,
that the CacheCleaner will just remove the newly cached payloads from Akonadi
again and so when ItemRetriever finishes, many items have empty payload in
Akonadi again. ItemRetriever nor handlers are aware of this howerver, so they
will just make copies or moves of empty items, causing data loss.

This patch introduces CacheCleanerInhibitor, a class which when it is created
will pause the timer in CacheCleaner and resume the timer again when it's
destroyed (so usually when it goes out of scope). Also, this patch adds the
inhibitor to all handlers that use ItemRetriever, so that the the situation
described above does not happen.

The current solution is not perfect because it pauses the entire CacheCleaner
while I think it would be better to be able to temporarily 'blacklist' only
specific collections or items. That would however require much more complex
code and changes, which makes it unsuitable for 1.12 release.

I tried moving 78 000 emails from one maildir to another and all emails were
moved correctly. Move itself has many other problems (CPU, IO, memory, KMail
responsivness, etc.) but that's out of scope of this fix.

File Changes

Modified 10 files
  • /src
  •   server/cachecleaner.cpp
  •   server/cachecleaner.h
  •   server/collectionscheduler.cpp
  •   server/collectionscheduler.h
  •   server/handler/colcopy.cpp
  •   server/handler/colmove.cpp
  •   server/handler/copy.cpp
  •   server/handler/fetch.cpp
  •   server/handler/modify.cpp
  •   server/handler/move.cpp
10 files changed in total