Move the activity simulation AFTER removing idle timeouts. Otherwise, timeouts might be
triggered out when the actions are still being unloaded, and hence still listening to them.