Fix hiding of expired/revoked keys

While at it: if "show only public keys" is also set don't check already hidden keys twice.