Oh. Yet another idea: An option so passive unlock is geo-gated, so it can be disabled while at home, but works while out and about.
Even though it's been working quite reliable for me with my mobile phone key, I've just decided I'm going to have to keep it disabled because while I'm walking...