Just try it. You don't need to turn off passive unlock to use the door handle. Triggering the door handle does trigger the car to look for a key, whether you have passive unlock on or not.
Instead of waiting 30 seconds and whispering angry things under your breath, just push the handle. If that...