What we are talking about is the total time it takes to process an input and send a command to the car. The less memory bandwidth you have (the more you use for sensors), and running VLA models means that your output latency is very, very high.
High latency means bad reaction time, which prevents you from braking for a pedestrian that pops out from behind a car.
This is simply not how VLA models work. As mentioned before, the VLA model acts as a planner, charting out the next few seconds of driving, with adjustments 10x per second on Nvidia's latest model. Notably, that is about 3x quicker than the avergage person's reaction time. Also, this means there wouldn't be "ping-ponging" in a lane because there is a planned trajectory.
This planner is completely separate from the low latency systems that are coupled with VLA models and adjust as much as 100x per second. This is what is actually executing the VLA model's planned trajectory. For the brief moments where the planned VLA path is no longer feasible, like due to something jumping in front of your car, the separate system computes a new trajectory.
In other words, this concept that systems utilizing VLA models have "very high latency" and "bad reaction times" is not grounded in reality.