Mercs have processing power of 254 TOPS while Cosmos has 2000 TOPS, night and day difference
Tesla AI5 has around 2000 for a single chip. It is a half reticle design, so when it comes to the car if they use 2 of them, that's more than 4000.
Memory bandwidth is about 8x what nvidia thor has as there are 12 memory modules per chip. 72gb total memory x 2
his. Tesla has to use a lot more memory than people would think because it has no depth sensors, doesn't leverage HD maps and doesn't have ultra-precise GPS.
Like literally everybody else. Everybody is doing depth estimation from video. Waymo does this. Every self driving car on earth does this. Now you have to use lidar as well. That's more compute, more memory bandwidth, not less.
people forget the lidar in consumer cars is not high res and has low FOV. That means it covers a very small part of your FOV and you can't even see cross traffic with this lidar. It also has similar failure modes to cameras and can produce a noisy input. And these E2E neural networks already process video.
There's a reason wayve drives pretty much using vision only, despite claiming their tech is sensor agnostic.
While that may seem slow, that is just the vehicle "planner", with a separate, low latency perception/safety loop.
What we are talking about is the total time it takes to process an input and send a command to the car. The less memory bandwidth you have (the more you use for sensors), and running VLA models means that your output latency is very, very high.
High latency means bad reaction time, which prevents you from braking for a pedestrian that pops out from behind a car.
You can see with the chinese systems they have very slow reaction time and nonexistent accident avoidance.
The slower you react the more obvious mistakes are when the car is driving like a tourist. Tesla drives with very incorrect map data, they just fix it in real-time so quickly you never notice these mistakes.
The other simple thing which people don't realize is the output control (steering/brake/throttle) is actually forward predicted to the moment these controls are actually sent to the car (vs. when the sensor inputs are received). The higher the latency, the more likely there are going to be errors in the command you are sending to the car.
This is actually the cause of a lot of lane centering systems "ping-ponging" across the lanes. It's because latency is so high, that the position the car is in when it sends the steering command is wrong and then the car is constantly overcorrecting.