Mercedes MB.DRIVE ASSIST PRO Public Demo (or what Lucid's Dream Drive Pro might look like)

I point out issues like memory bandwidth with the nvidia thor and people gloss over it like it's not some real issue

Sure, memory bandwidthb is important, but it isn't the end-all be-all.

There are serious drawbacks to the way the Tesla system is designed, and in some ways they are brute forcing their way to the current level of FSD.

There are other architectural designs in Thor that allow it to (theoretically) make better use of the memory bandwidth, and do so at much lower power (and thus heat) levels than Tesla.

Nvidia has a LOT more experience designing and then wringing maximum performance out of compute/graphics chipsets and systems than Tesla. They haven't dominated both the gaming and workstation GPU market by being stupid.

Is it a guarantee that it will exceed or even meet the overall performance that Tesla has? No, of course not, it's a complicated, emerging technology and there's never a guarantee a particular new way of doing things will work out.

But you're insane to pretend they don't know what they're doing, and I wouldn't bet against Thor being a very successful platform.
 
I think this video shows tremendous improvement with Mercedes autonomy in the last year. It’s promising for Alpamayo for Lucid too. Also, notice how much better the Mercedes assistant is than Lucid - would be nice to see a more capable release of this for Air and Gravity soon.
 
And this is not everyone on this forum?
No.

People want to believe what they want to believe. The difference is that there are others who have followed this for a long time without some kind of bias about what is the "right" or "wrong" way.
1794902133614.gif
 
FYI, when Cosmos gets released, it will be on the top 3-5 cars of processing power. Gravity, Teslas and Merc are going to look like a joke in comparison.

Mercs have processing power of 254 TOPS while Cosmos has 2000 TOPS, night and day difference
 
Sure, memory bandwidthb is important, but it isn't the end-all be-all.

There are serious drawbacks to the way the Tesla system is designed, and in some ways they are brute forcing their way to the current level of FSD.
This. Tesla has to use a lot more memory than people would think because it has no depth sensors, doesn't leverage HD maps and doesn't have ultra-precise GPS.
 
Also, Nvidia released their distilled VLA model that operates at 10 hz (refreshes 10x per second) in vehicles. While that may seem slow, that is just the vehicle "planner", with a separate, low latency perception/safety loop.
 
Mercs have processing power of 254 TOPS while Cosmos has 2000 TOPS, night and day difference
Tesla AI5 has around 2000 for a single chip. It is a half reticle design, so when it comes to the car if they use 2 of them, that's more than 4000.

Memory bandwidth is about 8x what nvidia thor has as there are 12 memory modules per chip. 72gb total memory x 2

his. Tesla has to use a lot more memory than people would think because it has no depth sensors, doesn't leverage HD maps and doesn't have ultra-precise GPS.

Like literally everybody else. Everybody is doing depth estimation from video. Waymo does this. Every self driving car on earth does this. Now you have to use lidar as well. That's more compute, more memory bandwidth, not less.

people forget the lidar in consumer cars is not high res and has low FOV. That means it covers a very small part of your FOV and you can't even see cross traffic with this lidar. It also has similar failure modes to cameras and can produce a noisy input. And these E2E neural networks already process video.

There's a reason wayve drives pretty much using vision only, despite claiming their tech is sensor agnostic.

While that may seem slow, that is just the vehicle "planner", with a separate, low latency perception/safety loop.
What we are talking about is the total time it takes to process an input and send a command to the car. The less memory bandwidth you have (the more you use for sensors), and running VLA models means that your output latency is very, very high.

High latency means bad reaction time, which prevents you from braking for a pedestrian that pops out from behind a car.

You can see with the chinese systems they have very slow reaction time and nonexistent accident avoidance.

The slower you react the more obvious mistakes are when the car is driving like a tourist. Tesla drives with very incorrect map data, they just fix it in real-time so quickly you never notice these mistakes.

The other simple thing which people don't realize is the output control (steering/brake/throttle) is actually forward predicted to the moment these controls are actually sent to the car (vs. when the sensor inputs are received). The higher the latency, the more likely there are going to be errors in the command you are sending to the car.

This is actually the cause of a lot of lane centering systems "ping-ponging" across the lanes. It's because latency is so high, that the position the car is in when it sends the steering command is wrong and then the car is constantly overcorrecting.
 
Last edited:
Everybody is doing depth estimation from video. Waymo does this. Every self driving car on earth does this. Now you have to use lidar as well. That's more compute, more memory bandwidth, not less.
Yes, but others are also using the direct distance from Radar and LiDAR.

people forget the lidar in consumer cars is not high res and has low FOV. That means it covers a very small part of your FOV and you can't even see cross traffic with this lidar. It also has similar failure modes to cameras and can produce a noisy input. And these E2E neural networks already process video.
The failure modes between optical, radar and LiDAR are not the same. In fact this is an advantage for multiple types of senors.

Comparing optical and LiDAR:
Optical resolution degrades at night where LiDAR does not.
Strong sunlight degrades optical cameras but much less so for LiDAR.
Pedestrian wearing dark clothes at night are difficult for cameras due to reduced contrast. LiDAR does not suffer from this problem
Both optical and LiDAR are degraded by heavy rain, fog and dust storms.
Optical cameras have better resolution than LiDAR.

Radar has lower resolution than Optical or LiDAR but has an advantage in rain, fog and dust storms.
 
Comparing optical and LiDAR:
Optical resolution degrades at night where LiDAR does not.
Except camera detection is already much higher res than lidar
Strong sunlight degrades optical cameras but much less so for LiDAR.
As if strong sunlight is really a problem? I've never seen any evidence of robotaxi having any issue with strong sunlight. With a properly cleaned windshield I've seen no evidence of any version of modern FSD having an issue with sun glare. There has never been a failure of modern FSD since 2024 (e2e neural networks) that has been a failure of HW4 cameras but a failure of camera placement, or them simply being dirty.

With camera cleaning and geofencing (which tesla robotaxis have and consumer cars do not have) you can remedy this.

There is zero evidence of HW4 FSD cameras having any failure in fog, heavy rain, night time etc. at any levels that is a failure of the camera unable to see something. There are object classification issues like say identifying a pothole.

Pedestrian wearing dark clothes at night are difficult for cameras due to reduced contrast. LiDAR does not suffer from this problem
But then you need to classify objects (hard to see) with lidar only. Identifying pedestrians from lidar only is very expensive. Waymo does object classification with lidar but it is not cheap.
Radar has lower resolution than Optical or LiDAR but has an advantage in rain, fog and dust storms.
Yes this is correct

Yes, but others are also using the direct distance from Radar and LiDAR.
But at the cost of extra memory bandwidth utilization, 2-3x the TOPS requirement for early sensor fusion, and higher latency

Which the problem is you have a theoretical increase in accident avoidance (only at the edge case level that will always improve with cameras) while providing a non-theoretical decrease in safety due to higher latency.

The other thing is that with vehicle to vehicle communication, the difference with lidar/camera will be even less as vehicles ahead will warn of obstacles miles before you get to them

What you're also failing to understand is that robotaxis use lidar placed HIGH above the vehicle. Like a hat. This allows you to see over all cars and track pedestrians from behind a vehicle before they pop out. This reduces the need for low latency but at the cost of requiring lots of compute for independently classifying objects from lidar and having your vehicle look "ugly."

Consumer solid state lidars are sleek but they are low res, low FOV and not positioned for performance, but purely aesthetics
 
Last edited:
Optical resolution degrades at night where LiDAR does not. Except camera detection is already much higher res than lidar
You fail to see the point and advantage of complimentary sensors is that each have strengths and weaknesses and having multiple sensor types minimizes the weaknesses.
There is zero evidence of HW4 FSD cameras having any failure in fog, heavy rain, night time etc. at any levels that is a failure of the camera unable to see something.
Are you saying that rain and fog do not degrade optical cameras? Maybe you should study light reflection from weather. The fact is that camera images do degrade in fog, rain and dust. The question is how much degradation is acceptable and safe.
But then you need to classify objects (hard to see) with lidar only. Identifying pedestrians from lidar only is very expensive. Waymo does object classification with lidar but it is not cheap.
I never said that you need to classify objects with just LiDAR. I am not advocating a LiDAR only system, just pointing out where LiDAR has a physical advantage over cameras.
The other thing is that with vehicle to vehicle communication, the difference with lidar/camera will be even less as vehicles ahead will warn of obstacles miles before you get to them
I was not aware that this was available yet.
What you're also failing to understand is that robotaxis use lidar placed HIGH above the vehicle. Like a hat. This allows you to see over all cars and track pedestrians from behind a vehicle before they pop out. This reduces the need for low latency but at the cost of requiring lots of compute for independently classifying objects from lidar and having your vehicle look "ugly."
You have no clue to what I understand or do not understand. Please tame your arrogant attitude, it is not polite. Early fusion versus late fusion is an architecture decision. Either can be done with multiple sensor types.
 
You fail to see the point and advantage of complimentary sensors is that each have strengths and weaknesses and having multiple sensor types minimizes the weaknesses.
The way I see it is if you have a system with 20 sensors, and with early sensor fusion it takes about 250ms total latency, and a system with a handful of cameras has 120ms of latency, you drive around with a net decrease in safety 99.99% of the time because faster reaction will be safer than being slow. The other problem is that 0.001% of the time the actual decrease in accidents is not quantifiable because you can overcome limitations with fast reaction, and more importantly cameras will always improve. And the fact that the majority of the time your system is slower.

On the other hand, when you drive at lower speeds (in say snow driving) then the increase in latency is more acceptable and so it is useful to have both. There's always going to be tradeoffs to every approach. Late sensor fusion works better for identifying independent objects from one sensor only and is more computationally efficient.

Late sensor fusion does not work well with the low quality sensors and placement found in consumer cars.

At parking lot speeds, more sensors does increase safety to a point, but not once you start moving faster than 10mph because reaction does matter here. But for taking off from park, tesla would benefit from more sensors around and underneath the car.
Are you saying that rain and fog do not degrade optical cameras? Maybe you should study light reflection from weather. The fact is that camera images do degrade in fog, rain and dust. The question is how much degradation is acceptable and safe.
As far as now, robotaxi does not shut down for seemingly any rain levels. For what it's worth tesla has billions of miles to base their decision on what rain levels are acceptable. Fog driving also seems fine but it is not part of robotaxi's ODD declaration in Texas. So at this moment rain is not a problem, but tesla does seem to have limits on fog which is not exactly clear where those limits are.


Waymo has never operated in this level of rain. Their software doesn't currently allow it


I never said that you need to classify objects with just LiDAR. I am not advocating a LiDAR only system, just pointing out where LiDAR has a physical advantage over cameras.

I was not aware that this was available yet.
You have to do this to do the things you are suggesting with lidar. Robotaxis do this everyday. Waymo does this. Zoox does this.
 
Last edited:
FYI, when Cosmos gets released, it will be on the top 3-5 cars of processing power. Gravity, Teslas and Merc are going to look like a joke in comparison.

Mercs have processing power of 254 TOPS while Cosmos has 2000 TOPS, night and day difference
Other manufacturers will be increasing compute power as time goes on... We'll see what everyone is doing with their mid 2027 vehicles
 
Yes, but others are also using the direct distance from Radar and LiDAR.


The failure modes between optical, radar and LiDAR are not the same. In fact this is an advantage for multiple types of senors.

Comparing optical and LiDAR:
Optical resolution degrades at night where LiDAR does not.
Strong sunlight degrades optical cameras but much less so for LiDAR.
Pedestrian wearing dark clothes at night are difficult for cameras due to reduced contrast. LiDAR does not suffer from this problem
Both optical and LiDAR are degraded by heavy rain, fog and dust storms.
Optical cameras have better resolution than LiDAR.

Radar has lower resolution than Optical or LiDAR but has an advantage in rain, fog and dust storms.
On a slight tangent I've also read some interesting papers that took inspiration from the Transformer architecture for sensor fusion
 
Please share the links.
Sure, here are the original papers that got me down that rabbit hole:






Though be aware these are pretty old papers, so I wouldn't be surprised if the state of the art has change drastically since then
 
If you think about rain driving for a second, let's imagine the worst case scenario. You are driving at night 50mph, and the heaviest rain possible. Your camera image is full of noise (because it's dark), and everything is very distorted. There's a human standing on the freeway.

if you take the image from the clearest of the two front facing cameras, demosaic it, apply noise reduction and gain the image up (so it's very clear) go to the point where the human is most visible. If you label the human in reverse, slowly (as the car is getting very far away), then you can clearly see where the human is in the more distorted rain clip as well. Take these two clips, invert the labeling so that it would be applied to the non-demosaiced, non-denoised raw sensor data, which to a human looks like nonsense at this point. This is the clear advantage of training on videos. You label the videos in reverse, always starting from the clearest point

The neural network doesn't care and across 10s of thousands of examples of these clips (which you can only get from large scale data collection), the neural network learns what the camera noise looks like and filters it out, it learns how to see past these distortion patterns in the windshield, etc.

So what looks like nonsense to a human the AI is very good at seeing past

That's why if you ever use FSD in the heaviest possible rain at night, you notice no degradation in performance. It's kind of alarming how well the car already sees.
 
I'll say this about the arguments around ADAS. Tesla's FSD, no matter how ahead it is right now, is not a moat for the company. All you have to do is look at the timelines of the major competitors. Google started their self-driving project in 2009. It took them until 2015 to produce that first 2-seat self-driving bubble vehicle. Public trials started in 2017. Tesla announced full self driving capabilities (not even officially named FSD) in 2016. Beta release in 2020 but current capabilities only possible with HW4 released in 2022. Nvidia launches Drive platform in 2015 having already spent an unknown number of years beforehand on it. First version of Alpamayo is launched in 2024. Alpamayo 2 is launched this year with point to point set for launch in 2027.

Timelines are shrinking thanks to increases in processing power and model development. Once something is shown to be possible, it opens the door for others to follow at a faster pace.

The automobile industry is too large for one company to dominate it. There's too much money in this for Tesla to take it all. In the end, this is just technology, and history has shown again and again that technology will get cheaper and better faster than people think.
 
Timelines are shrinking thanks to increases in processing power and model development. Once something is shown to be possible, it opens the door for others to follow at a faster pace.
This ignores the fact that the safety of all these systems is not equivalent. Many of these companies have been working on it as long as FSD and their systems are not that good.

if it is not a data collection or training cost issue, the evidence does not show that. It's edge case safety that matters. Driver assist is one thing but driving unsupervised is entirely different
The automobile industry is too large for one company to dominate it. There's too much money in this for Tesla to take it all. In the end, this is just technology, and history has shown again and again that technology will get cheaper and better faster than people think.
The problem is the cost for unsupervised driving doesn't simply work on a small scale.

it costs 50 billion all-in to develop a bespoke self driving system, and then you're probably look at 10 billion a year to maintain it. Perfect, real-time maps of the U.S. with road closures updated by the minute, vehicle to vehicle communication, a service center within 30 miles of every part of the U.S. (to enable monthly vehicle inspections/service), 10,000 support agents to support millions of cars driving all at once, as perfect as possible connectivity (starllink?), etc.

you can spend 50 billion developing a self driving system that simply doesn't work (VW spent billions on adas with nothing to show for it).

Even if you have a tech that works, individual automakers simply don't have the money for this. If you're 4 years behind, it's not exactly fast to scale to millions of cars, and the part that is actually hard cannot be shortcutted in the way software can.

you'll always be behind in monthly cost, featureset, odd, safety, etc.
 
Except camera detection is already much higher res than lidar
And much like memory bandwidth, camera resolution is not the end-all be-all of object recognition. All the pixels in the world won't make a difference if it's a picture of a black thing in front of a black background.

If you can't even acknowledge the concept that multiple sensor types have the potential to reliably detect a wider range of objects across a side range of environmental situations, you simple prove your lack of critical thinking.

Yes, it may require more computing power to integrate the different data, but that doesn't mean it's not worth working towards. The computing power WILL catch up to the needs, if not today, then tomorrow, and self driving systems WILL be better for it.

There's literally no valid argument against that statement that doesn't involve the complete cessation of technological improvements.

The way I see it is if you have a system with 20 sensors, and with early sensor fusion it takes about 250ms total latency, and a system with a handful of cameras has 120ms of latency, you drive around with a net decrease in safety 99.99% of the time because faster reaction will be safer than being slow.

Sure, I can make up numbers too. Doesn't make it a valid argument.
 
Back
Top