Having worked in software development and QA for over 30 years in some of the leading software and hardware development companies, I would NOT expect QA to catch the issue we’re discussing.
That’s not to say that QA can’t make a suggestion or log an issue with the design team. As we have all observed how this works, yes it’s poor, but I bet it works as designed. Yes it should be redesigned by the design/development team. But to say that QA/validation should have caught it is not correct.
QA and development often have arguments as to what’s a bug and what isn’t. In the end, a group such as product management has the final say as to how a feature should work.
QA and Validation are two separate but related subjects.
Let me use some straight-forward (though not all encompassing) examples to illustrate.
QA: when I go pick up my Lucid on delivery day, if there are scratches on the paint, mis-aligned panels, trunk mats that are not properly secured (which happened in my case), those are QA issues. In other words, if Lucid inspected the car carefully before delivery, they would have picked up on these defects and corrected these issues before the customer arrives to take delivery. These are random defects, not systematic issues.
Validation: this is a completely different animal. First, there are multiple levels of validation. Let’s start with SW and HW. Each one has to be validated (i.e., testing the functionality against the specification) on its own. Thereafter, the SW and HW have to be tested together, under a range of circumstances (e.g., at different temperature, humidity, etc.) to make sure they work together as intended.
Let’s use the Homelink garage door opener as an example. So, there is a “module” (HW and SW) that performs the Homelink function (programming the garage door open code, multiple users, multiple garage doors, multiple brands of garage door openers, temperature extremes, etc.) . Let’s just say the engineers tested all of these and the Homelink Module performed faultlessly. Is the Homelink module therefore “Validated”? Yes, validated at the module level (i.e., if that’s the only standalone function), So, is it good-to-go. NO!. We still don’t know if it will it work in conjunction with all the other functions (modules) in the rest of the car under all circumstances.
As many of us (myself included) experienced, the Homelink module does not work properly when the Lucid is in reverse backing out of the garage (even though Homelink works perfectly on its own). The reason why this happens is (likely) Lucid might not have tested these two functions (Homelink and backing up) in conjunction (i.e., as a system). Now, we have a “bug”, an unanticipated interaction between two independent functions (Homelink and backup) that interacted in a manner we did not anticipate.
OK, so what! Now we know this unintended interaction, let’s fix this bug and move on. (Let’s say) We did an OTA , now it is fixed. But, is it? What happens if I try to close the garage door with Homelink while backing up at the same time, but it is raining and it is dark, so I have my headlights on and the wipers going., will the “bug-fix” for the Homelink-Backup still work? The answers is, I have no idea! The unintended interaction between independent functions is virtually impossible to anticipate and they generate scenarios that cannot be easily predicted.
Hence, we have to do SYSTEM VALIDATION…i.e., test the SYSTEM (the whole car, all its functions, at all kinds of systematic and random sequences, under different environmental conditions, temperature, rain, night/day, etc.). It this kind of testing sounds very arduous, you got it right, it is!
System manufactures (computers, airplanes, cars, space crafts, etc.) have learned that even though each individual module works faultlessly on their own, there is no guarantee they will work harmoniously TOGETHER. More likely than not, they won’t. That’s why we do SYSTEM VALIDATION. It is hard work arduous, and it is a grind to run all those regressions. And it is a never-ending task. Many a times, we are luck enough we could fix some of these unintended interactions with SW fixes (OTA). That’s great. SW fixes are easy to implement as opposed to hardware fixes. Many a times, people get confused and call them Quality issues, they are not! These are architecture/design/system bugs that were inadvertently baked into the system. We call them “bug”/”defects” because they manifest themselves “randomly”. In reality, they are not random, though they are hard to pin down. They only manifest themselves when the stars and the moon align.
I don’t want to come across as a pessimistic doomsayer. I just want us to be realistic. Building a world-class, reliable electro-mechanical-computerized system on wheels going at 70-80 mph is not for the faint of heart. A lot more work has to be done in validation to get the Lucid system to be robust.
