Firmware Over-the-air (FOTA) Is Hard

metalpro2021

Well-Known Member
First Name
Chris
Joined
Apr 5, 2021
Threads
11
Messages
138
Reaction score
112
Location
Amsterdam
Vehicles
Mach-e 1st edition
Occupation
Software architect
Country flag
My 2 cents as a 'more than knowledgeable person on the matter' : it is not so easy as some suggest. But, if the delivery-foundation is well-designed (and not set up in a hurry....) it is manageable. However, based on the update-stories 'going around' I get the feeling that a lot of cars are running different versions of the modules which require ' firmware’ updates. Another feeling I am having: there SEEMS to be version dependency on different modules which should have been designed to work independently. In other words: module A requires module B to have version Z. Things get really complicated from there, because if you want to do major updates (like blue cruise), you have to control an indefinite number of upgrade paths(because every car can have an unique set of versions for all the modules) for the different modules which can fail individually.....
Sponsored

 

mpshizzle

Well-Known Member
Joined
Mar 22, 2021
Threads
65
Messages
1,274
Reaction score
1,589
Location
Utah
Vehicles
Mustang Mach E 4X
Country flag
My 2 cents as a 'more than knowledgeable person on the matter' : it is not so easy as some suggest. But, if the delivery-foundation is well-designed (and not set up in a hurry....) it is manageable. However, based on the update-stories 'going around' I get the feeling that a lot of cars are running different versions of the modules which require ' firmware’ updates. Another feeling I am having: there SEEMS to be version dependency on different modules which should have been designed to work independently. In other words: module A requires module B to have version Z. Things get really complicated from there, because if you want to do major updates (like blue cruise), you have to control an indefinite number of upgrade paths(because every car can have an unique set of versions for all the modules) for the different modules which can fail individually.....
This may be an ignorant thought, but FDRS seems to handle these dependencies and version differences just fine. So if that isn't a problem for an external system on a laptop, I can't see why using an internal system to do the same thing is any different
 

metalpro2021

Well-Known Member
First Name
Chris
Joined
Apr 5, 2021
Threads
11
Messages
138
Reaction score
112
Location
Amsterdam
Vehicles
Mach-e 1st edition
Occupation
Software architect
Country flag
The number of people / dealerships doing FDRS is far lower than all the cars out there. Maybe the FDRS is not able to update all different scenarios as well. We simply do not know. We have heard stories of dealerships not able to update through FDRS have not we?
 

mpshizzle

Well-Known Member
Joined
Mar 22, 2021
Threads
65
Messages
1,274
Reaction score
1,589
Location
Utah
Vehicles
Mustang Mach E 4X
Country flag
The number of people / dealerships doing FDRS is far lower than all the cars out there. Maybe the FDRS is not able to update all different scenarios as well. We simply do not know. We have heard stories of dealerships not able to update through FDRS have not we?
As far as I've seen, the only trouble with the dealerships is the techs are afraid to do the updates/refusing to do them. That and the OTA updates aren't made available in FDRS until later.

If you actually get to FDRS though, I think success rate is near 100%

Having performed FDRS updates myself, it handles dependencies very well. If you choose to perform an update that has dependencies, it automatically downloads and installs all of the modules software in the right order.
 

Jimrpa

Well-Known Member
First Name
Jim
Joined
Sep 10, 2020
Threads
231
Messages
7,119
Reaction score
9,478
Location
Wayne, PA
Vehicles
2021 Infinite Blue Premium Mustang Mach E ER AWD
Occupation
Retied (formerly tried to herd highly technical, independent cats)
Country flag
My 2 cents as a 'more than knowledgeable person on the matter' : it is not so easy as some suggest. But, if the delivery-foundation is well-designed (and not set up in a hurry....) it is manageable. However, based on the update-stories 'going around' I get the feeling that a lot of cars are running different versions of the modules which require ' firmware’ updates. Another feeling I am having: there SEEMS to be version dependency on different modules which should have been designed to work independently. In other words: module A requires module B to have version Z. Things get really complicated from there, because if you want to do major updates (like blue cruise), you have to control an indefinite number of upgrade paths(because every car can have an unique set of versions for all the modules) for the different modules which can fail individually.....
That’s exactly what I was getting at when I spoke of “collections”. Yes, it makes sense.
 


Chuck

Well-Known Member
First Name
Chuck
Joined
May 13, 2021
Threads
64
Messages
988
Reaction score
1,397
Location
SoCal
Vehicles
Mustang Mach E 2021
Country flag
The issue is not that OTA is hard. It is the fact that ford is radio silent on the issue. Zero communication. They also have miss multiple deadlines they originally announced and us early adaptors are feeling really screwed.
Here, I'll break the silence:

"Hi, there folks. We're in the process of developing some great features for your Mach E. In fact, they're all ready to go but we have this one little problem. Should it happen that your vehicle gets bricked after an update, and we admit that this is a rare thing but could happen, you will need to have your car towed to a dealer to get fixed. "

"But here's the problem: A lot of dealers don't like electric cars. They see themselves as being cut out of the lucrative repair business because electric cars take much less maintenance work. Many refuse to get trained in repairs and they don't want to support this path. So, if your car gets bricked your local dealer may not know how to fix it and, worse, may not want to fix it. Your car could sit on their lot for months. And you can bet they'll make up a lot of excuses mostly blaming Ford for the problem."

"So, folks, we're going to make sure these OTA updates work. Every time. We're going to roll it out slowly and the areas where the Ford dealers cooperate will be the ones getting them first. Sorry, but if the dealer you marked as your "favorite" isn't a cooperative one then you're pretty much screwed."

So, is that what you want them to communicate? You can bet your bottom dollar there's a war going on between Ford and dealers right now.
 

Trick.Mach-E

Well-Known Member
First Name
Trick
Joined
Mar 8, 2021
Threads
6
Messages
291
Reaction score
502
Location
Earth
Vehicles
Old Raptor, Mach-E4x, New Raptor, F-250, Taycan.
Occupation
Computer Nerd
Country flag
This is the way I understand how Ford does things after 3+ years of working with doing software/firmware updates along with using unsupported firmwares, etc on various modules. My business revolves around this.

I don't think it is as hard as that article leads us to believe...I think Ford is just being extra safe to make sure they have all of their bases covered to avoid failed updates causing people to freak out or go to dealers prematurely. The gateway module acts like FDRS and a VCM. It downloads the update files and flashes them. I'm not saying this is simple, but I'm saying they're being EXTRA conservative.

The way Ford does modules you get the firmware and software in the same package in *most* modules. The only exception I know of for a fact is the APIM. It has a separate firmware and software package.

When a Ford module is updated it does the secondary boot loader first, followed by firmware and then software files. When a PMI is done after an update it configures the as built data, VIN, etc - which is generally stored separately with other configuration information in an EEPROM on the module.

For erase and replace modules if the update fails mid update it will be bricked....temporarily. All you have to do is do the firmware/software flash again and complete it successfully. If you put the wrong files on you can simply re-flash. The main boot loader is designed to allow you to always be able to flash new software/firmware even if it is corrupted.

For ABA swap modules if it fails it will not even try to boot that firmware and the module will still work correctly. These are not risky to try and update OTA since a failed flash wouldn't cause a problem. Both firmware AND software are on separate ROMs and can independently run the module.
Having programmed 16 VCR's in the 80's to stop the flashing 12:00 I can relate to what you said. Totally legit!

Ford's approach to the updates is frustrating but seeing the reaction of 21,000+ people on a 23,000+ computer network when there are problems after various updates I can appreciate Ford's concern.
 

GoGoGadgetMachE

Well-Known Member
First Name
Michael
Joined
Jan 23, 2020
Threads
153
Messages
5,614
Reaction score
12,656
Location
Ohio
Vehicles
2021 Mach-E 1st Ed., 2022 Lightning Platinum
Occupation
Professional forum cheerleader and fanboy
Country flag
If it stays hard for over 4 hours, go see a doctor.
Where's that ticket book? It's around here somewhere...
Sponsored

 
 




Top