hack-e

Well-Known Member
First Name
Mark
Joined
Mar 5, 2022
Threads
14
Messages
892
Reaction score
782
Location
Austin, TX
Vehicles
2022 Prem AWD ER MME
Occupation
Robotics Engineer
Country flag
Are you sure the SVS is due to the HVBJB code(s)?
Yes, I have the first DTC listed over in the other thread. And I just tested while driving home and indeed I actually do have reduced power. I was mistaken.
Sponsored

 
OP
OP
RobbertPatrison

RobbertPatrison

Well-Known Member
Joined
Mar 24, 2022
Threads
13
Messages
187
Reaction score
492
Location
Silicon Valley, California
Vehicles
Ice White Mustang Mach-E
Country flag
The heat is extracted through the contact terminals. The bus bars in the HVBJB act like heat sinks. This was discussed recently on another topic, but it’s my opinion that the bus bar near the contactors is not thick enough to remove enough heat. It’s only 3 mm thick and I think it should be 5+ mm thick to provide enough thermal mass. Tesla puts very thick busbars on their contactors (6+mm?) for comparison:

Tesla Model Y contactors, note thick busbars:

s-l1600-3.webp


s-l1600-2.webp



I’ve measured a failed Mach-E contactor at 5+ mΩ, which is very high and would generate a lot of heat. If anything, the recall software was not sensitive enough, and could only detect a failing contactor under a high amp draw, such as full throttle acceleration for several seconds. It may not detect a failing contactor if you never floor the vehicle or only do so briefly.

I think another factor besides the HVBJB design was the part quality. Some of these original contactors just decide they’re going to stick even though they’ve lived an easy life and haven’t necessarily ever been overheated. No software will detect that ahead of time.

It’s my theory the new contactors switched to using Inconel or other high heat resistant contact material, and were hopefully subjected to more rigorous initial quality testing to exclude defects.

I would like to see a new HVBJB with thicker bus plates and a switch to a different contactor manufacturer such as Gigavac instead of TE.
Interesting, thanks for the details. 5 mOhm is 10x the rated worst resistance, which will obviously fry the part. Thicker bus bars is the way to go, not only for heat transfer but also for lower resistance. Mach-lee’s excellent analysis shows a solid BVBJB design but the bars seem on the thin side indeed.

Looking on the TE product page, I cannot find an alternative contactor with a higher rating than 500A and in the same form factor. Internally it is the same thing with a springs and rod that makes contact. Is there a different part number and it is rated for more than this EVC 500 part ? It could indeed be sloppy manufacturing causing defects, but at their core relays are just very simple things with few variations on the contact side.
 

Tom L

Well-Known Member
First Name
Tom
Joined
Nov 5, 2021
Threads
7
Messages
391
Reaction score
418
Location
Chippewa Falls, WI
Vehicles
2018 Honda Clarity
Occupation
Retired
Country flag
The heat is extracted through the contact terminals. The bus bars in the HVBJB act like heat sinks. This was discussed recently on another topic, but it’s my opinion that the bus bar near the contactors is not thick enough to remove enough heat. It’s only 3 mm thick and I think it should be 5+ mm thick to provide enough thermal mass. Tesla puts very thick busbars on their contactors (6+mm?) for comparison:

Tesla Model Y contactors, note thick busbars:

s-l1600-3.jpg


s-l1600-2.jpg



I’ve measured a failed Mach-E contactor at 5+ mΩ, which is very high and would generate a lot of heat. If anything, the recall software was not sensitive enough, and could only detect a failing contactor under a high amp draw, such as full throttle acceleration for several seconds. It may not detect a failing contactor if you never floor the vehicle or only do so briefly.

I think another factor besides the HVBJB design was the part quality. Some of these original contactors just decide they’re going to stick even though they’ve lived an easy life and haven’t necessarily ever been overheated. No software will detect that ahead of time.

It’s my theory the new contactors switched to using Inconel or other high heat resistant contact material, and were hopefully subjected to more rigorous initial quality testing to exclude defects.

I would like to see a new HVBJB with thicker bus plates and a switch to a different contactor manufacturer such as Gigavac instead of TE.
This is further evidence that Mach-Lee is technological wizard from another planet secretly sent to help us as we bumble and fumble our way into a transportation revolution.
 

DevSecOps

Well-Known Member
First Name
Todd
Joined
Sep 22, 2021
Threads
69
Messages
4,764
Reaction score
11,624
Location
Sacramento, CA
Vehicles
'21 Audi SQ5 / '23 Rivian R1T / '23 M3P
Occupation
CISO
Country flag
Rather, it attempts to detect the deterioration of the contactors by measuring increased resistance as a sign of trouble. When higher resistance is detected, the MME will go into a 50% low-power mode and the MIL light will switch on.
While resistance is one component of what's monitored, the SOBDMC is also supposedly monitoring temperatures. Additionally, it reads as if the temperature is the first trigger while resistance is the second.

This is per Ford's communication with Federal Investigators. I'll quote directly from NHTSA:
"The remedy in this recall was a Secondary On-Board Diagnostic Control Module (SOBDMC) software update to monitor contactor temperature and reduce battery power to prevent damage to the contactor, and a Battery Energy Control Module (BECM) software update to monitor contactor resistance to identify an overheated contactor and reduce vehicle power to prevent further damage."
 

dtbaker61

Well-Known Member
First Name
Dan
Joined
May 11, 2020
Threads
126
Messages
4,822
Reaction score
4,524
Location
santa fe,nm
Website
www.envirokarma.org
Vehicles
MME (delivered 2/26/21), DIY eMiata BEV
Occupation
Solar Sales/install
Country flag
This is of the 1st generation, but the second generation contactor looks exactly the same. The issue is to extract the inevitable heat energy that occurs with high current. If it has nowhere to go due to a lack of heat conductivity, it will fail again. As far as I can see from the teardown by Weber Auto, there is no cooling in this part of the battery box. If that issues remains unaddressed, I am afraid that the HVBJB will continue to fail in all MMEs.
The failure probability depends on how hard the car is driven.
exactly.

there is no sensor I know of in proximity to the Contactor(s). The closest ones I think are showing up as the AC and DC charger temps basically on the same bus bar, but physically outside the HVBJB.

until there is improved cooling inside or under the HVBJB and contactors (like a battery 'chill plate' with a little additional plumbing of coolant on the way to the battery chillers), and a local sensor to provide direct feedback for any required current limiting I expect we will continue to see failures.... and programmatically limited current under 'certain conditions' which may be modified by OTA changes but don't really fix the issue.
 


ChasingCoral

Well-Known Member
First Name
Mark
Joined
Feb 3, 2020
Threads
502
Messages
14,306
Reaction score
28,653
Location
Maryland
Vehicles
2021 GB E4X FE, 2022 F-150 Lightning Lariat ER
Occupation
Retired oceanographer
Country flag
From the new text, it is clear that Ford admits that this strategy does not work well. Too many vehicles still fry their contactors without the software detecting increased resistance beforehand. This is not surprising IMHO. Ford does think that the regular battery with its lower current rating reduces the probability of a fried HVBJB.
I'll give Ford more credit than you are. I think the strategy has worked in that the 22S41 software fix is preventing complete failures. Better to limp home than completely lose power on the road.

I am not saying it is a perfect fix but my understanding is Ford did replace the contactors with a more robust contactor. I also understand many now are just rebuilds of the old parts. Also everyone realized that the recall was not a fix of the problem but that Ford thought that it would provide a soft landing and allow owners to continue driving and then service the vehicle. In the vast majority of issues reported on the forums that was the case. ...
Agreed. That is what happened to me: wrench thrown, power reduced, limped home.

... Obviously that was not the case for all issues and some continued to fail as a SSN vs the soft landing SVS. The new recall appears to contain an update to the software as well as a proactive change out of the HVBJB. My guess is that will significantly reduce if not eliminate the HVBJB issue.
Yes, it looks like part of the next remedy will include software adjustments to other modules beyond what was included in 22S41.

ooh boy I'm almost at 22k lol
Mine was just over 30,000.

Since this an EV component, warranty is 8 years/100,000 miles. It will be some time before there are significant post-warranty failures.
That's good for those of us with regular use patterns. A few cars have already rolled over 100,000 miles. For them, the recall provides a safety net.

Marlin just got a new HVBJB. The recall provides an alternate HVBJB resolution in addition to the warranty, probably even after reaching the end of the warranty, depending on how the remedy reads.
 

ChasingCoral

Well-Known Member
First Name
Mark
Joined
Feb 3, 2020
Threads
502
Messages
14,306
Reaction score
28,653
Location
Maryland
Vehicles
2021 GB E4X FE, 2022 F-150 Lightning Lariat ER
Occupation
Retired oceanographer
Country flag
exactly.

there is no sensor I know of in proximity to the Contactor(s). The closest ones I think are showing up as the AC and DC charger temps basically on the same bus bar, but physically outside the HVBJB.

until there is improved cooling inside or under the HVBJB and contactors (like a battery 'chill plate' with a little additional plumbing of coolant on the way to the battery chillers), and a local sensor to provide direct feedback for any required current limiting I expect we will continue to see failures.... and programmatically limited current under 'certain conditions' which may be modified by OTA changes but don't really fix the issue.
Agreed, although thicker busbars might suffice.
 

helium89

Well-Known Member
Joined
Mar 1, 2022
Threads
4
Messages
89
Reaction score
122
Location
Colorado
Vehicles
Forester
Country flag
I'll give Ford more credit than you are. I think the strategy has worked in that the 22S41 software fix is preventing complete failures. Better to limp home than completely lose power on the road.
The whole reason for this new recall is that 22S41 doesn’t prevent complete failures. Our car is currently in the shop waiting for a new HVBJB after experiencing a complete failure. Enough people with similar experiences filed complaints with the NHTSA that they opened an investigation, and Ford is now being forced to do what they should have done in the first place.
 

ChasingCoral

Well-Known Member
First Name
Mark
Joined
Feb 3, 2020
Threads
502
Messages
14,306
Reaction score
28,653
Location
Maryland
Vehicles
2021 GB E4X FE, 2022 F-150 Lightning Lariat ER
Occupation
Retired oceanographer
Country flag
The whole reason for this new recall is that 22S41 doesn’t prevent complete failures. Our car is currently in the shop waiting for a new HVBJB after experiencing a complete failure. Enough people with similar experiences filed complaints with the NHTSA that they opened an investigation, and Ford is now being forced to do what they should have done in the first place.
I think we're just splitting hairs here. I agree 22S41 was not sufficient. However, it appears 22S41 dramatically improved the odds of a soft landing rather and limp home.
 

Mach-Lee

Well-Known Member
First Name
Lee
Joined
Jul 16, 2021
Threads
262
Messages
11,344
Reaction score
24,963
Location
Wisconsin
Vehicles
2022 Mach-E Premium AWD
Occupation
Sci/Eng
Country flag
While resistance is one component of what's monitored, the SOBDMC is also supposedly monitoring temperatures. Additionally, it reads as if the temperature is the first trigger while resistance is the second.

This is per Ford's communication with Federal Investigators. I'll quote directly from NHTSA:
"The remedy in this recall was a Secondary On-Board Diagnostic Control Module (SOBDMC) software update to monitor contactor temperature and reduce battery power to prevent damage to the contactor, and a Battery Energy Control Module (BECM) software update to monitor contactor resistance to identify an overheated contactor and reduce vehicle power to prevent further damage."
Except the first part is impossible because there are no temp sensors in the HVBJB. They can try to model it, but due to part variation a thermal model could be way off. As far as I’m concerned, they only implemented the second part of that statement.

Using a thermal model won’t be fun either, means we will be subject to more derating and jail bars.
 

DevSecOps

Well-Known Member
First Name
Todd
Joined
Sep 22, 2021
Threads
69
Messages
4,764
Reaction score
11,624
Location
Sacramento, CA
Vehicles
'21 Audi SQ5 / '23 Rivian R1T / '23 M3P
Occupation
CISO
Country flag
Except the first part is impossible because there are no temp sensors in the HVBJB. They can try to model it, but due to part variation a thermal model could be way off. As far as I’m concerned, they only implemented the second part of that statement.
I know you've said that many times before, but I'm just quoting what they apparently told NHTSA. If Ford lied to them, well that's on them. Just kinda strange that they even list the module supposedly doing the temperature monitoring.
 
OP
OP
RobbertPatrison

RobbertPatrison

Well-Known Member
Joined
Mar 24, 2022
Threads
13
Messages
187
Reaction score
492
Location
Silicon Valley, California
Vehicles
Ice White Mustang Mach-E
Country flag
While resistance is one component of what's monitored, the SOBDMC is also supposedly monitoring temperatures. Additionally, it reads as if the temperature is the first trigger while resistance is the second.

This is per Ford's communication with Federal Investigators. I'll quote directly from NHTSA:
"The remedy in this recall was a Secondary On-Board Diagnostic Control Module (SOBDMC) software update to monitor contactor temperature and reduce battery power to prevent damage to the contactor, and a Battery Energy Control Module (BECM) software update to monitor contactor resistance to identify an overheated contactor and reduce vehicle power to prevent further damage."
Even though the NHTSA text mentions “monitor contactor temperature” , there simply is no temperature sensor in the HVBJB. Not in the original, nor in the replacements that are being installed. So the text is a liberal interpretation of a software feature.

The software guesses the temperature based on the load over the past minutes and the voltage drops it sees. I’m not even sure what good that does as the voltage drop itself is enough of an indicator. A higher voltage drop under load means more heat which is the signature sign of a hardware problem with the contactor. When that is detected the MME turns on the MIL and restricts motor power.

The problem is that this is not accurate enough to prevent frying contactors. It is just hard to precisely determine such very low resistances accurately. So I am not surprised that this was not sufficient in detecting problematic contactors before they fully break down.

The contactor breaks down as a result of heat, which is caused by a manufacturing defect in combination with improper thermal design of the HVBJB.
 
Last edited:

DevSecOps

Well-Known Member
First Name
Todd
Joined
Sep 22, 2021
Threads
69
Messages
4,764
Reaction score
11,624
Location
Sacramento, CA
Vehicles
'21 Audi SQ5 / '23 Rivian R1T / '23 M3P
Occupation
CISO
Country flag
I think we're just splitting hairs here. I agree 22S41 was not sufficient. However, it appears 22S41 dramatically improved the odds of a soft landing rather and limp home.
The defend Ford at all cost strategy is really tiresome.

The point that myself and others made since the software "fix" was released was that it was not a fix. If you pay a plumber to come to your house because your toilet doesn't work and he gives you a bucket and says "there's your fix" I bet you would be upset. That's what Ford did, they gave us a bucket to shit in. It should have been a recall to replace the components that they knowingly designed incorrectly from day one.

I'm really confused why anyone would defend 22S41. Yes, it was better than nothing, but it has harmed the reputation of Ford and burdened owners for the last 1.5 years. Defending 22S41 is defending Ford instead of sticking up for all MME owners. It was the wrong thing to do and I, for one, have always criticized Ford for 22S41.
 

AKgrampy

Well-Known Member
First Name
Mike
Joined
Jan 29, 2022
Threads
7
Messages
3,519
Reaction score
3,590
Location
Fairbanks, Alaska
Vehicles
Ford Expedition, Ford F-150, Mach E GT
Occupation
Retired
Country flag
I know you've said that many times before, but I'm just quoting what they apparently told NHTSA. If Ford lied to them, well that's on them. Just kinda strange that they even list the module supposedly doing the temperature monitoring.
Not supporting Ford but they did state monitoring and not measuring and obviously they must be calculating both and not actually measuring them. If they calculate a certain resistance then they can also calculate a temperature rise across that resistance over time base on amps flowing through it. Actually a good software protection to add but obviously not an acceptable fix. They probably won’t say anything but I wonder what they are changing software-wise? I often wondered if some of the HVBJB alarms may have been false positives. Or is it being fine tuned to catch the few that have slipped through to go to SSN?
 

ChasingCoral

Well-Known Member
First Name
Mark
Joined
Feb 3, 2020
Threads
502
Messages
14,306
Reaction score
28,653
Location
Maryland
Vehicles
2021 GB E4X FE, 2022 F-150 Lightning Lariat ER
Occupation
Retired oceanographer
Country flag
Even though the NHTSA text mentions “monitor contactor temperature” , there simply is no temperature sensor in the HVBJB. Not in the original, nor in the replacements that are being installed.
At least not in the HVBJBs that are currently being installed due to failures. We still don't know Ford's remedy for this recall. They MIGHT have a new, more capable HVBJB in the works that could include heavier busbars, temperature monitors, or other improvements. I realize that may be wishful thinking.
Sponsored

 
 







Top