PSA - DO NOT UPDATE to 3.35.0 on Android if using PaaK without a FOB

[06VistaGT]

I have been using PAAK exclusively for the last several months. Galaxy s10. I am on version 3.33.0 now and just turned off auto updates. Thanks for the PSA!

@dsuyang, nice to see we have the right folks engaged to help figure out the issue. Please let us know what you find...within what you are allowed to divulge, obviously.

Sponsored

 

[Gimme_my_MME]

If you really are part of the team:

I am 99% sure Suyang is part of the team

 

[Tarynem]

I don't get it. No user acceptance testing for the Android group?? How does this happen? I know all products have issues... but this is major.

 

[chrisGT]

I am 99% sure Suyang is part of the team

I really doubt it.
When you work for a big company typically you are not allowed to make public comments or statements in forums or social media. Everything has to go through the PR department.

Also major software companies have oncall software engineers who look on these major issues ASAP, they don't wait till next day as he said. If he is truly a lead at Ford, and what he is saying is true they really have to step up their software operations. They need to add real-time monitoring for failures and issues instead of depending on forums, they need SREs (Site Reliability Engineers) who can stop or revert new releases and they need software engineers oncall to fix what SREs cannot.
Also software should be rolled slowly to a test subgroup, then gradually ramp up the release. Of course you need solid monitoring to know if there are problems in the test group.

 

[DevSecOps]

I am 99% sure Suyang is part of the team

Sus ?

My singular investigative brain cell is ticking! I see what's going on here.

 

[GoGoGadgetMachE]

The error is interestingly different when you try to add a code with a fob available :

We know sort of that there's multiple certificates for mutual authentication and all kinds of stuff going on in different places. It feels like some kind of weirdo expiration issue or something especially since apparently nuke and pave works.

I literally just set this phone up yesterday so whatever happened wasn't tied to length of setup.

Anyway I guess Ford is now aware and we can hope that some kind of testing for this failure mode will be put in place. Doesn't help anyone now of course.

 

[DevSecOps]

I really doubt it.
When you work for a big comoany typically you are not allowed to make public comments or statements in forums or social media. Everything has to go through the PR department.

He could be. I would rather give someone the benefit of doubt. I also reached out to some people I know at Ford and raised this issue with them who said they were in contact with the team in charge. So it could very well be.

 

[GoGoGadgetMachE]

Hey @Ford Motor Company you have the ability to validate @dsuyang.

It's possible others on here could at least validate the name in the company address book.

 

[GoGoGadgetMachE]

Hey @dsuyang if you have any way to instant kill the update for now on the Play side that would be a good idea too. ?

 

[chrisGT]

He could be. I would rather give someone the benefit of doubt. I also reached out to some people I know at Ford and raised this issue with them who said they were in contact with the team in charge. So it could very well be.

I updated my previous post. Basically if that is true, it implies they are far behind in software operations compared to major software companies. See my post for why.

 

[chrisGT]

That's alright. I ask my wife to leave her MacheE key fob at home and keep her faith in the Paak. I rely on her to give me the feedback. (I work from home). She is asleep now with kid, otherwise I would grab her phone to test ...

If you are truly the lead of the Android dev team, don't you have access to monitoring data to see number of successful Paak usages today? Do you see a drop in the number after the release?
Also do you have a test environment/simulator? Can you test the scenario where you have Paak set up in the old version, then you update the app (not fresh install).

 

[GoGoGadgetMachE]

If you are truly the lead of the Android dev team, don't you have access to monitoring data to see number of successful Paak usages today? Do you see a drop in the number after the release?
Also do you have a test environment/simulator? Can you test the scenario where you have Paak set up in the old version, then you update the app (not fresh install).

"zero successes before today, zero today" ??

As someone in the software industry as well, reality often doesn't match testing for lots of unexpected reasons. Not saying this is a good thing but saying I understand how these things can happen. The number of setup and upgrade failures I have personally fought because of the field having some unexpected thing versus the test environment could fill a book, I suspect.

Now, let's see how fast this is fixed, and if it ever happens again.

 

[BMT1071]

Another thing I will try is the batteryless fob. I have three fobs. I will take tha battery out of one and hide it somewhere. I think even a fob with a dead battery will start the car if inserted in the console holder spot. Never actually tried that though.....now I am curious

That's alright. I ask my wife to leave her MacheE key fob at home and keep her faith in the Paak. I rely on her to give me the feedback. (I work from home). She is asleep now with kid, otherwise I would grab her phone to test ...

I had some time so I just went to my garage to test a bit. The car is not making the PaaK BT connection.
Made sure FordPass was up and running.
Entered the garage and the car had no idea I was there with my phone.
No illuminated ring on the door button, no welcome lighting.
Pressed the door button and heard the usual relay/contactor clicking.
Several more button presses did nothing.
Entered the car using the pillar keypad.
Put my phone on the charging pad.
Foot on the brake, pressed the start button.
No Key Detected message in the IPC.
Sync screen came on as it would in accessory or ready mode.
AA connected.
Switched to BT connection to Sync without a problem.
Checked my phone's BT connections. Mach E PaaK pairing was listed but inactive.
Tapped on that pairing and got a message "An app is required to use this device".
After another press of the Start button the password screen came up and I was able to enter my code and start the car.
Shut the car off and walked away. No walk away lock.

 

[timbop]

Yeah the security failure guess seems reasonable.

Things break. Pretending fobs are perfect is also objectively dumb. I have been burned by that. Hell, I have been burned by a key breaking in a car door lock.

When all the 12V nonsense was happening did anyone with any sense of reality say "your stubborn refusal to accept that a battery might die will burn you"? Of course not.

It's asinine to blame the victim.

Please; I was blaming him for the next time if he doesn't adjust his thinking after getting stuck this time. After this debacle forum members who've been here a while have to recognize the reality that PaaK breaks with alarming regularity and adjust their philosophy. Paak has been broken by AT LEAST 4 different versions of FP since April. This is now the second time that the passcodes got wiped/broken. Given that there have been about 14 updates since cars started getting delivered in February that's around a 25% serious failure rate and 15% catastrophic failure rate.

At some point in time you have to use that extra intelligence that separates us from the other primates and recognize that risk/reward involved. You can take the fob with you and STILL have paak on your phone for remote start or whatever for very little inconvenience, or you can put yourself in a position to get stranded. It's no different than @Mopey recognizing he had to adapt his plan to do an extreme cold test or risk getting stuck in the freezing cold if he stubbornly held on to his original plan.

Having a "cool" feature that means you "only need your phone" is a stupid reason to get stuck when you know it has a track record of failure - which at this point all of you do.

Fobs are a proven technology, but .0000001% of the time they break. The largest cause of failure by far is a dead battery in a fob has a corresponding warning, and the worst case is going to the CVS or hardware store around the corner to replace the battery. Oh, and if you have Paak AND a fob you're double covered.

 

[Kamuelaflyer]

I am 99% sure Suyang is part of the team

There are a lot of Ford employees and employees of Ford contractors lurking here. I’m personally inclined to believe him until proven otherwise. I hope this is a quick fix regardless of why it happened or which program, app, module or planetary alignment caused it.

Sponsored