Original Message:
Sent: Mar 28, 2023 01:15 PM
From: Shannon Hebert
Subject: WARNING: Serious ZDU bug impacting AE 21.0.5
Hi All,
21.0.5 HF2 is now available for download. This announcement is on its way out to our customers: https://support.broadcom.com/web/ecx/support-content-notification/-/external/content/ReleaseAnnouncements/0/21991
Thank you,
-Shannon
------------------------------
_______________________________________
Shannon Hebert
Head of AOD Automation Support
Broadcom Software | Agile Operations Division
_______________________________________
Original Message:
Sent: Mar 23, 2023 10:55 AM
From: Shannon Hebert
Subject: WARNING: Serious ZDU bug impacting AE 21.0.5
Hi Stefan,
Keld asked a question on a non-ZDU topic, which is what my last reply addressed.
The ZDU issues will be addressed in 21.0.5 HF2.
Thank you,
-Shannon
------------------------------
_______________________________________
Shannon Hebert
Head of AOD Automation Support
Broadcom Software | Agile Operations Division
_______________________________________
Original Message:
Sent: Mar 23, 2023 09:27 AM
From: Stefan Lerch
Subject: WARNING: Serious ZDU bug impacting AE 21.0.5
Hi Shannon
Your lines do confuse me. Is the fix for ZDU included in (the released) HF1 (software download) and just the "release documentation" was not updated or do we have to wait for another HF(2?) that will include the fix for the ZDU problem?
Kind Regards
Stefan
------------------------------
Stefan Lerch
Senior Consultant
at www.systempartners.ch
For AUTOMIC trainings please check https://www.systempartners.ch/en/homee/services/training/trainingautomic/
Original Message:
Sent: Mar 21, 2023 03:28 PM
From: Shannon Hebert
Subject: WARNING: Serious ZDU bug impacting AE 21.0.5
Hi Keld,
This issue was a special case, fixed for 21.0.6 and then backported to 21.0.5 HF1. I did confirm the fix is included, and we intend to publish all released fixes in the release notes. I'll follow up on this process.
Thank you,
-Shannon
------------------------------
_______________________________________
Shannon Hebert
Head of AOD Automation Support
Broadcom Software | Agile Operations Division
_______________________________________
Original Message:
Sent: Mar 20, 2023 06:45 AM
From: Keld Mollnitz
Subject: WARNING: Serious ZDU bug impacting AE 21.0.5
Hi @Shannon Hebert
The issue mentioned in this KB article is not mentioned in the Bug Fixes section of the v21.0.5 HF1 release notes. Do you think that, in the future, all bug fixes will be mentioned in the Release Notes...?
Original Message:
Sent: Mar 14, 2023 10:47 AM
From: Shannon Hebert
Subject: WARNING: Serious ZDU bug impacting AE 21.0.5
Hi All,
21.0.5 HF1 was released today: https://support.broadcom.com/external/content/ReleaseAnnouncements/0/21894
The link above links to the download and fixes within.
Thank you,
-Shannon
------------------------------
_______________________________________
Shannon Hebert
Head of AOD Automation Support
Broadcom Software | Agile Operations Division
_______________________________________
Original Message:
Sent: Mar 07, 2023 12:05 PM
From: Shannon Hebert
Subject: WARNING: Serious ZDU bug impacting AE 21.0.5
Hi All,
21.0.5 HF1 was planned to be released last week and has been delayed due to architectural changes within our internal environment. We will notify our customers as soon as it is released. Our current plan is to release the ZDU fix in HF2, which is due out soon after HF1.
I appreciate your patience.
-Shannon
------------------------------
_______________________________________
Shannon Hebert
Head of AOD Automation Support
Broadcom Software | Agile Operations Division
_______________________________________
Original Message:
Sent: Feb 28, 2023 04:28 PM
From: Shannon Hebert
Subject: WARNING: Serious ZDU bug impacting AE 21.0.5
Hi Michael and All,
During our initial testing of 21.0.5, the ZDU was functional. With the details you supplied through the support ticket, we could reproduce the issue internally and will incorporate these differences in our testing methods. The ZDU issues have been corrected and are scheduled to be included in 21.0.5 HF2, due out within the next two weeks.
I agree that we can improve by raising awareness in our customer base as soon as we confirm an issue that can impact the efforts underway.
Best,
-Shannon
------------------------------
_______________________________________
Shannon Hebert
Head of AOD Automation Support
Broadcom Software | Agile Operations Division
_______________________________________
Original Message:
Sent: Feb 24, 2023 09:36 AM
From: Michael Lowry
Subject: WARNING: Serious ZDU bug impacting AE 21.0.5
Summary: Do not upgrade to v21.0.5 using Zero Downtime Upgrade without first contacting Broadcom Support .
I have not seen an announcement about this so I thought I would share our experience in the hopes others may be spared needless trouble.
Eleven days ago, Broadcom released AWA 21.0.5. We had been waiting for bug fixes delivered in this specific release, so we were understandably eager to install the new version. Unfortunately, we ran into a problem when using the Zero Downtime Upgrade feature to upgrade one of our systems from 12.3.9 HF2 to 21.0.5.
Everything went OK until step 4. Connections in the ZDU Wizard. During that step, we clicked Disconnect All Agents and then Disconnect All Users. This closed connections to the old-version Communication Processes, and triggered the switch of the PWP to the new version. As soon as the switch happened, the problem manifested. Normal (non-Java) Work Processes began to die with segmentation faults, writing UC4Dump files and forced traces.
Some WPs would stay up for a few minutes, but others would die almost immediately after starting. Restarting the WPs worked fine, but they would soon die again. Eventually, more WPs were down than up. Also, the WP traces were beginning to fill up the file system, so we chose to stop the system and call Broadcom Support for help.
Broadcom Support is currently investigating the problem. Their preliminary guess at a root cause is a known bug in 21.0.5. The bug is apparently related to ZDU, and a hotfix is expected soon.
If you intend to upgrade to v21.0.5 using Zero Downtime Upgrade, I strongly recommend that you contact Broadcom Support for advice first.
One more thing: I believe Broadcom should take a more proactive stance in notifying customers of known serious problems affecting releases that are publicly available. The Release Notes tool on the Automic Downloads site lists no known problems for Automation Engine 21.0.5. I searched for relevant KB articles and found only one partial hit: KB article 258694. (That bug is supposedly PostgreSQL-specific though, and we use Oracle; also, the symptoms described in the article are somewhat different from the ones we experienced.) We had been waiting for many months for bug fixes included in v21.0.5, and had reported several of these bugs; so Broadcom Support knew or should have known that we were intending to upgrade to this version. AE v12.0.5 was released almost two weeks ago, but Broadcom informed us that they knew about this problem only after we had run into the problem and opened a support case.