DX NetOps

 View Only
Expand all | Collapse all

Spectrum upgrade failure

  • 1.  Spectrum upgrade failure

    Posted Apr 04, 2023 11:15 AM
      |   view attached

    Hi,

    I'm using Spectrum 10.4.1 installed on 2 Windows 2016 servers (1 SS and 1 OC). I've upgraded SS successfully to v22.2.5 but OC upgrade failed. In the log files it can be seen that MySQL has been upgraded but it cannot start. I cannot start it manually either. Eventually this causes upgrade to fail.

    Log files are attached.

    Can you please help me find out the reason of failure?


    Regards

    Attachment(s)

    rar
    logs.rar   158 KB 1 version


  • 2.  RE: Spectrum upgrade failure

    Broadcom Employee
    Posted Apr 04, 2023 11:34 AM

    There was an issue running the mysql.cus script during the install:

    ********** Started: Tue Apr  4 13:51:15 2023 **********
    running Custom Script Install-Tools/CUS/mysql.cus
     Error(s) occurred. Please see D:/win32app/Spectrum/Install-Tools/LOGS/22.2.5.0.36_20230404/mysql.log.

    Please upload a copy of the D:/win32app/Spectrum/Install-Tools/LOGS/22.2.5.0.36_20230404/mysql.log for review.

    Joe




  • 3.  RE: Spectrum upgrade failure

    Posted Apr 04, 2023 11:39 AM
      |   view attached

    Here it is.


    Attachment(s)

    log
    mysql.log   4 KB 1 version


  • 4.  RE: Spectrum upgrade failure

    Broadcom Employee
    Posted Apr 04, 2023 12:17 PM

    I see the following error in the log file:

    Final attempt to reach mysql... 
    **^G Error during mysqladmin --defaults-file=D:/win32app/Spectrum/mysql/my-spectrum.cnf --login-path=myclient ping

    Please check out the following two knowledge articles for resolution.

    https://knowledge.broadcom.com/external/article?articleId=236536
    https://knowledge.broadcom.com/external/article?articleId=188298

    Joe




  • 5.  RE: Spectrum upgrade failure

    Posted Apr 04, 2023 12:28 PM

    First link doesn't apply because mysqld is not 0 bytes and there is no $SPECROOT/mysql/new/bin dir.
    I'm checking about AV software on the server but I wouldn't say this is the issue because SS upgrade went well and MySQL was successfully upgraded.




  • 6.  RE: Spectrum upgrade failure

    Broadcom Employee
    Posted Apr 04, 2023 01:00 PM

    The AV may not cause an issue on one system but may on another.

    As a point of general practice, AV should be disabled for the install.

    Joe




  • 7.  RE: Spectrum upgrade failure

    Posted Apr 04, 2023 03:54 PM

    Apart from AV, is there something else that could be the cause here?




  • 8.  RE: Spectrum upgrade failure

    Broadcom Employee
    Posted Apr 04, 2023 04:00 PM

    Not that I am aware of.

    Is mysql running after the install?

    If so, try rerunning the installation.

    Joe




  • 9.  RE: Spectrum upgrade failure

    Posted Apr 04, 2023 04:04 PM

    No, it's not running and it cannot be started.




  • 10.  RE: Spectrum upgrade failure

    Broadcom Employee
    Posted Apr 04, 2023 04:08 PM

    Unfortunately the only two things I am aware of are the ones in the knowledge articles I sent previously.

    If you are running AV software, please disable it and retry the install.

    Joe




  • 11.  RE: Spectrum upgrade failure

    Posted Apr 06, 2023 03:41 AM
      |   view attached

    AV software was disabled but upgrade again failed. MySQL cannot start. I'm attaching the MySQL err file, maybe it can give some clue.


    Attachment(s)

    txt
    oc1.err.txt   634 KB 1 version


  • 12.  RE: Spectrum upgrade failure

    Broadcom Employee
    Posted Apr 06, 2023 09:40 AM

    This is a different error.

    Please review knowledge article https://knowledge.broadcom.com/external/article?articleId=261563 for the solution.

    Joe




  • 13.  RE: Spectrum upgrade failure

    Posted Apr 12, 2023 02:52 AM

    So I need to move the $SPECROOT/mysql directory to mysql_old and try to install Spectrum?




  • 14.  RE: Spectrum upgrade failure

    Broadcom Employee
    Posted Apr 12, 2023 07:02 AM

    Correct




  • 15.  RE: Spectrum upgrade failure

    Posted Apr 18, 2023 04:41 AM

    This procedure succeeded:
    1. Took a mysqldump of the reporting db in 10.4.1.
    ./mysqldump.exe --opt --routines --ignore-table=reporting.v_active_user_model --ignore-table=reporting.v_alarm_activity --ignore-table=reporting.v_ncm_config_diff --ignore-table=reporting.v_security_string_accessibility_by_landscape--ignore-table=reporting.v_user_report_security --compress -uroot -p<root_pw> --databases reporting > reporting_backup.sql
    2. Moved the $SPECROOT/mysql directory to mysql_old and upgraded Spectrum.
    3. Imported reporting db dump.
    ./mysql.exe -uroot -p<root_pw> reporting < reporting_backup.sql




  • 16.  RE: Spectrum upgrade failure

    Broadcom Employee
    Posted Apr 18, 2023 07:28 AM

    Outstanding!