Clarity

Expand all | Collapse all

How to roll back a failed upgrade

  • 1.  How to roll back a failed upgrade

    Posted 11-27-2018 05:34 AM

    How can i roll back a failed upgrade. I recently tried to upgrade clarity 143. to 15.3. The size on the database server, i saw was just 9MB left, when upgrade was happening. I would also see that the query it was executing on the database was rolled back. 

     

    I Just wanted to rollback, increase space and run upgrade again.

     

    thanks,

    Samad Abdul Aleem



  • 2.  Re: How to roll back a failed upgrade

    Broadcom Employee
    Posted 11-30-2018 12:28 PM

    15.3 was our first release to include the new auto-backup feature in between "hops" from 14.3 to 14.4 to 15.1 to 15.2 before landing at 15.3. 

     

    This is what I could find:

     

    15.3 and Higher Upgrades

     

    When you attempt an upgrade from Release 15.1 or a previous release to this release (15.3 or higher), the upgrade offers to save database backups for you. One backup is stored at a time for each release-level step. For example, you create a backup of your 14.4 database and then start an upgrade to 15.3. The upgrade attempts to save a backup of your temporary 15.1 database. The upgrade continues, purges that backup, and then attempts to backup your temporary 15.2 database. If you encounter a problem before the final 15.3 upgrade finishes, you have a more recent backup.

     

    Tip: Depending on the size of your database, each step-level backup can potentially take several hours for each release in the upgrade path. We recommend that you attempt the upgrade with database backups in a sandbox environment. When you have resolved any upgrade issues, perform the production upgrade and skip the backups.

     

    Before starting the upgrade, to allow backups on Oracle databases only, grant the required Oracle permissions to the database schema user. If you elect to allow the upgrade installer to perform the backups, the approximate disk space on the database server is specified so that you can make an informed decision and necessary arrangements before proceeding. The installer then prompts you for your choice of backup location on the same database server (it must be on the same server). 

    Requirement: To avoid errors, when selecting a backup directory, verify that it is on the same server as the Oracle home directory.

     

    The following step applies on Oracle databases only; however, the automated backups during upgrade feature works on both Oracle and MS-SQL databases. To permit the upgrade to make automatic backups, grant the application database schema user the CREATE ANY DIRECTORY permission and SELECT permission on DBA_DATAPUMP_JOBS. Grant the same permissions for the data warehouse schema user if you are upgrading from a previous release of the data warehouse. These are for Oracle only.


        grant CREATE ANY DIRECTORY to <app schema user>;
       grant SELECT on DBA_DATAPUMP_JOBS to<app schema user>;

       grant CREATE ANY DIRECTORY to <dwh schema user>;
       grant SELECT on DBA_DATAPUMP_JOBS to <dwh schema user>;

     

    In some environments where the application and database are on the same server, running the service stop all command also stops the listener.

     

    1. To check the status of the listener, enter lsnrctl status.
    2. If this returns error code 61, issue the command lsnrctl start.

     

    Recovery/Restore

     

    Recovery/restore requires a few checks to be done. The file install.log provides information on where the upgrade failed. In order to restore from the last successful backup (both application and database), find the last successful backup message as indicated above. Once the last successful database backup has been identified, look for the corresponding application binaries. As indicated above those will be named like:

    c:\clarity_14.4.0.234

     

    Delete the current install directory. After the current install directory has been deleted, rename the identified application binary to the install directory. For example:

    c:\clarity_14.4.0.234 renamed to c:\clarity

     

    After the application binary has been restored, you can now proceed to restore the database.

     

    The process is similar to how a fresh install is initialized, by creating the database user, granting permissions (including new ones for db export) and restoring the database from the backup created by the upgrade install.

     

    NOTE: It is possible that the upgrade failed and there is also an additional folder with the post-fix "_prev". As always before restoring and application binary, look in the following file to identify the correct version:

    <install dir>_prev/.setup/version.property

     

    In that file look for something like below to confirm the version:

    # Full version number
    version=15.3.0.83

     

    If a directory with "_prev" exists then see if the db exported version matches with the version specified in "version.property" file. The significant numbers are the first 3 numbers. The last number can be ignored for comparison.

     

    Issues when recovering from a failed upgrade

    • When upgrading PPM, if an intermediate install is 14.2, then the attribute "home" for the element "applicationServer" is set to tomcat 8. If the upgrade install fails on 14.2 then on restoring it make sure to modify the entry to tomcat 7 home directory location
    • After restoring a database, make sure to reset the password as specified in properties.xml and provide privileges as mentioned above
    • Ensure that services (beacon, nsa) are added and deployed. It is not important that they are running and should be shut down if running (except db).

     

    AEC-CA

         Please let Suman and I know how it goes.

     

    Kind Regards,

     Damon



  • 3.  Re: How to roll back a failed upgrade

    Broadcom Employee
    Posted 11-27-2018 05:42 AM

    I hope you have taken the backup of database and app folder prior to upgrade and thats how you should roll back. 



  • 4.  Re: How to roll back a failed upgrade

    Posted 11-27-2018 10:34 AM

    With since 14.x I mean versions after a specific 14 and after that including all 15 versions.



  • 5.  Re: How to roll back a failed upgrade

    Posted 12-02-2018 12:41 AM

    Thank you Damon,

     

    We successfully upgraded ca clarity  to 15.3 from 14.2. We successfully applied our test cases on one of the instance of 15.3 which we upgraded from the test server.

    In production we are using four different servers for clarity, which are application server , database server, integration server and jaspersoft(report) server.

    We cloned the production server and upgraded last week these four servers. We will test all the scenarios again. Once we get a sign off we will upgrade the production as well.

     

    When backup of database was prompted during upgrade, I selected NO, as I already had the VM backup.

     

    Thank you very much for your reply. It is very informative.

     

    Regards,

    Samad Abdul Aleem



  • 6.  Re: How to roll back a failed upgrade

    Posted 11-27-2018 07:42 AM

    YES..i have the VM backup as well



  • 7.  Re: How to roll back a failed upgrade

    Posted 11-27-2018 11:01 AM

    Thanks Urmas



  • 8.  Re: How to roll back a failed upgrade

    Broadcom Employee
    Posted 11-27-2018 08:18 AM

    Great so its best to revert back the restored VM



  • 9.  Re: How to roll back a failed upgrade

    Posted 11-27-2018 08:20 AM

    Yes, I was actually looking to do it via steps recommended by CA. Thanks.



  • 10.  Re: How to roll back a failed upgrade

    Posted 11-27-2018 09:56 AM

    The short answer is that since something like 14.x you do not roll back.

    You restore a backup.



  • 11.  Re: How to roll back a failed upgrade

    Posted 11-27-2018 09:59 AM

    thanks Urmas

    How about 15.x.