DX Infrastructure Manager

Expand all | Collapse all

Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

kiran_diwakar02-27-2015 02:23 PM

  • 1.  Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 07:16 AM

    Hello All,

     

    I just wanted to reach out and ensure I give you heads up for a key issue that our engineers discovered yesterday!

    The CA Spectrum OneClick Java JRE certificate will expire on Monday March 01 2015. This will impact all the customers who are currently on 9.2.x, 9.3.x, and 9.4.x using the supported versions of java.

    Because of this, while all other components/servers of Spectrum will work appropriately, OneClick Client will not launch for all the users whose security setting on the OC Client box is set to HIGH or VERY HIGH in the Java Control Panel.

    So you would need to apply the hot fix we are providing (plans to follow later in the day) or use the workaround to lower Java Control Panel security setting to medium on OneClick Client Box ONLY (other servers like OC Server, SpectroServer, Reporting etc can still remain on HIGH/VERYHIGH security)


    I acknowledge the fact that given that the certificate will expire and the problem will manifest itself on March 1, we could have given you a bit more advanced notice – it was a late catch on our side

    We are already working on ways to not let this happen in future at such short notice for us as well as for our users.

     

    Having said that, the good news is that we have a workaround for this that the team has put together and our support team will publish by end of day today EST.

    Additionally, we also have plans in place that will be published for fixes/solutions for the problem that we will be making available – and that you will need to move to. Please look out for that note/information.

     

    As the Spectrum group, we have done a good job in the last couple years with our releases, quality and our general things and I know the expectations from us are high. Hence, I personally apologize for the problem and inconvenience that you will be going through – please rest assured that we are doing our best to give you the best and the easiest way out from the myriad of possibilities we went through. The team has been up since last night getting a solution and our support teams are trying to get as many scenarios covered in their docs and ensuring as much heads up as possible internally/externally – we have all hands on deck for this.

     

    Please do not hesitate to reach out me or to your support contacts if you need more information or have any suggestions.

     

    Regards,

    Kiran Diwakar



  • 2.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 08:10 AM

    Thanks Kirwan for the update.

     

    I have sent a note regarding using Java 6 and from my searching the setting is not apparent. It may be useful to the community if the situation with Java 6 can be clarified.



  • 3.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 08:32 AM

    Hi Ian,

     

    Yep, I saw that and I see you are talking to the right people.

    You will hear back from us very soon

     

    Regards,

    Kiran Diwakar



  • 4.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 08:51 AM

    What about the other CA applications? (eHealth / SOI / ...)



  • 5.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-26-2015 12:19 PM

    A note with regard to all products: The workaround -- lowering the Java JRE security setting from High or Very High to Medium and using the exceptions list -- should work around this issue for most clients.

     

    -Margaret, eHealth PM



  • 6.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 08:58 AM

    Hi Frank,

     

    I am not sure about the other products. I don't believe there is an issue on their side, coz we would have heard. Anyway, I will check it and let this group know.

     

    One more thing - the workaround is to lessen the Java security (through Java control panel) only on the box/system that has the OC client.

    I am just trying to get a feel of how problematic would this be in your scenarios. I know there would be sensitive deployments - but given this is a client system (maybe outside of the NOC or Data Center), would this workaround only for client system be too difficult?

    Again, as I said, we will be having solutions for all platforms/releases by end of this week, but just wanted to make sure I have a feel for the workaround - so comments from people would be much appreciated.

     

    Regards,

    Kiran Diwakar



  • 7.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 09:04 AM

    All

     

    One of my customers is getting this.

     

    We removed cert checking and set security to lowest

     

    Still no good

     

    Windows 7

     

     

        <jar href="lib/contrib/clientcivpn



  • 8.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 10:02 AM

    Hi Dan,

     

    Have you tried adding the Spectrum OneClick server address to the exception list? Then there is  no need to change the security levels

     

    Regards,

    Dencel



  • 9.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 10:26 AM

    Hi Dan,


    The certificate will expire on March 1st - so not sure how you are running into the problem now. Did you do something else?

    If not, this could be a completely different problem and not related. Would request you to check with local support

     

    Regards,

    Kiran Diwakar



  • 10.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 12:22 PM

    Hi Kiran,

     

    The problem with the workaround will be that most users - for security reasons - don't have the rights/ permission on their computers to change Java related settings (or other settings).

    The Java settings on the Client PCs are configured via Group Policies and these Group Policies are company wide and have to match the company's Security Policy. I think it will be difficult to lower the Java security settings only for the group of Spectrum OneClick users and even more difficult to push through that the Java settings are changed for all users in the company.

     

    Especially with the latest news in mind that big companies are more and more attacked by hackers, lowering security settings is a no-go for most of our customers.

     

    Are there any news about the patches?

    While we are running out of time, the patches need to be easy to install. Will it be possible to update the current servers easily? Any information is very welcome to prepare the customer's server for the upgrade procedure.

     

    BTW: Best would be if the patch does not contain any further fixes, because there is no time left to test the patch in the customer's environment before installing. And negative impacts on the server due to fixes is not what we need now

     

    Best regards

    Kristian



  • 11.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 01:09 PM

    Agreed. We are in the same situation. Users don't have permission to perform the workaround so the patch is our only option.



  • 12.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 01:17 PM

    Hi Stuart,

     

    Right – we absolutely could think of atleast a couple dozen customers we work with closely where we know the workaround will not fly.

    As I said, the solution will be made available, this was a heads up for the team here to have a plan in place for this to be tried out.

    Over the day today, Rene will send out our plan that we are fine tuning and it went through multiple iterations.

     

    Please stay assured that we are pushing ourselves very hard to get this done in time for all of the customers. Thanks for the note and confirmation as well as the patience.

     

    Regards,

    Kiran Diwakar

    Director, Product Management

    Office: +78964 | Mobile: +91 97 66709995 | Kiran.Diwakar@ca.com

    <mailto:Kiran.Diwakar@ca.com>[CA][cid:image003.jpg@01CF96F2.15E42900]    <http://www.ca.com/us/default.aspx>

    <http://twitter.com/CAInc>[LinkedIn]<http://www.linkedin.com/company/1372?goback=.cps_1244823420724_1>[Facebook]<https://www.facebook.com/CATechnologies>[YouTube]<http://www.youtube.com/user/catechnologies>[Google]<https://plus.google.com/CATechnologies>[Slideshare]<http://www.slideshare.net/cainc>



  • 13.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 01:29 PM

    No problem. I'm sure you're all working diligently to fix this recently

    discovered problem. I work a 9/80 schedule, so I won't be here Friday and

    I'm taking a personal day (planned since last week) tomorrow. Hopefully

    something comes out today so I can get it patched.

     

    On Wed, Feb 25, 2015 at 12:18 PM, kiran_diwakar <



  • 14.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 01:37 PM

    If you are on 9.4, we possibly could have something today.

     

    Regards,

    Kiran Diwakar



  • 15.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 01:37 PM

    We are on 9.4.0. 



  • 16.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 01:15 PM

    Hi Kristian,

     

    Thanks so much for your response and we understand that it would be the case for many of our users.

    So the solution/patches will be available by end of the week for most versions.

    We have kept it absolutely minimal so that the changes in them are isolated to this certificate problem only.

     

    The notes that we will release with the solution will have all the details. In the meantime, the idea was to have the workaround wherever applicable.

    Please stay tuned and really appreciate all your patience with this.

     

    Regards,

    Kiran Diwakar

    Director, Product Management

    Office: +78964 | Mobile: +91 97 66709995 | Kiran.Diwakar@ca.com

    <mailto:Kiran.Diwakar@ca.com>[CA][cid:image003.jpg@01CF96F2.15E42900]    <http://www.ca.com/us/default.aspx>

    <http://twitter.com/CAInc>[LinkedIn]<http://www.linkedin.com/company/1372?goback=.cps_1244823420724_1>[Facebook]<https://www.facebook.com/CATechnologies>[YouTube]<http://www.youtube.com/user/catechnologies>[Google]<https://plus.google.com/CATechnologies>[Slideshare]<http://www.slideshare.net/cainc>



  • 17.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-26-2015 04:10 AM

    I have checked the certificates in SOI.

    SOI 3,1

    [From: Thu Feb 28 01:00:00 CET 2013  To: Mon Mar 02 00:59:59 CET 2015]

     

    SOI 3.3

    [From: Thu Feb 28 01:00:00 CET 2013,  To: Mon Mar 02 00:59:59 CET 2015] &

    [From: Thu Feb 28 01:00:00 CET 2013, To: Mon Mar 02 00:59:59 CET 2015]

     

    Am I correct that, SOI will be affected by this problem if the Java security settings are higher than medium on Mar 02 00:59:59 CET 2015?

     

    regards

     

    Andre



  • 18.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-26-2015 11:51 AM

    Hi Andre,

     

    The SOI team is also checking and let the community know.

    I am getting the SOI key product folks on this update

     

    Regards,

    Kiran Diwakar



  • 19.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 11:14 AM

    Kiran, thank you for notifying the community!



  • 20.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 12:55 PM

    Hi Kiran,

     

    Thanks a lot for the notice in the community.



  • 21.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 01:35 PM

    Is a mass-mail planned (similar to how the heartbleed email was delivered for eHealth) for when the hotfix(es) becomes available? Or is best to keep an eye on this thread for the release announcement?

     

    Thanks for the heads-up.



  • 22.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 02:15 PM

    We are in the process of using a new notification system that will use your email addresses if you have opened a support issue with use within the last 12 months.

     

    In addition the same advisory will be posted to the Spectrum product page and this community.

     

    --

    Rene’

     

    Rene’ Cantwell

    CA Technologies

    Support Delivery Manager

    603-334-2497

    rene.cantwell@ca.com<mailto:rene.cantwell@ca.com>



  • 23.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 04:10 PM

    Thank you for the insight!



  • 24.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 02:48 PM

    Is there a way for us to reproduce this issue ahead of March 1?

     

    I would like to see it break when set to High/Very High, and then work when set to Medium -- just to validate the workaround.



  • 25.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 02:51 PM

    Yes, you can change the system clock on the client box to a date after March 1st.

     

    The recommended workaround will be to use the exception list over changing the security setting.  You may want to try them both.

     

     

     

    --

    Rene’

     

    Rene’ Cantwell

    CA Technologies

    Support Delivery Manager

    603-334-2497

    rene.cantwell@ca.com<mailto:rene.cantwell@ca.com>



  • 26.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 03:34 PM

    Anxiously awaiting for what fix will be.

    Don't want to speculate here, but I'm hoping fix is in the form PTfs with updated JAR files, or simply plain jar files.

    The new jars would simply have updated signature certs (btw, not clear to me if those are ones affected).

    Changing security settings or updating Exceptions list will not work for many users .

    This is not just due PC admin rights but also to new JRE security restrictions, that will simply ignore those settings.



  • 27.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 05:16 PM

    The details are available in the Advisory which is now available from our Product Page News area or this link:

    https://support.ca.com/phpdocs/7/7832/7832_certificate-expiry_product-advisory.pdf

     

    One additional note to the workarounds recommended.  If the Exception Site List (workaround #1) is going to be used, you will have to add your BOXI server if on a different box than your OneClick Sever.  The entry will need the port number (at least on Linux) i.e. http://hostname:8088



  • 28.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 05:46 PM

    Rene,

     

    Thanks for the Advisory. After seeing the workaround, I think this issue is being caused by JRE Security changes which were introduced by Oracle with release of JRE 1.7_60ish, which changed the security from Medium to High. Users running OneClick server which has a lower JRE requirement and client machines running JRE release lower than 1.7_60ish might get away by just accepting a warning message, till the point they are able to apply new hotfixes. In our company we were aware of this JRE changes from couple of months and have already put our OneClick and CABI servers URL's in the exception list.

     

    Our Spectrum environment is 9.3 H01 on RHEL which has JRE requirement of 1.7_u51. Client computers run JRE 1.7u76 with Security set to High. I tested the certificate expiry scenario by changing the date on a client PC to March 2nd and with exception list in place there was no impact on launching OneClick. I just had to acknowledge that the certificate is Expired.

     

    Hope this helps some of the customers!!!

     

    jcp2.png



  • 29.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-26-2015 01:17 PM

    An additional problem that you may run into in trying to use the Exception List is if your site uses Deployment Rule sets. This is a JAR file, typically in C:\Windows\Sun|Java\Deployment\DeploymentRuleSet.jar which is distributed within an organization. If your Java control panel has "View the active Deployment Rule Set" as shown, you're one of the affected ones. The problem is (as Oracle states), that anything added to the Exception Site List is ignored.

     

    https://www.java.com/en/download/faq/exception_sitelist.xml

     

    Deployment Rule Set

    If an active deployment rule set is installed on the system, the deployment rules take
    precedence over the exception site list
    . The exception site list is
    considered only when the default rule applies. See Deployment
    Rule Set
    for more information about deployment rules.

    1.jpg2.jpg




  • 30.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-26-2015 09:25 PM

    Hello Al,

     

    Thanks – that is very good information.

    I have asked the engineers to review it and check if we can add it to our advisory/document.

    Thanks again!

     

    Regards,

    Kiran Diwakar



  • 31.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-26-2015 06:28 AM

    Anyone able to reproduce the situation after 1st march today by changing the system time?

     

    I tried on a win7 client box:

    - shut down windows time service

    - set date to 5th of march

    - set java sec to high

    - start oneclick > oneclick starts .. no problems about expired certitficate

     

    Spectrum 9.3



  • 32.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-25-2015 08:40 PM

    Hi Saurabh,


    Thanks for this information, it would be definitely beneficial for atleast some of the customers. Thanks.

     

    Regards,

    Kiran Diwakar



  • 33.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-26-2015 12:22 AM

    Hello All,

     

    Hopefully everyone is able to review the product advisory that was posted yesterday.

    As you see we will have solutions (HFs) in place and the one for 9.4.x is already available.

    I know many of you wanted to solution to be isolated to this problem and we tried our best.

    But given the solution is a cumulative (hotfix), we wanted to make sure that the critical patches that people have applied to their current environments, they do not miss those or lose those.

    So after heavy deliberation, we decided to merge a few critical fixes as part of this cumulative hotfix as well. The details are in the product advisory.

    As we mentioned earlier, we tried to keep as minimum as possible to not only get you over the hump of this security issue, but also keep you current with respect to the key patches that you might have already applied.

     

    I request you to please review the product advisory in detail and if you are on 9.4.x to take advantage of the available solution as soon as possible. Additionally, we would request you to review the potential workarounds that might be applicable and allowed in your organizations so that you could potentially take care of the problem before it happens.

     

    Another thing that aggravates this is the fact that March 1 is a Sunday and not everyone might NOT be in office. This could lead to calls/pages for you and/or your teams over the weekend - would request you to consider that scenario and plan accordingly as well.

    On our end we are having our on call engineers be aware of it, and putting contacts in place for specific scenarios.

     

    As always, comments/suggestions very welcome.

    I would definitely want to call out our appreciation for this group for your support, patience and understanding in this case - it has helped us stay focused on the problem at hand a lot better!! Thanks once again.


    Regards,

    Kiran Diwakar



  • 34.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-26-2015 05:01 AM

    Hello all,

     

    maybe a stupid question.

    We already upgraded an environment with 14 SpectroServers and two OC servers to 9.4.2

    I know that in Fixed_Issues.09.04.02.01.htm there is the statement:"The 09.04.02.01 OneClick is only compatible with the 09.04.02.01 SpectroSERVER"

    but in our opinion it is sufficient enough to upgrade only the OC servers to 9.4.2.1 because according to the release notes nothing changed besides the signed jars.

    We also not able to upgrade the 14 servers in time.

    So any comments about that?

     

    regards

    Torsten



  • 35.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-26-2015 09:15 AM

    Torsten,  I'll check into this for you.

     

    --

    Rene'




  • 36.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-26-2015 11:10 AM

    Any update on this?  We also upgraded to 9.4.2 yesterday.  Can we apply this to our One Click Servers only or will we have to apply to all Spectrum servers?



  • 37.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-26-2015 11:39 AM

    Hi Skip,

     

    If you have moved to 9.4.2, then it is easier for you.

    9.4.1 would be having only the security changes.

    Yes, it would need to applied to all servers – as it is a hot fix

     

    Please let me know if you need any clarifications or help

     

    Regards,

    Kiran Diwakar



  • 38.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-26-2015 11:47 AM

    Kiran,

    How is it easier if I am already at 9.4.2?   It appears to me this is an upgrade as opposed to a hotfix which means I will have to follow the same steps as an upgrade.  I have 8 servers to update and it took nearly 5 hours to accomplish



  • 39.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-26-2015 12:00 PM

    Hi Skip,

     

    I am sorry, what I was intending to say is that given you are on 9.4.2, there are no additional changes in 9.4.2.1 – so the risk and tests on your part would be minimal.

    But yes, you would need to update and run the installer on all the systems and I am extremely sorry for the additional work you would need to do here.

    This would be the case for all the customers who would want to use the patch and not the workaround.

    The reason is that the certificate needs to be updated for over 200 jar files and there is no way the patch mechanism allows us to do that.

    We were trying VERY HARD to try and get a PTF instead of a hot fix, but given the above we could not create a patch and hotfix was the only way.

    We understand it is extra work – but be assured we really tried all other possible solutions.

     

    For 9.3 & 9.2 as well we are trying to get as minimum changes in the HF as possible.

    While 9.3 solutions will be available by today or tomorrow, please refer the documentation. Our goal was not to get you to lose the critical patches you would have already applied that we would have given.

    We intend to get the 9.2 solution out by end of the week – over the weekend. I will send a detailed note for 9.2 solution.

    Please stay tuned

     

    Once again – I would want to earnestly remind and request you to create awareness within your users/operators about this impending problem so that there is no panic and additional frustration in your teams.

    Please do not hesitate to reach out to us if you need anything else

     

    Regards,

    Kiran Diwakar



  • 40.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read

    Posted 02-26-2015 12:18 PM

    Thanks for the explanation.  Looks like I will be busy tomorrow morning. 



  • 41.  Re: Important: Problems to Certificates on your CA Spectrum One Click Installs – Please Read