vCentre file upload failure (Certs are trusted)

[CoryIT]

Hello,

I have one client machine (WIndows 10, Chrome) that I am using to try and upload an ISO of about 3GB to a datatsore via the vCenter web UI.

When doing so, it gets about 66% of the way through the copy, and then fails citing a certificate error:

When I upload smaller files (between 1GB and 2GB), it seems to work without issue.

I have opened a case with vmware, and they have been unable to help me determine the cause of this issue.

The VC root certs are trusted, and if I browse directly to the host, those certs are also trusted.

Uploading directly to the host gets stuck part way through and seems to keep retrying the upload, but never completes nor does it ever fail.

Can anyone assist me determining why this one machine is struggling to complete this upload, even though all certs are already trusted?

Thanks

James

[daphnissov]

How large is this file? Are you using the Flex client as well?

[CoryIT]

File is 3.5GB (vCenter appliance ISO).

I have used both flash and HTML5 clients with the same issue in both cases.

[daphnissov]

And what version of vCenter here? Is this being uploaded to a host-local datastore?

[CoryIT]

Sorry, should have said, it's 6.5 U1.

The datastore is a shared datastore on a fiber channel SAN.

[IT_pilot]

Try using different browsers, different workstations.

If nothing works, then WinSCP, select /vmfs/volumes.

SSH will need to be enabled on the host.

[daphnissov]

I just ran a test from Windows 10 with Chrome uploading the latest vCSA ISO file to a SAN-backed datastore and it completed.

It's odd in your case that not only do you get failure but reason cited as having to do with certs. Normally, if certs are the issue, that upload will not even commence.

Do you have an old client integration plug-in installed, by chance? How about the enhanced auth plug-in?

[khaliqamar]

perhaps problem is not what it looks alike...

you can use winscp/scp/vsphere client  to upload your file.

================================

https://seimaxim.com

[CoryIT]

Other workstation was unaffected, also with Win 10 and Chrome

2012 R2 server also ok.

All three client machines have same certs pushed to them via GPO.

I also expected it to fail out right if it were a cert issue (VMware said that's not the case, some BS about 2GB leniance on un authenticated uploads, whatever that means).

I did at one point have CIP installed, but uninstalled it as VMware told me it was redundant. When working with support, they asked me to reinstall it, so it is currently installed but was happening both before and after installation.

Using WinSCP is fine, but I am trying to avoid having to switch to a different app to complete one simple task like this and would like to achieve through the web client.

Thanks all for everything so far.

[CoryIT]

Any further thoughts?

[daphnissov]

Have you tried multiple browsers (just to test) and also switching between their private browsing modes?

[CoryIT]

I have yes, and have the same problem with any browser I have tried, and their "private" modes.

Cheers

Eds

[daphnissov]

When you get that failure, have you checked logs of the vCenter web client to see what it says at that time? This is sounding like some strange config issue on that Windows 10 box, but I'd be curious to know what vCenter says about the failure from its perspective.

[CoryIT]

VMware support had a look at logs, but don't think they spotted anything unusual (otherwise we probably would have picked up on it and I wouldn't have posted here haha).

Do you have specific logs in mind that I should check?

I can then try and reproduce the issue and look at that log and paste the results here.

Thanks.

[daphnissov]

If you're uploading using the Flex client (https://<FQDN>/vsphere-client/) then grab the virgo log in the corresponding folder. If you're using the HTML5 client (https://<FQDN>/ui/) then use the Flex client for this test.

[CoryIT]

Sorry what's the full path to the log?

In /var/log/vmware I can see vsphere-ui and vsphere-client, but neither contain a virgo log file.

Thanks.

[daphnissov]

/var/log/vmware/vsphere-client/logs/vsphere_client_virgo.log

[CoryIT]

Ah, didn't realise there was a logs folder within.

Here's the output from start of upload to end:

[2018-04-30T12:32:27.533Z] [INFO ] http-bio-9090-exec-6         70002817 101297 201260 org.springframework.flex.servlet.MessageBrokerHandlerAdapter      Channel endpoint ds-core-amf received request.

[2018-04-30T12:32:28.683Z] [INFO ] http-bio-9090-exec-10        70002818 101297 201260 org.springframework.flex.servlet.MessageBrokerHandlerAdapter      Channel endpoint ds-core-amf received request.

[2018-04-30T12:32:39.593Z] [INFO ] http-bio-9090-exec-11        70002819 101297 201260 org.springframework.flex.servlet.MessageBrokerHandlerAdapter      Channel endpoint ds-core-amf received request.

[2018-04-30T12:33:27.573Z] [INFO ] health-status-51              com.vmware.vise.vim.cm.healthstatus.AppServerHealthService        Memory usage: used=271,494,368; max=557,842,432; percentage=48.66864770874941%. Status: GREEN

[2018-04-30T12:33:27.573Z] [INFO ] health-status-51              c.v.v.v.cm.HealthStatusRequestHandler$HealthStatusCollectorTask   Determined health status 'GREEN' in 0 ms

[2018-04-30T12:33:28.732Z] [INFO ] http-bio-9090-exec-13        70002820 101297 201260 org.springframework.flex.servlet.MessageBrokerHandlerAdapter      Channel endpoint ds-core-amf received request.

[2018-04-30T12:33:29.871Z] [INFO ] http-bio-9090-exec-13        70002821 101297 201260 org.springframework.flex.servlet.MessageBrokerHandlerAdapter      Channel endpoint ds-core-amf received request.

[daphnissov]

And you're uploading using the Flex client (not the HTML5 client that you showed in your screenshot in the first post)?

[CoryIT]

Correct

[daphnissov]

Just out of curiosity, have you tried to upload this same file to an ESXi host directly (maybe even try the same host as that which was targeted in the vCenter-based upload--this is knowable through the Events pane) using its embedded HTML5 host client? If not, try that and see what the result is.

[CoryIT]

I have yes, tried this with support.

In this instance, it doesn't error out as vCenter does, but seems to keep retrying over and over.

The percentage gets so far then stops, but if I refresh the datastore, I can see the file size in crease, then suddenly it drops down to only a meg or two, then increases again, then drops, then increases ad infinitum.

[daphnissov]

There's definitely something environmentally weird going on in this case of this system...

[CoryIT]

Yes I think that is absolutely true, but uploads to ESXi/vCenter are the only issue I have, and am now at a loss as to what else to look at to correct?