VCSA fails to Backup using FTPS

[insearchof]

vCenter6.5

VCSA appliance

Windows 2012 R2 Standard  IIS FTPS

I used this setup FTPS on my Windows servers

1. https://www.altaro.com/vmware/backing-up-vcsa-6-5-natively-using-ftps/

The Backup process starts and then runs for a long time with no data transferred and I get this error

BackupManager encountered an exception. Please check logs for details.

Which Logs? where are they located ?

Any thoughts?

Thank you

Tom

[msripada]

its is backup.log under /var/log/vmware/applmgmt

Thanks,

MS

[insearchof]

Msripada

here is the extract of the log

2018-07-27 04:12:31,860 [MainProcess:PID-29017] INFO: The location provided: 10.2.8.23:21/

2018-07-27 04:12:31,902 [MainProcess:PID-29017] INFO: Starting backup job...

2018-07-27 04:12:49,668 [MainProcess:PID-29017] INFO: The backup dir does not exist

2018-07-27 04:12:59,690 [MainProcess:PID-29017] ERROR: BackupManager encountered an exception: Failed to create FTP dir 'ftp://10.2.8.23:21/' at 'ftp://10.2.8.23:21/'; ret=28

stdOut:

stdErr: curl: (28) Connection time-out

　

2018-07-27 04:12:59,691 [MainProcess:PID-29017] INFO: Cleaning up the failed backup job.

2018-07-27 04:12:59,691 [MainProcess:PID-29017] INFO: Cleaning up all running backup child processes.

2018-07-27 04:12:59,788 [MainProcess:PID-29017] INFO: Cleaning up all sizeFiles.

2018-07-27 04:12:59,788 [MainProcess:PID-29017] INFO: Cleaning up ConfigFiles.

2018-07-27 04:12:59,789 [MainProcess:PID-29017] INFO: Cleaning up Lotus.

2018-07-27 04:12:59,789 [MainProcess:PID-29017] INFO: Cleaning up Vmdir. Running /usr/lib/vmware-vmdir/bin/vdcbackup -c

2018-07-27 04:13:00,239 [MainProcess:PID-29017] INFO: stdout: Cleanup: Setting vmdir state to VMDIRD_NORMAL

　

2018-07-27 04:13:00,239 [MainProcess:PID-29017] INFO: Cleaning up VCDB.

2018-07-27 04:13:00,240 [MainProcess:PID-29017] INFO: Finishing full database backup

2018-07-27 04:13:00,241 [MainProcess:PID-29017] INFO: Retrieving postgres server listening port

2018-07-27 04:13:00,241 [MainProcess:PID-29017] INFO: Executing command: netstat -plnt.

2018-07-27 04:13:00,283 [MainProcess:PID-29017] INFO: Canceling running pg_start_backup() process.

2018-07-27 04:13:00,284 [MainProcess:PID-29017] INFO: Executing command: ['/opt/vmware/vpostgres/current/bin/psql', '-U', 'postgres', '-p', '5432', '-At', '-c', "SELECT pg_cancel_backend(pid) FROM pg_stat_activity WHERE pid <> pg_backend_pid()  and query ~ '^select\\ pg\\_xlogfile\\_name\\(pg\\_start\\_backup\\(.*\\)\\)\\;' "].

2018-07-27 04:13:00,768 [MainProcess:PID-29017] INFO: No pg_start_backup() process is running.

2018-07-27 04:13:00,768 [MainProcess:PID-29017] INFO: Checking whether Postgres online backup still in progress

2018-07-27 04:13:00,769 [MainProcess:PID-29017] INFO: Executing command: /opt/vmware/vpostgres/current/bin/psql -U postgres -p 5432 -t -c "select pg_is_in_backup();".

2018-07-27 04:13:00,798 [MainProcess:PID-29017] INFO: Postgres on-line backup already finished. Skip stopping backup operation

2018-07-27 04:13:00,798 [MainProcess:PID-29017] INFO: Successfully cleaned up for VCDB backup.

2018-07-27 04:13:00,799 [MainProcess:PID-29017] INFO: Cleaning up ComponentScripts.

2018-07-27 04:13:00,799 [MainProcess:PID-29017] INFO: Cleaning up StatsMonitorDB.

2018-07-27 04:13:00,799 [MainProcess:PID-29017] INFO: Cleaning up failed backup files.

2018-07-27 04:13:10,822 [MainProcess:PID-29017] WARNING: Ignore to clean up failed backup files, due to issue: 'Failed to list FTP dir with cmd ['curl', '--fail', '-u', u'vcsauser:****', '--connect-timeout', '10', '--ssl-reqd', '-k', '--silent', '--show-error', '--list-only', u'ftp://10.2.8.23:21/']; ret=28

stdOut:

stdErr: curl: (28) Connection time-out

'.

2018-07-27 04:13:10,822 [MainProcess:PID-29017] ERROR: Failed to cleanup the backup job. Error: Failed to clean up the backup job.

2018-07-27 04:13:10,837 [MainProcess:PID-29017] INFO: Backup job failed.

~ ]#

Thoughts?

Tom

[Vijay2027]

Are you using default port 21?

[insearchof]

see my log below

[msripada]

Can you telnet 10.2.8.23:21 using the command curl -v telnet://10.2.8.23:21 and validate if you are getting response properly???

Also check the permissions on the ftp for the user vcsauser

2018-07-27 04:13:10,822 [MainProcess:PID-29017] WARNING: Ignore to clean up failed backup files, due to issue: 'Failed to list FTP dir with cmd ['curl', '--fail', '-u', u'vcsauser:****', '--connect-timeout', '10', '--ssl-reqd', '-k', '--silent', '--show-error', '--list-only', u'ftp://10.2.8.23:21/']; ret=28

stdOut:

stdErr: curl: (28) Connection time-out

'.

2018-07-27 04:13:10,822 [MainProcess:PID-29017] ERROR: Failed to cleanup the backup job. Error: Failed to clean up the backup job.

Thanks,

MS

[insearchof]

From the Appliance

Command> curl -v telnet://10.2.8.23:21

Unknown command: `curl'

From my windows 10 via cmd prompt

telnet 10.2.8.23 21

220 Microsoft FTP Service

The users permissions on FTP are Full Control

[msripada]

Please try to take ssh from vcsa and run the command.. If you are doing it from your local Windows remove curl - v and try the cmd

[insearchof]

Sorry if I was not clear in y last response.

From my windows 10 computer I can telnet in to the FTP server with no problem

I am SSH' into the VSCA and I cannot curl it fails

[sk84]

For using "curl" you need bash shell access. See: Enable and Access the Bash Shell from the Appliance Shell

And "connection timed out" errors are usually caused by firewalls or routing errors. Have you checked this or are the FTPS server and VCSA on the same subnet without firewalling and routing?

[insearchof]

Ok I was not in the bash shell

I get the same response as my Windows 10 computer

This is from the VCSA

~ ]# curl -v telnet://10.2.8.23:21

* Rebuilt URL to: telnet://10.2.8.23:21/

*   Trying 10.2.8.23...

* Connected to 10.2.8.23 (10.2.8.23) port 21 (#0)

220 Microsoft FTP Service

　

So what can I try next? It looks like it connects fine

[sk84]

Have a look at this: Setting up FTPS server for VCSA backup | vmninja

For FTPS, not only port 21/tcp must be open, since VCSA uses passive mode. I bet there's something wrong with a firewall (maybe Windows Firewall). That's why I asked for routing and firewall. So, again, is there any firewall in between?

[msripada]

Eddie's Blog: Configuring VCSA 6.5 Backup Lessons Learned

You are directly dumping the backup file to the FTP and not to any specific folder inside the ftp?

Is there any firewall in the FTPServer ? Can you try restarting ftpservice and also run the command curl -u <ftp user>:<password> -l <ftp server> from the above article to validate the same

Thanks,

MS

[insearchof]

The blog is good up to the last step and then it is unclear

I am not sure which ip address to add there?  "External IP address of Firewall?

This is a Windows 2012 R2 server running IIS do I use its IP address?

You are directly dumping the backup file to the FTP and not to any specific folder inside the ftp?

I created a folder like this

Mine is E:\FTPSVCSA

As noted above I have a Windows 2012 R2 which is using the built in Windows firewall no third party firewall here.

All my systems are on same subnet 10.2.8.0/22

I used the curl command here is what I got

~ ]# curl -u vcsauser -l 10.2.8.23

\Enter host password for user 'vcsauser':

<html>

<head>

<meta charset="UTF-8">

<meta name="description" content="Main page to the Grassi Web site with a link to my favorite sites including Lukes Wrestling history">

<title>TGCS Home Page</title>

<script type="text/javascript" src="js/GA-TGCSNET.js"></script>

</head>

<div id="fb-root"></div>

<script type="text/javascript" src="js/Facebook.js"></script>

<body background="images/starry-night.gif">

　

<bgsound src="sound/PF_Welcome_Machine.mid" loop="-1">

　

<font color=#00FFFF>

<h1 style="text-align:center;">Welcome to The Grassi Network</h1>

<font color=#FF00FF>

<h1 style="text-align:center;">T.G.C.S.</h1>

　

<p>You are the <script src="http://www.tgcsnet.com/counter/counter.php?page=home"></script>th person to visit this site</p>

　

<p><font color="#FFFFFF"><marquee align="bottom" bgcolor="#FF0000" border="0">To See Additional Sites Just Select the Hot Sites Button</marquee></font></p>

　

<script type="text/javascript" src="js/Get-Date.js"></script>

<p>

<div align="left">

<form style="width: 110px">

<input type="button" value="My Hot Sites" onClick="window.location.href='TGCS_HotSites.html'">

</form>

</div>

　

<html>

<body>

<div align="center" id="bottomright">

<form style="width: 103px">

<input type="button" value="Contact US" onClick="window.location.href='mailto:[e-mail]?subject=User requesting info'"></form>

<form style="width: 190px">

<input type="button" value="Sign Our Guest Book" onClick="window.location.href='GB/GuestBook.htm'">

</form>

<font color=#FF00FF>

　

<div align="center" class="fb-like" data-href="http://www.tgcsnet.com" data-layout="standard" data-action="like" data-show-faces="true" data-share="true" style="width: 613px"></div>

　

        </font>

</div>

<br>

<div align="center">

<a href="https://twitter.com/trgrassijr" class="twitter-follow-button" data-show-count="false">Follow US</a>

<script  type="text/javascript">!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id))

{js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>

<a href="https://twitter.com/share" class="twitter-share-button" data-via="trgrassijr">Tweet</a>

<script  type="text/javascript">!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id))

{js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>

</div>

</body>

</html>

~ ]#

This looks lime my default web site on this server

Thoughts?

[msripada]

If the folder is specified, I do not see the folder path specified which providing the FTP. In the Step 4, check the ftp info... It is specified as ftpserver/folder where the backup has to go.

Eddie's Blog: Configuring VCSA 6.5 Backup Lessons Learned

[insearchof]

I added the path and job still fails

2018-07-27 21:08:39,756 [MainProcess:PID-17647] INFO: The location provided: 10.2.8.23:21/FTPSVCSA

2018-07-27 21:08:40,009 [MainProcess:PID-17647] INFO: Starting backup job...

2018-07-27 21:08:51,621 [MainProcess:PID-17647] INFO: The backup dir does not exist

2018-07-27 21:09:01,638 [MainProcess:PID-17647] ERROR: BackupManager encountered an exception: Failed to create FTP dir 'ftp://10.2.8.23:21/FTPSVCSA/' at 'ftp://10.2.8.23:21/FTPSVCSA/'; ret=28

stdOut:

stdErr: curl: (28) Connection time-out

　

2018-07-27 21:09:01,665 [MainProcess:PID-17647] INFO: Cleaning up the failed backup job.

2018-07-27 21:09:01,666 [MainProcess:PID-17647] INFO: Cleaning up all running backup child processes.

2018-07-27 21:09:01,738 [MainProcess:PID-17647] INFO: Cleaning up all sizeFiles.

2018-07-27 21:09:01,739 [MainProcess:PID-17647] INFO: Cleaning up ConfigFiles.

2018-07-27 21:09:01,739 [MainProcess:PID-17647] INFO: Cleaning up Lotus.

2018-07-27 21:09:01,739 [MainProcess:PID-17647] INFO: Cleaning up Vmdir. Running /usr/lib/vmware-vmdir/bin/vdcbackup -c

2018-07-27 21:09:02,899 [MainProcess:PID-17647] INFO: stdout: Cleanup: Setting vmdir state to VMDIRD_NORMAL

　

2018-07-27 21:09:02,900 [MainProcess:PID-17647] INFO: Cleaning up VCDB.

2018-07-27 21:09:02,900 [MainProcess:PID-17647] INFO: Finishing full database backup

2018-07-27 21:09:02,901 [MainProcess:PID-17647] INFO: Retrieving postgres server listening port

2018-07-27 21:09:02,901 [MainProcess:PID-17647] INFO: Executing command: netstat -plnt.

2018-07-27 21:09:03,031 [MainProcess:PID-17647] INFO: Canceling running pg_start_backup() process.

2018-07-27 21:09:03,032 [MainProcess:PID-17647] INFO: Executing command: ['/opt/vmware/vpostgres/current/bin/psql', '-U', 'postgres', '-p', '5432', '-At', '-c', "SELECT pg_cancel_backend(pid) FROM pg_stat_activity WHERE pid <> pg_backend_pid()  and query ~ '^select\\ pg\\_xlogfile\\_name\\(pg\\_start\\_backup\\(.*\\)\\)\\;' "].

2018-07-27 21:09:03,553 [MainProcess:PID-17647] INFO: No pg_start_backup() process is running.

2018-07-27 21:09:03,553 [MainProcess:PID-17647] INFO: Checking whether Postgres online backup still in progress

2018-07-27 21:09:03,554 [MainProcess:PID-17647] INFO: Executing command: /opt/vmware/vpostgres/current/bin/psql -U postgres -p 5432 -t -c "select pg_is_in_backup();".

2018-07-27 21:09:03,584 [MainProcess:PID-17647] INFO: Postgres on-line backup already finished. Skip stopping backup operation

2018-07-27 21:09:03,584 [MainProcess:PID-17647] INFO: Successfully cleaned up for VCDB backup.

2018-07-27 21:09:03,584 [MainProcess:PID-17647] INFO: Cleaning up ComponentScripts.

2018-07-27 21:09:03,585 [MainProcess:PID-17647] INFO: Cleaning up StatsMonitorDB.

2018-07-27 21:09:03,585 [MainProcess:PID-17647] INFO: Cleaning up failed backup files.

2018-07-27 21:09:13,603 [MainProcess:PID-17647] WARNING: Ignore to clean up failed backup files, due to issue: 'Failed to list FTP dir with cmd ['curl', '--fail', '-u', u'vcsauser:****', '--connect-timeout', '10', '--ssl-reqd', '-k', '--silent', '--show-error', '--list-only', u'ftp://10.2.8.23:21/FTPSVCSA/']; ret=28

stdOut:

stdErr: curl: (28) Connection time-out

'.

2018-07-27 21:09:13,604 [MainProcess:PID-17647] ERROR: Failed to cleanup the backup job. Error: Failed to clean up the backup job.

2018-07-27 21:09:13,633 [MainProcess:PID-17647] INFO: Backup job failed.

[msripada]

This is a connection timeout message.. may be the backup time failing to establish connection within 10 seconds or may be 10 minutes i guess.... --connect-timeout', '10',...

where is the ftp server located? is it of same subnet as vcsa?

Thanks,

MS

[insearchof]

Still not working

[kurtd]

I've had similar issues with FTP\FTPS.  Webdav has been much more reliable for me.