DX NetOps

 View Only

  • 1.  Error Backup

    Posted May 07, 2026 09:21 PM

    Dear Community, could you guide me with this error that appears in the backup log? As background, it works correctly, I just don't know what changed.

    sdff@ithfhwds3 vbr]$ more vbr_2026-05-07-172229_XJV3DRDC.log
    2026-05-07 17:22:29 localhost vbr VBR log initialized.
    2026-05-07 17:22:29 localhost vbr Command line: /opt/vertica/bin/vbr.py --task backup --config-file /opt/vertica/bin/backup_diario_vertica_pm.ini
    2026-05-07 17:22:29 localhost vbr Getting cluster information.
    2026-05-07 17:22:30 localhost vbr Getting information for database: drdata.
    2026-05-07 17:22:32 localhost vbr Launching rsync daemon on "ithfhwdnpmrap03".
    2026-05-07 17:22:32 localhost backup_python ['ssh', '-T', '-x', '-o', 'ServerAliveInterval=60', '-o', 'BatchMode=yes', '-o', 'ExitOnForwardFailure=yes', '-o', 'PasswordAuthentication=no', '-o', 'StrictHostKeyChecking=no', '-oNoHostAuthe
    nticationForLocalhost=yes', 'ithfhwdnpmrap03', '/opt/vertica/oss/python/bin/python -V || true']
    2026-05-07 17:22:33 localhost backup_python ['ssh', '-T', '-x', '-o', 'ServerAliveInterval=60', '-o', 'BatchMode=yes', '-o', 'ExitOnForwardFailure=yes', '-o', 'PasswordAuthentication=no', '-o', 'StrictHostKeyChecking=no', '-oNoHostAuthe
    nticationForLocalhost=yes', 'ithfhwdnpmrap03', '/opt/vertica/oss/python/bin/python']
    2026-05-07 17:22:33 localhost rsyncd_launcher Output from rsync launcher:
    2026-05-07 17:22:33 localhost rsyncd_launcher Transfer Server process entry: pid 116838.
      Acquiring remoteServer mutex.
      Acquired remoteServer mutex.
      ps aux | grep rsync | grep -v grep | grep daemon | grep port=50000
      
      A new Rsync daemon has started.
      Released mutex.
      
    2026-05-07 17:22:33 localhost vsql /opt/vertica/bin/vsql -q -t -X -ddrdata -p5433 -Upmadmin -h10.59.2.210
    2026-05-07 17:22:33 localhost vbr Checking node connectivity.
    2026-05-07 17:22:36 localhost vbr All nodes reachable.
    2026-05-07 17:22:36 localhost vbr Making temp directories.
    2026-05-07 17:22:36 localhost vbr Making lock directories.
    2026-05-07 17:22:36 localhost vbr Locking nodes.
    2026-05-07 17:22:36 localhost vbr Locked nodes.
    2026-05-07 17:22:37 localhost backup_python ['ssh', '-T', '-x', '-o', 'ServerAliveInterval=60', '-o', 'BatchMode=yes', '-o', 'ExitOnForwardFailure=yes', '-o', 'PasswordAuthentication=no', '-o', 'StrictHostKeyChecking=no', '-oNoHostAuthe
    nticationForLocalhost=yes', 'ithfhwdnpmrap03', '/opt/vertica/oss/python/bin/python -V || true']
    2026-05-07 17:22:37 localhost backup_python ['ssh', '-T', '-x', '-o', 'ServerAliveInterval=60', '-o', 'BatchMode=yes', '-o', 'ExitOnForwardFailure=yes', '-o', 'PasswordAuthentication=no', '-o', 'StrictHostKeyChecking=no', '-oNoHostAuthe
    nticationForLocalhost=yes', 'ithfhwdnpmrap03', '/opt/vertica/oss/python/bin/python -V || true']
    2026-05-07 17:22:37 localhost backup_python ['ssh', '-T', '-x', '-o', 'ServerAliveInterval=60', '-o', 'BatchMode=yes', '-o', 'ExitOnForwardFailure=yes', '-o', 'PasswordAuthentication=no', '-o', 'StrictHostKeyChecking=no', '-oNoHostAuthe
    nticationForLocalhost=yes', 'ithfhwdnpmrap03', '/opt/vertica/oss/python/bin/python -V || true']
    2026-05-07 17:22:37 localhost backup_python ['ssh', '-T', '-x', '-o', 'ServerAliveInterval=60', '-o', 'BatchMode=yes', '-o', 'ExitOnForwardFailure=yes', '-o', 'PasswordAuthentication=no', '-o', 'StrictHostKeyChecking=no', '-oNoHostAuthe
    nticationForLocalhost=yes', 'ithfhwdnpmrap03', '/opt/vertica/oss/python/bin/python']
    2026-05-07 17:22:38 localhost backup_python ['ssh', '-T', '-x', '-o', 'ServerAliveInterval=60', '-o', 'BatchMode=yes', '-o', 'ExitOnForwardFailure=yes', '-o', 'PasswordAuthentication=no', '-o', 'StrictHostKeyChecking=no', '-oNoHostAuthe
    nticationForLocalhost=yes', 'ithfhwdnpmrap03', '/opt/vertica/oss/python/bin/python']
    2026-05-07 17:22:38 localhost backup_python ['ssh', '-T', '-x', '-o', 'ServerAliveInterval=60', '-o', 'BatchMode=yes', '-o', 'ExitOnForwardFailure=yes', '-o', 'PasswordAuthentication=no', '-o', 'StrictHostKeyChecking=no', '-oNoHostAuthe
    nticationForLocalhost=yes', 'ithfhwdnpmrap03', '/opt/vertica/oss/python/bin/python']
    2026-05-07 17:22:50 localhost vsql /opt/vertica/bin/vsql -q -t -X -ddrdata -p5433 -Upmadmin -h10.59.2.210
    2026-05-07 17:22:50 localhost location_locker Error terminating locker process: [Errno 3] No such process.
    2026-05-07 17:22:50 localhost location_locker Error terminating locker process: [Errno 3] No such process.
    2026-05-07 17:22:50 localhost location_locker Error terminating locker process: [Errno 3] No such process.
    2026-05-07 17:22:50 localhost vbr Error: Error locking backup location.  Another vbr task is currently running: unknown.
      Backup FAILED.
    [sfffdf@sdfdvbr]$ 

    I performed a manual execution to see if it would work



    -------------------------------------------


  • 2.  RE: Error Backup

    Broadcom Employee
    Posted 29 days ago
    Edited by Mohammed Alfakhrany 29 days ago

    Hi Jose,

    This error occurs because the Vertica Backup and Restore (vbr.py) utility detects that your backup destination is currently "locked."

    Whenever a Vertica backup runs, it places a hidden lock file in the backup destination directory to prevent two backups from running at the same time and corrupting your data. The message Another vbr task is currently running: unknown (especially with the No such process lines right above it) tells us that vbr sees a lock, but cannot find an active process attached to it.

    This issue is detailed in the following KB, Please go through the resolution in it:

    KB : Vertica backup fails with "Another vbr task is currently running" error in CA Performance Management (CAPM)

    After this, if there are still problems, then 

    • Check your configuration file to find your exact backup target directory. You can usually find it under the [Mapping] or [Misc] section:

      Bash
      cat /opt/vertica/bin/backup_diario_vertica_pm.ini
      
    • Navigate to that backup destination directory on your backup host  wherever the storage is mounted.

    • Look for a hidden .lock file. It will typically be located at the root of the backup directory.

    • Delete the orphaned lock file:

      Bash
      rm -f /path/to/your/backup/dir/.lock
      
    • Re-run your vbr.py command. It should now proceed normally.

    Thanks

     

    -------------------------------------------



  • 3.  RE: Error Backup

    Posted 28 days ago

    Thanks for your reply. The backup was running normally until a certain date. Regarding the .lock files, I found and deleted them, and I also checked the vbr and rsync processes to ensure they weren't interfering with the new manual run.

    In summary, my test was:

    Deleting the .lock files
    Killing the vbr and rsync processes to ensure a clean run
    Running the backup manually

    But it's still giving the error

    -------------------------------------------



  • 4.  RE: Error Backup

    Broadcom Employee
    Posted 28 days ago

    Hi Jose,

    What's the location of the backup? Is it local or Network Accessible Storage (NAS)? If NAS, has anything changed in permissions?

    Maybe take a look at /var/log/messages and search for vbr.py. See if any errors are generated at the OS level for file locking.

    -------------------------------------------



  • 5.  RE: Error Backup

    Posted 25 days ago

    The backup path is an NFS v3 drive. I can't see the log you mentioned because I don't have permission to view it with my user account. I was able to run a debug command, though.


    tKeyChecking=no', '-oNoHostAuthenticationForLocalhost=yes', 'ithfhwdnpmrap03', '/opt/vertica/oss/python/bin/python']
    2026-05-12 15:43:15 localhost backup_python ['ssh', '-T', '-x', '-o', 'ServerAliveInterval=60', '-o', 'BatchMode=yes', '-o', 'ExitOnForwardFailure=yes', '-o', 'PasswordAuthentication=no', '-o', 'StrictHo
    stKeyChecking=no', '-oNoHostAuthenticationForLocalhost=yes', 'ithfhwdnpmrap03', '/opt/vertica/oss/python/bin/python']
    2026-05-12 15:43:28 localhost vsql /opt/vertica/bin/vsql -q -t -X -ddrdata -p5433 -Upmadmin -h10.59.2.210
    2026-05-12 15:43:28 localhost vsql select log_vbr_invocations('Full Backup Task', '/tmp/vbr/vbr_2026-05-12-154307_13KEY366.log', 'KIHX40DU6GGFP75NK9AVKY5DO8MNZ71Q', 'Fail');
    2026-05-12 15:43:28 localhost vsql Read from vsql: Logged vbr invocations
    2026-05-12 15:43:28 10.59.2.208 vbr File unlocked: 6
    2026-05-12 15:43:28 10.59.2.209 vbr File unlocked: 6
    2026-05-12 15:43:28 10.59.2.210 vbr File unlocked: 6
    2026-05-12 15:43:28 localhost location_locker Error terminating locker process: [Errno 3] No such process.
    2026-05-12 15:43:28 localhost location_locker Error terminating locker process: [Errno 3] No such process.
    2026-05-12 15:43:28 localhost location_locker Error terminating locker process: [Errno 3] No such process.
    2026-05-12 15:43:28 localhost vbr Traceback (most recent call last):
        File "/opt/vertica/bin/vbr.py", line 9950, in main
          if vbr_task.run():
        File "/opt/vertica/bin/vbr.py", line 4915, in run
          self._setup_hosts()
        File "/opt/vertica/bin/vbr.py", line 5303, in _setup_hosts
          self._lock_backup_locations()
        File "/opt/vertica/bin/vbr.py", line 5426, in _lock_backup_locations
          self._backup_locker.lock_locations(self._identity)
        File "/opt/vertica/bin/vbr.py", line 9166, in lock_locations
          Parallel.foreach(lock_location, self._get_backup_lock_locations(self._database_info))
        File "/opt/vertica/bin/vbr.py", line 9632, in foreach
          cls.map(func, iterable, threads_num=threads_num)
        File "/opt/vertica/bin/vbr.py", line 9611, in map
          if not thr.join(Parallel.WAIT_QUANTUM):
        File "/opt/vertica/bin/vbr.py", line 9587, in join
          return self._thr.join(*args, **kwargs)
        File "/opt/vertica/bin/vbr.py", line 9541, in run
          super(Parallel.CancellableThread, self).run()
        File "/opt/vertica/oss/python/lib/python2.7/threading.py", line 763, in run
          self.__target(*self.__args, **self.__kwargs)
        File "/opt/vertica/bin/vbr.py", line 9575, in do_run
          self._result = func(val)
        File "/opt/vertica/bin/vbr.py", line 9154, in lock_location
          raise VbrError(err_msg)
      VbrError: Error locking backup location.  Another vbr task is currently running: unknown.

    2026-05-12 15:43:28 localhost vbr Error: Error locking backup location.  Another vbr task is currently running: unknown.

    -------------------------------------------



  • 6.  RE: Error Backup

    Posted 20 days ago

    Thank you for your response; it turned out to be the way they have NFS registered.

    -------------------------------------------