Failed SSH Check

[vcdnewbie]

Greeting,

Got error ` Failed SSH Check``

I did the steps which followed vmmark 2.5 manual page 108 for esxi 5.x

Please help me take a look what is wrong,

online waiting... result file attached

Thanks

newbie

[Rebecca Yo]

Hi,

Make sure you can ping your hosts from the prime client, using the hostnames
vm07.nubav.thinkon.net

vm08.nubav.thinkon.net

vm09.nubav.thinkon.net

vm10.nubav.thinkon.net

Right now your results indicate that your prime client cannot resolve these hostnames.

You may need to add the hostnames to your prime client's hosts file.

When you complete the instructions to set up passwordless SSH, make sure you are logging into the system using its hostname.

Afterward, test passwordless SSH by opening a cygwin window from the prime client and running

ssh -l root ESXiSystem

where ESXiSystem is the system hostname.

If passwordless SSH has been installed correctly, you should be taken directly to the command line of the remote host.

[vcdnewbie]

thanks, ping fixed

and the ssh, I got `Are you sure you want to continue connecting (yes/no)? `

I did answer: yes

     The authenticity of host 'vm09' can't be established.    

     RSA key fingerprint is 58:d3:bf:39:aa:ef:79:20:44:6a:91:1c:f4:c9:06:30.

     Are you sure you want to continue connecting (yes/no)? yes

then I got:

     Warning: Permanently added 'vm09' (RSA) to the list of known hosts.

     VMware offers supported, powerful system administration tools.  Please

     see www.vmware.com/go/sysadmintools for details.

     The ESXi Shell can be disabled by an administrative user. See the

     vSphere Security documentation for more information.

     ~ #

I think that is the console, just want to see if the yes/no question is normal or not?

start a new run now.

thanks

newbie

[Rebecca Yo]

Yes, that's correct. On your first time connecting via SSH to a host, you must say that 'yes' you want to continue connecting.

[vcdnewbie]

thanks,

still stop on the ssh.

thanks

[Rebecca Yo]

You should always be ssh-ing into your hosts using the fully qualified domain name, rather than 'vm09'.

From cygwin on your prime client, run

ssh-keygen -R vm07.nubav.thinkon.net

ssh-keygen -R vm08.nubav.thinkon.net

ssh-keygen -R vm09.nubav.thinkon.net

ssh-keygen -R vm10.nubav.thinkon.net

You can also just delete the file C:\cygwin\home\<your username>\.ssh\known_hosts

Then, test passwordless SSH again as I described above (using the fully qualified domain name) to make sure you don't have to enter a password to reach the remote host's command line. Answer 'yes' you want to continue connecting. Make sure the hosts file on your prime client includes the hosts' fully qualified domain name so it resolves to its IP address.

[vcdnewbie]

thanks, but I am not sure what you want me to do ?

remove the key do again?

I can ping the FQDN. the hosts file has the information for FQDN

I did test the ssh command on all the vmware hosts, all working.

ssh -l root ESXiSystem

vm07 and vm08 need to answer `YES`, vm09 and vm10 not

newbie

[Rebecca Yo]

It's ok, just delete the file on your prime client C:\cygwin\home\<your username>\.ssh\known_hosts

From cygwin, run ssh [e-mail] "vmware -v" and answer 'yes' at the prompt. Do the same for each of your hosts using the fully qualified domain name. You should see the host's version of ESX as output. If you do, re-run VMmark.

Thanks,

Rebecca

[vcdnewbie]

Rebecca,

on prime client:

I did not find ``C:\cygwin\home\<your username>\.ssh\known_hosts`

nothing in C:\cygwin\home .  just empty. so no delete action.

command:
ssh [e-mail] "vmware -v"

yes, I just launch it, do not need to answer yes,

it just show the hosts edition information

I did all the 4 hosts using FQDN. they all same, just show the host edition

I did not change anything, any way just try to launch it again

newbie

[vcdnewbie]

Rebecca,

Same error in ssh: ( Note: domain name masked to aaaa.bbbbbbb.ccc)

20130808-00:05:40    Info : vm08.aaaaa.bbbbbbb.ccc SSH Check 1 :

20130808-00:05:40    Info : vm07.aaaaa.bbbbbbb.ccc SSH Check 1 :

20130808-00:05:40    Info: machName = DS2WebC0

20130808-00:05:40    Info : vm10.aaaaa.bbbbbbb.ccc SSH Check 1 :

20130808-00:05:40    Info: machName = DS2DB0

20130808-00:05:40    Info : vm09.aaaaa.bbbbbbb.ccc SSH Check 1 :

20130808-00:06:50    VMmark2 PowerCLI Service Started Successfully

20130808-00:06:50    [u'Error : vm08.aaaaa.bbbbbbb.ccc Failed SSH Check :: ', u'Error : vm07.aaaaa.bbbbbbb.ccc Failed SSH Check :: ', u'Error : vm10.aaaaa.bbbbbbb.ccc Failed SSH Check :: ', u'Error : vm09.aaaaa.bbbbbbb.ccc Failed SSH Check :: ']

Could you please take a look.

thanks

newbie

[Rebecca Yo]

OK we are getting closer. SSH seems to be working correctly but something is odd on the prime client. The VMmark harness is expecting /home/Administrator/.ssh, which you should have created during the SSH setup, but it doesn't exist.

When you followed the directions in the Benchmarking Guide to set up passwordless SSH, you should have created the directory /~/root/.ssh, (which creates the directory C:\cygwin\home\Administrator\.ssh) and generated an SSH key on the prime client (id_rsa.pub) using ssh-keygen -t rsa.

The id_rsa.pub then must be transferred over to the host to enable passwordless SSH, which it appears you did.

However, you don't have C:\cygwin\home\Administrator\.ssh\ on your prime client. I assume the ssh directory is someplace else instead.

From a cygwin window on your prime client, can you please try running

ssh-keygen -t rsa

You will see the line "Enter file in which to save the key (path):

What is the listed "path"? You can then press control-c to abort the command.

Thanks,

Rebecca

[vcdnewbie]

here you are

[vcdnewbie]

do I need to create that folder (manual) then put file in it?

[Rebecca Yo]

Yes, please manually create the folder C:\cygwin\home\Administrator\.ssh

and then move the id_rsa and id_rsa.pub (the same one you transferred onto the hosts) into this folder and re-run.

[vcdnewbie]

re-run and look good, but it stop again

20130808-16:34:55    Error: Could not find all VMs needed for Storage VMotion : NumMissing 1

20130808-17:02:23    Could not complete Setup for the following 1 Wklds: ['svmotion failed Setup Phase']

result file attached.

Thanks

newbie

[Rebecca Yo]

Okay, good, so that solved your SSH issue.

The file VC-listVMs.txt shows that the VM needed for sVmotion, Standby0, is not present in the cluster. Is Standby0 located in a different cluster or powered off?

[vcdnewbie]

no standby0 is in the same cluster: vmmark as the others and it is powered on, second screenshot I did from prime client, you can see standby0 is in the vmmark cluster.

      

Will be possible some BUG?

I think I am very close to the final report.

Thanks

newbie

[vcdnewbie]

Greeting,

I did not change anything, but like that issue gone,

I spent about 3.45 hours, finish the run.

but with some errors.

try to attach the result file, but after zip still over 100mb, I can not attach it.

I can only attach the stax job file, all the error inside it

By the way, where is the report file?

I found a file "vcsupport-522de7cf-54a3-ad3f-3c8e-0cdb5b2ef999_Reporter"

newbie

[Rebecca Yo]

Hi vcdnewbie,

Your issue with Standby definitely seems odd; I've never seen something like that before. Let us know if it pops up again.

Your reporter files will be named

vm08-20130809-1843.tgz (where the timestamp will be slightly different).

and so on for your other hosts.

And vcsupport-*_Reporter is one of the reporter files too.

It's still important for you to provide the results zip file to us, because the STAX log alone leaves a lot of information out.

To attach the results file, simply delete as many reporter .tgz files from the results as you need to make the zip file small enough.

The results zip file will be small enough once you delete some or all of the reporter files.

Thanks,

Rebecca

[vcdnewbie]

result file attached

Thanks

newbie

[Rebecca Yo]

Hi,

Thanks for attaching your results. It's good that VMmark is running, but there are still things that will need to be fixed here. Here are some things to look at:

1. It looks like the DVDstore database has not been initialized. Please see the section ”Unknown database” Message in ConfigDS2DB Text File" in the VMmark Benchmarking Guide.

2. Your standby0 VM is not being detected sometimes. It looks like sometimes VC is recognizing it and sometimes it is not. As a result, the SVMotion working is sometimes working correctly, and other times showing an error. It looks like a problem specific to your VC.

3. Please check the java installation on your client, including the environment variables. You are showing errors such as "

'java' is not recognized as an internal or external command, operable program or batch file." from the files which originate on your client.

Thanks,

Rebecca

[vcdnewbie]

Hi Rebecca

thanks for your feedback.

1. I follow the Guide, on page 154 I did step3

         

  step 4

step 6

          

But for step 7 and 8, I do not know how to do that, yes, I did use template virtual machine

Could you please give me the detail commands?

2. VC issue will check later.

3. JAVA issue I think you point to standby0, if yes, there is Java there, and I have java_home point to c:\program files\java\jre7

thanks

newbie

[Rebecca Yo]

Hi Newbie,

Keep in mind that all instructions in the VMmark Benchmarking guide need to be followed exactly for a successful result.

Yes, I can give detailed commands.

7

vi /etc/fstab

Press 'i', then using the cursor, delete the line that contains the comment "Remove after DB setup"

Press Esc

":wq" Press enter.

8

In the web client, right click on DS2DB VM and click "Edit Settings..."

Hover the mouse cursor over the 20GB hard disk, then click the little X on the right to remove the temporary disk.

Select the checkbox that says "Delete files from datastore".

Click OK.

2. OK.

3. The same error appears in Standby0.wrf and OlioWeb0.wrf. The error is coming from the client. The VMmark benchmarking guide has an entry in the troubleshooting section for this error, see "Error: ‘java’ is not recognized as an internal or external command..." You may have installed the the 32-bit version of STAF on a 64-bit operating system. If that is not the problem, double check your java installation on your client.

Thanks,

Rebecca

[vcdnewbie]

thanks, for the vi /etc/fstab,  will do that soon,

I can verify the standby0 again for the java issue, but how about the olioweb0? it was pre-build, how can I change it?

newbie

[Rebecca Yo]

The java error is coming from the client, not Standby0 or OlioWeb0.

On your client0, you need to verify that

1. You installed the correct version of STAF for your client0 operating system (Did you install 32 bit staf on a 64 bit operating system?)

and

2. That you installed the correct version of Java on client0 (correct bitness) and that you set the environment variable JAVA_HOME correctly. See instructions "Install a Java JRE" in the benchmarking guide.

To double check Java installation, open a command prompt and run java -version.

[vcdnewbie]

Rebecca,

What is Client0? Is it prime client?

if it is, then here is the information: I think it is 64 bit

below is from standby0

which one is correct?  jre7\bin or only \jre7?

Am I good to start a new run?

newbie

[Rebecca Yo]

For you, JAVA_HOME should be C:\Program Files\Java\jre7.

Yes, the operation on DS2DB0 is correct.

According to your results files, you are running virtualclient2 with your tile0 VMs. Normally, you should run client0 with tile0 VMs, to keep things organized.

The Prime Client contains the VMmark harness and has STAX installed. The Prime Client must be a physical machine.

The Client0 places load onto the tile0 workload VMs. The Client0 can be virtual or physical.

The Prime Client and Client0 can be combined into one machine/VM, or they may be two separate machines/VMs. You decided whether to make them the same or separate when you set up VMmark. So when you ask "Is Client0 the prime client?", the answer may be yes or no depending on how you set up VMmark. The role of client0 is different from prime client but they can be combined into the same machine.

For you, virtualclient2 is serving the role of client0 because it is placing load onto the tile0 VMs.

So, on your virtualclient2, please verify that:

1. You installed the correct version of STAF for your client0 operating system (Did you install 32 bit staf on a 64 bit operating system?)

and

2. That you installed the correct version of Java on client0 (correct bitness) and that you set the environment variable JAVA_HOME correctly. See instructions "Install a Java JRE" in the benchmarking guide.

To double check Java installation, open a command prompt and run java -version.

[vcdnewbie]

Greeting,

May I know where I can find the report as below:

I did check c:\vmmakrk2\results, can not find any file like this.

Thanks

[vcdnewbie]

Could you please give me the detail process on how to patch vmmark? I think I may need to patch it.

[vcdnewbie]

Error message

20130816-03:49:37    STAXProcessStartError signal raised. Continuing job.

===== XML Information =====

File: c:\vmmark2\xml\generic_functions.xml, Machine: local

Line 653: Error in element type "process".

The process failed to start, RC: 16, STAFResult: STAFConnectionProviderConnect: Timed out connecting to endpoint: select() timeout: 22, Endpoint: ssl://standby0

===== Call Stack for STAX Thread 160 =====

[

  sequence: 2/2 (Line: 1206, File: C:\VMmark2\xml\vmmark2_main.xml, Machine: local://local)

  if: workloadtype == 'WinWorkload' (Line: 1215, File: C:\VMmark2\xml\vmmark2_main.xml, Machine: local://local)

  sequence: 1/1 (Line: 1216, File: C:\VMmark2\xml\vmmark2_main.xml, Machine: local://local)

  function: GetWindowsGuestInfo (Line: 638, File: c:\vmmark2\xml\generic_functions.xml, Machine: local)

  sequence: 3/18 (Line: 647, File: c:\vmmark2\xml\generic_functions.xml, Machine: local)

]

Results file attached

Thanks

newbie

[Rebecca Yo]

Regarding Table 1-3 and Table 1-5: You can find scores for Mail server (mailserver), Olio server (olio), Standby server (standby), DS2WebA (dvdstoreA), DS2WebB (dvdstoreB), DS2WebC (dvdstoreC), Clone and deploy (deploy), vMotion (vmotion) and Storage vMotion (svmotion) all inside Score_N_Tile_Test.out in your results folder. You can find the operations/transactions/actions per minute in the table titled "TILE_0_Scores".

Yes, the patch version, 2.5.1 is the latest version of VMmark which you should be running.

To patch VMmark, please download VMmark2.5.1-Patch-06062013 here, and follow the instructions in the ReadMe.txt. Also, please use the 2.5.1 version of the VMmark Benchmarking Guide.

Regarding the error "The process failed to start, RC: 16, STAFResult: STAFConnectionProviderConnect: Timed out connecting to endpoint: select() timeout: 22, Endpoint: ssl://standby0"

In your VMMARK2.CONFIG file, you designate Standby0 as the deployVM. Standby0 cannot be the deploy VM; try "DeployVM1". As the Benchmarking guide states "This specification must allow a newly deployed virtual machine to be able to be customized and then pinged by the prime client, not conflicting with any other virtual machine names or IP addresses." The Deploy VM needs to have its own VM name, hostname, and IP address that is completely different from Standby0. Please review the VMmark Benchmarking guide "Prepare the vSphere vCenter Server for the Deploy Infrastructure Operations".

Also, virtualclient0 is still showing issues with Java, so please troubleshoot it as I described in the previous post.

Thanks,

Rebecca

[vcdnewbie]

thanks, yes, find that Score_N_Tile_Test.out file but how to open it. (which software need to use to open it, get the result clear) ?

thanks

newbie

will make all the change then run again

by the way to patch it is good idea or not? what I need to backup in case some thing wrong?

[Rebecca Yo]

Great, you should open Score_N_Tile_Test.out in a text editor of your choice such as notepad or wordpad.

[vcdnewbie]

thank, but some errors there.

patch it is good idea or not? what I need to backup in case some thing wrong?

[Rebecca Yo]

Yes, you should run the patch. The patch is very safe, but if you still want to backup, you can always copy your C:\VMmark2 folder to C:\VMmark2_backup. Then run the patch which modifies C:\VMmark2.

[vcdnewbie]

is the patch only run on prime client?

[vcdnewbie]

very very strange, screenshot took from prime client, you can see I had java installed and JAVA_HOME, path, but when I try to patch it I got error:

"Could not find or load main class"

all the files from patch ZIP file in the current folder

Thanks

newbie

[Rebecca Yo]

Yes, please unzip the patch file on the prime client and follow the enclosed directions on the prime client only.

Also, when you run the 2.5.1 patch, the VMMARK2.CONFIG file will be overwritten. You will want to backup your VMMARK2.CONFIG so you can copy your configuration information out of it into the new VMMARK2.CONFIG file (note, do not use the old VMMARK2.CONFIG file)

[vcdnewbie]

could you please check my previous post, I can not patch

[Rebecca Yo]

You'll want to start the .jar from the windows GUI rather than command prompt. Please follow the instructions in ReadMe which specify to double click on the .jar file, thank you.

[vcdnewbie]

reason why I am using DOS. because in windows GUI, nothing happened. a black box opened then close.

[Rebecca Yo]

Try running java -jar VMmark2.5.1-Patcher.jar.

[vcdnewbie]

works

[vcdnewbie]

too hard,my understand the prime client is 64 bit

the ptd-wind-x86.exe I need the run is x86, from "readme.txt"

  6.  Upgrading the ptd-windows-x86.exe binary to 1.6.0 requires manual user interaction.

      Copy the ptd-windows-x86.exe file from the ManualUpgrades folder, found in same directory

      as this readme.txt, to each of the PTD_Clients in your environment at

      C:\vclient\ptd\ptd-windows-x86.exe.

but when I run it got error

is it for the power only? if yes, I can skip it I think

newbie

[Rebecca Yo]

You're just supposed to copy the file ptd-windows-x86.exe from ManualUpgrades into C:\vclient\ptd\ptd-windows-x86.exe on each of your clients, to update the ptd-windows-x86.exe file version.

You're not supposed to run ptd-windows-x86.exe manually. The VMmark harness is responsible for running ptd-windows-x86.exe.

[vcdnewbie]

patched to latest edition.

start a new run, it stop, I can not find anything in the results folder, what is wrong?

thanks

newbie

File in c:\vmmark2\tmp\ VMmarkMSG-warnings

VMmark Msg::

Message TimeOut : 300seconds

Mon Aug 19 15:19:08 EDT 2013

Message Contents : Terminating job due configuration Errors : Error: Could not resolve VMMARK/standby0/SERVERS

Mon Aug 19 15:24:13 EDT 2013

Message TimedOut

---------------------------------------------------

20130819-15:19:08    Error: Could not resolve VMMARK/standby0/SERVERS

20130819-15:19:08    Default: VMMARK/CONFIGDIR = c:\vmmark2

20130819-15:19:08    Default: VMMARK/TMPDIR = c:\vmmark2\tmp

20130819-15:19:08    Default: VMMARK/CLIENTROOTDIR = C:\\vclient

20130819-15:19:08    Terminating job due configuration Errors : Error: Could not resolve VMMARK/standby0/SERVERS

20130819-15:19:08    VMmark2Msg Terminating job due configuration Errors : Error: Could not resolve VMMARK/standby0/SERVERS

20130819-15:19:08    VMmark2Msg : Automatic Message TimeOut in 300 Seconds

[Rebecca Yo]

This topic is being continued in a different thread:

https://communities.vmware.com/thread/455019