couple of vms in vApp can not power on

[vcdnewbie]

Greeting,

Just create a new vApp. include 19 windows base vms, 14 vms power on no any issue, but the other 5 can not power on.

I got error message "Internal server error"

I did restart vcloud service, no luck and even reboot the vcloud director machine, still same

can someone give me some hit where is the start point?

thanks

newbie

[Sree]

Hi,

    Internal server error is a generic error.The best way to start would be reproduce the issue and check the VCD logs carefully(vcloud-container-debug.log).Every task will have a JOB ID and you would see the same in the VCD GUI,make a note of the task and "grep" with the same in Logs for easy analysis(I have attached a screenshot of the same).On a lighter note,do you see any task getting failed in VC for those 5 VM's when you are trying to power-on?

VMware KB: Location of vCloud Director log files

[vcdnewbie]

Thanks Sreec,

in vcenter there nothing about this power on task.

I will get some log file and let you know, as I remember there are lots of java command there.,

newbie

[vcdnewbie]

thanks for the help.

I got the log file for this task as below.

Task 'JOBID' is now running

Job object - Object : APPV32-3(com.vmware.vcloud.entity.vm:710fd45c-69be-40af-980e-36127d2dd6fd) operation name: VAPP_DEPLOY

Update last job       : [APPV32-3(com.vmware.vcloud.entity.vm:710fd45c-69be-40af-980e-36127d2dd6fd)], status=[1], [null]

Target for task with job ID 'JOBID' is '710fd45c-69be-40af-980e-36127d2dd6fd'

IP address not available for VM(VC: e21018e6-125b-422a-bd96-2343ca30aaf0-Moref: vm-8910) MAC: 00:50:56:01:01:5e

IP address not found for VM(VC: e21018e6-125b-422a-bd96-2343ca30aaf0-Moref: vm-8910) MAC: 00:50:56:01:01:5e

VAppManager.deployVM(710fd45c-69be-40af-980e-36127d2dd6fd)

IP address not available for VM(VC: e21018e6-125b-422a-bd96-2343ca30aaf0-Moref: vm-8910) MAC: 00:50:56:01:01:5e

IP address not found for VM(VC: e21018e6-125b-422a-bd96-2343ca30aaf0-Moref: vm-8910) MAC: 00:50:56:01:01:5e

vApp: Deploying id=1e5416ed-aeb7-410c-85be-e44982ef7454, name=Ticket 192065, creationStatus=RESOLVED, configurationType=1, orgId=e23466ac-c2ea-4cf3-8816-b1e56e4aeb99, vdc=[id=e101551d-d044-48b1-a23c-49fdc981ca45, name=oVDC-Microsoft, enabled=true, fastProvisioning=true, thinProivisioning=true, org=[id=e23466ac-c2ea-4cf3-8816-b1e56e4aeb99, name=MS, displayName=MS], providerVdc=[id=0039f07b-30ee-40b0-9719-8a5f1d149bed, name=pVDC-Comp, highestSupportedHwVersion=9, vcenter=[[Id = e21018e6-125b-422a-bd96-2343ca30aaf0, url = https://IP/sdk, cell_instance_id = 224, uuid = 91708083-359D-41BF-8BBB-5981B1F499E7, status = READY]]]] with deployment options: com.vmware.ssdc.backend.interfaces.DeployOptions@7329aa83, honorBootOrder=false, deployPowerAction=poweron. VMs to be deployed: 1 out of 17

VM APPV32-3 has supported tools version: 9,216

VAppVm: Deploying: id=710fd45c-69be-40af-980e-36127d2dd6fd, name=APPV32-3, fabricIds=[svmId=15f18015-df20-41df-b89f-0038acf0058e, cvmId=81cffc3f-2d61-4543-aaee-0023daaba8cd, nvmId=c8a655a7-0db0-416d-b984-441f265434ee], vapp=[id=1e5416ed-aeb7-410c-85be-e44982ef7454, name=Ticket 192065, creationStatus=RESOLVED, configurationType=1, orgId=e23466ac-c2ea-4cf3-8816-b1e56e4aeb99, vdc=[id=e101551d-d044-48b1-a23c-49fdc981ca45, name=oVDC-Microsoft, enabled=true, fastProvisioning=true, thinProivisioning=true, org=[id=e23466ac-c2ea-4cf3-8816-b1e56e4aeb99, name=MS, displayName=MS ], providerVdc=[id=0039f07b-30ee-40b0-9719-8a5f1d149bed, name=pVDC-Comp, highestSupportedHwVersion=9, vcenter=[[Id = e21018e6-125b-422a-bd96-2343ca30aaf0, url = https://IP/sdk, cell_instance_id = 224, uuid = 91708083-359D-41BF-8BBB-5981B1F499E7, status = READY]]]]]

IP address not available for VM(VC: e21018e6-125b-422a-bd96-2343ca30aaf0-Moref: vm-8910) MAC: 00:50:56:01:01:5e

IP address not found for VM(VC: e21018e6-125b-422a-bd96-2343ca30aaf0-Moref: vm-8910) MAC: 00:50:56:01:01:5e

Failed to deploy VM

Is really not IP ?

By the way, I tried 1 of 5 vms, totally remove the NIC still can not power on and error is " Internal Server error"

Thanks

[Sree]

Hi,

     Since you mentioned there are no tasks getting populated in VC. Can you please follow below steps in same order.I believe this is not in production yet :smileyhappy:

1)Stop the vCD cell vCD service - service vmware-vcd stop.

2)Restart VCNS web manager GUI or complete appliance.

3)Restart the VC service(Do check if there is any back-trace kind of errors in VPXD logs)

4)Log into VCNS GUI via and update VC IP/Hostname once more(To ensure VCNS is in sync with VC)

5)Start the vCD cell vCD service – service vmware-vcd start

May i know what type of network pools you are using in this Org-VDC?

[vcdnewbie]

May I know any impact?

I have another VDC is working well in the same vcloud director.

Thanks

[Sree]

Hi,

     There will be certainly an impact(Portal Access,VC service availability) if you perform below tasks.Since you mentioned there is another VDC working flawlessly i would request you to not to try what i suggested until i have some precise information.

1.What Kind of Network pools are used in both the VDC?

2.VM's which are failing to power on,can you tell me how you created those VM's(Import/Catalog/Manual etc)

3.If you create a new VM in both the VDC,does it fail with same error or are you sure that issue is specific to the VDC what you reported?

[vcdnewbie]

my understand

A1. VLAN-backed

A2. VMs are create by the templates with is the same process with the other working vms

A3. Other Organization no problem, I can still power vms on.

      on this "Problem" Organization, I did try same process to create 2 new vms, still same problem, can not "Power on" and "Internal Server Error"

Thanks

[Sree]

Hi,

    Ideally we will see No IP found errors when the network Pool(In your case vlan) is running shot of networks.But you also mentioned you tried removing the NIC's completely from the VM and it still failed to Power on.Can you post me the logs for those tasks?

1.Ensure you have enough networks in VLAN network pool that you have defined for the failing Org_VDC.

2.Create a new VM(not from template),with MB's of Memory/Storage without any NIC's and try to power on.

3.If the above task is failing please do post fresh logs for the failed JOB-ID.

[vcdnewbie]

I just create a new vapp and one vm in the "Problem" VDC, I manually create it, NO NIC. attached the windows 7 ISO file as CD. try to turn on got the same issue, "Internal Server error"

vm name: 'vmjx20150205'' (976b3d0f-dd51-4355-9276-3a358bd9cee5)

Job ID: 8fe40806-e856-4f57-940a-107e248b6f1c

below is what i found in the log file. start from the first Job ID.

Created new task with job ID '8fe40806-e856-4f57-940a-107e248b6f1c'

Trying to set entities [vApp_jxu(com.vmware.vcloud.entity.vapp:958ce135-a619-4fae-9eac-1c14e45ef619), vmjx20150205(com.vmware.vcloud.entity.vm:976b3d0f-dd51-4355-9276-3a358bd9cee5)] as busy for task ref VAPP_DEPLOY(com.vmware.vcloud.entity.task:8fe40806-e856-4f57-940a-107e248b6f1c)

executing local task VAPP_DEPLOY(com.vmware.vcloud.entity.task:8fe40806-e856-4f57-940a-107e248b6f1c)

Executing task '8fe40806-e856-4f57-940a-107e248b6f1c' with operation VAPP_DEPLOY

submitted task LocalTask [taskRef=VAPP_DEPLOY(com.vmware.vcloud.entity.task:8fe40806-e856-4f57-940a-107e248b6f1c)] to local scheduler. current scheduled count=1

preProcessTask requested VAPP_DEPLOY(com.vmware.vcloud.entity.task:8fe40806-e856-4f57-940a-107e248b6f1c)

No active collaboration found for task VAPP_DEPLOY(com.vmware.vcloud.entity.task:8fe40806-e856-4f57-940a-107e248b6f1c).

Authorizing method: public abstract com.vmware.vcloud.api.presentation.entity.common.TaskSpec com.vmware.vcloud.api.presentation.service.TaskService.getTaskSpec(com.vmware.vcloud.api.presentation.entity.common.EntityRef).

No enabled endpoints for task VAPP_DEPLOY(com.vmware.vcloud.entity.task:8fe40806-e856-4f57-940a-107e248b6f1c). Skip pre-processing.

Transaction rolled back; session has been closed

Task '8fe40806-e856-4f57-940a-107e248b6f1c' is now running

Job object - Object : vmjx20150205(com.vmware.vcloud.entity.vm:976b3d0f-dd51-4355-9276-3a358bd9cee5) operation name: VAPP_DEPLOY

Update last job       : [vmjx20150205(com.vmware.vcloud.entity.vm:976b3d0f-dd51-4355-9276-3a358bd9cee5)], status=[1], [null]

Target for task with job ID '8fe40806-e856-4f57-940a-107e248b6f1c' is '976b3d0f-dd51-4355-9276-3a358bd9cee5'

VAppManager.deployVM(976b3d0f-dd51-4355-9276-3a358bd9cee5)

vApp: Deploying id=958ce135-a619-4fae-9eac-1c14e45ef619, name=vApp_jxu, creationStatus=RESOLVED, configurationType=1, orgId=e23466ac-c2ea-4cf3-8816-b1e56e4aeb99, vdc=[id=e101551d-d044-48b1-a23c-49fdc981ca45, name=oVDC-Microsoft, enabled=true, fastProvisioning=true, thinProivisioning=true, org=[id=e23466ac-c2ea-4cf3-8816-b1e56e4aeb99, name=MS, displayName=MSSS ], providerVdc=[id=0039f07b-30ee-40b0-9719-8a5f1d149bed, name=pVDC-Compugen, highestSupportedHwVersion=9, vcenter=[[Id = e21018e6-125b-422a-bd96-2343ca30aaf0, url = https://vcenterIP:443/sdk, cell_instance_id = 224, uuid = 91708083-359D-41BF-8BBB-5981B1F499E7, status = READY]]]] with deployment options: com.vmware.ssdc.backend.interfaces.DeployOptions@2c72dd15, honorBootOrder=false, deployPowerAction=poweron. VMs to be deployed: 1 out of 1

VAppVm: Deploying: id=976b3d0f-dd51-4355-9276-3a358bd9cee5, name=vmjx20150205, fabricIds=[svmId=462d4f63-dfa6-41a7-b9a1-e59a4712f9fe, cvmId=ab1860bf-2d0b-4c2a-b1da-0ca2f9ba4e44, nvmId=aa434df6-57a6-44d1-b341-45be385fb674], vapp=[id=958ce135-a619-4fae-9eac-1c14e45ef619, name=vApp_jxu, creationStatus=RESOLVED, configurationType=1, orgId=e23466ac-c2ea-4cf3-8816-b1e56e4aeb99, vdc=[id=e101551d-d044-48b1-a23c-49fdc981ca45, name=oVDC-Microsoft, enabled=true, fastProvisioning=true, thinProivisioning=true, org=[id=e23466ac-c2ea-4cf3-8816-b1e56e4aeb99, name=MS, displayName=MSSS ], providerVdc=[id=0039f07b-30ee-40b0-9719-8a5f1d149bed, name=pVDC-Compugen, highestSupportedHwVersion=9, vcenter=[[Id = e21018e6-125b-422a-bd96-2343ca30aaf0, url = https://vcenterIP:443/sdk, cell_instance_id = 224, uuid = 91708083-359D-41BF-8BBB-5981B1F499E7, status = READY]]]]]

Failed to deploy VM

Thanks for your time

[Sree]

Hi,

    Is the VC inventory still syncing? VCD Manage & Monitor in System Organization, one of the entries continually shows the Synchronizing Inventory ? Please do reconnect the VC connectivity from VCD and let me know the results.

[vcdnewbie]

sorry, may I have a screenshot for you are looking for? thanks

[Sree]

Hi,

    You can find the attached.Firstly,confirm the Status is showing Green tick mark or Syncing.Adding to that right click on VC name and Reconnect the same and re-try your operations.

[vcdnewbie]

here you are,

did reconnect. result same as the picture

[Sree]

Does it still fail ? From what i know this is a VC connectivity issue "No enabled endpoints for task" .How feasible is it for you to reboot the VC machine OR restart the VC service?

Are you using vCloud Director appliance ? Please post the below

1) VCD,VC,VCNS versions.

[vcdnewbie]

if you can help me understand the log file that will be Great!