virtualeverything

thoughts and musings regarding virtualizing IT

Getting the VMware VSA running in a nested ESXi environment

Posted by veverything on August 17, 2011

In the previous VSA article we took at a look at the storage architecture of the appliance, as well as some of the caveats and considerations when deploying it. In this article, we’ll take a look at how to get it up and running in a nested ESXi environment as well as some of the functions the VSA provides.

First, in order to create a nested ESXi 5.0 environment, have a look at this great article.

When creating your environment, my recommendation is to create 4 individual vDS port groups or individual standard 4 vSwitches for the environment. You will assign each to a vNIC of the vESXi host to simulate connecting each pNIC to a physical switch in a real deployment.

Be sure to configure the vSwitches (or vDS port groups) with promiscuous mode enabled and create 2 vESXi VMs with 4 NICs minimum and a SINGLE VMFS volume (this is important or else the VSA will not install). I recommend a thin provisioned volume of about 200GB for testing.

You should end up with something like this:

Same applies if you are using standard vSwitches in your environment.

Now you need a Windows based vCenter 5.0 instance to manage this environment. Install the VSA manager software onto that vCenter which will then expose the VSA manager plug-in/tab on the vCenter client once you click on a vSphere data center:

In normal installations, you would then click on the VSA manager tab and follow the instructions to install. The problem is that since we are installing in a nested vESXi environment, “EVC” does not work with nested vESXi and is a requirement the installer checks for, thus you will not be able to proceed:

Thus far I have not been able to find a workaround for this for the GUI based install. However, after lots of lab time I found there is a way around this problem: in order to install the VSA in nested ESXi and bypass the EVC requirement, we need to tweak a configuration file and then do the installation via command line. Download the full zipfile which includes the command line installer if you haven’t already and unzip that onto your system.

Here is the minimum syntax to get it going:

install.exe -u root -p <password_to_ESX_hosts> -si <start_address_for_VSA_front_end_IPs> -nh

Recall that the VSA has a front-end network and a back-end network. The “-si” switch tells it what public IPs to use for the front-end. You can specify a “-bs” start range for the back-end IPs, but it will default to 192.168.0.1 as the start range if you do not specify anything. You can also specify netmasks and VLANs. See the manual for details.

The “-nh” tells it not to join the hosts into a high availability cluster and this will be important to help bypass the EVC check. If we execute this command this will be the result:

As you can see the automated command line installer runs an audit stage and it fails for the same EVC reason!

Well, after much lab time, I figured a way around this problem. We need to change a parameter in C:\Program Files\VMware\Infrastructure\tomcat\webapps\VSAManager\WEB-INF\classes\dev.properties. Search for this line:

evc.config=true

and change it to

evc.config=false

This will effectively bypass the audit check for EVC. Cool huh?

Now re-run the install.exe command, and it should complete:

And you end up with this in your nested ESXi environment:

The result is 2x 100GB data stores, which correlates with each VSA having 200GB of RAW storage, for a total of 400GB RAW or 200GB usable after RAID10 internal to the VSA.

 

Here is a peak at the networking the VSA installer sets up:

There are front-end and back-end port groups that live on separate vSwitches and pNICs. You are now free to customize the networking however you see fit, but it HAS to have a default configuration starting out or else the install WILL fail.

Now that the VSA is installed, you can continue to manage it through the VSA plug-in in vCenter. We only needed to do the hack and command line to get it up and running. Again, its important to note this would not be required in a real installation, it was required due to the limitations of nested ESXi.

End result:

In the next article, we’ll take a look at some administrative tasks, and testing out some of the failure scenarios and how the VSA handles it from a downtime/uptime/reliability perspective.

Advertisement

5 Responses to “Getting the VMware VSA running in a nested ESXi environment”

  1. Hi There,

    Tremendous post, thank you!

    I am trying to set up the VSA in a nested ESXi VM, only difference is that its inside VMware Fusion 4.0.1. I have followed the steps you have mentioned, but run into a problem with the following error : Scripted installation failed com.vmware.vim25.DuplicateName

    I would truly appreciate some assistance. Thanks in advance!

    Microsoft Windows [Version 6.1.7600]
    Copyright (c) 2009 Microsoft Corporation. All rights reserved.

    C:\Users\Administrator>d:

    D:\>install.exe -u root -p password -si 192.168.20.161 -ei 192.168.20.151 192.16
    8.20.153 -nh
    Current user: Administrator
    Current IP: 192.168.20.200
    Virtual Center is already installed
    The VMware VirtualCenter Management Webservices service is stopping.
    The VMware VirtualCenter Management Webservices service was stopped successfully
    .

    Installing VSA Manager, please wait …
    The requested service has already been started.

    More help is available by typing NET HELPMSG 2182.

    2011-09-22 16:27:04,910
    2011-09-22 16:27:04,956 Start to scan ESX hosts, please wait …
    2011-09-22 16:27:05,003 localhost netmask: /24
    2011-09-22 16:27:05,010 localhost broadcast: /192.168.20.255
    2011-09-22 16:27:09,693 Total IP in subnet: 255
    2011-09-22 16:27:09,694 start IP: 192.168.20.0
    2011-09-22 16:27:09,695 end IP: 192.168.20.255
    2011-09-22 16:27:25,439 Create Datacenter : vsaDC
    2011-09-22 16:27:25,464 Add host to SVA Datacenter, please wait …
    2011-09-22 16:27:26,702 Scripted installation failed.
    com.vmware.vim25.DuplicateName
    at com.vmware.vim25.DuplicateName$Factory.parse(DuplicateName.java:571)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
    java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
    sorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.vmware.vim25.ExtensionMapper.getTypeObject(ExtensionMapper.java:2
    8)
    at com.vmware.vim25.MethodFault$Factory.parse(MethodFault.java:738)
    at com.vmware.vim25.LocalizedMethodFault$Factory.parse(LocalizedMethodFa
    ult.java:652)
    at com.vmware.vim25.TaskInfo$Factory.parse(TaskInfo.java:2367)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
    java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
    sorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.vmware.vim25.ExtensionMapper.getTypeObject(ExtensionMapper.java:2
    8)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
    java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
    sorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.vmware.vim25.ConverterUtil.getAnyTypeObject(ConverterUtil.java:15
    89)
    at com.vmware.vim25.PropertyChange$Factory.parse(PropertyChange.java:762
    )
    at com.vmware.vim25.ObjectUpdate$Factory.parse(ObjectUpdate.java:891)
    at com.vmware.vim25.PropertyFilterUpdate$Factory.parse(PropertyFilterUpd
    ate.java:830)
    at com.vmware.vim25.UpdateSet$Factory.parse(UpdateSet.java:818)
    at com.vmware.vim25.WaitForUpdatesResponse$Factory.parse(WaitForUpdatesR
    esponse.java:419)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
    java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
    sorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.vmware.vim25.VimServiceStub.fromOM(VimServiceStub.java)
    at com.vmware.vim25.VimServiceStub.waitForUpdates(VimServiceStub.java:45
    606)
    at com.vmware.vim25.util.PropertyCollectorHelper.waitForUpdates(Property
    CollectorHelper.java:338)
    at com.vmware.vim25.util.PropertyCollectorHelper.access$000(PropertyColl
    ectorHelper.java:34)
    at com.vmware.vim25.util.PropertyCollectorHelper$1.run(PropertyCollector
    Helper.java:51)
    at java.lang.Thread.run(Thread.java:662)
    2011-09-22 16:27:26,709 Start to clean up, please wait …
    2011-09-22 16:27:26,710 Remove VSA cluster …
    2011-09-22 16:27:27,514 Remove VSA datacenter …
    2011-09-22 16:27:27,555 Clean up done.
    System error 109 has occurred.

    The pipe has been ended.

    The VMware VirtualCenter Management Webservices service is starting.
    The VMware VirtualCenter Management Webservices service was started successfully
    .

    vSphere Client is already installed

    AFTER THIS STEP, I MODIFY THE C:\Program Files\VMware\Infrastructure\tomcat\webapps\VSAManager\WEB-INF\classes\dev.properties file to set evc.config=false

    D:\>install.exe -u root -p password -si 192.168.20.161 -ei 192.168.20.151 192.16
    8.20.153 -nh
    Current user: Administrator
    Current IP: 192.168.20.200
    Virtual Center is already installed
    VSA Manager is already installed
    2011-09-22 16:29:41,207
    2011-09-22 16:29:41,212 Start to scan ESX hosts, please wait …
    2011-09-22 16:29:41,213 localhost netmask: /24
    2011-09-22 16:29:41,213 localhost broadcast: /192.168.20.255
    2011-09-22 16:29:41,215 Total IP in subnet: 255
    2011-09-22 16:29:41,215 start IP: 192.168.20.0
    2011-09-22 16:29:41,216 end IP: 192.168.20.255
    2011-09-22 16:29:43,927 Create Datacenter : vsaDC
    2011-09-22 16:29:43,953 Add host to SVA Datacenter, please wait …
    2011-09-22 16:29:44,232 Scripted installation failed.
    com.vmware.vim25.DuplicateName
    at com.vmware.vim25.DuplicateName$Factory.parse(DuplicateName.java:571)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
    java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
    sorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.vmware.vim25.ExtensionMapper.getTypeObject(ExtensionMapper.java:2
    8)
    at com.vmware.vim25.MethodFault$Factory.parse(MethodFault.java:738)
    at com.vmware.vim25.LocalizedMethodFault$Factory.parse(LocalizedMethodFa
    ult.java:652)
    at com.vmware.vim25.TaskInfo$Factory.parse(TaskInfo.java:2367)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
    java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
    sorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.vmware.vim25.ExtensionMapper.getTypeObject(ExtensionMapper.java:2
    8)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
    java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
    sorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.vmware.vim25.ConverterUtil.getAnyTypeObject(ConverterUtil.java:15
    89)
    at com.vmware.vim25.PropertyChange$Factory.parse(PropertyChange.java:762
    )
    at com.vmware.vim25.ObjectUpdate$Factory.parse(ObjectUpdate.java:891)
    at com.vmware.vim25.PropertyFilterUpdate$Factory.parse(PropertyFilterUpd
    ate.java:830)
    at com.vmware.vim25.UpdateSet$Factory.parse(UpdateSet.java:818)
    at com.vmware.vim25.WaitForUpdatesResponse$Factory.parse(WaitForUpdatesR
    esponse.java:419)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
    java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
    sorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.vmware.vim25.VimServiceStub.fromOM(VimServiceStub.java)
    at com.vmware.vim25.VimServiceStub.waitForUpdates(VimServiceStub.java:45
    606)
    at com.vmware.vim25.util.PropertyCollectorHelper.waitForUpdates(Property
    CollectorHelper.java:338)
    at com.vmware.vim25.util.PropertyCollectorHelper.access$000(PropertyColl
    ectorHelper.java:34)
    at com.vmware.vim25.util.PropertyCollectorHelper$1.run(PropertyCollector
    Helper.java:51)
    at java.lang.Thread.run(Thread.java:662)
    2011-09-22 16:29:44,241 Start to clean up, please wait …
    2011-09-22 16:29:44,241 Remove VSA cluster …
    2011-09-22 16:29:44,478 Remove VSA datacenter …
    2011-09-22 16:29:44,500 Clean up done.
    The VMware VirtualCenter Management Webservices service is stopping.
    The VMware VirtualCenter Management Webservices service was stopped successfully
    .

    The VMware VirtualCenter Management Webservices service is starting.
    The VMware VirtualCenter Management Webservices service was started successfully
    .

    vSphere Client is already installed

  2. Hi there, follow up. I have removed the hosts, and the datacenter, and allowed the VSA installer to create the DC. It looks ok, however, now I have the error : Audit failed – Not a greenfield host, invalid number of vSwitches.

    I have installed 4 standard vSwitches, each with one uplink, with promiscuous mode enabled, first for management, and the rest named Trunk 1,2,3. Any ideas would be welcome!

  3. I continue to get stuck at a certain place during the install and can’t get passed it.

    “Cannot create VSA cluster: java.lang.Exception: Failed to ping VSA service on VM 172.16.100.32″

    My vESXi hosts both have 4 NICs, each one is connected to a separate distributed port group, and all have promiscuous mode set to Accept. I’m only using two physical NICs on the dvSwitch.

    During the install i’m able to ping the VSA Cluster Service IP address and both vmkernel interfaces that are setup for vMotion.

    Here is an excerpt from the log:

    2011-10-02 23:00:21,305 193 [Utils] [Thread-178] INFO – Retrieve dev.property: network.config=true
    2011-10-02 23:01:03,285 88 [SVAManager] [http-127.0.0.1-8080-8] INFO – VSA Manager dummyPing invoked.
    2011-10-02 23:01:16,842 193 [Utils] [Thread-179] INFO – Retrieve dev.property: svaservice.detect.time=3
    2011-10-02 23:01:16,842 315 [SVAVMConfigurator] [Thread-179] INFO – Detecting VSA service waiting time is 3 min(s).
    2011-10-02 23:01:18,870 193 [Utils] [Thread-178] INFO – Retrieve dev.property: svaservice.detect.time=3
    2011-10-02 23:01:18,870 315 [SVAVMConfigurator] [Thread-178] INFO – Detecting VSA service waiting time is 3 min(s).
    2011-10-02 23:04:55,508 334 [SVAVMConfigurator] [Thread-178] ERROR – Cannot access VSA VM 172.16.100.34
    2011-10-02 23:04:55,508 391 [SVAVMConfigurator] [Thread-178] ERROR – Failed to ping VSA service on VM 172.16.100.34
    2011-10-02 23:04:55,508 242 [SVAVMConfigurator] [Thread-178] ERROR – VSA service is not running.
    2011-10-02 23:04:55,508 447 [SVAVMConfigurator] [Thread-178] ERROR – Customize VM on host host-326 failed
    java.lang.Exception: Failed to ping VSA service on VM 172.16.100.34
    at com.vmware.sva.manager.config.SVAVMConfigurator.connectToSVAService(SVAVMConfigurator.java:392)
    at com.vmware.sva.manager.config.SVAVMConfigurator.attemptSVAConnect(SVAVMConfigurator.java:240)
    at com.vmware.sva.manager.config.SVAVMConfigurator.initializeVM(SVAVMConfigurator.java:175)
    at com.vmware.sva.manager.config.SVAVMConfigurator.customizeSvaVM(SVAVMConfigurator.java:509)
    at com.vmware.sva.manager.config.SVAVMConfigurator.call(SVAVMConfigurator.java:445)
    at com.vmware.sva.manager.config.SVAVMConfigurator.call(SVAVMConfigurator.java:61)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    2011-10-02 23:04:55,508 334 [SVAVMConfigurator] [Thread-179] ERROR – Cannot access VSA VM 172.16.100.32
    2011-10-02 23:04:55,508 391 [SVAVMConfigurator] [Thread-179] ERROR – Failed to ping VSA service on VM 172.16.100.32
    2011-10-02 23:04:55,508 242 [SVAVMConfigurator] [Thread-179] ERROR – VSA service is not running.
    2011-10-02 23:04:55,508 447 [SVAVMConfigurator] [Thread-179] ERROR – Customize VM on host host-323 failed
    java.lang.Exception: Failed to ping VSA service on VM 172.16.100.32
    at com.vmware.sva.manager.config.SVAVMConfigurator.connectToSVAService(SVAVMConfigurator.java:392)
    at com.vmware.sva.manager.config.SVAVMConfigurator.attemptSVAConnect(SVAVMConfigurator.java:240)
    at com.vmware.sva.manager.config.SVAVMConfigurator.initializeVM(SVAVMConfigurator.java:175)
    at com.vmware.sva.manager.config.SVAVMConfigurator.customizeSvaVM(SVAVMConfigurator.java:509)
    at com.vmware.sva.manager.config.SVAVMConfigurator.call(SVAVMConfigurator.java:445)
    at com.vmware.sva.manager.config.SVAVMConfigurator.call(SVAVMConfigurator.java:61)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    2011-10-02 23:04:55,539 271 [VCUserSession] [Thread-560] ERROR – Customize VM on host host-326failed.
    java.util.concurrent.ExecutionException: java.lang.Exception: Failed to ping VSA service on VM 172.16.100.32
    at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
    at java.util.concurrent.FutureTask.get(FutureTask.java:83)
    at com.vmware.sva.manager.dao.VCUserSession.installSvaVms(VCUserSession.java:264)
    at com.vmware.sva.manager.service.CreateClusterThread.run(CreateClusterThread.java:156)
    Caused by: java.lang.Exception: Failed to ping VSA service on VM 172.16.100.32
    at com.vmware.sva.manager.config.SVAVMConfigurator.connectToSVAService(SVAVMConfigurator.java:392)
    at com.vmware.sva.manager.config.SVAVMConfigurator.attemptSVAConnect(SVAVMConfigurator.java:240)
    at com.vmware.sva.manager.config.SVAVMConfigurator.initializeVM(SVAVMConfigurator.java:175)
    at com.vmware.sva.manager.config.SVAVMConfigurator.customizeSvaVM(SVAVMConfigurator.java:509)
    at com.vmware.sva.manager.config.SVAVMConfigurator.call(SVAVMConfigurator.java:445)
    at com.vmware.sva.manager.config.SVAVMConfigurator.call(SVAVMConfigurator.java:61)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    2011-10-02 23:04:55,539 273 [VCUserSession] [Thread-560] ERROR – Cancelling the running customization tasks
    2011-10-02 23:04:55,555 193 [Utils] [Thread-560] INFO – Retrieve dev.property: vm.rollback=true
    2011-10-02 23:04:55,555 291 [VCUserSession] [Thread-560] INFO – Deleting all the installed VM
    2011-10-02 23:05:12,606 188 [CreateClusterThread] [Thread-560] ERROR – Create cluster failed:
    java.util.concurrent.ExecutionException: java.lang.Exception: Failed to ping VSA service on VM 172.16.100.32
    at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
    at java.util.concurrent.FutureTask.get(FutureTask.java:83)

    Any help that anyone can provide is much appreciated

    • Datto said

      In the dev.properties file located in c:\program files\VMware\infrastructure\tomcat\webapps\VSAManager\web-inf\classes

      change the value of this line from the default of 3 to 30 — the problem may be that your VSAs are not coming on line fast enough to meet the default timeout value (because your nested of ESX hosts are slower than what VMware thought would be the time necessary to get the VSAs up and running).

      svaservice.detect.time=30

      Datto

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Connecting to %s

 
Follow

Get every new post delivered to your Inbox.

Join 29 other followers