Onboarding Brownfield vSphere Environments to VMware Cloud Foundation – Part 1 – Management Workload Domain

VMware Cloud Foundation 5.2 was released last day and one of the new features was the introduction of VCF Import Tool. VCF Import Tool is used for onboarding existing brownfield vSphere deployments to VCF without needing a full rebuild. In addition to onboarding vSphere environments, VCF Import tool can also prepare the imported vSphere clusters with NSX on vCenter port groups (DVPGs), thereby offering distributed firewalling and other security features available in NSX.

If you haven’t seen the release notes yet, please check out below:

https://docs.vmware.com/en/VMware-Cloud-Foundation/5.2/rn/vmware-cloud-foundation-52-release-notes/index.html

In this two-part blog series on the VCF Import Tool, we will cover the below scenarios:

Part 1 : Onboarding a brownfield vSphere environment as management workload domain in VCF
Part 2 : Onboarding a brownfield vSphere environment as VI compute workload domain in VCF

Let’s get started:

Considerations for onboarding brownfield vSphere deployments as VCF management workload domain

Taken from the official documentation at https://docs.vmware.com/en/VMware-Cloud-Foundation/5.2/vcf-admin/GUID-41CEC8AD-73D1-4FBD-9063-994EA26D2C69.html , the below considerations apply while we onboard brownfield vSphere environments as management workload domain in VCF.

The existing brownfield vSphere environment needs to be upgraded to the minimum VCF 5.2 bill-of-materials (BOM). That means, all clusters must be running vSphere 8.0U3 with consistent build numbers.
For onboarding to the management workload domain, vCenter Server virtual machine should be co-located on the cluster that it manages. For the VI workload domain, the vCenter server on the existing brownfield deployment can be either co-located or hosted on the management workload domain.
vCenter server of existing vSphere environment should be not present as a compute manager in any NSX instance.
Existing vSphere clusters should not be prepared with an NSX instance. However, after onboarding the clusters to VCF, we can prepare it with NSX on vCenter port groups.
Onboarded clusters in VCF can only leverage NSX on vCenter port groups (DVPGs). Overlay backed networking is currently not supported.
Onboarded clusters in VCF cannot host NSX edge clusters. They also don’t have VCF application virtual networks (AVN)
All the ESXi hosts in the existing brownfield cluster should have identical network configuration.
- They should be configured with VDS port groups. Standard switches are not supported.
- vmkernel ports for management, vSAN and vMotion should be configured with static IP address. DHCP is not supported.
- All ESXi hosts must have identical number of uplinks (minimum 2)
- No LACP configuration on the uplinks.
DRS mode for the cluster should be set to fully-automated.
Standalone hosts in the vCenter inventory are not supported. They either need to be removed or moved to a cluster.
vSphere clusters with vSAN should have three nodes at a minimum. Stretched vSAN clusters are not supported.
vSphere clusters enabled with workload management (vSphere with Tanzu) are not supported for convert / import to VCF.
And finally, all hosts and appliances should be configured with proper NTP settings.

Existing brownfield management vSphere environment walkthrough

Now let’s do a walkthrough of the existing management vSphere brownfield environment and confirm that all the pre-requisites and considerations stated above are met before we onboard the environment to VCF.

Our existing vSphere environment has two vCenter servers, each managing a single vSphere cluster:

Management vCenter “vxdc01-vcenter01.vxplanet.int” managing the vSphere cluster “VxDC01-C01-MGMT”
Compute vCenter “vxdc01-vcenter02.vxplanet.int” managing the vSphere cluster “VxDC01-C02-Compute”

The vCenter server VMs are deployed on the vSphere management cluster “VxDC01-C01-MGMT”. ie, for the management workload domain import, the vCenter server is co-located with the cluster that it manages, which is a requirement.

Both the management and compute vSphere environments are running vSphere 8.0U3.

The management vSphere cluster has a single VDS with multiple VDS backed port groups for workloads, vSAN and vMotion. It is not prepared with NSX and doesn’t have any NSX backed segments. It also doesn’t have any vSphere standard switches.

The management cluster VDS “VxDC01-C01-VDS01” has two uplinks and that is consistent across all the ESXi hosts in the cluster.

All the vmkernel ports are configured with static IP addresses, and no DHCP is in place.

The DRS mode of the cluster is set to fully-automated.

We don’t have a stretched vSAN cluster. All the four vSAN nodes are in a single site with no fault domains configured.

and finally, we don’t have workload management (vSphere with Tanzu) enabled:

At this point, we have met all the pre-requisites for a successful VCF import. Now let’s move on and download the tools needed for VCF import.

Downloading the tools for VCF Import

Let’s login to the Broadcom support portal and download the necessary tools and ova files needed for the VCF import process. We will download the below files:

VCF Import tool
SDDC manager appliance OVA
VCF install bundle for NSX manager

Perform pre-checks on the management vCenter server

VCF Import tool has a pre-check script to be run on the vCenter server that will be onboarded to the VCF management workload domain. Let’s login to the management vCenter “vxdc01-vcenter01.vxplanet.int” and open the bash terminal as root.

We will copy the VCF import bundle and extract it.

and then we will run the pre-checks script.

Success!!! All the pre-checks have passed, and we are good to proceed.

Deploying SDDC Manager

We will now deploy SDDC manager using the OVA file we downloaded previously. SDDC manager is deployed on the management vCenter cluster.

Let’s create a DNS host record for the SDDC manager FQDN.

and import the SDDC manager OVA to the management cluster “VxDC01-C01-MGMT”

We will deploy to the management network “VxDC01-C01-VDS01-MGMT-V1001”, and provide values to the OVA properties.

Once the appliance is deployed successfully, we will power on and wait for the shell to initialize. The UI will not initialize at this moment, and we need to wait until the management workload domain import is completed.

Performing Import checks from SDDC manager

Let’s login to SDDC manager cli as user ‘vcf’, upload the VCF Import bundle and run import checks against the management vCenter.

We see that there are 4 failed checks, reviewing the ‘failed_guardrails_csv’ will give more information about the issue.

These 4 warnings are related to lifecycle manager policy mismatch between the management vCenter and SDDC manager. As per the official documentation below, these warnings can be ignored.

https://docs.vmware.com/en/VMware-Cloud-Foundation/5.2/vcf-admin/GUID-458B6715-3ED6-4423-B093-64B1A2963CC0.html

Remediating the above warnings are optional and will not cause the onboarding process to fail, but if we still want to remediate it, the below settings from vSphere Lifecycle manager can be adjusted as per the recommendations in the csv file.

Reviewing the ‘all_guardrails_csv’ will give details about all the successful checks performed against the management vCenter.

Uploading NSX bundles and generating NSX deployment spec

Deployment of NSX manager cluster can be performed either along with the VCF import process or later, as a separate procedure after the VCF import is completed.

For this blog post, we will deploy NSX manager cluster along with the workload domain import process. This workflow covers the below tasks:

Deploy a three-node NSX manager cluster
Assign cluster VIP to the NSX manager cluster
Add the management vCenter server as a compute manager in NSX
Prepares the management vSphere cluster with NSX on DVPG (formerly NSX-Security only deployment)

There is no overlay configuration on the imported clusters, and as such they don’t host NSX edges and AVNs (Application Virtual Networks).

Let’s create the DNS host records for the NSX management cluster.

We need to import the VCF bundle for NSX to the SDDC manager. The workflow will extract the bundle and deploys NSX management cluster on the management vSphere cluster to the same vSphere resource pool where the vCenter VM is deployed.

Next, we will define the NSX deployment specification that will be supplied as part of the VCF import process. The json template is available in the official documentation at:

https://docs.vmware.com/en/VMware-Cloud-Foundation/5.2/vcf-admin/GUID-29B000D1-1452-45FC-82FC-02FF24E381BD.html

Onboarding brownfield vSphere environment as management workload domain in SDDC manager

At this stage, we are all good to run the VCF import tool and start the onboarding of existing brownfield vSphere deployment to the management workload domain in VCF.

Let’s login to the SDDC manager cli as user ‘vcf’ and run the tool with the ‘convert’ option.

As we saw during import checks, we had 4 warnings that can be safely ignored, and we will now acknowledge this to continue.

Unfortunately, the import process has Failed !!! ☹ Let’s pause for a moment while I troubleshoot and identify the root cause.

After doing few troubleshooting and reviewing the log file in the SDDC manager at /var/log/vmware/vcf/domainmanager/domainmanager.log, I figured out that this was due to a time skew between the management vCenter server and SDDC manager, causing a token to expire, thereby failing the script execution.

I have restarted NTP service on the vCenter server and re-ran the Import tool, and the workflow proceeded without issues.

We see that the three NSX manager appliances are getting deployed on the management cluster in the same resource pool as the vCenter server VM.

and the VCF import operation has succeeded.

Success!!! Our brownfield vSphere management deployment is now a VCF management workload domain with NSX on VLAN port groups. In my home lab, it took approx. 3 hours to complete (possibly because it’s a nested environment).

Now let’s restart the SDDC manager services and wait for the UI to initialize.

Validating the onboarded management workload domain

We will now perform a quick walkthrough of the SDDC manager console and validate the imported management workload domain.

Let’s run prechecks on the management workload domain for general upgrade readiness and see if we get any critical errors or warnings.

Reviewing the errors mostly show results with missing backup, licenses, incompatibilities etc which are expected as this is a nested home lab.

Now let’s review the NSX deployment on the management workload domain.

We see that the three node NSX management cluster is formed successfully with the cluster VIP.

The management workload domain vCenter is added as a compute manager to the NSX cluster.

The management cluster is prepared for NSX on DVPGs. This was formerly called NSX-Security only deployment. This means that we are now able to apply NSX security features to the management plane components that are deployed on vCenter port groups. Before we work on NSX security policies, make sure that the critical management VMs are whitelisted or excluded, in order to avoid a lockdown situation.

We see that the workflow has defined a DFW exclusion list for the critical management plane VMs. We can define custom exclusion list for non-VCF management components if needed.

Congratulations!!! If you are still reading, we have successfully achieved our goal of onboarding a brownfield vSphere deployment to a VCF management workload domain. In the next article, we will onboard a brownfield vSphere environment to a VCF VI workload domain with NSX VLAN networking. Stay tuned!!!

I hope the article was informative. Thanks for reading.