VMware Cloud Foundation stands as a beacon of versatility, offering the capability to seamlessly run Kubernetes natively atop vSphere. This integration empowers customers to harness the same robust platform for both virtual machines and cutting-edge applications built on Kubernetes.
A few years back, I wrote a detailed guide of the setup process for NSX Advanced Load Balancer, illuminating its role in delivering robust load balancing services to Kubernetes environments. Fast forward to today, while the underlying process remains largely unchanged, the interface itself has undergone a significant facelift in version 30.x., so an updated post is much deserved;
Lets start examining my lab environment: 3 fully routable network segments with no DHCP designed as follow:
- Management Network: where all my management components are placed, vCenter, NSX, ESXi and now AVI Controller and Service Engines;
- Service Network: where the Kubernetes Services (Load Balancer) will be allocated to;
- Workload Network: where the Tanzu Kubernetes Cluster Nodes will be placed;
Every environment is unique, so it's imperative you take sometime to go through the topologies and requirements before standing up your own solution.
First thing first, NSX Advanced Load Balancer OVA
deployment, also known as AVI Controller; it's the central control
plane of the solution, responsable for creation, configuration,
management of Service Engines and services that are created on demand by
developers.
Deploying an OVA is a pretty straightforward operation
that you probably have done a thousand times during the years, so I'll skip it.
Once it's done, just power it on and wait a few minutes to the startup process finishes the configuration (it might take around 10 minutes, depending on each environment).
Just open up a browser and hit the IP address you just specified during OVA deployment.
- You will be tasked to create an admin account and set the password;
- On System Settings section create a Passphare, it's used when backing up and restoring the controller, also setup your DNS and DNS domain,SMTP information, which I skip because I don’t have it on my environment.
- On Multi-Tenant section keep the default, and click SAVE;
Now that you can access the platform, let's create some profiles which will be used latter when configuring Virtual Services;
- Select Templates, expand Profiles and click IPAM/DNS Profiles;
- Click, Create and select IPAM Profile;
- Give it a Name, select Allocate IP in VRF and the Cloud account for your vSphere environment (we will configure it later) click ADD under Usable Networks and select the PortGroup designted to host the K8s services, click SAVE;
- Back on profiles page, lets create a DNS profile;
- Give it a Name, click ADD under DNS Service Domain and fill out with your domain information; click SAVE;
Now that the system is ready to go, let’s make the system more secure replacing the self-signed certificates generated during deployment.
- On the menu select Administration, System Settings and hit the Edit;
- On Access tab, scroll to SSL/TLS Certtificate and remove the certificates by clicking the X;
- Create a new certificate, clicking on the ellipses and selecting Create;
- Give it a Name;
As Type make sure it's Self Signed. If you want you can also generate a CSR to be assigned by your certificate authority.
Common name shoud be your controller FQDN, fill out the remaining information accordingly.
As a best practice you can add alternative names, such as the short name and IP address of the controller, when done click SAVE;
Make sure your new certificate is listed and click SAVE;
After chancing the certificates you will be logged out of the system and you will need to login again.
Now back to the System Settings, on DNS/NTP section is an opportunity to review your DNS information, but more important adjust NTP settings, click SAVE;
Up to this point we just did basic system configuration, it's time to configure the Cloud accounts, the endpoints holding the resources for your workloads;
Under Infrastructure, Clouds there's a default cloud already created. It's Ok if you want create another one, just to remember to go back and ajust the IPAM profile we created previously.
The Default-Cloud has no type associate with it yet,lets fix it clicking on the pencil to edit;
- Select VMware vCenter/vSphere ESX as type;
- Scroll down and enable Prefer Static Routes vs Directly Connected Network
- Scroll down to Credentials and click SET CREDENTIALS;
- Type the address of your vCente, the username and password for the user with the
required privileged on vCenter and click Connect;
Back to the cloud account details page, select the vCenter datacenter where the services engines will be provisioned to;
I decided no to use Content Library, so I unchecked it. If you do, just fill out with the Content Library name, click SAVE & RELAUNCH
Once the page is reloaded, scroll to Management Network section, select the PortGroup
designed for the management traffic, fill out the information about the subnet and add
a range of IP address
When creating Service Engines on-demand, those IPs will
be management address assigned to them.
On the IPAM/DNS sectrion, select the profiles we just created and click SAVE;
Give a few minutes to a vCenter collection occurs, If everything is fine you should see a green light next to it.
Service
Engines are the componentes providing the load balance services, they are grouped together providing a concise configuration for
easier/faster management.
Service Engine Group rules how the service engines are placed, their configuration and quantity.
- On the Infrastructure tab expand Cloud Resources and select Service Engine Group, there’s an already default group created, just click on the pencil to edit it;
There are hundres of features and settings which are not part of this tutorial, so other then what I'm showing just stick with the default options
- On Placement section, select your High Avalability Mode, which depends of the license your are entitle to:
- Essential license only allows you to Active/Standby mode.
- Enterprise license allows you to select Active/Active and buffer mode in addition to Active/Standby.
- On Resources section, select the vCenter folder where the engines appliance will be placed, that's great for easier identification and also a prefix to be added on every appliance;
- The Scope section controls the engine appliances placement, you can define which cluster or host will be used to provision the engines and what datastore will be used as well, once done click SAVE;
When a Kubernetes Services is created by a developer, a free IP will be pulled from the IP Pool and allocated to the service engines backing up the service;
To configure this IP Pool navigate to Infrastructure, expand Cloud Resources and select Networks;
You will see all Port Groups discovered on the vCenter you just assigned on the Cloud section.
- Click on the pencil to edit the PortGroup providing the Kubernetes Service or VIP;
- On the Subnets section click ADD; there might be cases where your Portgroup is already being used and it might be automatically discovered, so it's just a matter of editingt it;
- Add the subnet prefix, make sure Use Static IP Address for VIPs and SE is checked;
- Add the range of free IP address which will be used by the K8S services and click SAVE;
Since I'm using a fully routable network I need to specify how the Services network reach my Workload network, where the K8s nodes are placed, It's done by creating a static route.
- On Infrastructure section, expand Cloud Resources and click on VRF Context;
There's an already global created, just click on the pencil to edit it;
- Under Static Route section click ADD, fill the fields with the Workload subnet information and the gateway on the Service Network, click SAVE;
If you got this far, Congratulations, now your system is ready to enable vSphere with Tanzu
with NSX Advanced Load Balanced.
If you are not sure how to enable it, just check my post on how to do it.
