Thursday, March 1, 2018

Pivotal Container Service and NSX-T integration

I have to admit, since the launch of VMware Pivotal Container Service (PKS), I was very anxious to start creating and managing production grade Kubernetes clusters from within.
As soon as the product gets released I just grabbed my copy and started playing with that without reading anything about it (yeah I know), a few minutes late I realized there were some concepts that I needed to grasp if I want to succeed.

The automation PKS provides is amazing and make the platform deployment very easy to consume, but if you don’t know exactly what the input parameters are there’s a huge chance you get yourself into problems.

Thinking about that I decided to share what I learned about the network aspects of PKS, specially NSX-T integration and how to set it up.

I'm not showing how to configure NSX-T components, like T0, Logical Switch, etc.. but how do you consume them from PKS.

Once Ops Manager OVA has been deployed; 
The first thing you need to configure is the Ops Manager Director Tile.

Right on the vCenter Config section, you will see an option to configure the integration with NSX-T

LEAVE it to Standard vCenter Networking option, I know the anxiety to start using PKS with NSX-T, but it’s not there yet, in fact, this option is to allow other Pivotal solutions to communicate with NSX-T, like PAS.


Jumping to Network section, you need to create the networks where your components will be hooked to;
I created two networks;

- one for management components, like Ops Director, PKS broker and Harbor
- another one for Service components, like Kubernetes Master, ETCD and nodes VMs
the only difference between them is that on the service network you select the service check box.
 Don't forget to configure what vSphere Network (Port Group or Logical Switches) the VMs will be connected to, CIDR and others network parameters accordingly.

Once you are done with Ops Director it’s time to configure Pivotal Container Service Tile.

On the Networking section is where you configure your PKS integration with NSX-T, just provide your NSX-T Manager hostname and credentials

Scrolling down a little bit you will see the fields for the NSX-T integration details

1 – T0 Router ID, this one is easy, If you remember about how PKS works (when integrated with NSX) every time you create a new Kubernetes namespace a new T1 router will be created to segregate and secure this new namespace workloads, in order to allow communication this T1 will be connected with a T0, that’s why it needs to know T0 ID.

2 – IP Block ID, that’s the range of IPs to be assigned to your PODs. Through the use of NSX-T Container Plugin, those address will be configured through the use of Container Network Interface (CNI).

On NSX go to DDI/IPAM Section and create a new IP Block, the CIDR recommend is

3 – Floating IP Pool ID, that’s the range of IPs assigned to NSX-T Load Balance when Kubernetes Services and Ingress be created.

On NSX go to Inventory/Groups and create a new IP Pool with the desired range of IPs
 No doubt there's a lot to learn and understand yet, but I hope this post reduce a little the burden to get your PKS up and running.

See you

No comments:

Post a Comment

Who am I

My photo
I’m an IT specialist with over 15 years of experience, working from IT infrastructure to management products, troubleshooting and project management skills from medium to large environments. Nowadays I'm working for VMware as a Consulting Architect, helping customers to embrace the Cloud Era and make them successfully on their journey. Despite the fact I'm a VMware employee these postings reflect my own opinion and do not represents VMware's position, strategies or opinions. Reach me at @dumeirell

Most Viewed Posts

Blog Archive