Monday, March 5, 2018

VMware Pivotal Container Service - Bosh CLI

While VMware Pivotal Container Service UI is great, there are a few things you still need to perform on the command line (or API requests), like troubleshooting and stuff.

Of course, you could just SSH into Ops Manager and run every command from there, but giving access to this crucial resource to others is far from ideal.
That’s why I decided to create some basic tutorials on how to access and perform PKS tasks remotely throughout the CLI.

The tutorials will cover:
  • Bosh CLI
  • UAAC CLI (coming soon)
  • PKS CLI    (coming soon)

Bosh CLI is intended to manage Bosh resources, tasks and objects; In other for PKS to be able to instantiate VMs (K8s Master and Nodes) on vCenter, it needs a broker which has specific vSphere CPI, that’s the Ops Director you first deploy on your PKS environment.

After you deploy it you can connect to the Bosh Director service running within and see the VMs and tasks it’s managing.

*** Installing Bosh CLI ***

- Download bosh cli;   
  Run: wget

- Make it executable;
  Run: chmod +x bosh-cli-2.0.48-linux-amd64

- Move the CLI to the bin directory;
  Run: cp bosh-cli-2.0.28-linux-amd64 /usr/local/bin/bosh

- You can test if it's installed properly;
  Run: bosh -v

*** Connecting to Bosh Director ***
Once you get the bosh CLI installed you can point it to your Bosh Director and start issuing commands.

If you are using a self-signed certificate, don’t forget to first download the root CA certificate
- Go to PKS Settings;

On the Advanced option, download the Root CA Certificate;

- Create an environment alias for future reference;
  Run: bosh alias-env "alias" -e "Ops-Director" --ca-cert "CA_CERT_Path"

- Login with the desired credentials (for this purpose I’m using director);
  Run: bosh -e “alias” log-in

You can get the credentials from the Ops Director’s Credentials tab

You are ready to go !! It's possible to create as many environments as you need and you just need to specify which environment the command will run against, like:
bosh -e "alias" tasks

But, If you have a single environment it’s easier to set up a system environment and then you can omit the parameter.

*** Bosh CLI examples ***

Here are a few commands to get you started

Checking all tasks performed on the system;
Run: bosh tasks -ar

If you need details about a specific task;
Run: bosh tasks "ID" 

To list all VMs provisioned by the system;
Run:  bosh vms

SSH into a specific VM without providing any credentials;
Run: bosh ssh -e “alias” -d “deployment_ID” “vm_Instance”

You can get the deployment ID and VM instance from the bosh vms command.

that’s it, stay tuned for the next basic PKS tutorials.

Thursday, March 1, 2018

Pivotal Container Service and NSX-T integration

I have to admit, since the launch of VMware Pivotal Container Service (PKS), I was very anxious to start creating and managing production grade Kubernetes clusters from within.
As soon as the product gets released I just grabbed my copy and started playing with that without reading anything about it (yeah I know), a few minutes late I realized there were some concepts that I needed to grasp if I want to succeed.

The automation PKS provides is amazing and make the platform deployment very easy to consume, but if you don’t know exactly what the input parameters are there’s a huge chance you get yourself into problems.

Thinking about that I decided to share what I learned about the network aspects of PKS, specially NSX-T integration and how to set it up.

I'm not showing how to configure NSX-T components, like T0, Logical Switch, etc.. but how do you consume them from PKS.

Once Ops Manager OVA has been deployed; 
The first thing you need to configure is the Ops Manager Director Tile.

Right on the vCenter Config section, you will see an option to configure the integration with NSX-T

LEAVE it to Standard vCenter Networking option, I know the anxiety to start using PKS with NSX-T, but it’s not there yet, in fact, this option is to allow other Pivotal solutions to communicate with NSX-T, like PAS.


Jumping to Network section, you need to create the networks where your components will be hooked to;
I created two networks;

- one for management components, like Ops Director, PKS broker and Harbor
- another one for Service components, like Kubernetes Master, ETCD and nodes VMs
the only difference between them is that on the service network you select the service check box.
 Don't forget to configure what vSphere Network (Port Group or Logical Switches) the VMs will be connected to, CIDR and others network parameters accordingly.

Once you are done with Ops Director it’s time to configure Pivotal Container Service Tile.

On the Networking section is where you configure your PKS integration with NSX-T, just provide your NSX-T Manager hostname and credentials

Scrolling down a little bit you will see the fields for the NSX-T integration details

1 – T0 Router ID, this one is easy, If you remember about how PKS works (when integrated with NSX) every time you create a new Kubernetes namespace a new T1 router will be created to segregate and secure this new namespace workloads, in order to allow communication this T1 will be connected with a T0, that’s why it needs to know T0 ID.

2 – IP Block ID, that’s the range of IPs to be assigned to your PODs. Through the use of NSX-T Container Plugin, those address will be configured through the use of Container Network Interface (CNI).

On NSX go to DDI/IPAM Section and create a new IP Block, the CIDR recommend is

3 – Floating IP Pool ID, that’s the range of IPs assigned to NSX-T Load Balance when Kubernetes Services and Ingress be created.

On NSX go to Inventory/Groups and create a new IP Pool with the desired range of IPs
 No doubt there's a lot to learn and understand yet, but I hope this post reduce a little the burden to get your PKS up and running.

See you

Friday, February 16, 2018

vSphere Integrated Containers and VMware NSX better together

The dynamics and agile nature of container world are constantly challenging us that need to deal with securing our environments, it’s clear manual operations are just not enough to keep up with innovations pace this new world brings us.

But How can I protect my production container workloads in a dynamic and agile way?
VMware NSX has the answer.

Leveraging Security Groups and dynamic membership you can create rules that match specific vSphere objects, which is a perfect match for vSphere Integrated Containers and its container-vm constructs, allowing or blocking traffic on-demand whenever a new workload is created or deleted, providing the agility developers love without giving up on security that infrastructure guys need.

Let me give you a couple of use cases:

- Protect containers based on name
VIC allows you to expose a container service directly on the network with the use of container networks (just covered it on another post), so you can protect them just allowing certain services based on the container-vm name, like access to HTTP to only container-vms starting with web. (check the video below for a quick demo).

- Container to container communication
You could also use the container-vm names to create rules between containers, like, just container-vms starting with app could communicate with database container-vms (starting with db).

- Protect tenant containers
I might be pushing a little bit on that one, it’s not a real tenant construct OK, but it serves my point.
Image 2 distinct developers or projects, you can provision a VCH for each one of them leveraging distinct name prefixes/suffixes (check my post about it if you are not sure), that way you can create a security boundary based on the prefixes, where they can communicate freely between the container-vms with the same one, but not with the others, this way providing isolation and security between projects.

You guys are clever than me and I’m sure you can come up with some new and innovative ideas to use NSX to protect the containers, so tell us about it, leave your comment below.

Monday, January 22, 2018

vSphere Integrated Containers – VCH Wizard

With the increase of user’s adoption of vSphere Integrated Containers, it became clear a new consumption model of Virtual Container Hosts (VCH) is needed. There was nothing wrong with the VIC Engine Bundle command line, but due to VIC powerful features, sometimes, VIC creation command line string is just too long.

In order to provide an easier, faster and agile way for initial setup, VIC 1.3 brought a new VCH creation/deletion Wizard !!!

This Wizard is part of the new VIC Plugin for vCenter HTML5 UI, so, don't forget to install/upgrade the plugin to version 1.3 before start using it.

A new “+  New Virtual Container Host” action is available on the Virtual Container Hosts tab where you can initiate the creation of your VCHs any time.

Once started, the Wizard will guide you throughout all the sections related to VCH and on the right will be shown the specifics about each section.

Some sections also provide advanced parameters to be configured, just expand it to configure them, otherwise default values will be used instead.

On the Storage section, a good practice is to always have one volume named “Default”, eliminating the need to explicitly tell datastore names during container’s volume use.

The network section is where you have the most interesting things, don't forget to hit Advanced to see it all

There you can find the options to configure Container networks, firewall behavior and optional VCH networks, like management and client networks.

There's also an entire section dedicated to protect your VCH with TLS.

You can also grant the required privileges of the operations user on demand.

Double check the details on the Summary, if everything is fine just hit Finish

One nice touch is during the creation of VCH, you can watch the log live, just expands the VCH details.

If you are a fan of VIC Engine Bundle, don’t worry the script still available and you can still use it to create VCHs.

Who am I

My photo
I’m an IT specialist with over 15 years of experience, working from IT infrastructure to management products, troubleshooting and project management skills from medium to large environments. Nowadays I'm working for VMware as a Consulting Architect, helping customers to embrace the Cloud Era and make them successfully on their journey. Despite the fact I'm a VMware employee these postings reflect my own opinion and do not represents VMware's position, strategies or opinions. Reach me at @dumeirell

Most Viewed Posts

Blog Archive