Thursday, April 28, 2016

Hadoop cluster nodes password

During a Big Data Extension cluster creation, you choose what the node's password will be, either a specified one or a random generated one, them all nodes will be created using the same password.

It’s not unusual that sometimes you need to login directly on the nodes’ console through SSH to perform some troubleshooting, run commands or scripts, but in order to execute this, first you need to properly login with a username and password.
Well, if you specified the password during creation it’s easy, but what about the random password ?

The random password is not shown on any GUI option. To identify the random password you will need to open the node’s console through vSphere Web Client.

There’s it, the random password will be shown on the hosts’s banner.

You might have realized already that it’s not the most secure method since everybody with a vSphere privilege could open the node’s console and check the password for your nodes. 
Some companies have restrict security policies which would not allow this behavior. 

So how do we change the password on all nodes of a cluster ?

       -    login to BDE Management Server CLI
-       run: Cluster_Name ‘echo New_Password | sudo passwd serengeti –stdin’

If you forget the password, just change it again !!!

Tuesday, April 19, 2016

VMware NSX and the dropped packes tale

Day two of a VMware NSX implementation and I was surrounded by angry network guys asking me: “What have you done ?
As scare as it looks like I kept my pace and started to work with them to understanding what has been going on.

What they have seen was an increase of network packages drops on some network ports connected to physical servers and, of course, got the conclusion it was the NSX implementation causing the problem……well, yes and no !!! Let me explain why.

Backing to the dropped packets: using a tool like Wireshark they were seeing packets originated from a VMware prefix MAC Address with a type 0x8922, it was clear something on VMware side was generating those packets, I could not blame them to turn it on us.

(obs: I could not get the original screen, so I use this as an example; on the original one you would see the MAC address of a physical server on the destination)

0x8922 type is generally related to the Beacon Probing fail detection mechanism, so we checked vDS to make sure it was not using Beacon Probing, in fact it was not.

Then we realized that as part of NSX implementation we increased the MTU size to 1600 in order to enable VXLAN utilization on the ESXi host's network ports, as required, we also changed the MTU size of it's vDS accordingly.

The client’s architecture is a single vDS with two 10Gb uplinks handling all port groups and vmkernels, vSphere Distributed Switch Health Check is also enabled on the vDS for VLAN and MTU.

The way vSphere Distributed Switch Health Check works is sending broadcasts packets the size of the MTU configured on vDS to all it’s port groups throughout ESXi hosts uplinks.

Since we increased the MTU size on only ESXi hosts(client's decision), physical hosts connected to the same segment (VLAN id) of some portp groups were receiving those bigger packets and as they could not handle those, they started to dropped it. BINGO !!!

In reality it was not causing any issue other than polluting their monitoring tool screen.

To help you on how to avoid or remediate a situation like that here are some advice:

1 – Keep the MTU size the same on all your environment. (it's a fix)
2 – Change the default health check interval, it will reduce the number of dropped packets you see over time. (it just remediates)
3 – Disable vSphere Distributed Switch Health Check on vDS, not desirable as you will loose the capability of being warned if there’s something broken on the underlay network. (it's a fix)
4 – Create a vDS only for VXLAN traffic, harder to accomplish as it requires spare uplinks on the hosts. (it's a fix)

As you could see it was not NSX related, but some changes required by the NSX lead to the situation.
See you guys

Wednesday, April 6, 2016

VMware’s “game-ified” learning platform

How about get recognized by the things you know, learn new stuff, expand you network and earning prizes along the way ?!?!?
That’s what CloudCredibility is all about. The VMware’s learning platform that allows you to complete tasks and make scores that can later become prizes.

It involves just 4 steps. Sign up, Join a team, complete tasks and get rewards.
Could not be easier !!!

Let me show you how easy it is:

-        Once signed;
-        Go to tasks, there’s about 1590 tasks grouped by several categories;

-       - Select the task you wanna take;

-       - Click Start Task

Now, just complete the task to earn the points.

The first time you join CloudCred you will automatically be joined to a group, you can change it later through your profile, creating your own or joining others,  mine is “PSOBrazil”, fell free to join ; )

Still have doutbs about what CloudCred is all about, may be this video can help you.

Who am I

My photo
I’m an IT specialist with over 15 years of experience, working from IT infrastructure to management products, troubleshooting and project management skills from medium to large environments. Nowadays I'm working for VMware as a Consulting Architect, helping customers to embrace the Cloud Era and make them successfully on their journey. Despite the fact I'm a VMware employee these postings reflect my own opinion and do not represents VMware's position, strategies or opinions. Reach me at @dumeirell

Most Viewed Posts

Blog Archive