Tuesday, December 29, 2015

Long Live the vCloud Director


Some time ago VMware decided to give vCloud Director an API only strategy, that means the new features would only be available through APIs call and the graphical user interface (GUI) would not be updated to reflect these enhancements anymore, so for the clients be able to use these new features they would need to create their own mechanism to use them, like creating it’s own portal.

Everyone who knows me a little bit know that among all VMware’s products the one I like the most is vCloud Director and despite the fact if I do agree or not with this API only strategy, VMware has decided to change vCD’s strategy back to a full feature parity between GUI and API, I could not be more happy.

It would take some time until all the past features like, Organization Virtual Datacenter Templates, Virtual Machine monitoring metrics, Virtual Machine disk level storage profile and more, be available on the GUI but we are working on it.


Long Live the vCloud Director !!!

Tuesday, December 22, 2015

VMware License Tool Tracker


From time to time I stumble with some VMware’s Tools that make my life easier.
It just happened again past week when I need to validate a client’s license without access to MyVMware.

License Tracker Tool allows you to check the quantity, contract, status, support level and upgrade history just with your license serial number.

- Just login on MyVMware;

 
- Type the serial number and click Tracker;

 As you can see on my example I have all the information I need about this serial.

 
Learning one new thing every day…….

Monday, December 14, 2015

VMware’s products release on December 2015


This month VMware has released some new versions of it's products, covering new models, architectures,features, compatibility enhancement and bug fixes.
I know it’s a lot to digest, my suggestion is to you focus on the products you already have deployed today and, if time permits, check the ones you see as a candidate to compound your solution in the near future.

I challenge you to read it and don’t find anything that you would not benefit from.

It’s a huge change for vRA, I would say a remarkable one that would definitely makes vRA a reliable and efficient solution for private clouds.
The improvements I most enjoyed are the streamlined  and automated wizard install for single and distributed environments, new Blueprint authoring, allowing on a single pane the creation of VMs provisioning (IaaS) with applications installation (former Application Services) and a lot more, here’s a good blog postt about the enhancements.

vRO introduces Control Center, which enables a centralized server administration, easier cluster configuration, troubleshooting and runtime metrics and some plugin improvements.
 
Although it’s not the latest’s version of vCD, It adds support for Internet Explorer 11 along with some bug fixes, like when accessing media files and some AD issues with federation.

There are a lot of improvements on this new version of BDE, things like multi template allowing you to choose different templates for each Hadoop cluster, customization of virtual disks and controllers , support for spark clusters, enhanced compatibility with Cloudera CDH 5.4, HortonWorks HDP 2.3, Ambari 2.1 and Pivotal PHD 3.0.

It had some improved architecture changes as access to VMware Identity Management from vRA and separate remote data collectors for remote sites and more on areas like pricing support, cloud comparison and user interface.

It’s a management release, which upgraded Apache Tomcat to 7.0.65 and JRE to 1.8.0_51, Showback Management was also removed.

Now you can download drivers, tools, open source software and custom ISOs, along with some bug fixes that, in some cases, prevented you from downloading the binaries you need.
If you are not familiar with Software Manager check this post out.

For Hyperic it’s just a maintenance release, upgrading it’s internal components like:, Postgres to 9.1.15, Tomcat to version 8 and others, but I believe people are moving to vROPs where Hyperic is alredy built in

It’s just a maintenance release fixing some bugs, but also increases the security disabling SSLv3 and enabling by default TLS 1.1 and 1.2. On the functionality side, now Linux desktop servers support clipboard redirection, single sing-on and smart-card redirection

It introduces the in-place OS migration from Windows 7 to Windows 10, here’s a nice post about this migration.

What a scalability enhancement going from 1.000 clients to 10.000 per server ?!?! also a more intuitive interface for editing policies and a better error reporting and updating certificates.

good reading..

Thursday, November 19, 2015

Big Data Extension – GUI vs CLI


Nowadays, usability is one of the keys success for any product, nobody likes a product that’s complex, hard to interact with and have a big learning curve, no matter how good and amazing the product might be.

Thinking about that, VMware, provides two interfaces to interact with Big Data Extension, the Graphical User Interface (GUI) and Command Line Interface (CLI).
Each one has it’s own merits and fits the needs of all users kind, let’s explore them a little bit.

The GUI is the most intuitive and easy to use of them, based on the already know vSphere Web Interface it takes almost no time to learn how to operate it, being the recommend tool for the majority of Big Data Users.

Once BDE plug-in is register, it will show up on the vSphere Web Client like any other feature.


From there you can manage all features of BDE as, adding/removing resources, configuring additional Application Managers and the most important on: creating/deleting and managing Hadoop Clusters.


The cluster creation is intuitive, just click “Add New Cluster” and the Wizard will guide you through it.


 In the other hand we have CLI, which is the most powerful and flexible tool to manage BDE, recommended for Big Data Hard Users. It allows the same functionalities of the GUI but also allows the creation of more advanced, customized and complex clusters topologies based on JSON configuration files, extending the functionalities provided by the GUI.

To start with, you will need to download the CLI from the BDE server, from the address: http://”bde_server”/cli

Just download the zip file and extract it to your local drive.

To initiate the CLI run the following command:
java –jar serengeti-cli-x.x.x.jar

obs: you must have java already installed.

Once started you will need to connect to your BDE server, run:
connect --host “bde_server":8443

To learn the commands and syntaxes available you can just run help, but I strongly recommend you to read the vSphere Big DataExtensions Command-Line Interface Guide for full understanding of all features.

Here’s an extra bonus tip, besides of GUI and CLI and you leverage vRealize Automation to provide to your internal users a Hadoop as a Service Offering, leveraging the power of BDE and the friendly user interface of vRA.

 I’ll show you how on a future post, keep watching.

Here's the link for the post using vRA as a Interface for Hadoop as a Service Offering.

 

Wednesday, October 21, 2015

Managing Distributions on BDE


One of the Big Data Extension strengths is the flexibility to provision Hadoop clusters based on the distributions that best suites your taste. Although BDE comes only with Bigtop distribution, it’s very easy to add the new ones you need.

To add a new distro just use the /opt/serengeti/sbin/config-distro.rb command.
BDE Administration Guide cover it nicely, so there’s no point to duplicate the details here, just take a look at the guide for the right syntax and I'm sure you will be good to go.

In fact what I want to cover here is how to remove a distro from BDE !!!
You might have realized already that using config-distro commando to add a distro is pretty straightforward, but removing it is not that intuitive. 
Let's see how to accomplish it then !

As you can see, I have added Hortonworks (HDP-2.2) to my environment.


When you add a distro, it's details are published to a manifest file for later consumption.
To remove it, login on BDE and edit the manifest file to remove the distro information from it.

Don’t forget to create a backup file first.
run: cp /opt/serengeti/www/distros/manifest /opt/serengeti/www/distros/manifest.bkp

Now we are ready to edit it.
run: vi /opt/serengeti/www/distros/manifest

Clean up the entire section of the distro you want to remove, in my case, HDP 2.2


Save the file and restart BDE services to the changes take effect.

run: service tomcat restart

As you can see the Hortonworks is no longer available.


If anything goes wrong just restore the backup file and restart tomcat service again.

Now you are ready to manage the distributions on your BDE solution !!!


Friday, October 9, 2015

BDE template default password

Essential role of the VMware Big Data Extension is the node provisioning automation function, which uses a virtual machine belonging to BDE’s vApp as template for new Hadoop cluster nodes.



It’s not unusual that companies need to change their template’s configuration to adhere to internal security requirements, installing agents like for  monitoring and antivirus or tweaking  some security parameter, which will all be inherited by the new nodes.

The template provided by VMware as part BDE 2.2 is a CentOS 6.5 64-bits, so if you want to change anything on it first you would need login on (obviously).
The thing is you can look for it's username and password through all the documentation and you wont find it.

Since it took me some time to find it, I thought it would be a good idea to share with you guys.

Without further ado here’s the BDE’s Hadoop template default password:

User: serengeti
Password: password

You can leverage sudo if you need to run any command with a higher privileges.

One last reminder, after your change is completed, don’t forget to remove all configs from /etc/sysconfig/network-scripts/ifcfg-eth (otherwise the new nodes wont get the correct IP) and delete the VM’s snapshot.

If you got this far and still have doubts about what BDE is really about, check my BDE The basics post.
 

Tuesday, September 22, 2015

VMware vForum 2015 - Brazil



VMworld Americas and Europe are not the only events you can attend. VMware holds local events called vForum, where you can interact and listen from specialists about Cloud Management, optimization, real use cases, network and storage virtualization along with technical and business sessions.

On October 13th will be São Paulo – Brazil time , come and join this free one-day event that offers a unique opportunity. Ohh yes, you hear, it’s FREE !!

All you need to do is make your registration !!!

Bellow you can find vForums around the world

See you there !!!
 

Tuesday, September 15, 2015

VMware’s releases after VMworld 2015

Past week VMware released updates to a variety of products, the majority of them were driven by VMworld 2015 announcements while others carry compatibility enhancements, new features and even bug fixes.

I’ll highlight the features I think are the coolest ones, but you are welcome to read the full release notes.
I don’t expect you guys to read it all, but focus on the products you have deployed on your environment today. I’m sure you will find some great news…

This release of vCenter brings some security enhancements like, disabling support for SSLv3 and vCSA Kerberos authentication against AD supporting only AES256-CTS/AES128-CTS/RC4-HMAC encryption along with vSAN and Fault Tolerance compatibility.

The host ESXi leverage the same security enhancement listed for vCenter above, but the most commented feature seems to be the Stretch Cluster support with vSAN 6.0 U1.

That’s just amazing, Stretched Cluster support based on vSAN Datastores, how great is to protect your work load across two geographic locations ?!?
Also a welcome to branch officers all around the word is the ability to have vSAN clusters of only two nodes. Last but not least the integrated Health Check service that monitors your vSAN cluster for hardware compatibility, network configuration and operations.

That’s just a minor release to support some new endpoints as:
- OpenStack Juno
- vCloud Director 5.5.2
- Orchestrator 6.0.3
- vSphere 5.5 U3 and 6.0 U1
What about this minor new feature ?!?! The ability to disable snapshot for all users including business group managers and support users.

This release brings support to vSphere 6 along with NSX 6.1.4.
But more interesting is the feature of creating Organization Virtual Data Centers Templates that can be offered on a self-service basis.

SRM 6.1 is aligned with vSphere 6.0 U1 support, but it also debuts the storage policy base protection, protecting VMs based on their policies and not their underling datastores.

Along with some UI enhancements to provide more simplified wizard the greatest news is the 5 minutes RPO (only when you have both source and destination vSAN datastores).

This release of VIO comes with Kilo release, the last one for Openstack and also the capability of upgrade roll-back in case of anything goes wrong. Load Balance as a Service with Neutron service has been add to this release as well. Resource Measurement and utilization has been added as TechPreview based on Ceilometer service. Auto Scaling service through Heat is also there, nice, right ?!?

I think the most notable news here is that Hyperic functionality is built in on vROps core product, meaning you don’t need to deploy and configure the Hyperic Appliance anymore, but you still need to install the agent on the VMs you want to monitor.

With more than 20 bug fixes this release focus on stability and reliability, check the list of fixes, it might has one that will solve a problem on your environment.

The price and details of deleted VMs can now be viewed on the current month. Another feature is the capability of mapping business units to vRA Tenant Administrator.

This version on Log insight was focus more on scalability, supporting now two times more ingest data per second, 15.000 events per second per node and the number of nodes per cluster has doubled, this is huge.

New extensibilities over here, invoke published Blueprints from release pipeline and Code Stream plug-in for Jenkins continuous integration dashboard have been added..

Microsoft Windows 10 support as host as long as a guest operation system. Support for USB 3.0 redirect and Microsoft DirectX10 support are just a sample of the Horizon Flex features.

Hard to download all these new releases ? Take a look at VMware Software Manager...

Friday, September 11, 2015

Big Data Extension – The Basics

Do you know that 90% of the world’s data has been produced in the past two years ? That’s an impressive number, right ?!

 

Companies of all sizes understand that there are opportunities to enhance their customer relationships, product quality and to understand trends through analyzing this huge amount of data (that’s why the term “Big Data”). The problem is that many of those data sources are unstructured data that cannot be analyzed with traditional tools. For instance the data can be in form of text, video, images, social medias, you name it.

 

Driven by this challenge, a new application platform has been born, Hadoop.              In a nutshell, this platform uses a divide and conquers approach, using distributed components to allow the processing of large amount of data. 

 

The main roles within Haddop 1.0 architecture are:

NameNodes: in charge of managing the Hadoop File System namespace (HDFS) and data block placement;

DataNodes: are the ones storing the data and retrieving them when necessary, you can scale the nodes to storage more data and provide data resiliency;

JobTracker: Jobs are submitted to JobTracker, which splits the job into tasks and send them to the TaskTracker for execution, controlling and scheduling the jobs execution;

TaskTracker: runs on each worker node, processing the tasks and requesting the data results from the DataNodes.

 

This architecture had some scalability and efficiency problems, that’s why an architectural change has been made with Hadoop 2.0; The main architectural change was the split of JobTracker functionalities into two new roles: 

 

ResourceManager: is now in charge of tracking resource usage, node liveness, enforces allocation invariants and contention among tenants;

ApplicationMaster: is responsible for coordinating job execution plans, requesting resources from the ResourceManager and coordinating the execution of tasks.

 

Of course there's more about the architecture and how it works behind the scenes, if you want to know more here's a good paper to digest

 

As you could imagine creating those Hadoop clusters, with dozens of nodes, followed by installation and configuration of the application on each one, takes a lot of time !! That’s where VMware comes into play.

 

VMware vSphere Big Data Extensions (BDE), is the VMware’s solution to the problems of managing and deploying Hadoop efficiently and in an agile way. No matter how complex your Hadoop solution could be, BDE can deploy it,  automating the nodes creation, application installation and configuration in a much shorter time than the traditional way, saving you hours if not days of manual tasks.


To make it even better, BDE can work with the distribution of your taste: Apache Hadoop, Cloudera, Pivotal, Hortonworks and MapR.

 


Now that we know the basics of BDE, we are ready to explore more advanced topics:

- Hadoop default template password

- Graphical Interface vs command line

- Managing Disks and Controller Types

- Topology Matters

 

 Coming Soon

- IP Allocation and Name resolution

 

Tuesday, September 1, 2015

vSphere Fault Tolerance - Redundant Storage


vSphere Fault Tolerance (FT), one of the most cool and less implemented feature of VMware vSphere had a significant improvement on vSphere 6, at this point you probably know what FT is about, just in case you don’t know yet, here’s a brief explanation: 

FT is a feature that enables you to protect your mission critical applications creating a secondary VM and keep them synchronized, so in case of a failure of the primary VM the secondary VM will take place almost instantly. Cool right ?!?

 

Problem was, until vSphere 5.5, you could only protect VMs with a single vCPU !!! Good luck finding a mission critical VM running with a single vCPU.

As you could imagine, support for multiple vCPUs VM has been asked for years and finally on vSphere 6 it’s now possible.

But this post is not to talk about FT on multiple vCPUs, there is a bunch of blogs about it already, I wanna talk about another new resource of FT, Redundant Storage.

Now with vSphere 6 you have the option to also replicate the data to another datastore, eliminating the single point of failure that datastores has been being so far.

 

 FT will create a secondary copy of vmdk, vmx and the Tie Breaker file on a separate datastore and keep the data synchronized between them, so in case of a datastore failure the secondary VM will take place without any problem.

 

There's not trick to use it, just Turn On Fault Tolerance on the VM.

 

 

Then browse for the datastore where will want to place each file.


FT is a precious and limited resource, be aware of it’s limitation of 4 VMs per host or 8 vCPUs, on this sum take into account primary and secondary VMs.

 

Are you willing to use FT more often now ???

 

Who am I

My photo
I’m an IT specialist with over 15 years of experience, working from IT infrastructure to management products, troubleshooting and project management skills from medium to large environments. Nowadays I'm working for VMware as a Consulting Architect, helping customers to embrace the Cloud Era and make them successfully on their journey. Despite the fact I'm a VMware employee these postings reflect my own opinion and do not represents VMware's position, strategies or opinions.

Most Viewed Posts

Blog Archive