Wednesday, October 21, 2015

Managing Distributions on BDE

One of the Big Data Extension strengths is the flexibility to provision Hadoop clusters based on the distributions that best suites your taste. Although BDE comes only with Bigtop distribution, it’s very easy to add the new ones you need.

To add a new distro just use the /opt/serengeti/sbin/config-distro.rb command.
BDE Administration Guide cover it nicely, so there’s no point to duplicate the details here, just take a look at the guide for the right syntax and I'm sure you will be good to go.

In fact what I want to cover here is how to remove a distro from BDE !!!
You might have realized already that using config-distro commando to add a distro is pretty straightforward, but removing it is not that intuitive. 
Let's see how to accomplish it then !

As you can see, I have added Hortonworks (HDP-2.2) to my environment.

When you add a distro, it's details are published to a manifest file for later consumption.
To remove it, login on BDE and edit the manifest file to remove the distro information from it.

Don’t forget to create a backup file first.
run: cp /opt/serengeti/www/distros/manifest /opt/serengeti/www/distros/manifest.bkp

Now we are ready to edit it.
run: vi /opt/serengeti/www/distros/manifest

Clean up the entire section of the distro you want to remove, in my case, HDP 2.2

Save the file and restart BDE services to the changes take effect.

run: service tomcat restart

As you can see the Hortonworks is no longer available.

If anything goes wrong just restore the backup file and restart tomcat service again.

Now you are ready to manage the distributions on your BDE solution !!!


Unknown said...

Hello, Thanks for the post, I have BDE 2.3.0 and 2.3.1 also seperate cloudera manager installed in Different VM mapped it as application manger in both BDE plug-in's wheren I'm able to create successfull cluster with BDE 2.3.0 but not with 2.3.1 any ideas/suggestions would be greate appriciated. Thanks Sharan

Eduardo Meirelles da Rocha said...

Hello Sharan, I would need more information to help you, but I suggest you look at BDE's log to see what's the cause.

Post a Comment

Who am I

My photo
I’m an IT specialist with over 15 years of experience, working from IT infrastructure to management products, troubleshooting and project management skills from medium to large environments. Nowadays I'm working for VMware as a Consulting Architect, helping customers to embrace the Cloud Era and make them successfully on their journey. Despite the fact I'm a VMware employee these postings reflect my own opinion and do not represents VMware's position, strategies or opinions. Reach me at @dumeirell

Most Viewed Posts

Blog Archive