Real World Hadoop – Deploying Hadoop with Cloudera Manager
Move to the next step from the Cloudera QuickStart VM. Install a DEV Hadoop Environment with Enterprise Tooling
Course Access
You can access all the Big Data / Spark courses for one low monthly fee. Currently the membership site houses courses that covers deploying Hadoop with Cloudera and Hortonworks as well as installing and working with Spark 2.0.
- POC-d membership site : POC-D Membership site
This course can be purchased from
“Big Data” technology is a hot and highly valuable skill to have – and this course will teach you how to quickly deploy a Hadoop Cluster using the Cloudera stack.
Cloudera allows you to download a QuickStart Virtual machine which is great for developers, but this is of no use for the Operations team to start the planning and the building out of DEV / UAT and PROD environments within their organizations. What assumptions were made when the QuickStart VM was put together?
In addition, hosting all of Cloudera’s processes as well as Hadoop’s processes on one VM is not a model that any large organization can or should follow. The Hadoop services need to be split out across multiple VMs/Servers. In fact that’s the whole point out Hadoop!
Distributed Data and Distributed Compute.
After all, if you are developing against or operating a distributed environment, it needs to be tested. Tested in terms of the forcing various failure modes within the cluster and ensuing that the cluster can still respond to user requests. Killing the QuickStart VM destroys the entire cluster!
You’ll learn the same techniques these large enterprise guys use to move to the next step in building out an enterprise grade Hadoop cluster.
If you are a developer, the operations team can build out that centralized cluster in which you are truly testing against a distributed cluster. Testing code against the Quickstart VM may work, but as any experienced distributed developer knows, verifying code against a pseudo cluster on a single machine is different than verifying against code against a truly distributed cluster.
As an example bottlenecks in Networks or CPU cycles will come to light. In addition, this will also assist in capacity planing of the UAT / PROD cluster as initial metrics can be acquired.
If you are in operations then this gives the operations team an environment for the team to start learning how to jointly operate the cluster. Here the team can start to understand cluster metrics, adding/removing cluster nodes, managing the various Hadoop services (Zookeeper, HDFS, YARN and Spark) and a lot more. We also look at managing Cloudera Hadoop Parcels as well as changing Hadoop versions once a cluster is deployed.
The operation team can start to develop procedures and change management documentation ready for Production operation of a Hadoop cluster.
Recommended Cloudera Manager curriculum path. If you already have Cloudera Manager installed, you do not need to access the first three courses
