You can modify this script or create your own script to install other versions of Spark. In this topic, we use a Script Action custom script to install Spark on an HDInsight cluster.
#How to install apache spark 1.5 on lnux how to#
This topic provides instructions on how to customize an HDInsight cluster to install Spark. Also, Spark is compatible with the Hadoop Distributed File System (HDFS) and Azure Blob storage so the existing data can easily be processed via Spark. Spark improves the traditional MapReduce framework by avoiding writes to disk in the intermediate stages. Spark can also be used to perform conventional disk-based data processing. Spark's in-memory computation capabilities make it a good choice for iterative algorithms in machine learning and graph computations. What is Spark?Īpache Spark is an open-source parallel processing framework that supports in-memory processing to boost the performance of big-data analytic applications. For more information, see Get Started with Apache Spark on HDInsight. Using the Spark cluster type, you get a Windows-based HDInsight version 3.2 cluster with Spark version 1.3.1. However, this is limited to Windows-based clusters currently. HDInsight also provides Spark as a cluster type, which means you can now directly provision a Spark cluster without modifying a Hadoop cluster. Once you have installed Spark, you'll also learn how to run a Spark query on HDInsight clusters. For more information, see Customize HDInsight cluster using Script Action. Script Action lets you run scripts to customize a cluster, only when the cluster is being created. In this document, you will learn how to install Spark by using Script Action. Install and use Spark on HDInsight Hadoop clusters
0 kommentar(er)
