Dataconomy
  • News
  • AI
  • Big Data
  • Machine Learning
  • Trends
    • Blockchain
    • Cybersecurity
    • FinTech
    • Gaming
    • Internet of Things
    • Startups
    • Whitepapers
  • Industry
    • Energy & Environment
    • Finance
    • Healthcare
    • Industrial Goods & Services
    • Marketing & Sales
    • Retail & Consumer
    • Technology & IT
    • Transportation & Logistics
  • Events
  • About
    • About Us
    • Contact
    • Imprint
    • Legal & Privacy
    • Newsletter
    • Partner With Us
    • Writers wanted
Subscribe
No Result
View All Result
Dataconomy
  • News
  • AI
  • Big Data
  • Machine Learning
  • Trends
    • Blockchain
    • Cybersecurity
    • FinTech
    • Gaming
    • Internet of Things
    • Startups
    • Whitepapers
  • Industry
    • Energy & Environment
    • Finance
    • Healthcare
    • Industrial Goods & Services
    • Marketing & Sales
    • Retail & Consumer
    • Technology & IT
    • Transportation & Logistics
  • Events
  • About
    • About Us
    • Contact
    • Imprint
    • Legal & Privacy
    • Newsletter
    • Partner With Us
    • Writers wanted
Subscribe
No Result
View All Result
Dataconomy
No Result
View All Result

Learn Basic Codes in This Apache Spark Tutorial

by Evgeniya Panova
May 20, 2020
in Data Science, Education
Home Topics Data Science
Share on FacebookShare on TwitterShare on LinkedInShare on WhatsAppShare on e-mail

Whether you are experienced or thinking about getting your hands on Apache Spark, this Apache Spark tutorial will guide you through:

  • downloading and running Spark
  • launching Spark’s consoles
  • Spark’s basic architecture
  • Spark’s language APIs
  • DataFrames and SQL
  • Spark’s Toolset

What is Apache Spark?

Apache Spark is a unified computing engine and a set of libraries for parallel data processing on computer clusters. Spark is the most actively developed open-source engine for this task; making it the de facto tool for any developer or data scientist interested in big data. Spark supports multiple widely-used programming languages (Python, Java, Scala and R), includes libraries for diverse tasks ranging from SQL to streaming and machine learning, and runs anywhere from a laptop to a cluster of thousands of servers. This makes it an easy system to start with and scale up to big data processing or an incredibly large scale.

Here’s a simple illustration of all that Spark has to offer an end user.

apache spark
Apache Spark Structure / Source: Databricks

Let’s break down our description of Apache Spark – a unified computing engine and set of libraries for big data – into its key components.

1. Unified: Spark’s key driving goal is to offer a unified platform for writing big data applications. Spark is designed to support a wide range of data analytics tasks, ranging from simple data loading and SQL queries to machine learning and streaming computation, over the same computing engine and with a consistent set of APIs.


Join the Partisia Blockchain Hackathon, design the future, gain new skills, and win!


2. Computing Engine: At the same time that Spark strives for unification, Spark carefully limits its scope to a computing engine. By this, we mean that Spark only handles loading data from storage systems and performing computation on it, not permanent storage as the end itself.

3. Libraries: Spark’s final component is its libraries, which build on its design as a unified engine to provide a unified API for common data analysis tasks. Spark supports both standard libraries that ship with the engine, and a wide array of external libraries published as third-party packages by the open-source communities.

Spark’s Language APIs

Spark’s language APIs allow you to run Spark code from other languages. For the most part, Spark presents some core “concepts” in every language and these concepts are translated into Spark code that runs on the cluster of machines.

Scala

Spark is primarily written in Scala, making it Spark’s “default” language. This book will include Scala code examples wherever relevant.

Java

Even though Spark is written in Scala, Spark’s authors have been careful to ensure that you can write Spark code in Java. This book will focus primarily on Scala but will provide Java examples where relevant.

Python

Python supports nearly all constructs that Scala supports. This book will include Python code examples whenever we include Scala code examples and a Python API exists.

apache spark language api
Apache Spark language APIs / Source: Databricks

SQL

Spark supports ANSI SQL 2003 standard. This makes it easy for analysts and non-programmers to leverage the big data powers of Spark. This book will include SQL code examples wherever relevant

R

Spark has two commonly used R libraries, one as a part of Spark core (SparkR) and another as an R community driven package (sparklyr).

Download the full free Apache Spark tutorial here.

Editor’s note: Article includes introductory information about Apache Spark from the Databricks free ebook: “A Gentle Introduction to Apache Spark”

Related Posts

Claiming to be the most humane AI chatbot, Replika AI wants to be your empathetic pal

Claiming to be the most humane AI chatbot, Replika AI wants to be your empathetic pal

May 26, 2023
Meet Inflection AI Pi chatbot that earned acclaim from Bill Gates

Meet Inflection AI Pi chatbot that earned acclaim from Bill Gates

May 25, 2023
Exploring the dynamic fusion of AI and the IoT

Exploring the dynamic fusion of AI and the IoT

May 25, 2023
Photoshop AI generative fill: Check out Adobe’s latest AI feature

Photoshop AI generative fill: Check out Adobe’s latest AI feature

May 24, 2023
Who co-pilots the co-pilots? Why AI needs cloud support

Who co-pilots the co-pilots? Why AI needs cloud support

May 24, 2023
AI-generated images are taking over the world and you can’t do anything about it

AI-generated images are taking over the world and you can’t do anything about it

May 23, 2023

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

LATEST ARTICLES

Navigating the path to generative AI success across industries: A Grid Dynamics crawl-walk-run strategy

Boosting productivity and efficiency with workload automation

Claiming to be the most humane AI chatbot, Replika AI wants to be your empathetic pal

Meet Inflection AI Pi chatbot that earned acclaim from Bill Gates

Exploring the dynamic fusion of AI and the IoT

Photoshop AI generative fill: Check out Adobe’s latest AI feature

Dataconomy

COPYRIGHT © DATACONOMY MEDIA GMBH, ALL RIGHTS RESERVED.

  • About
  • Imprint
  • Contact
  • Legal & Privacy
  • Partnership
  • Writers wanted

Follow Us

  • News
  • AI
  • Big Data
  • Machine Learning
  • Trends
    • Blockchain
    • Cybersecurity
    • FinTech
    • Gaming
    • Internet of Things
    • Startups
    • Whitepapers
  • Industry
    • Energy & Environment
    • Finance
    • Healthcare
    • Industrial Goods & Services
    • Marketing & Sales
    • Retail & Consumer
    • Technology & IT
    • Transportation & Logistics
  • Events
  • About
    • About Us
    • Contact
    • Imprint
    • Legal & Privacy
    • Newsletter
    • Partner With Us
    • Writers wanted
No Result
View All Result
Subscribe

This website uses cookies. By continuing to use this website you are giving consent to cookies being used. Visit our Privacy Policy.
x
x