Learning PySpark : build data-intensive applications locally and deploy at scale using the combined powers of Python and Spark 2.0 by Drabas, Tomasz

Table of Contents: “…-- Simple streaming application using DStreams -- A quick primer on global aggregations -- Introducing Structured Streaming -- Summary -- Chapter 11: Packaging Spark Applications -- The spark-submit command -- Command line parameters -- Deploying the app programmatically -- Configuring your SparkSession -- Creating SparkSession -- Modularizing code -- Structure of the module -- Calculating the distance between two points -- Converting distance units -- Building an egg -- User defined functions in Spark -- Submitting a job -- Monitoring execution -- Databricks Jobs -- Summary -- Index.…”


Request