Tome / Bhattacharjee / Radford Data Engineering with Scala and Spark

Build streaming and batch pipelines that process massive amounts of data using Scala
1. Auflage 2024
ISBN: 978-1-80461-432-7
Verlag: De Gruyter
Format: EPUB
Kopierschutz: 0 - No protection

E-Book, Englisch, 300 Seiten

No detailed description available for "Data Engineering with Scala and Spark".

Autoren/Hrsg.

Tome, Eric

Bhattacharjee, Rupam

Radford, David

Fachgebiete

• Mathematik | Informatik EDV | Informatik Informatik Mensch-Maschine-Interaktion Informationsarchitektur
• Mathematik | Informatik EDV | Informatik Daten / Datenbanken Data Warehouse
• Mathematik | Informatik EDV | Informatik Daten / Datenbanken Datenbankdesign & Datenbanktheorie
• Mathematik | Informatik EDV | Informatik Daten / Datenbanken Automatische Datenerfassung, Datenanalyse

Inhaltsverzeichnis

Table of Contents - Scala Essentials for Data Engineers
- Environment Setup
- An Introduction to Apache Spark and Its APIs – DataFrame, Dataset, and Spark SQL
- Working with Databases
- Object Stores and Data Lakes
- Understanding Data Transformation
- Data Profiling and Data Quality
- Test-Driven Development, Code Health, and Maintainability
- CI/CD with GitHub
- Data Pipeline Orchestration
- Performance Tuning
- Building Batch Pipelines Using Spark and Scala
- Building Streaming Pipelines Using Spark and Scala

Über Autor(innen)

Tome Eric:

Eric Tome has over 25 years of experience working with data. He has contributed to and led teams that ingested, cleansed, standardized, and prepared data used by business intelligence, data science, and operations teams. He has a background in mathematics and currently works as a senior solutions architect at Databricks, helping customers solve their data and AI challenges.Bhattacharjee Rupam:

Rupam Bhattacharjee works as a lead data engineer at IBM. He has architected and developed data pipelines, processing massive structured and unstructured data using Spark and Scala for on-premises Hadoop and K8s clusters on the public cloud. He has a degree in electrical engineering.Radford David:

David Radford has worked in big data for over 10 years, with a focus on cloud technologies. He led consulting teams for several years, completing a migration from legacy systems to modern data stacks. He holds a master's degree in computer science and works as a senior solutions architect at Databricks.