Antonio Boutaour

Joined: 2021-02-17
Articles: 4

Data Engineering

Kafka + Spark for Batch processing

Batch processing is a methodology widely used in the world of Big Data. Throughout this post, we will analyse how technologies typically used for streaming -such as Apache Kafka and Apache Spark– can ve leveraged for batch data processing. ETL…

Antonio Boutaour
2021-12-16

Data Engineering

Deploying Airflow: CeleryExecutor on Kubernetes

How to deploy Airflow Celery Executor on Kubernetes

What is Apache Airflow and how does it work? One of the work processes of a data engineer is called ETL (Extract, Transform, Load), which allows organisations to have the capacity to load data from different sources, apply an appropriate…

Antonio Boutaour
2021-02-17