Testing in Apache Airflow
Today we are going to talk about two ways of testing in Apache Airflow. Historically, testing in Airflow has been something that has been a headache for all users of the famous framework. The coupling of the code with the…
Category Software
Types of Schedulers in Apache Airflow: Timetables and Datasets
Apache Airflow is an open source tool designed for workflow orchestration especially useful in the field of data engineering. DAGs are defined in Python files and set the relationship and dependencies between the tasks to be executed. You can take…
Apache Airflow with LocalStack
Usually, when we start working on a new integration where it is necessary to connect to AWS services in the early stages of development, it is easier and faster to work only locally. For this, we can make use of…
Introduction to Pycharm Templates
PyCharm is an IDE developed by JetBrains for Python. As an IDE, its goal is to provide the programmer with a set of features that make code development in this language more efficient. For example, it automatically writes import lines…
Apache Airflow dockerization and customization
In the world of data engineering, the efficient organization and structuring of process flows plays a crucial role. At this point, Apache Airflow has positioned itself as one of the most efficient tools to achieve this task. However, to maximize…
Master function Overloading in Python with simple examples
Function overloading is a very interesting tool that allows several different functions to share the same identifier. This makes the readability and interpretability of our code better in some scenarios. This tool is available by default in many of today’s…
Introduction to software testing
What is software testing? Software tests are tests that are performed on a piece of code or an entire software to validate that the behavior of the software is as expected. The main purpose of testing is to detect errors…
Sensors in Apache Airflow: What they are and practical examples
Apache Airflow is an open source tool for workflow orchestration widely used in the field of data engineering. You can take a look at this other blog post where we made an introduction to Basics on Apache Airflow. In this…
Automate code quality checks with Pre-commit in Python
When reviewing code, we often spend time going over small bugs or stylistic details that distract us from what’s really important. In this post we will introduce pre-commit, a tool that attacks this problem by automatically correcting our code. What…
Introduction to Elasticsearch
The idea of this article is to save some time for those interested in Elasticsearch and to share some useful concepts and resources. What is Elasticsearch? Elasticsearch is a free, open source, distributed search engine developed in Java capable of…