In the world of data engineering, the efficient organization and structuring of process flows plays a crucial role. At this point, Apache Airflow has positioned itself as one of the most efficient tools to achieve this task. However, to maximize…
Function overloading is a very interesting tool that allows several different functions to share the same identifier. This makes the readability and interpretability of our code better in some scenarios. This tool is available by default in many of today’s…
What is software testing? Software tests are tests that are performed on a piece of code or an entire software to validate that the behavior of the software is as expected. The main purpose of testing is to detect errors…
Apache Airflow is an open source tool for workflow orchestration widely used in the field of data engineering. You can take a look at this other blog post where we made an introduction to Basics on Apache Airflow. In this…
When reviewing code, we often spend time going over small bugs or stylistic details that distract us from what’s really important. In this post we will introduce pre-commit, a tool that attacks this problem by automatically correcting our code. What…
The idea of this article is to save some time for those interested in Elasticsearch and to share some useful concepts and resources. What is Elasticsearch? Elasticsearch is a free, open source, distributed search engine developed in Java capable of…
In the last few years, we have seen a great evolution in the Python programming language as it has gained popularity to become one of the most widely used programming languages. The involvement of the community with the development of…
What is MongoDB? MongoDB is an open source NOSQL database. This means that data does not necessarily have to follow a schema. All data is stored in a JSON similar document format known as BSON or Binary JSON and will…
Since our Pentaho PDI plugin for Apache Airflow release, we have seen an industry shift towards the usage of Apache Hop for data processing. What is Apache Hop? Apache Hop started (late 2019) as a fork of Kettle PDI, is…
Apache Spark is an open source framework that allows us to process large volumes of data in a distributed way. How? By dividing the large volumes of data, impossible to process in one machine, and distributing them among the different…