Introduction This post is about the impact that blocking code can have on an application and the importance of using libraries that natively support asynchronous programming. The goal is to understand that a blocking call is always blocking. In the…
Introducción Since our Pentaho PDI plugin for Apache Airflow release, we have seen an industry shift towards the usage of Apache Hop for data processing. What is Apache Hop? Apache Hop started (late 2019) as a fork of Kettle PDI,…
Introduction If you have ever shared code, it is quite likely that you have said “well, it works on my machine” when you see how others have difficulties running it. Incorrect configuration, version differences or uninstalled dependencies are often some…
Today I would like to deal with a topic that, from my point of view, is very important and is probably the holy grail of data engineering projects. However, we rarely reach the necessary level of maturity to be able…
When we imagine a simple programming algorithm, it is logical to think about a succession of instructions that are executed sequentially, where the next instruction will not be executed until the one immediately preceding it has been completed. However, depending…
ETL process. Central piece of the Big Data project Collecting, ingesting, integrating, processing, storing and analyzing large volumes of information are the fundamental activities of a Big Data project. It is the union of these tasks that provides a solid…
