In the last few years, we have seen a great evolution in the Python programming language as it has gained popularity to become one of the most widely used programming languages. The involvement of the community with the development of…
What is MongoDB? MongoDB is an open source NOSQL database. This means that data does not necessarily have to follow a schema. All data is stored in a JSON similar document format known as BSON or Binary JSON and will…
Since our Pentaho PDI plugin for Apache Airflow release, we have seen an industry shift towards the usage of Apache Hop for data processing. What is Apache Hop? Apache Hop started (late 2019) as a fork of Kettle PDI, is…
Apache Spark is an open source framework that allows us to process large volumes of data in a distributed way. How? By dividing the large volumes of data, impossible to process in one machine, and distributing them among the different…
Apache Drools is a software used to manage business rules of any kind. As it is a very extensive framework, we limit this article to directly apply a simple use case with Scala, thus leaving aside the theoretical explanations that…
What is Apache Kafka? Apache Kafka is an open source distributed event system. It was originally developed by LinkedIn, in order to cover the needs caused by its rapid growth, and moved to a microservices-based infrastructure. It is also an…
One of the most outstanding new features of Airflow 2.3.0 is Dynamic Task Mapping. This new feature adds the possibility of creating tasks dynamically at runtime. Thanks to this we can change the number of such tasks in our DAG…
DevOps is a set of practices that aim to streamline the software development lifecycle by coordinating the development (Dev) and operations (Ops) departments. Teams with an integrated DevOps culture are able to continuously integrate and deliver software (CI/CD) by automating…
It has been about a year since I started developing some things in Go. At first, it was an unknown language to me. I only knew that it was a “typed” language, compiled and memory safe through a GC (Garbage…
Apache Airflow is a free workflow orchestration software, which are created through Python scripts, and can be monitored using its user interface. Some examples of workflows in which this tool could be used are the scheduling of ETL (Extract, Transform,…