DDD Tactical Design
It is time to dive into the concept of Domain-Driven Design (DDD). We are going to develop it from the point of view of software architecture. In this post, we will try to synthesise the concepts and explain the essence…
E2E testing with SeleniumBase
In a previous article, we analysed what is end-to-end testing, its scope, types, best practices, vulnerabilities, security risks and some interesting technologies. This time, we are going to work with SeleniumBase, a Python testing framework based on Selenium. First we…
What is end-to-end testing?
Software testing is an essential mechanism to guarantee the correct functioning of our applications. There are different types of tests, each of them focused on evaluating a different characteristic or with a particular approach. Within the repertoire of tests that…
SQL Optimisation with SQLFluff
When developing code, it is not only important to consider its functionality, but also its readability. This is especially important in working environments where a team of people are working on the same code. Readable code that is unified by…
Introduction to Apache Arrow
In this article we are going to see what this tool is and how it works. Apache Arrow is present in most of the frameworks in our environment, such as Spark, Dask, Drill, Tensorflow, Kudu, etc. All of them are…
APISIX: Authentication, security and traffic
In Introduction to Apache APISIX we studied the different components of this platform and the advantages of using it. Additionally, we saw a way to dockerise it and we configured a simple route with the openid-connect plugin, integrating authentication through…
Index optimisation in relational databases
When we are working with relational databases, it is usual for some queries to be extremely slow. This can be due to different reasons, but the most common is that the data access pattern that the query demands is not…
New features in DBT 1.9
The developers of DBT have set themselves the goal of releasing a new major release every four months, and so far they are delivering. Last October, version 1.9 of this popular framework was released, and has been undergoing corrections. In…
Sensitive data management in BigQuery – Part II
This is a continuation of what was described in the first part of this series of articles on Sensitive data management in BigQuery. Working with data in this tool can require control over who accesses it and how it is…
Tutorial DataHub 4 – API
In this new part of the DataHub tutorial saga, we are going to work on the connectivity with the platform through the API. As data engineers, the goal is to incorporate DataHub as a Data Governance tool in our ecosystem.…