Over the past two years, in one way or another, almost all companies have integrated generative AI into their operations. Most of them have gone through the process of testing a chatbot to handle research, summaries, and the drafting of…
In data science, forecasting is a widely used technique in data analysis. In this post, we’ll explore what NeuralProphet is and how it works—an open-source library designed to perform forecasting tasks. What is forecasting? In the fields of statistics and…
SQLFluff is an SQL code linting and auto-formatting tool that we’ve already covered in previous articles. In SQL optimization with SQLFluff, we discussed how to install it from scratch and walked through the first steps for using it. Implementing SQLFluff…
In a previous post, we analyzed from a theoretical point of view which are the main libraries for data science and machine learning used by data scientists to treat data efficiently and extract its full potential. In this post, we…
With the recent release of the DBT v2 beta, the data transformation tool has undergone its biggest update to date. Given the significant nature of this change, it’s worth taking a closer look at what it entails and what has…
Currently, there are more than 400 million code repositories on platforms such as GitHub. With such a large ecosystem, documenting a project well has become crucial for any developer who wants to use or review the code to understand how…
What is OpenCode? OpenCode is an open-source artificial intelligence (AI) code agent that is deployed locally. It is designed to assist developers with code development and other day-to-day tasks. OpenCode integrates various large language models (LLMs), both locally and via…
The post Process Discovery: Methods and a Practical Example already explored this popular technique for deriving a process model from logs of the original system. However, there is much more that can be done with those process logs. For example,…
Testing is a key component of any data engineering process. It allows us to validate that the data we are generating or transforming meets the expectations defined at each stage. Furthermore, it helps ensure the system’s robustness by facilitating the…
In one of the latest projects we’re working on in Damavis, we began building an ETL process from scratch. On one hand, a team member was setting up the cloud architecture and all its components. Meanwhile, I was analszing the…