Damavis Summary of week 9, 2021
Compilation of the news and news shared this week in our networks
Introduction to Apache YARN
Basic Single Node Configuration Note: the code of this post has been tested using Apache Hadoop 2.10.1. Please check out our previous post, Introduction to Apache Hadoop, to configure this version of Hadoop, in case you have not done it…
Damavis Summary of week 7, 2021
Compilation of the news and news shared this week in our networks
Cross Compiling In Java
A situation that occurs frequently is having to write code for a project that is in an old version of java. In Damavis, we always like to make use of the latest tools added to the language, so in these…
Damavis Summary of week 7, 2021
Compilation of the news and news shared this week in our networks
Deploying Airflow: CeleryExecutor on Kubernetes
How to deploy the Apache Airflow process orchestrator on Kubernetes What is Apache Airflow and how does it work? One of the work processes of a data engineer is called ETL (Extract, Transform, Load), which allows organisations to have the…
Summary of week 6 2021
Compilation of the news and news shared this week in our networks
Introduction to Apache Hadoop
Single Node Configuration Without Yarn Sometimes it might be a bit overwhelming to understand the role of the most common open source technologies used in big data contexts. For example, probably most of you have heard about tools such as…
First steps with Pyspark and Pycharm
Definitive guide to configure the Pyspark development environment in Pycharm; one of the most complete options. Spark has become the Big Data tool par excellence, helping us to process large volumes of data in a simplified, clustered and fault-tolerant way.…
Graphs – Finding optimal routes
An example of how the use of graphs can help us find optimal routes to solve various problems. A graph system can be used for multiple purposes, being in some cases very useful to solve complex problems. In this article…