2023
|
Hoffmann, Renato Barreto; Faé, Leonardo; Manssour, Isabel; Griebler, Dalvan Analyzing C++ Stream Parallelism in Shared-Memory when Porting to Flink and Storm Inproceedings doi In: International Symposium on Computer Architecture and High Performance Computing Workshops (SBAC-PADW), pp. 1-8, IEEE, Porto Alegre, Brazil, 2023. @inproceedings{HOFFMANN:SBAC-PADW:23,
title = {Analyzing C++ Stream Parallelism in Shared-Memory when Porting to Flink and Storm},
author = {Renato Barreto Hoffmann and Leonardo Faé and Isabel Manssour and Dalvan Griebler},
url = {https://doi.org/10.1109/SBAC-PADW60351.2023.00017},
doi = {10.1109/SBAC-PADW60351.2023.00017},
year = {2023},
date = {2023-10-01},
booktitle = {International Symposium on Computer Architecture and High Performance Computing Workshops (SBAC-PADW)},
pages = {1-8},
publisher = {IEEE},
address = {Porto Alegre, Brazil},
series = {SBAC-PADW'23},
abstract = {Stream processing plays a crucial role in various information-oriented digital systems. Two popular frameworks for real-time data processing, Flink and Storm, provide solutions for effective parallel stream processing in Java. An option to leverage Java's mature ecosystem for distributed stream processing involves porting legacy C++ applications to Java. However, this raises considerations on the adequacy of the equivalent Java mechanisms and potential degradation in throughput. Therefore, our objective is to evaluate programmability and performance when converting stream processing applications from C++ to Java while also exploring the parallelization capabilities offered by Flink and Storm. Furthermore, we aim to assess the throughput of Flink and Storm on shared-memory manycore machines, a hardware architecture commonly found in cloud environments. To achieve this, we conduct experiments involving four different stream processing applications. We highlight challenges encountered when porting C++ to Java and working with Flink and Storm. Furthermore, we discuss throughput, latency, CPU, and memory usage results.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Stream processing plays a crucial role in various information-oriented digital systems. Two popular frameworks for real-time data processing, Flink and Storm, provide solutions for effective parallel stream processing in Java. An option to leverage Java's mature ecosystem for distributed stream processing involves porting legacy C++ applications to Java. However, this raises considerations on the adequacy of the equivalent Java mechanisms and potential degradation in throughput. Therefore, our objective is to evaluate programmability and performance when converting stream processing applications from C++ to Java while also exploring the parallelization capabilities offered by Flink and Storm. Furthermore, we aim to assess the throughput of Flink and Storm on shared-memory manycore machines, a hardware architecture commonly found in cloud environments. To achieve this, we conduct experiments involving four different stream processing applications. We highlight challenges encountered when porting C++ to Java and working with Flink and Storm. Furthermore, we discuss throughput, latency, CPU, and memory usage results. |
Faé, Leonardo; Hoffmann, Renato Barreto; Griebler, Dalvan Source-to-Source Code Transformation on Rust for High-Level Stream Parallelism Inproceedings doi In: XXVII Brazilian Symposium on Programming Languages (SBLP), pp. 41-49, ACM, Campo Grande, Brazil, 2023. @inproceedings{FAE:SBLP:23,
title = {Source-to-Source Code Transformation on Rust for High-Level Stream Parallelism},
author = {Leonardo Faé and Renato Barreto Hoffmann and Dalvan Griebler},
url = {https://doi.org/10.1145/3624309.3624320},
doi = {10.1145/3624309.3624320},
year = {2023},
date = {2023-09-01},
booktitle = {XXVII Brazilian Symposium on Programming Languages (SBLP)},
pages = {41-49},
publisher = {ACM},
address = {Campo Grande, Brazil},
series = {SBLP'23},
abstract = {Utilizing parallel systems to their full potential can be challenging for general-purpose developers. A solution to this problem is to create high-level abstractions using Domain-Specific Languages (DSL). We create a stream-processing DSL for Rust, a growing programming language focusing on performance and safety. To that end, we explore Rust’s macros as a high-level abstraction tool to support an existing DSL language named SPar and perform source-to-source code transformations in the abstract syntax tree. We aim to assess the Rust source-to-source code transformations toolset and its implications. We highlight that Rust macros are powerful tools for performing source-to-source code transformations for abstracting structured stream processing. In addition, execution time and programmability results are comparable to other solutions.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Utilizing parallel systems to their full potential can be challenging for general-purpose developers. A solution to this problem is to create high-level abstractions using Domain-Specific Languages (DSL). We create a stream-processing DSL for Rust, a growing programming language focusing on performance and safety. To that end, we explore Rust’s macros as a high-level abstraction tool to support an existing DSL language named SPar and perform source-to-source code transformations in the abstract syntax tree. We aim to assess the Rust source-to-source code transformations toolset and its implications. We highlight that Rust macros are powerful tools for performing source-to-source code transformations for abstracting structured stream processing. In addition, execution time and programmability results are comparable to other solutions. |
Leonarczyk, Ricardo; Griebler, Dalvan; Mencagli, Gabriele; Danelutto, Marco Evaluation of Adaptive Micro-batching Techniques for GPU-accelerated Stream Processing Inproceedings In: Euro-ParW 2023: Parallel Processing Workshops, pp. 1-8, Springer, Limassol, 2023. @inproceedings{LEONARCZYK:Euro-ParW:23,
title = {Evaluation of Adaptive Micro-batching Techniques for GPU-accelerated Stream Processing},
author = {Ricardo Leonarczyk and Dalvan Griebler and Gabriele Mencagli and Marco Danelutto},
url = {https://doi.org/},
year = {2023},
date = {2023-08-01},
booktitle = {Euro-ParW 2023: Parallel Processing Workshops},
pages = {1-8},
publisher = {Springer},
address = {Limassol},
series = {Euro-ParW'23},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
|
Dopke, Luan; Griebler, Dalvan Estudo Sobre Spark nas Aplicações de Processamento de Log e Análise de Cliques Inproceedings doi In: Anais da XXIII Escola Regional de Alto Desempenho da Região Sul, pp. 85-88, Sociedade Brasileira de Computação, Porto Alegre, Brazil, 2023. @inproceedings{larcc:DOPKE:ERAD:23,
title = {Estudo Sobre Spark nas Aplicações de Processamento de Log e Análise de Cliques},
author = {Luan Dopke and Dalvan Griebler},
url = {https://doi.org/10.5753/eradrs.2023.229298},
doi = {10.5753/eradrs.2023.229298},
year = {2023},
date = {2023-05-01},
booktitle = {Anais da XXIII Escola Regional de Alto Desempenho da Região Sul},
pages = {85-88},
publisher = {Sociedade Brasileira de Computação},
address = {Porto Alegre, Brazil},
abstract = {O uso de aplicações de processamento de dados de fluxo contínuo vem crescendo cada vez mais, dado este fato o presente estudo visa mensurar a desempenho do framework Apache Spark Strucutured Streaming perante o framework Apache Storm nas aplicações de fluxo contínuo de dados, estas sendo processamento de logs e análise de cliques. Os resultados demonstram melhor desempenho para o Apache Storm em ambas as aplicações.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
O uso de aplicações de processamento de dados de fluxo contínuo vem crescendo cada vez mais, dado este fato o presente estudo visa mensurar a desempenho do framework Apache Spark Strucutured Streaming perante o framework Apache Storm nas aplicações de fluxo contínuo de dados, estas sendo processamento de logs e análise de cliques. Os resultados demonstram melhor desempenho para o Apache Storm em ambas as aplicações. |
Fim, Gabriel Rustick; Griebler, Dalvan Implementação e Avaliação do Paralelismo de Flink nas Aplicações de Processamento de Log e Análise de Cliques Inproceedings doi In: Anais da XXIII Escola Regional de Alto Desempenho da Região Sul, pp. 69-72, Sociedade Brasileira de Computação, Porto Alegre, Brazil, 2023. @inproceedings{larcc:FIM:ERAD:23,
title = {Implementação e Avaliação do Paralelismo de Flink nas Aplicações de Processamento de Log e Análise de Cliques},
author = {Gabriel Rustick Fim and Dalvan Griebler},
url = {https://doi.org/10.5753/eradrs.2023.229290},
doi = {10.5753/eradrs.2023.229290},
year = {2023},
date = {2023-05-01},
booktitle = {Anais da XXIII Escola Regional de Alto Desempenho da Região Sul},
pages = {69-72},
publisher = {Sociedade Brasileira de Computação},
address = {Porto Alegre, Brazil},
abstract = {Este trabalho visou implementar e avaliar o desempenho das aplicações de Processamento de Log e Análise de Cliques no Apache Flink, comparando o desempenho com Apache Storm em um ambiente computacional distribuído. Os resultados mostram que a execução em Flink apresenta um consumo de recursos relativamente menor quando comparada a execução em Storm, mas possui um desvio padrão alto expondo um desbalanceamento de carga em execuções onde algum componente da aplicação é replicado.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Este trabalho visou implementar e avaliar o desempenho das aplicações de Processamento de Log e Análise de Cliques no Apache Flink, comparando o desempenho com Apache Storm em um ambiente computacional distribuído. Os resultados mostram que a execução em Flink apresenta um consumo de recursos relativamente menor quando comparada a execução em Storm, mas possui um desvio padrão alto expondo um desbalanceamento de carga em execuções onde algum componente da aplicação é replicado. |