Publication: Smart Data Placement for Big Data Pipel...
Master data
Title: | Smart Data Placement for Big Data Pipelines: An Approach based on the Storage-as-a-Service Model |
Subtitle: | |
Abstract: | The development of big data pipelines is a challenging task, especially when data storage is considered as part of the data pipelines. Local storage is expensive, hard to maintain, comes with several challenges (e.g., data availability, data security, and backup). The use of cloud storage, i.e., Storageas-a-Service (StaaS), instead of local storage has the potential of providing more flexibility in terms of such as scalability, fault tolerance, and availability. In this paper, we propose a generic approach to integrate StaaS with data pipelines, i.e., computation on an on-premise server or on a specific cloud, but integration with StaaS, and develop a ranking method for available storage options based on five key parameters: cost, proximity, network performance, the impact of server-side encryption, and user weights. The evaluation carried out demonstrates the effectiveness of the proposed approach in terms of data transfer performance and the feasibility of dynamic selection of a storage option based on four primary user scenarios. |
Keywords: | Storage-as-a-service, big data pipelines, data locality, data placement strategies, software containers |
Publication type: | Article in Proceedings (Authorship) |
Publication date: | 14.03.2023 (Online) |
Published by: |
UCC 2022 Proceedings of the IEEE/ACM 15th International Conference on Utility and Cloud Computing
UCC 2022 Proceedings of the IEEE/ACM 15th International Conference on Utility and Cloud Computing
(
IEEE Xplore Digital Library;
)
to publication |
Title of the series: | - |
Volume number: | - |
First publication: | Yes |
Version: | - |
Page: | pp. 317 - 320 |
Versionen
Keine Version vorhanden |
Publication date: | 12.2022 |
ISBN: |
|
ISSN: | - |
Homepage: | https://ieeexplore.ieee.org/document/10061837 |
Publication date: | 14.03.2023 |
ISBN (e-book): | - |
eISSN: | - |
DOI: | http://dx.doi.org/10.1109/ucc56403.2022.00056 |
Homepage: | https://ieeexplore.ieee.org/document/10061837 |
Open access |
|
Wibi-relevant version (Admin) |
Authors
Akif Quddus Khan (external) |
Nikolay Nikolov (external) |
Mihhail Matskin (external) |
Radu Aurel Prodan (internal) |
Hui Song (external) |
Dumitru Roman (external) |
Ahmet Soylu (external) |
Assignment
Organisation | Address | ||||
---|---|---|---|---|---|
Fakultät für Technische Wissenschaften
Institut für Informationstechnologie
|
AT - 9020 Klagenfurt am Wörthersee |
Categorisation
Subject areas | |
Research Cluster | No research Research Cluster selected |
Peer reviewed |
|
Publication focus |
Classification raster of the assigned organisational units:
|
working groups |
|
Cooperations
Organisation | Address | ||||
---|---|---|---|---|---|
Norwegian University of Science and Technology
|
NO
- 7491
Trondheim |
||||
SINTEF Digital
|
NO
Oslo |
||||
Royal Institute of Technology
|
SE
Stockholm |
||||
Oslo Metropolitan University
|
NO - 0130 Oslo |
Research activities
Projects: |
|
Publications: | No related publications |
Events: |
|
Lectures: | No related lectures |