ASPIDE Project: Perspectives on the Sca...

Extreme Data is an incarnation of Big Data concept distinguished by the massive amounts of data that must be queried, communicated and analyzed in (near) real-time by using a very large number of memory/storage elements of both, the converging Cloud and Pre-Exascale computing systems. Notable examples are the raw high energy physics data produced at a rate of hundreds of gigabits-per-second that must be filtered, stored and analyzed in a fault-tolerant fasion, multi-scale brain imaging data analysis and simulations, complex networks data analyses, driven by the social media systems. To handle such amounts of data multi-tierung architectures are introduced, including scheduling systems and distributed storage systems, ranging from in-memory databases to tape libraries. The ASPIDE project is contributing with the definition of a new programming paradigm, APIs, runtime tools and methodologies for expressing data intensive tasks on the converging large-scale systems , which can pave the way for the exploitation of parallelism policies over the various models of the system architectures, promoting high performance and efficiency, and offering powerful operations and mechanisms for processing extreme data sources at high speed and / or real-time.