Adaptive utilization of communication and computational resources in high-performance distributed systems: the EMOP approach

Current pixel-array detectors produce diffraction images at extreme data rates (of up to 2 TB h−1) that make severe demands on computational resources. New multiprocessing frameworks are required to achieve rapid data analysis, as it is important to be able to inspect the data quickly in order to guide the experiment in real time. By utilizing readily available web-serving tools that interact with the Python scripting language, it was possible to implement a high-throughput Bragg-spot analyzer (cctbx.spotfinder) that is presently in use at numerous synchrotron-radiation beamlines. Similarly, Python interoperability enabled the production of a new data-reduction package (cctbx.xfel) for serial femtosecond crystallography experiments at the Linac Coherent Light Source (LCLS). Future data-reduction efforts will need to focus on specialized problems such as the treatment of diffraction spots on interleaved lattices arising from multi-crystal specimens. In these challenging cases, accurate modeling of close-lying Bragg spots could benefit from the high-performance computing capabilities of graphics-processing units.

Download Full-text

Flow control for fully adaptive routing††Part of this research was first presented at the 18th IEEE International Symposium on High Performance Computer Architecture (HPCA-2012) [32]. The journal extension edition of the HPCA-2012 paper was published in IEEE Transactions on Parallel and Distributed Systems [33].

Networks-On-Chip ◽

10.1016/b978-0-12-800979-6.00006-8 ◽

2015 ◽

pp. 175-214

Keyword(s):

Distributed Systems ◽

Flow Control ◽

International Symposium ◽

Computer Architecture ◽

High Performance ◽

Adaptive Routing ◽

Fully Adaptive ◽

High Performance Computer ◽

Ieee International Symposium

Download Full-text

Adaptive Threshold Based Scheduler for Batch of Independent Jobs for Cloud Computing System

Research Anthology on Architectures, Frameworks, and Integration Strategies for Distributed and Cloud Computing ◽

10.4018/978-1-7998-5339-8.ch110 ◽

2021 ◽

pp. 2246-2266

Author(s):

TAJ ALAM ◽

PARITOSH DUBEY ◽

ANKIT KUMAR

Keyword(s):

Cloud Computing ◽

Distributed Systems ◽

High Performance ◽

Large Scale ◽

Real Life ◽

Interval Estimation ◽

Computing System ◽

Adaptive Threshold ◽

Batch Simulation ◽

Heterogeneous Distributed Systems

Distributed systems are efficient means of realizing high-performance computing (HPC). They are used in meeting the demand of executing large-scale high-performance computational jobs. Scheduling the tasks on such computational resources is one of the prime concerns in the heterogeneous distributed systems. Scheduling jobs on distributed systems are NP-complete in nature. Scheduling requires either heuristic or metaheuristic approach for sub-optimal but acceptable solutions. An adaptive threshold-based scheduler is one such heuristic approach. This work proposes adaptive threshold-based scheduler for batch of independent jobs (ATSBIJ) with the objective of optimizing the makespan of the jobs submitted for execution on cloud computing systems. ATSBIJ exploits the features of interval estimation for calculating the threshold values for generation of efficient schedule of the batch. Simulation studies on CloudSim ensures that the ATSBIJ approach works effectively for real life scenario.

Download Full-text

GITIRBio: A Semantic and Distributed Service Oriented- Architecture for Bioinformatics Pipeline

Journal of Integrative Bioinformatics ◽

10.1515/jib-2015-255 ◽

2015 ◽

Vol 12 (1) ◽

pp. 1-15 ◽

Cited By ~ 1

Author(s):

Luis F. Castillo ◽

Germán López-Gartner ◽

Gustavo A. Isaza ◽

Mariana Sánchez ◽

Jeferson Arango ◽

...

Keyword(s):

High Performance ◽

Service Oriented Architecture ◽

Command Line ◽

Bioinformatics Pipeline ◽

Front End ◽

First Approximation ◽

Service Oriented ◽

Computational Resources ◽

Multiple Sequences ◽

Performance Computing

Summary The need to process large quantities of data generated from genomic sequencing has resulted in a difficult task for life scientists who are not familiar with the use of command-line operations or developments in high performance computing and parallelization. This knowledge gap, along with unfamiliarity with necessary processes, can hinder the execution of data processing tasks. Furthermore, many of the commonly used bioinformatics tools for the scientific community are presented as isolated, unrelated entities that do not provide an integrated, guided, and assisted interaction with the scheduling facilities of computational resources or distribution, processing and mapping with runtime analysis. This paper presents the first approximation of a Web Services platform-based architecture (GITIRBio) that acts as a distributed front-end system for autonomous and assisted processing of parallel bioinformatics pipelines that has been validated using multiple sequences. Additionally, this platform allows integration with semantic repositories of genes for search annotations. GITIRBio is available at: http://c-head.ucaldas.edu.co:8080/gitirbio

Download Full-text

Service for parallel applications based on JINR cloud and HybriLIT resources

EPJ Web of Conferences ◽

10.1051/epjconf/201921407012 ◽

2019 ◽

Vol 214 ◽

pp. 07012 ◽

Cited By ~ 1

Author(s):

Nikita Balashov ◽

Maxim Bashashin ◽

Pavel Goncharov ◽

Ruslan Kuchumov ◽

Nikolay Kutovskiy ◽

...

Keyword(s):

High Performance ◽

Cloud Service ◽

Parallel Applications ◽

Cloud Infrastructure ◽

Modular Architecture ◽

Practical Applications ◽

Speed Up ◽

Scientific Results ◽

Computational Resources ◽

Performance Computing

Cloud computing has become a routine tool for scientists in many fields. The JINR cloud infrastructure provides JINR users with computational resources to perform various scientific calculations. In order to speed up achievements of scientific results the JINR cloud service for parallel applications has been developed. It consists of several components and implements a flexible and modular architecture which allows to utilize both more applications and various types of resources as computational backends. An example of using the Cloud&HybriLIT resources in scientific computing is the study of superconducting processes in the stacked long Josephson junctions (LJJ). The LJJ systems have undergone intensive research because of the perspective of practical applications in nano-electronics and quantum computing. In this contribution we generalize the experience in application of the Cloud&HybriLIT resources for high performance computing of physical characteristics in the LJJ system.

Download Full-text