HOPS: high-performance library for (non-)uniform sampling of convex-constrained models

2020 ◽  
Author(s):  
Johann F Jadebeck ◽  
Axel Theorell ◽  
Samuel Leweke ◽  
Katharina Nöh
Keyword(s):  
High Performance ◽  
State Of The Art ◽  
Source Code ◽  
Third Party ◽  
Supplementary Information ◽  
Scalable Algorithms ◽  
Uniform Sampling ◽  
Non Uniform Sampling ◽  
Constrained Models ◽  
Performance Gains

Abstract Summary The C++ library Highly Optimized Polytope Sampling (HOPS) provides implementations of efficient and scalable algorithms for sampling convex-constrained models that are equipped with arbitrary target functions. For uniform sampling, substantial performance gains were achieved compared to the state-of-the-art. The ease of integration and utility of non-uniform sampling is showcased in a Bayesian inference setting, demonstrating how HOPS interoperates with third-party software. Availability and implementation Source code is available at https://github.com/modsim/hops/, tested on Linux and MS Windows, includes unit tests, detailed documentation, example applications and a Dockerfile. Contact [email protected] Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Temporal network alignment via GoT-WAVE

2019 ◽  
Vol 35 (18) ◽  
pp. 3527-3529 ◽  
Author(s):  
David Aparício ◽  
Pedro Ribeiro ◽  
Tijana Milenković ◽  
Fernando Silva
Keyword(s):  
User Interface ◽  
State Of The Art ◽  
Source Code ◽  
Network Alignment ◽  
Supplementary Information ◽  
Temporal Network ◽  
Temporal Networks ◽  
Supplementary Data ◽  
Node Similarity ◽  
User Friendly

Abstract Motivation Network alignment (NA) finds conserved regions between two networks. NA methods optimize node conservation (NC) and edge conservation. Dynamic graphlet degree vectors are a state-of-the-art dynamic NC measure, used within the fastest and most accurate NA method for temporal networks: DynaWAVE. Here, we use graphlet-orbit transitions (GoTs), a different graphlet-based measure of temporal node similarity, as a new dynamic NC measure within DynaWAVE, resulting in GoT-WAVE. Results On synthetic networks, GoT-WAVE improves DynaWAVE’s accuracy by 30% and speed by 64%. On real networks, when optimizing only dynamic NC, the methods are complementary. Furthermore, only GoT-WAVE supports directed edges. Hence, GoT-WAVE is a promising new temporal NA algorithm, which efficiently optimizes dynamic NC. We provide a user-friendly user interface and source code for GoT-WAVE. Availability and implementation http://www.dcc.fc.up.pt/got-wave/ Supplementary information Supplementary data are available at Bioinformatics online.

SOAPMetaS: profiling large metagenome datasets efficiently on distributed clusters

2020 ◽  
Author(s):  
Shixu He ◽  
Zhibo Huang ◽  
Xiaohan Wang ◽  
Lin Fang ◽  
Shengkang Li ◽  
...  
Keyword(s):  
Big Data ◽  
Large Volume ◽  
Machine Tools ◽  
High Performance ◽  
Marker Gene ◽  
Source Code ◽  
Large Datasets ◽  
Supplementary Information ◽  
Supplementary Data ◽  
Multiple Sample

Abstract Summary Rapid increase of the data size in metagenome researches has raised the demand for new tools to process large datasets efficiently. To accelerate the metagenome profiling process in the scenario of big data, we developed SOAPMetaS, a marker gene-based multiple-sample metagenome profiling tool built on Apache Spark. SOAPMetaS demonstrates high performance and scalability to process large datasets. It can process 80 samples of FASTQ data, summing up to 416 GiB, in around half an hour; and the accuracy of species profiling results of SOAPMetaS is similar to that of MetaPhlAn2. SOAPMetaS can deal with a large volume of metagenome data more efficiently than common-used single-machine tools. Availability and implementation Source code is implemented in Java and freely available at https://github.com/BGI-flexlab/SOAPMetaS. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals PATO: Pangenome Analysis Toolkit

2021 ◽  
Author(s):  
Miguel D. Fernández-de-Bobadilla ◽  
Alba Talavera-Rodríguez ◽  
Lucía Chacón ◽  
Fernando Baquero ◽  
Teresa M. Coque ◽  
...  
Keyword(s):  
Population Structure ◽  
Statistical Analysis ◽  
Core Genome ◽  
State Of The Art ◽  
Source Code ◽  
Supplementary Information ◽  
Complete Analysis ◽  
Large Set ◽  
Supplementary Data ◽  
Desktop Computer

AbstractMotivationComparative genomics is a growing field but one that will be eventually overtaken by sample size studies and the increase of available genomes in public databases. We present the Pangenome Analysis Toolkit (PATO) designed to simultaneously analyze thousands of genomes using a desktop computer. The tool performs common tasks of pangenome analysis such as core-genome definition and accessory genome properties and includes new features that help characterize population structure, annotate pathogenic features and create gene sharedness networks. PATO has been developed in R to integrate with the large set of tools available for genetic, phylogenetic and statistical analysis in this environment.ResultsPATO can perform the most demanding bioinformatic analyses in minutes with an accuracy comparable to state-of-the-art software but 20–30x times faster. PATO also integrates all the necessary functions for the complete analysis of the most common objectives in microbiology studies. Lastly, PATO includes the necessary tools for visualizing the results and can be integrated with other analytical packages available in R.AvailabilityThe source code for PATO is freely available at https://github.com/irycisBioinfo/PATO under the GPLv3 [email protected] informationSupplementary data are available at Bioinformatics online

scholarly journals High-performance Python for crystallographic computing

2019 ◽  
Vol 52 (4) ◽  
pp. 882-897 ◽  
Author(s):  
A. Boulle ◽  
J. Kieffer
Keyword(s):  
Programming Language ◽  
High Performance ◽  
Scientific Computing ◽  
State Of The Art ◽  
Third Party ◽  
Improve Performance ◽  
Python Language ◽  
Python Programming Language ◽  
Numerical Computing ◽  
Python Programming

The Python programming language, combined with the numerical computing library NumPy and the scientific computing library SciPy, has become the de facto standard for scientific computing in a variety of fields. This popularity is mainly due to the ease with which a Python program can be written and executed (easy syntax, dynamical typing, no compilation etc.), coupled with the existence of a large number of specialized third-party libraries that aim to lift all the limitations of the raw Python language. NumPy introduces vector programming, improving execution speeds, whereas SciPy brings a wealth of highly optimized and reliable scientific functions. There are cases, however, where vector programming alone is not sufficient to reach optimal performance. This issue is addressed with dedicated compilers that aim to translate Python code into native and statically typed code with support for the multi-core architectures of modern processors. In the present article it is shown how these approaches can be efficiently used to tackle different problems, with increasing complexity, that are relevant to crystallography: the 2D Laue function, scattering from a strained 2D crystal, scattering from 3D nanocrystals and, finally, diffraction from films and multilayers. For each case, detailed implementations and explanations of the functioning of the algorithms are provided. Different Python compilers (namely NumExpr, Numba, Pythran and Cython) are used to improve performance and are benchmarked against state-of-the-art NumPy implementations. All examples are also provided as commented and didactic Python (Jupyter) notebooks that can be used as starting points for crystallographers curious to enter the Python ecosystem or wishing to accelerate their existing codes.

Search for Compatible Source Code

2021 ◽  
Vol 31 (03) ◽  
pp. 477-502
Author(s):  
Fuqi Cai ◽  
Changjing Wang ◽  
Qing Huang ◽  
Zhengkang Zuo ◽  
Yunyan Liao
Keyword(s):  
Programming Language ◽  
State Of The Art ◽  
Source Code ◽  
The State ◽  
Third Party ◽  
Search Model ◽  
Code Search ◽  
Art Methods ◽  
Local Programming ◽  
Cosine Distance

Third-party libraries always evolve and produce multiple versions. Lucene, for example, released ten new versions (from version 7.7.0 to 8.4.0) in 2019. These versions confuse the existing code search methods to retrieve the source code that is not compatible with local programming language. To solve this issue, we propose DCSE, a deep code search model based on evolving information (i.e. evolved code tokens and evolution description). DCSE first deeply excavates evolved code tokens and evolution description in the code evolution process; then it takes evolved code tokens and evolution description as one feature of source code and code description, respectively. With such fuller representation, DCSE embeds source code and its code description into a high-dimensional shared vector space, and makes the cosine distance of their vectors closer. For the ever-evolving third-party libraries like Lucene, the experimental results show that DCSE could retrieve the source code that is compatible with local programming language, it outperforms the state-of-the-art methods (e.g. CODEnn) by 56.9–60.9[Formula: see text] in RFVersion. For the rarely-evolving third-party libraries, DCSE outperforms the state-of-the-art methods (e.g. CODEnn) by 4–11[Formula: see text] in Precision.

scholarly journals Low-Cost and Programmable CRC Implementation based on FPGA (Extended Version)

2020 ◽  
Author(s):  
Huan Liu ◽  
Zhiliang Qiu ◽  
Weitao Pan ◽  
Jun Li ◽  
Ling Zheng ◽  
...  
Keyword(s):  
Resource Utilization ◽  
Error Detection ◽  
High Performance ◽  
State Of The Art ◽  
Low Cost ◽  
Source Code ◽  
Experimental Results ◽  
Extended Version ◽  
Cyclic Redundancy Check

Cyclic redundancy check (CRC) is a well-known error detection code that is widely used in Ethernet, PCIe, and other transmission protocols. The existing FPGA-based implementation solutions are faced with the problem of excessive resource utilization in high-performance scenarios. The padding zeros problem and the introduction of programmability further exacerbate this problem. In this brief, the stride-by-5 algorithm is proposed to achieve the optimal utilization of FPGA resources. The pipelining go back algorithm is proposed to solve the padding zeros problem. The method of reprogramming by HWICAP is proposed to realize programmability with a small and constant resource utilization. The experimental results show that the resource utilization of proposed non-segmented architecture is 80.7%-87.5% and 25.1%-46.2% lower than those of two state-of-the-art FPGA-based CRC implementations, and the proposed segmented architecture has a lower resource utilization by 81.7%-85.9% and 2.9%-20.8% compared wtih the two state-of-the-art architectures; meanwhile, the throughput and programmability are guaranteed. We made the source code available on GitHub.

scholarly journals Low-Cost and Programmable CRC Implementation based on FPGA (Extended Version)

2020 ◽  
Author(s):  
Huan Liu ◽  
Zhiliang Qiu ◽  
Weitao Pan ◽  
Jun Li ◽  
Ling Zheng ◽  
...  
Keyword(s):  
Resource Utilization ◽  
Error Detection ◽  
High Performance ◽  
State Of The Art ◽  
Low Cost ◽  
Source Code ◽  
Experimental Results ◽  
Extended Version ◽  
Cyclic Redundancy Check

Cyclic redundancy check (CRC) is a well-known error detection code that is widely used in Ethernet, PCIe, and other transmission protocols. The existing FPGA-based implementation solutions are faced with the problem of excessive resource utilization in high-performance scenarios. The padding zeros problem and the introduction of programmability further exacerbate this problem. In this brief, the stride-by-5 algorithm is proposed to achieve the optimal utilization of FPGA resources. The pipelining go back algorithm is proposed to solve the padding zeros problem. The method of reprogramming by HWICAP is proposed to realize programmability with a small and constant resource utilization. The experimental results show that the resource utilization of proposed non-segmented architecture is 84.1% and 37.6% lower than those of two state-of-the-art FPGA-based CRC implementations, and the proposed segmented architecture has a lower resource utilization by 83.9% and 8.9% compared wtih the two state-of-the-art architectures; meanwhile, the throughput and programmability are guaranteed. We made the source code available on GitHub.

scholarly journals MolRep: A Deep Representation Learning Library for Molecular Property Prediction

2021 ◽  
Author(s):  
Jiahua Rao ◽  
Shuangjia Zheng ◽  
Ying Song ◽  
Jianwen Chen ◽  
Chengtao Li ◽  
...  
Keyword(s):  
State Of The Art ◽  
Source Code ◽  
Representation Learning ◽  
Supplementary Information ◽  
Data Sets ◽  
Supplementary Data ◽  
Property Prediction ◽  
Average Rank ◽  
Benchmark Data ◽  
Classification Tasks

AbstractSummaryRecently, novel representation learning algorithms have shown potential for predicting molecular properties. However, unified frameworks have not yet emerged for fairly measuring algorithmic progress, and experimental procedures of different representation models often lack rigorousness and are hardly reproducible. Herein, we have developed MolRep by unifying 16 state-of-the-art models across 4 popular molecular representations for application and comparison. Furthermore, we ran more than 12.5 million experiments to optimize hyperparameters for each method on 12 common benchmark data sets. As a result, CMPNN achieves the best results ranked the 1st in 5 out of 12 tasks with an average rank of 1.75. Relatively, ECC has good performance in classification tasks and MAT good for regression (both ranked 1st for 3 tasks) with an average rank of 2.71 and 2.6, respectively.AvailabilityThe source code is available at: https://github.com/biomed-AI/MolRepSupplementary informationSupplementary data are available online.

scholarly journals Nubeam-dedup: a fast and RAM-efficient tool to de-duplicate sequencing reads without mapping

2020 ◽  
Vol 36 (10) ◽  
pp. 3254-3256 ◽  
Author(s):  
Hang Dai ◽  
Yongtao Guan
Keyword(s):  
Hash Function ◽  
Reference Genome ◽  
State Of The Art ◽  
Source Code ◽  
Supplementary Information ◽  
Supplementary Data ◽  
Efficient Tool ◽  
Cpu Time ◽  
Products Of Matrices

Abstract Summary We present Nubeam-dedup, a fast and RAM-efficient tool to de-duplicate sequencing reads without reference genome. Nubeam-dedup represents nucleotides by matrices, transforms reads into products of matrices, and based on which assigns a unique number to a read. Thus, duplicate reads can be efficiently removed by using a collisionless hash function. Compared with other state-of-the-art reference-free tools, Nubeam-dedup uses 50–70% of CPU time and 10–15% of RAM. Availability and implementation Source code in C++ and manual are available at https://github.com/daihang16/nubeamdedup and https://haplotype.org. Supplementary information Supplementary data are available at Bioinformatics online.

CoCoNet: an efficient deep learning tool for viral metagenome binning

2021 ◽  
Author(s):  
Cédric G Arisdakessian ◽  
Olivia D Nigro ◽  
Grieg F Steward ◽  
Guylaine Poisson ◽  
Mahdi Belcaid
Keyword(s):  
Deep Learning ◽  
Viral Genome ◽  
High Performance ◽  
Source Code ◽  
Supplementary Information ◽  
Biological Processes ◽  
Bacterial Genomes ◽  
Large Dataset ◽  
Sequence Composition ◽  
Rigorous Framework

Abstract Motivation Metagenomic approaches hold the potential to characterize microbial communities and unravel the intricate link between the microbiome and biological processes. Assembly is one of the most critical steps in metagenomics experiments. It consists of transforming overlapping DNA sequencing reads into sufficiently accurate representations of the community’s genomes. This process is computationally difficult and commonly results in genomes fragmented across many contigs. Computational binning methods are used to mitigate fragmentation by partitioning contigs based on their sequence composition, abundance or chromosome organization into bins representing the community’s genomes. Existing binning methods have been principally tuned for bacterial genomes and do not perform favorably on viral metagenomes. Results We propose Composition and Coverage Network (CoCoNet), a new binning method for viral metagenomes that leverages the flexibility and the effectiveness of deep learning to model the co-occurrence of contigs belonging to the same viral genome and provide a rigorous framework for binning viral contigs. Our results show that CoCoNet substantially outperforms existing binning methods on viral datasets. Availability and implementation CoCoNet was implemented in Python and is available for download on PyPi (https://pypi.org/). The source code is hosted on GitHub at https://github.com/Puumanamana/CoCoNet and the documentation is available at https://coconet.readthedocs.io/en/latest/index.html. CoCoNet does not require extensive resources to run. For example, binning 100k contigs took about 4 h on 10 Intel CPU Cores (2.4 GHz), with a memory peak at 27 GB (see Supplementary Fig. S9). To process a large dataset, CoCoNet may need to be run on a high RAM capacity server. Such servers are typically available in high-performance or cloud computing settings. Supplementary information Supplementary data are available at Bioinformatics online.

Sign in / Sign up

Export Citation Format

Share Document