A Service for Parallel Processing of Protected Information Queries in a Multi-Server Environment

The principles of organizing an Internet service designed to implement parallel processing of protected queries in distributed information systems that are oriented to work in complex network environments with many processing servers are considered. Distinctive feature of the service consists in a possibility to combine sequential ("pipelined") and parallel query processing in a multi-server environment. The service architecture is based on the concept of a "Protected message" corresponding to a container for electronic documents (information query or response) that can be provided with one or more electronic digital signatures. In contrast, for example, to the Web services technology, the described service is based not on the model of calling methods of remote objects, but on the message exchange model. In this case, this means that all service processing functions (methods) have the same strict specification: they receive an array of objects of the "Protected message" program class as a parameter and return an array of objects of the same class. In sequential processing, both arrays usually contain one "Protected message" object each. However, when using parallel processing, arrays can contain multiple elements that correspond to the results of processing by multiple software threads. These processing functions are grouped into one or more dynamic libraries, which are connected to the PMS server when it starts working (each library can be considered as a remote analogue of a Web service), and become available to clients.

A technique for parallel query optimization using MapReduce framework and a semantic-based clustering method

Query optimization is the process of identifying the best Query Execution Plan (QEP). The query optimizer produces a close to optimal QEP for the given queries based on the minimum resource usage. The problem is that for a given query, there are plenty of different equivalent execution plans, each with a corresponding execution cost. To produce an effective query plan thus requires examining a large number of alternative plans. Access plan recommendation is an alternative technique to database query optimization, which reuses the previously-generated QEPs to execute new queries. In this technique, the query optimizer uses clustering methods to identify groups of similar queries. However, clustering such large datasets is challenging for traditional clustering algorithms due to huge processing time. Numerous cloud-based platforms have been introduced that offer low-cost solutions for the processing of distributed queries such as Hadoop, Hive, Pig, etc. This paper has applied and tested a model for clustering variant sizes of large query datasets parallelly using MapReduce. The results demonstrate the effectiveness of the parallel implementation of query workloads clustering to achieve good scalability.

Improving the performance of query processing using proposed resilient distributed processing technique

PurposeResilient distributed processing technique (RDPT), in which mapper and reducer are simplified with the Spark contexts and support distributed parallel query processing.Design/methodology/approachThe proposed work is implemented with Pig Latin with Spark contexts to develop query processing in a distributed environment.FindingsQuery processing in Hadoop influences the distributed processing with the MapReduce model. MapReduce caters to the works on different nodes with the implementation of complex mappers and reducers. Its results are valid for some extent size of the data.Originality/valuePig supports the required parallel processing framework with the following constructs during the processing of queries: FOREACH; FLATTEN; COGROUP.

DFTHR: A Distributed Framework for Trajectory Similarity Query Based on HBase and Redis

In recent years positioning sensors have become ubiquitous, and there has been tremendous growth in the amount of trajectory data. It is a huge challenge to efficiently store and query massive trajectory data. Among the typical operation over trajectories, similarity query is an important yet complicated operator. It is useful in navigation systems, transportation optimizations, and so on. However, most existing studies have focused on handling the problem on a centralized system, while with a single machine it is difficult to satisfy the storage and processing requirements of mass data. A distributed framework for the similarity query of massive trajectory data is urgently needed. In this research, we propose DFTHR (distributed framework based on HBase and Redis) to support the similarity query using Hausdorff distance. DFTHR utilizes a segment-based data model with a number of optimizations for storing, indexing and pruning to ensure efficient querying capability. Furthermore, it adopts a bulk-based method to alleviate the cost for adjusting partitions, so that the incremental dataset can be efficiently supported. Additionally, DFTHR introduces a co-location-based distributed strategy and a node-locality-based parallel query algorithm to reduce the inter-worker cost overhead. Experiments show that DFTHR significantly outperforms other schemes.