Construction of a multi-source heterogeneous hybrid platform for big data

2021 ◽  
pp. 1-10
Author(s):  
Ying Wang ◽  
Yiding Liu ◽  
Minna Xia
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Forest Fire ◽  
Original Data ◽  
Big Data Analysis ◽  
Multiple Sources ◽  
Data Types ◽  
Fire Monitoring ◽  
Data Platform

Big data is featured by multiple sources and heterogeneity. Based on the big data platform of Hadoop and spark, a hybrid analysis on forest fire is built in this study. This platform combines the big data analysis and processing technology, and learns from the research results of different technical fields, such as forest fire monitoring. In this system, HDFS of Hadoop is used to store all kinds of data, spark module is used to provide various big data analysis methods, and visualization tools are used to realize the visualization of analysis results, such as Echarts, ArcGIS and unity3d. Finally, an experiment for forest fire point detection is designed so as to corroborate the feasibility and effectiveness, and provide some meaningful guidance for the follow-up research and the establishment of forest fire monitoring and visualized early warning big data platform. However, there are two shortcomings in this experiment: more data types should be selected. At the same time, if the original data can be converted to XML format, the compatibility is better. It is expected that the above problems can be solved in the follow-up research.

scholarly journals Dislocations of the acromioclavicular and sternoclavicular joint in children and adolescents: A retrospective clinical study and big data analysis of routine data

PLoS ONE ◽  
2020 ◽  
Vol 15 (12) ◽  
pp. e0244209
Author(s):  
Ralf Kraus ◽  
Joern Zwingmann ◽  
Manfred Jablonski ◽  
M. Sinan Bakir
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Pediatric Trauma ◽  
Routine Data ◽  
Big Data Analysis ◽  
Sternoclavicular Joint ◽  
Specific Method ◽  
Retrospective Clinical Study ◽  
Dislocation Type

Background Dislocations of the sternoclavicular joint (anterior/posterior) and acromioclavicular joint (SCJ and ACJ, respectively) are rare injuries in childhood/adolescence, each having its own special characteristics. In posterior SCJ dislocation, the concomitant injuries in the upper mediastinum are most important complication, while in anterior SCJ dislocation there is a risk of permanent or recurrent instability. Methods In a retrospective analysis from seven pediatric trauma centers under the leadership of the Section of Pediatric Traumatology of the German Trauma Society, children (<18 years) were analyzed with focus on age, gender, trauma mechanism, diagnostics, treatment strategy and follow-up results. Additional epidemiological big data analysis from routine data was done. Results In total 24 cases with an average age of 14.4 years (23 boys, 1 girl) could be evaluated (7x ACJ dislocation type ≥ Rockwood III; 17x SCJ dislocation type Allman III, including 12 posterior). All ACJ dislocations were treated surgically. Postoperative immobilization lasted 3–6 weeks, after which a movement limit of 90 degrees was recommended until implant removal. Patients with SCJ dislocation were posterior dislocations in 75%, and 15 of 17 were treated surgically. One patient had a tendency toward sub-dislocation and another had a relapse. Conservatively treated injuries healed without complications. Compared to adults, SCJ injuries were equally rarely found in children (< 1% of clavicle-associated injuries), while pediatric ACJ dislocations were significantly less frequent (p<0.001). Conclusions In cases of SCJ dislocations, our cohort analysis confirmed both the heterogeneous spectrum of the treatment strategies in addition to the problems/complications based on previous literature. The indication for the operative or conservative approach and for the specific method is not standardized. In order to be able to create evidence-based standards, a prospective, multicenter-study with a sufficiently long follow-up time would be necessary due to the rarity of these injuries in children. The rarity was emphasized by our routine data analysis.

Big Data Analysis of Massive PMU Datasets: A Data Platform Perspective

2021 ◽  
Author(s):  
Vijay S. Kumar ◽  
Tianyi Wang ◽  
Kareem S. Aggour ◽  
Pengyuan Wang ◽  
Philip J. Hart ◽  
...  
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Big Data Analysis ◽  
Data Platform

scholarly journals THE MODELING OF "MUSTAHIQ" DATA USING K-MEANS CLUSTERING ALGORITHM AND BIG DATA ANALYSIS (CASE STUDY: LAZ)

2021 ◽  
Vol 13 (2) ◽  
pp. 213-230
Author(s):  
Nurhayati Buslim ◽  
Rayi Pradono Iswara ◽  
Fajar Agustian
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Clustering Algorithm ◽  
System Modeling ◽  
Big Data Analysis ◽  
Graphical Form ◽  
Data Types ◽  
Online Data ◽  
Combining Data ◽  
Data Survey

There are a lot of Mustahiq data in LAZ (Lembaga Amil Zakat) which is spread in many locations today. Each LAZ has Mustahiq data that is different in type from other LAZ. There are differences in Mustahiq data types so that data that is so large cannot be used together even though the purpose of the data is the same to determine Mustahiq data. And to find out whether the Mustahiq data is still up to date (renewable), of course it will be very difficult due to the types of data types that are not uniform or different, long time span, and the large amount of data. To give zakat to Mustahiq certainly requires speed of information. So, in giving zakat to Mustahiq, LAZ will find it difficult to monitor the progress of the Mustahiq. It is possible that a Mustahiq will change his condition to become a Muzaki. This is the reason for the researcher to take this theme in order to help the existing LAZ to make it easier to cluster Mustahiq data. Furthermore, the data already in the cluster can be used by LAZ managers to develop the organization. This can also be a reference for determining the zakat recipient cluster to those who are entitled later. The research is "Modeling using K-Means Algorithm and Big Data analysis in determine Mustahiq data ". We got data Mustahiq with random sample from online and offline survey. Online data survey with Google form and Offline Data survey we got from BAZNAS (National Amil Zakat Agency) in Indonesia and another zakat agency (LAZ) in Jakarta. We conducted by combining data to analyzed using Big Data and K-Means Algorithm. K-Means algorithm is an algorithm for cluster n objects based on attributes into k partitions according to criteria that will be determined from large and diverse Mustahiq data. This research focuses on modeling that applies K-Means Algorithms and Big Data Analysis. The first we made tools for grouping simulation test data. We do several experimental and simulation scenarios to find a model in mapping Mustahiq data to developed best model for processing the data. The results of this study are displayed in tabular and graphical form, namely the proposed Mustahiq data processing model at Zakat Agency (LAZ). The simulation result from a total of 1109 correspondents, 300 correspondents are included in the Mustahiq cluster and 809 correspondents are included in the Non-Mustahiq cluster and have an accuracy rate of 83.40%. That means accuracy of the system modeling able to determine data Mustahiq. Result filtering based on Gender is “Male” accuracy 83.93%, based on Age is ”30-39” accuracy 71,03%, based on Job is “PNS” accuracy 83.39%, based on Education is “S1” accuracy 83.79%. The advantaged of research expected to be able to determine quickly whether the person meets the criteria as a mustahik or Muzaki for LAZ (Amil Zakat Agency). The result of modeling is K-Means clustering algorithm application program can be used if UIN Syarif Hidayatullah Jakarta want to develop LAZ (Amil Zakat Agency) too.

scholarly journals Smart Home based Big Data Analysis for Healthcare Applications

2020 ◽  
Vol 9 (1) ◽  
pp. 275-282
Keyword(s):  
Health Care ◽  
Big Data ◽  
Data Analysis ◽  
Smart Home ◽  
Health Care Services ◽  
Activity Patterns ◽  
Big Data Analysis ◽  
City Centre ◽  
Data Types ◽  
Healthcare Applications

At present, there is a constant migration of people is encountered in urban regions. Health care services are considered as a confronting challenging factors, there is an extremely influenced by huge arrival of people to city centre. Subsequently, places all around the world are spending in digital evolution in an attempt to offer healthy eco-system for huge people. With this transformation, enormous homes are equipped with smarter devices (for example, sensors, smart sensors and so on) which produce huge amount of indexical data and fine-grained that is examined to assist smart city services. In this work, a model has been anticipated to utilize smart home big data analysis as a discovering and learning human activity patterns for huge health care applications. This work describes and highlights the experimentation with the analysis of vigorous data analysis process that assists healthcare analytics. This procedure comprises of subsequent stages: understanding, collection, cleaning, validation, enrichment, integration and storage. It has been resourcefully utilized to processing of data types variety comprising clinical data from EHR.

scholarly journals POLYMORPHIC SBD PREPROCESSOR: A PREPROCESSING APPROACH FOR SOCIAL BIG DATA

2020 ◽  
Vol 11 (6) ◽  
pp. 953-961
Author(s):  
Amit K. Jadiya ◽  
Archana Chaudhary ◽  
Ramesh Thakur
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Missing Values ◽  
Data Preprocessing ◽  
Big Data Analysis ◽  
Analysis Approach ◽  
Product Reviews ◽  
Multiple Sources ◽  
Data Set ◽  
Social Big Data

In recent years, the social media has become a powerful tool for sharing people thoughts and feelings. As a result data is being generated, analyzed and used with a tremendous growth rate. The data generated by numerous updates, comments, news, opinions and product reviews in social websites is very useful for getting insights. As there are multiple sources, the size, speed and formats of the gathered data affects the overall quality of information. To achieve quality information, preprocessing step is very important and decides future roadmap for efficient big data analysis approach. In context to social big data we are addressing the preprocessing phase which includes cleaning of data, identifying noise, data normalization, data transformation, handling missing values and data integration. In this paper we have proposed a new approach polymorphic SBD (Social Big Data) preprocessor which provides efficient results with multiple social big data sets. Also available data preprocessing methods for big data are presented in this paper. After efficient and successful data preprocessing steps, the output data set will be efficient, well formed and suitable source for any big data analysis approach to be applied afterwards. The paper also presents an example case and evaluates min-max normalization, z-score normalization and data mapping for the case presented.

scholarly journals Overview of Construction and Application of Big Data Analysis Platform

2021 ◽  
Vol 4 (6) ◽  
Author(s):  
Difei Zhang
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Resource Sharing ◽  
Data Exchange ◽  
Big Data Analysis ◽  
Time Data ◽  
Data Platform ◽  
Complete Data Analysis ◽  
Analysis Platform ◽  
Big Data Technology

Big data technology is widely spread around the world, and is constantly developing and applying. In order to enhance the application value of big data analysis platform, it is necessary to constantly improve the data analysis and processing capacity of big data platform, so as to build a complete data analysis platform, realize resource sharing and real-time data collection. As a key point of contemporary information development, big data analysis platform is of great significance to promote social data exchange. Based on this, this paper focuses on two aspects: first, it describes the construction process and content of big data platform; second, it summarizes the relevant application and development of big data platform for reference.

Sign in / Sign up

Export Citation Format

Share Document