scholarly journals A Microservice-Based Big Data Analysis Platform for Online Educational Applications

2020 ◽  
Vol 2020 ◽  
pp. 1-13
Author(s):  
Kehua Miao ◽  
Jie Li ◽  
Wenxing Hong ◽  
Mingtao Chen
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Data Science ◽  
Modular Design ◽  
Science Research ◽  
Big Data Analysis ◽  
Research Field ◽  
Traditional Work ◽  
Educational Applications ◽  
Analysis Platform

The booming development of data science and big data technology stacks has inspired continuous iterative updates of data science research or working methods. At present, the granularity of the labor division between data science and big data is more refined. Traditional work methods, from work infrastructure environment construction to data modelling and analysis of working methods, will greatly delay work and research efficiency. In this paper, we focus on the purpose of the current friendly collaboration of the data science team to build data science and big data analysis application platform based on microservices architecture for education or nonprofessional research field. In the environment based on microservices that facilitates updating the components of each component, the platform has a personal code experiment environment that integrates JupyterHub based on Spark and HDFS for multiuser use and a visualized modelling tools which follow the modular design of data science engineering based on Greenplum in-database analysis. The entire web service system is developed based on spring boot.

Design and Implementation of Financial Big Data Visualization Analysis Platform

2019 ◽  
Vol 3 (1) ◽  
Author(s):  
Xi Chen ◽  
Bo Fan ◽  
Jie Zheng ◽  
Hongyan Cui
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Data Visualization ◽  
Production Efficiency ◽  
Big Data Analysis ◽  
Research Field ◽  
Visualization Analysis ◽  
Front End ◽  
Big Data Visualization ◽  
Analysis Platform

At present, it has become a hot research field to improve production efficiency and improve life experience through big data analysis. In the process of big data analysis, how to vividly display the results of the analysis is crucial. So, this paper introduces a set of big data visualization analysis platform based on financial field. The platform adopts the MVC system architecture, which is mainly composed of two parts: the background and the front end. The background part is built on the Django framework, and the front end is built with html5, css3, and JavaScript. The chart is rendered by Echarts. The platform can realize the classification of customers' savings potential through bank data, and make portraits of customers with different savings levels. The data analysis results can be dynamically displayed and interact wit

Data Science and Big Data Analytics in Financial Services

Web Services ◽  
2019 ◽  
pp. 1301-1329
Author(s):  
Suren Behari ◽  
Aileen Cater-Steel ◽  
Jeffrey Soar
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Financial Services ◽  
Data Science ◽  
Big Data Analytics ◽  
Disruptive Innovation ◽  
Big Data Analysis ◽  
Future Trends ◽  
The Right

The chapter discusses how Financial Services organizations can take advantage of Big Data analysis for disruptive innovation through examination of a case study in the financial services industry. Popular tools for Big Data Analysis are discussed and the challenges of big data are explored as well as how these challenges can be met. The work of Hayes-Roth in Valued Information at the Right Time (VIRT) and how it applies to the case study is examined. Boyd's model of Observe, Orient, Decide, and Act (OODA) is explained in relation to disruptive innovation in financial services. Future trends in big data analysis in the financial services domain are explored.

scholarly journals Hemi- and Homonyms in the Big Data Era

Diversity ◽  
2020 ◽  
Vol 12 (12) ◽  
pp. 472
Author(s):  
Jorge Rubén Sánchez-González
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Biological Monitoring ◽  
Data Science ◽  
General Rule ◽  
Big Data Analysis ◽  
Species List ◽  
Statistical Software ◽  
Independence Principle ◽  
Near Future

The issue of hemi- and homonyms is an unsolved topic in the Big Data era, where informatics and technicians, rather than biologists or taxonomists, analyze huge datasets. Nowadays, taxonomic nomenclature is ruled by four independent international codes, and according to them, the existence of hemihomonyms and homonyms is accepted under some conditions as an exception to the general rule. This situation entails confusion, disagreements, and a plethora of problems whose consequences could worsen in the near future within the framework of the big data era. Moreover, the increasing use of big databases and analyses, data science, bioinformatics, biological monitoring, and bioassessment has shown such exceptions to be inconvenient, since these exceptions to homonyms are considered as duplicates by databases and statistical software, which are handled by non-taxonomist experts. International Codes of Nomenclature must change within the new context of big data analysis. This work aims to propose the elimination of any exception to the presence of homonyms and to evaluate whether the Independence Principle makes sense within this new context. Increasing coordination between several independent nomenclatural systems is essential and, perhaps, we must conduct our efforts towards a universal species list, finishing with the historical schism between Codes.

scholarly journals BDAP: A Big Data Analysis Platform Based On Spark

2018 ◽  
Vol 1060 ◽  
pp. 012023
Author(s):  
Zhixiang Wang ◽  
Yao Bu ◽  
Demeng Bai ◽  
Bin Wu ◽  
Jiafeng Qin
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Big Data Analysis ◽  
Analysis Platform

Research of Big Data Processing Platform

2014 ◽  
Vol 484-485 ◽  
pp. 922-926
Author(s):  
Xiang Ju Liu
Keyword(s):  
Cloud Computing ◽  
Big Data ◽  
Data Analysis ◽  
Network Architecture ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Big Data Analysis ◽  
Computing Platform ◽  
Operational Characteristics ◽  
Analysis Platform

This paper introduces the operational characteristics of the era of big data and the current era of big data challenges, and exhaustive research and design of big data analytics platform based on cloud computing, including big data analytics platform architecture system, big data analytics platform software architecture , big data analytics platform network architecture big data analysis platform unified program features and so on. The paper also analyzes the cloud computing platform for big data analysis program unified competitive advantage and development of business telecom operators play a certain role in the future.

Establishing the Advanced Disaster Reduction Management System by Fusion of Real-Time Disaster Simulation and Big Data Assimilation

2016 ◽  
Vol 11 (2) ◽  
pp. 164-174 ◽  
Author(s):  
Shunichi Koshimura ◽  
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Data Assimilation ◽  
Real Time ◽  
Big Data Analysis ◽  
Disaster Mitigation ◽  
Observation Data ◽  
Real Time Simulation ◽  
Time Simulation ◽  
Analysis Platform

A project titled “Establishing the advanced disaster reduction management system by fusion of real-time disaster simulation and big data assimilation,” was launched as Core Research for Evolutional Science and Technology (CREST) by the Japan Science and Technology Agency (JST). Intended to save as many lives as possible in future national crises involving earthquake and tsunami disasters, the project works on a disaster mitigation system of the big data era, based on cooperation of large-scale, high-resolution, real-time numerical simulations and assimilation of real-time observation data. The world’s most advanced specialists in disaster simulation, disaster management, mathematical science, and information science work together to create the world’s first analysis platform for real-time simulation and big data that effectively processes, analyzes, and assimilates data obtained through various observations. Based on quantitative data, the platform designs proactive measures and supports disaster operations immediately after disaster occurrence. The project was launched in 2014 and is working on the following issues at present.Sophistication and fusion of simulations and damage prediction models using observational big data: Development of a real-time simulation core system that predicts the time evolution of disaster effect by assimilating of location information, fire information, and building collapse information which are obtained from mobile terminals, satellite images, aerial images, and other new observation data in addition to sensing data obtained by the undersea high-density seismic observation network.Latent structure analysis and major disaster scenario creation based on a huge amount of simulation results: Development of an analysis and extraction method for the latent structure of a huge amount of disaster scenarios generated by simulation, and creation of severe scenarios with minimum “unexpectedness” by controlling disaster scenario explosion (an explosive increase in the number of predicted scenarios).Establishment of an earthquake and tsunami disaster mitigation big data analysis platform: Development of an earthquake and tsunami disaster mitigation big data analysis platform that realizes analyses of a huge number of disaster scenarios and increases in speed of data assimilation, and clarifies the requirements for operation of the platform as a disaster mitigation system.The project was launched in 2014 as a 5-year project. It consists of element technology development and system fusion, feasibility study as a next-generation disaster mitigation system (validation with/without introduction of the developed real-time simulation and big data analysis platform) in the affected areas of the Great East Japan Earthquake, and test operations in affected areas of the Tokyo metropolitan earthquake and the Nankai Trough earthquake.

Sign in / Sign up

Export Citation Format

Share Document