scholarly journals Accelerated Stochastic Gradient for Nonnegative Tensor Completion and Parallel Implementation

2021 ◽  
Author(s):  
Ioanna Siaminou ◽  
Ioannis Marios Papagiannakos ◽  
Christos Kolomvakis ◽  
Athanasios P. Liavas
Keyword(s):  
Parallel Implementation ◽  
Stochastic Gradient ◽  
Tensor Completion ◽  
Nonnegative Tensor

Iterative algorithm for the symmetric and nonnegative tensor completion problem

2018 ◽  
Vol 67 (8) ◽  
pp. 1579-1595
Author(s):  
Xuefeng Duan ◽  
Jianheng Chen ◽  
Chunmei Li ◽  
Qingwen Wang
Keyword(s):  
Iterative Algorithm ◽  
Tensor Completion ◽  
Nonnegative Tensor ◽  
Completion Problem

scholarly journals Nesterov-Based Alternating Optimization for Nonnegative Tensor Factorization: Algorithm and Parallel Implementation

2018 ◽  
Vol 66 (4) ◽  
pp. 944-953 ◽  
Author(s):  
Athanasios P. Liavas ◽  
Georgios Kostoulas ◽  
Georgios Lourakis ◽  
Kejun Huang ◽  
Nicholas D. Sidiropoulos
Keyword(s):  
Parallel Implementation ◽  
Tensor Factorization ◽  
Nonnegative Tensor ◽  
Alternating Optimization ◽  
Nonnegative Tensor Factorization ◽  
Factorization Algorithm

scholarly journals Nonnegative Tensor Completion via Low-Rank Tucker Decomposition: Model and Algorithm

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 95903-95914 ◽  
Author(s):  
Bilian Chen ◽  
Ting Sun ◽  
Zhehao Zhou ◽  
Yifeng Zeng ◽  
Langcai Cao
Keyword(s):  
Low Rank ◽  
Tensor Completion ◽  
Nonnegative Tensor ◽  
Tucker Decomposition ◽  
Decomposition Model

scholarly journals Parallel Implementation on FPGA of Support Vector Machines Using Stochastic Gradient Descent

Electronics ◽  
2019 ◽  
Vol 8 (6) ◽  
pp. 631 ◽  
Author(s):  
Felipe F. Lopes ◽  
João Canas Ferreira ◽  
Marcelo A. C. Fernandes
Keyword(s):  
Support Vector Machines ◽  
Gradient Descent ◽  
Parallel Implementation ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Support Vector ◽  
Data Set ◽  
Viable Solution ◽  
Vector Machines ◽  
Field Programmable

Sequential Minimal Optimization (SMO) is the traditional training algorithm for Support Vector Machines (SVMs). However, SMO does not scale well with the size of the training set. For that reason, Stochastic Gradient Descent (SGD) algorithms, which have better scalability, are a better option for massive data mining applications. Furthermore, even with the use of SGD, training times can become extremely large depending on the data set. For this reason, accelerators such as Field-programmable Gate Arrays (FPGAs) are used. This work describes an implementation in hardware, using FPGA, of a fully parallel SVM using Stochastic Gradient Descent. The proposed FPGA implementation of an SVM with SGD presents speedups of more than 10,000× relative to software implementations running on a quad-core processor and up to 319× compared to state-of-the-art FPGA implementations while requiring fewer hardware resources. The results show that the proposed architecture is a viable solution for highly demanding problems such as those present in big data analysis.

scholarly journals Asynchronous Proximal Stochastic Gradient Algorithm for Composition Optimization Problems

2019 ◽  
Vol 33 ◽  
pp. 1633-1640 ◽  
Author(s):  
Pengfei Wang ◽  
Risheng Liu ◽  
Nenggan Zheng ◽  
Zhefeng Gong
Keyword(s):  
Convergence Rate ◽  
Optimization Problem ◽  
Variance Reduction ◽  
Optimization Problems ◽  
Parallel Implementation ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Composition Gradient ◽  
Composition Optimization ◽  
Asynchronous Parallel

In machine learning research, many emerging applications can be (re)formulated as the composition optimization problem with nonsmooth regularization penalty. To solve this problem, traditional stochastic gradient descent (SGD) algorithm and its variants either have low convergence rate or are computationally expensive. Recently, several stochastic composition gradient algorithms have been proposed, however, these methods are still inefficient and not scalable to large-scale composition optimization problem instances. To address these challenges, we propose an asynchronous parallel algorithm, named Async-ProxSCVR, which effectively combines asynchronous parallel implementation and variance reduction method. We prove that the algorithm admits the fastest convergence rate for both strongly convex and general nonconvex cases. Furthermore, we analyze the query complexity of the proposed algorithm and prove that linear speedup is accessible when we increase the number of processors. Finally, we evaluate our algorithm Async-ProxSCVR on two representative composition optimization problems including value function evaluation in reinforcement learning and sparse mean-variance optimization problem. Experimental results show that the algorithm achieves significant speedups and is much faster than existing compared methods.

Sign in / Sign up

Export Citation Format

Share Document