Iterative Collaborative Filtering for Sparse Matrix Estimation

Matrix estimation or completion has served as a canonical mathematical model for recommendation systems. More recently, it has emerged as a fundamental building block for data analysis as a first step to denoise the observations and predict missing values. Since the dawn of e-commerce, similarity-based collaborative filtering has been used as a heuristic for matrix etimation. At its core, it encodes typical human behavior: you ask your friends to recommend what you may like or dislike. Algorithmically, friends are similar “rows” or “columns” of the underlying matrix. The traditional heuristic for computing similarities between rows has costly requirements on the density of observed entries. In “Iterative Collaborative Filtering for Sparse Matrix Estimation” by Christian Borgs, Jennifer T. Chayes, Devavrat Shah, and Christina Lee Yu, the authors introduce an algorithm that computes similarities in sparse datasets by comparing expanded local neighborhoods in the associated data graph: in effect, you ask friends of your friends to recommend what you may like or dislike. This work provides bounds on the max entry-wise error of their estimate for low rank and approximately low rank matrices, which is stronger than the aggregate mean squared error bounds found in classical works. The algorithm is also interpretable, scalable, and amenable to distributed implementation.

Exploring the Direct and Indirect Use of ICT Measurements in DODME (Dynamic OD Matrix Estimation)

The estimation of the network traffic state, its likely short-term evolution, the prediction of the expected travel times in a network, and the role that mobility patterns play in transport modeling is usually based on dynamic traffic models, whose main input is a dynamic origin–destination (OD) matrix that describes the time dependencies of travel patterns; this is one of the reasons that have fostered large amounts of research on the topic of estimating OD matrices from the available traffic information. The complexity of the problem, its underdetermination, and the many alternatives that it offers are other reasons that make it an appealing research topic. The availability of new traffic data measurements that were prompted by the pervasive penetration of information and communications technology (ICT) applications offers new research opportunities. This study focused on GPS tracking data and explored two alternative modeling approaches regarding how to account for this new information to solve the dynamic origin–destination matrix estimation (DODME) problem, either including it as an additional term in the formulation model or using it in a data-driven modeling method to propose new model formulations. Complementarily, independently of the approach used, a key aspect is the quality of the estimated OD, which, as recent research has made evident, is not well measured by the conventional indicators. This study also explored this problem for the proposed approaches by conducting synthetic computational experiments to control and understand the process.

A General Framework of Remote Sensing Epipolar Image Generation

Epipolar images can improve the efficiency and accuracy of dense matching by restricting the search range of correspondences from 2-D to 1-D, which play an important role in 3-D reconstruction. As most of the satellite images in archives are incidental collections, which do not have rigorous stereo properties, in this paper, we propose a general framework to generate epipolar images for both in-track and cross-track stereo images. We first investigate the theoretical epipolar constraints of single-sensor and multi-sensor images and then introduce the proposed framework in detail. Considering large elevation changes in mountain areas, the publicly available digital elevation model (DEM) is applied to reduce the initial offsets of two stereo images. The left image is projected into the image coordinate system of the right image using the rational polynomial coefficients (RPCs). By dividing the raw images into several blocks, the epipolar images of each block are parallel generated through a robust feature matching method and fundamental matrix estimation, in which way, the horizontal disparity can be drastically reduced while maintaining negligible vertical disparity for epipolar blocks. Then, stereo matching using the epipolar blocks can be easily implemented and the forward intersection method is used to generate the digital surface model (DSM). Experimental results on several in-track and cross-track images, including optical-optical, SAR-SAR, and SAR-optical pairs, demonstrate the effectiveness of the proposed framework, which not only has obvious advantages in mountain areas with large elevation changes but also can generate high-quality epipolar images for flat areas. The generated epipolar images of a ZiYuan-3 pair in Songshan are further utilized to produce a high-precision DSM.