Crop Type Mapping from Optical and Radar Time Series Using Attention-Based Deep Learning

Crop maps are key inputs for crop inventory production and yield estimation and can inform the implementation of effective farm management practices. Producing these maps at detailed scales requires exhaustive field surveys that can be laborious, time-consuming, and expensive to replicate. With a growing archive of remote sensing data, there are enormous opportunities to exploit dense satellite image time series (SITS), temporal sequences of images over the same area. Generally, crop type mapping relies on single-sensor inputs and is solved with the help of traditional learning algorithms such as random forests or support vector machines. Nowadays, deep learning techniques have brought significant improvements by leveraging information in both spatial and temporal dimensions, which are relevant in crop studies. The concurrent availability of Sentinel-1 (synthetic aperture radar) and Sentinel-2 (optical) data offers a great opportunity to utilize them jointly; however, optimizing their synergy has been understudied with deep learning techniques. In this work, we analyze and compare three fusion strategies (input, layer, and decision levels) to identify the best strategy that optimizes optical-radar classification performance. They are applied to a recent architecture, notably, the pixel-set encoder–temporal attention encoder (PSE-TAE) developed specifically for object-based classification of SITS and based on self-attention mechanisms. Experiments are carried out in Brittany, in the northwest of France, with Sentinel-1 and Sentinel-2 time series. Input and layer-level fusion competitively achieved the best overall F-score surpassing decision-level fusion by 2%. On a per-class basis, decision-level fusion increased the accuracy of dominant classes, whereas layer-level fusion improves up to 13% for minority classes. Against single-sensor baseline, multi-sensor fusion strategies identified crop types more accurately: for example, input-level outperformed Sentinel-2 and Sentinel-1 by 3% and 9% in F-score, respectively. We have also conducted experiments that showed the importance of fusion for early time series classification and under high cloud cover condition.

Human-Machine Cooperative Decision Making Outperforms Individualism and Autonomy

<div>The experiment reported in this paper provides a first experimental evaluation of human-machine cooperation on decision level: It explicitly focuses on the interaction of human and machine in cooperative decision making situations for which a suitable experimental design is introduced. Furthermore, it challenges conventional leader-follower approaches by comparing them to newly proposed automation designs based on cooperative decision making models. These models originate from negotiation theory and game theory and allow for an investigation of cooperative decision making between equal partners. This equality is motivated by similar approaches on the action level of human-machine cooperation. <br></div><div>The experiment’s results indicate an added value of the proposed automation designs in terms of objective cooperative performance as well as human trust in and satisfaction with the cooperation. Hence, the experiment yields the same insight on decision level as already observed on action level: it may be beneficial to design machines as equal cooperation partners and in accordance to models of emancipated human-machine cooperation.</div>

Human-Machine Cooperative Decision Making Outperforms Individualism and Autonomy

<div>The experiment reported in this paper provides a first experimental evaluation of human-machine cooperation on decision level: It explicitly focuses on the interaction of human and machine in cooperative decision making situations for which a suitable experimental design is introduced. Furthermore, it challenges conventional leader-follower approaches by comparing them to newly proposed automation designs based on cooperative decision making models. These models originate from negotiation theory and game theory and allow for an investigation of cooperative decision making between equal partners. This equality is motivated by similar approaches on the action level of human-machine cooperation. <br></div><div>The experiment’s results indicate an added value of the proposed automation designs in terms of objective cooperative performance as well as human trust in and satisfaction with the cooperation. Hence, the experiment yields the same insight on decision level as already observed on action level: it may be beneficial to design machines as equal cooperation partners and in accordance to models of emancipated human-machine cooperation.</div>

Decision tree-based classification in coastal area integrating polarimetric SAR and optical data

PurposeAs one of the world's most productive ecosystems, ecological land plays an important role in regional and global environments. Utilizing advanced optical and synthetic aperture radar (SAR) data for land cover/land use research becomes increasingly popular. This research aims to investigate the complementarity of fully polarimetric SAR and optical imaging for ecological land classification in the eastern coastal area of China.Design/methodology/approachFour polarimetric decomposition methods, namely, H/Alpha, Yamaguchi3, VanZyl3 and Krogager, were applied to Advanced Land Observing Satellite (ALOS) SAR image for scattering parameter extraction. These parameters were merged with ALOS optical parameters for subsequent classification using the object-based quick, unbiased, efficient statistical tree decision tree method.FindingsThe experimental results indicate that an improved classification performance was obtained in the decision level when merging the two data sources. In fact, unlike classification using only optical images, the proposed approach allowed to distinguish ecological land with similar spectrum but different scattering. Moreover, unlike classification using only polarimetric information, the integration of polarimetric and optical data allows to accurately distinguish reed from artemisia and sand from salt field and therefore achieve a detailed classification of the coastal area characteristics.Originality/valueThis research proposed an integrated classification method for coastal ecological land with polarimetric SAR and optical data. The object-based and decision-level fusion enables effective ecological land classification in coastal area was verified.