A Method for Gradient Differentiable Network Architecture Search by Selecting and Clustering Candidate Operations

The current evolution of deep learning requires further optimization in terms of accuracy and time. From the perspective of new requirements, AutoML is an area that could provide possible solutions. AutoML has a neural architecture search (NAS) field. DARTS is a widely used approach in NAS and is based on gradient descent; however, it has some drawbacks. In this study, we attempted to overcome some of the drawbacks of DARTS by improving the accuracy and decreasing the search cost. The DARTS algorithm uses a mixed operation that combines all operations in the search space. The architecture parameter of each operation comprising a mixed operation is trained using gradient descent, and the operation with the largest architecture parameter is selected. The use of a mixed operation causes a problem called vote dispersion: similar operations share architecture parameters during gradient descent; thus, there are cases where the most important operation is disregarded. In this selection process, vote dispersion causes DARTS performance to degrade. To cope with this problem, we propose a new algorithm based on DARTS called DG-DARTS. Two search stages are introduced, and the clustering of operations is applied in DG-DARTS. In summary, DG-DARTS achieves an error rate of 2.51% on the CIFAR10 dataset, and its search cost is 0.2 GPU days because the search space of the second stage is reduced by half. The speed-up factor of DG-DARTS to DARTS is 6.82, which indicates that the search cost of DG-DARTS is only 13% that of DARTS.

Not toeing the number line for simple arithmetic: Two large-n conceptual replications of Mathieu et al. (Cognition, 2016, Experiment 1)

We conducted two conceptual replications of Experiment 1 in Mathieu, Gourjon, Couderc, Thevenot, and Prado (2016, https://doi.org/10.1016/j.cognition.2015.10.002). They tested a sample of 34 French adults on mixed-operation blocks of single-digit addition (4 + 3) and subtraction (4 – 3) with the three problem elements (O1, +/-, O2) presented sequentially. Addition was 34 ms faster if O2 appeared 300 ms after the operation sign and displaced 5° to the right of central fixation, whereas subtraction was 19 ms faster when O2 was displaced to the left. Replication Experiment 1 (n = 74 recruited at the University of Saskatchewan) used the same non-zero addition and subtraction problems and trial event sequence as Mathieu et al., but participants completed blocks of pure addition and pure subtraction followed by the mixed-operation condition used by Mathieu et al. Addition RT showed a 32 ms advantage with O2 shifted rightward relative to leftward but only in mixed-operation blocks. There was no effect of O2 position on subtraction RT. Experiment 2 (n = 74) was the same except mixed-operation blocks occurred before the pure-operation blocks. There was an overall 13 ms advantage with O2 shifted right relative to leftward but no interaction with operation or with mixture (i.e., pure vs mixed operations). Nonetheless, the rightward RT advantage was statistically significant for both addition and subtraction only in mixed-operation blocks. Taken together with the robust effects of mixture in Experiment 1, the results suggest that O2 position effects in this paradigm might reflect task specific demands associated with mixed operations.