Recommendations for the Empirical Assessment of Human-AI Work Systems: A Contribution to AI Measurement Science

This Report is a companion to the Report titled "Requirements for the Evaluation of Human-AI Work Systems." Whereas that Report focused on the minimum necessary empirical requirements for the assessment of AI systems, this Report provides additional recommendations and technical details to assist the developers of AI systems. Recommendations are presented covering study design, research methods, measurement, statistical analyses, and online experimentation. This guidance should be applicable to all research intended to evaluate the effectivity of AI systems.

Requirements for the Empirical Assessment of Human-AI Work Systems: A Contribution to AI Measurement Science

The development of AI systems represents a significant investment. But to realize the promise of that investment, performance assessment is necessary. Empirical evaluation of Human-AI work systems must adduce convincing empirical evidence that the work method and its AI technology are learnable, usable, and useful. The theme to this Report is the notion that AI assessment must be effective but must also be efficient. Bench testing of a prototype of an AI system cannot require extensive series of experiments with complex designs. Thus, the empirical requirements that are presented in this Report involve escaping some of the constraints that are imposed in traditional laboratory research. Also, there is a recognition of new constraints that are unique to AI evaluation contexts. Empirical requirements are presented covering study design, research methods, statistical analyses, and online experimentation. The 15 requirements presented in this Report should be applicable to all research intended to evaluate the effectivity of AI systems.

Separation Theorems in Econometrics and Psychometrics: Rasch, Frisch, Two Fishers and Implications for Measurement

In 1959, Ragnar Frisch prompted Georg Rasch to formalise a separability theorem that continues today to serve as the basis of a wide range of theoretical and applied developments in psychological and social measurement. Previously unnoted are the influences on Rasch exerted by Frisch’s concerns for data autonomy, model identification and necessary and sufficient conditions. Although Rasch acknowledged Frisch’s prompting towards a separability theorem, he did not acknowledge any substantive, intellectual debt to him, nor to Irving Fisher, but only to Ronald Fisher. Rasch appears to have developed a special interest in sufficiency and identified models when studying with Frisch in 1935, and in 1947, when Rasch accompanied Tjalling Koopmans to the University of Chicago and the Cowles Commission for Research in Economics. I. Fisher’s separation theorem continues to be relevant in econometrics, and interest in Rasch’s separability theorem is growing as the measurement models based on it are adopted in metrological theory and practice. The extensive interrelations between measurement science, metrological standards and economics suggest paths towards lower transaction costs and more efficient markets for individualised exchanges of human, social and natural capital. Equally, if not more, surprising are the implications for a poetic art of complex, harmonised relationships played out via creative improvisations expressed using instruments tuned to shared scales. JEL: B41, C10, C13, C20, C42, D70, E60, H54, I11, I21, I31, P11

A phytopathometry glossary for the twenty-first century: towards consistency and precision in intra- and inter-disciplinary dialogues

Phytopathometry can be defined as the branch of plant pathology (phytopathology) that is concerned with estimation or measurement of the amount of plant disease expressed by symptoms of disease or signs of a pathogen on a single or group of specimens. Phytopathometry is critical for many reasons, including analyzing yield loss due to disease, breeding for disease resistance, evaluating and comparing disease control methods, understanding coevolution, and studying disease epidemiology and pathogen ecology. Phytopathometry underpins all activities in plant pathology and extends into related disciplines, such as agronomy, horticulture, and plant breeding. Considering this central role, phytopathometry warrants status as a formally recognized branch of plant pathology. The glossary defines terms and concepts used in phytopathometry based on disease symptoms or visible pathogen structures and includes those terms commonly used in the visual estimation of disease severity and sensor-based methods of disease measurement. Relevant terms from the intersecting disciplines of measurement science, statistics, psychophysics, robotics, and artificial intelligence are also included. In particular, a new, broader definition is proposed for “disease severity,” and the terms “disease measurement” and “disease estimate” are specifically defined. It is hoped that the glossary contributes to a more unified cross-discipline approach to research in, and application of the tools available to phytopathometry.

Application of Oscilloscope Technology in the Early 21st Century: A Systematic Literature Review

Background: An oscilloscope is a basic tool for measuring electrical quantities, as well as studying all types of waveforms. Generally, measurement of a quantity is regarding peak voltage, frequency, phase difference, pulse width, delay time, etc. However, what about the oscilloscope technology that existed in the early the 21st-century? Purpose: Aim of this study is to identify and analyze the application and use of current oscilloscopes in the early 21st-century. Method: The research methodology used is qualitative research with a systematic literature review by taking journals from 2019-2020 on the IEEE, IOP, Science Direct, ArXiv, and Measurement Science Review journal databases. Results: The results obtained are a batch of 12 articles showing the application and use of oscilloscope technology at the beginning of the 21st century. Conclusion: The conclusions applied to the application of oscilloscope technology in the early 21st-century are centered on the field of education in the measurement of physical quantities and the use of measurement technology which has high efficiency and low cost.