Automatic food detection in egocentric images using artificial intelligence technology

Abstract Objective To develop an artificial intelligence (AI)-based algorithm which can automatically detect food items from images acquired by an egocentric wearable camera for dietary assessment. Design To study human diet and lifestyle, large sets of egocentric images were acquired using a wearable device, called eButton, from free-living individuals. Three thousand nine hundred images containing real-world activities, which formed eButton data set 1, were manually selected from thirty subjects. eButton data set 2 contained 29 515 images acquired from a research participant in a week-long unrestricted recording. They included both food- and non-food-related real-life activities, such as dining at both home and restaurants, cooking, shopping, gardening, housekeeping chores, taking classes, gym exercise, etc. All images in these data sets were classified as food/non-food images based on their tags generated by a convolutional neural network. Results A cross data-set test was conducted on eButton data set 1. The overall accuracy of food detection was 91·5 and 86·4 %, respectively, when one-half of data set 1 was used for training and the other half for testing. For eButton data set 2, 74·0 % sensitivity and 87·0 % specificity were obtained if both ‘food’ and ‘drink’ were considered as food images. Alternatively, if only ‘food’ items were considered, the sensitivity and specificity reached 85·0 and 85·8 %, respectively. Conclusions The AI technology can automatically detect foods from low-quality, wearable camera-acquired real-world egocentric images with reasonable accuracy, reducing both the burden of data processing and privacy concerns.

Download Full-text

Introduction to the special issue on constraint satisfaction for planning and scheduling

The Knowledge Engineering Review ◽

10.1017/s0269888917000017 ◽

2016 ◽

Vol 31 (5) ◽

pp. 415-416

Author(s):

Miguel A. Salido ◽

Roman Barták

Keyword(s):

Artificial Intelligence ◽

Resource Allocation ◽

Constraint Satisfaction ◽

Real World ◽

Real Life ◽

Special Issue ◽

Planning And Scheduling ◽

Artificial Intelligence Planning ◽

Life Problems ◽

Real World Problems

AbstractThe areas of Artificial Intelligence planning and scheduling have seen important advances thanks to the application of constraint satisfaction models and techniques. Especially, solutions to many real-world problems need to integrate plan synthesis capabilities with resource allocation, which can be efficiently managed by using constraint satisfaction techniques. Constraint satisfaction plays an important role in solving such real life problems, and integrated techniques that manage planning and scheduling with constraint satisfaction are particularly useful.

Download Full-text

Success Factors of Artificial Intelligence Implementation in Healthcare

Frontiers in Digital Health ◽

10.3389/fdgth.2021.594971 ◽

2021 ◽

Vol 3 ◽

Author(s):

Justus Wolff ◽

Josch Pauling ◽

Andreas Keck ◽

Jan Baumbach

Keyword(s):

Artificial Intelligence ◽

Economic Impact ◽

Real World ◽

Success Factors ◽

Real Life ◽

Data Access ◽

World Health ◽

Success Factor ◽

Real World Application ◽

Life Years

Background: Artificial Intelligence (AI) in healthcare has demonstrated high efficiency in academic research, while only few, and predominantly small, real-world AI applications exist in the preventive, diagnostic and therapeutic contexts. Our identification and analysis of success factors for the implementation of AI aims to close the gap between recent years' significant academic AI advancements and the comparably low level of practical application in healthcare.Methods: A literature and real life cases analysis was conducted in Scopus and OpacPlus as well as the Google advanced search database. The according search queries have been defined based on success factor categories for AI implementation derived from a prior World Health Organization survey about barriers of adoption of Big Data within 125 countries. The eligible publications and real life cases were identified through a catalog of in- and exclusion criteria focused on concrete AI application cases. These were then analyzed to deduct and discuss success factors that facilitate or inhibit a broad-scale implementation of AI in healthcare.Results: The analysis revealed three categories of success factors, namely (1) policy setting, (2) technological implementation, and (3) medical and economic impact measurement. For each of them a set of recommendations has been deducted: First, a risk adjusted policy frame is required that distinguishes between precautionary and permissionless principles, and differentiates among accountability, liability, and culpability. Second, a “privacy by design” centered technology infrastructure shall be applied that enables practical and legally compliant data access. Third, the medical and economic impact need to be quantified, e.g., through the measurement of quality-adjusted life years while applying the CHEERS and PRISMA reporting criteria.Conclusions: Private and public institutions can already today leverage AI implementation based on the identified results and thus drive the translation from scientific development to real world application. Additional success factors could include trust-building measures, data categorization guidelines, and risk level assessments and as the success factors are interlinked, future research should elaborate on their optimal interaction to utilize the full potential of AI in real world application.

Download Full-text

Neuropsychology in the Real World

Zeitschrift für Neuropsychologie ◽

10.1024/1016-264x/a000139 ◽

2014 ◽

Vol 25 (4) ◽

pp. 233-238 ◽

Cited By ~ 2

Author(s):

Martin Peper ◽

Simone N. Loeffler

Keyword(s):

Neuropsychological Assessment ◽

Real World ◽

Ecological Validity ◽

Real Life ◽

Emotional States ◽

Context Sensitive ◽

Traditional Assessment ◽

Life Data ◽

Real Life Data ◽

Assessment And Treatment

Current ambulatory technologies are highly relevant for neuropsychological assessment and treatment as they provide a gateway to real life data. Ambulatory assessment of cognitive complaints, skills and emotional states in natural contexts provides information that has a greater ecological validity than traditional assessment approaches. This issue presents an overview of current technological and methodological innovations, opportunities, problems and limitations of these methods designed for the context-sensitive measurement of cognitive, emotional and behavioral function. The usefulness of selected ambulatory approaches is demonstrated and their relevance for an ecologically valid neuropsychology is highlighted.

Download Full-text

Stress testing reveals gaps in clinic readiness of image-based diagnostic artificial intelligence models

npj Digital Medicine ◽

10.1038/s41746-020-00380-6 ◽

2021 ◽

Vol 4 (1) ◽

Author(s):

Albert T. Young ◽

Kristen Fernandez ◽

Jacob Pfau ◽

Rasika Reddy ◽

Nhat Anh Cao ◽

...

Keyword(s):

Artificial Intelligence ◽

Real World ◽

Stress Testing ◽

Operating Characteristic ◽

Characteristic Curve ◽

Skin Lesions ◽

Stress Tests ◽

Performance Metric ◽

Intelligence Models ◽

Artificial Intelligence Models

AbstractArtificial intelligence models match or exceed dermatologists in melanoma image classification. Less is known about their robustness against real-world variations, and clinicians may incorrectly assume that a model with an acceptable area under the receiver operating characteristic curve or related performance metric is ready for clinical use. Here, we systematically assessed the performance of dermatologist-level convolutional neural networks (CNNs) on real-world non-curated images by applying computational “stress tests”. Our goal was to create a proxy environment in which to comprehensively test the generalizability of off-the-shelf CNNs developed without training or evaluation protocols specific to individual clinics. We found inconsistent predictions on images captured repeatedly in the same setting or subjected to simple transformations (e.g., rotation). Such transformations resulted in false positive or negative predictions for 6.5–22% of skin lesions across test datasets. Our findings indicate that models meeting conventionally reported metrics need further validation with computational stress tests to assess clinic readiness.

Download Full-text

Dealing with multiple experts and non-stationarity in inverse reinforcement learning: an application to real-life problems

Machine Learning ◽

10.1007/s10994-020-05939-8 ◽

2021 ◽

Author(s):

Amarildo Likmeta ◽

Alberto Maria Metelli ◽

Giorgia Ramponi ◽

Andrea Tirinzoni ◽

Matteo Giuliani ◽

...

Keyword(s):

Reinforcement Learning ◽

Real World ◽

Real Life ◽

User Preferences ◽

Inverse Reinforcement Learning ◽

Water Release ◽

Reward Function ◽

Model Free ◽

Conflicting Objectives ◽

Multiple Experts

AbstractIn real-world applications, inferring the intentions of expert agents (e.g., human operators) can be fundamental to understand how possibly conflicting objectives are managed, helping to interpret the demonstrated behavior. In this paper, we discuss how inverse reinforcement learning (IRL) can be employed to retrieve the reward function implicitly optimized by expert agents acting in real applications. Scaling IRL to real-world cases has proved challenging as typically only a fixed dataset of demonstrations is available and further interactions with the environment are not allowed. For this reason, we resort to a class of truly batch model-free IRL algorithms and we present three application scenarios: (1) the high-level decision-making problem in the highway driving scenario, and (2) inferring the user preferences in a social network (Twitter), and (3) the management of the water release in the Como Lake. For each of these scenarios, we provide formalization, experiments and a discussion to interpret the obtained results.

Download Full-text

Improving the performance of a radio-frequency localization system in adverse outdoor applications

EURASIP Journal on Wireless Communications and Networking ◽

10.1186/s13638-021-02001-6 ◽

2021 ◽

Vol 2021 (1) ◽

Author(s):

Marcelo N. de Sousa ◽

Ricardo Sant’Ana ◽

Rigel P. Fernandes ◽

Julio Cesar Duarte ◽

José A. Apolinário ◽

...

Keyword(s):

Random Forest ◽

Ray Tracing ◽

Real World ◽

Practical Implication ◽

Real Life ◽

Simulated Data ◽

Real Data ◽

Gradient Boosting ◽

Real World Data ◽

Localization Accuracy

AbstractIn outdoor RF localization systems, particularly where line of sight can not be guaranteed or where multipath effects are severe, information about the terrain may improve the position estimate’s performance. Given the difficulties in obtaining real data, a ray-tracing fingerprint is a viable option. Nevertheless, although presenting good simulation results, the performance of systems trained with simulated features only suffer degradation when employed to process real-life data. This work intends to improve the localization accuracy when using ray-tracing fingerprints and a few field data obtained from an adverse environment where a large number of measurements is not an option. We employ a machine learning (ML) algorithm to explore the multipath information. We selected algorithms random forest and gradient boosting; both considered efficient tools in the literature. In a strict simulation scenario (simulated data for training, validating, and testing), we obtained the same good results found in the literature (error around 2 m). In a real-world system (simulated data for training, real data for validating and testing), both ML algorithms resulted in a mean positioning error around 100 ,m. We have also obtained experimental results for noisy (artificially added Gaussian noise) and mismatched (with a null subset of) features. From the simulations carried out in this work, our study revealed that enhancing the ML model with a few real-world data improves localization’s overall performance. From the machine ML algorithms employed herein, we also observed that, under noisy conditions, the random forest algorithm achieved a slightly better result than the gradient boosting algorithm. However, they achieved similar results in a mismatch experiment. This work’s practical implication is that multipath information, once rejected in old localization techniques, now represents a significant source of information whenever we have prior knowledge to train the ML algorithm.

Download Full-text

A Survey on Bias and Fairness in Machine Learning

ACM Computing Surveys ◽

10.1145/3457607 ◽

2021 ◽

Vol 54 (6) ◽

pp. 1-35

Author(s):

Ninareh Mehrabi ◽

Fred Morstatter ◽

Nripsuta Saxena ◽

Kristina Lerman ◽

Aram Galstyan

Keyword(s):

Artificial Intelligence ◽

Machine Learning ◽

Deep Learning ◽

Real World ◽

State Of The Art ◽

Future Directions ◽

Discriminatory Behavior ◽

Real World Applications ◽

Near Future ◽

Different Sources

With the widespread use of artificial intelligence (AI) systems and applications in our everyday lives, accounting for fairness has gained significant importance in designing and engineering of such systems. AI systems can be used in many sensitive environments to make important and life-changing decisions; thus, it is crucial to ensure that these decisions do not reflect discriminatory behavior toward certain groups or populations. More recently some work has been developed in traditional machine learning and deep learning that address such challenges in different subdomains. With the commercialization of these systems, researchers are becoming more aware of the biases that these applications can contain and are attempting to address them. In this survey, we investigated different real-world applications that have shown biases in various ways, and we listed different sources of biases that can affect AI applications. We then created a taxonomy for fairness definitions that machine learning researchers have defined to avoid the existing bias in AI systems. In addition to that, we examined different domains and subdomains in AI showing what researchers have observed with regard to unfair outcomes in the state-of-the-art methods and ways they have tried to address them. There are still many future directions and solutions that can be taken to mitigate the problem of bias in AI systems. We are hoping that this survey will motivate researchers to tackle these issues in the near future by observing existing work in their respective fields.

Download Full-text

Training and deploying a deep learning model for endoscopic severity grading in ulcerative colitis using multicenter clinical trial data

Therapeutic Advances in Gastrointestinal Endoscopy ◽

10.1177/2631774521990623 ◽

2021 ◽

Vol 14 ◽

pp. 263177452199062

Author(s):

Benjamin Gutierrez Becker ◽

Filippo Arcadu ◽

Andreas Thalhammer ◽

Citlalli Gamez Serna ◽

Owen Feehan ◽

...

Keyword(s):

Artificial Intelligence ◽

Ulcerative Colitis ◽

Clinical Trials ◽

Deep Learning ◽

Mayo Clinic ◽

Automated System ◽

Phase Iii ◽

Plain Language ◽

Data Set ◽

Endoscopic Videos

Introduction: The Mayo Clinic Endoscopic Subscore is a commonly used grading system to assess the severity of ulcerative colitis. Correctly grading colonoscopies using the Mayo Clinic Endoscopic Subscore is a challenging task, with suboptimal rates of interrater and intrarater variability observed even among experienced and sufficiently trained experts. In recent years, several machine learning algorithms have been proposed in an effort to improve the standardization and reproducibility of Mayo Clinic Endoscopic Subscore grading. Methods: Here we propose an end-to-end fully automated system based on deep learning to predict a binary version of the Mayo Clinic Endoscopic Subscore directly from raw colonoscopy videos. Differently from previous studies, the proposed method mimics the assessment done in practice by a gastroenterologist, that is, traversing the whole colonoscopy video, identifying visually informative regions and computing an overall Mayo Clinic Endoscopic Subscore. The proposed deep learning–based system has been trained and deployed on raw colonoscopies using Mayo Clinic Endoscopic Subscore ground truth provided only at the colon section level, without manually selecting frames driving the severity scoring of ulcerative colitis. Results and Conclusion: Our evaluation on 1672 endoscopic videos obtained from a multisite data set obtained from the etrolizumab Phase II Eucalyptus and Phase III Hickory and Laurel clinical trials, show that our proposed methodology can grade endoscopic videos with a high degree of accuracy and robustness (Area Under the Receiver Operating Characteristic Curve = 0.84 for Mayo Clinic Endoscopic Subscore ⩾ 1, 0.85 for Mayo Clinic Endoscopic Subscore ⩾ 2 and 0.85 for Mayo Clinic Endoscopic Subscore ⩾ 3) and reduced amounts of manual annotation. Plain language summary Patient, caregiver and provider thoughts on educational materials about prescribing and medication safety Artificial intelligence can be used to automatically assess full endoscopic videos and estimate the severity of ulcerative colitis. In this work, we present an artificial intelligence algorithm for the automatic grading of ulcerative colitis in full endoscopic videos. Our artificial intelligence models were trained and evaluated on a large and diverse set of colonoscopy videos obtained from concluded clinical trials. We demonstrate not only that artificial intelligence is able to accurately grade full endoscopic videos, but also that using diverse data sets obtained from multiple sites is critical to train robust AI models that could potentially be deployed on real-world data.

Download Full-text

An Adaptive Game-Based Learning Strategy for Children Road Safety Education and Practice in Virtual Space

Sensors ◽

10.3390/s21113661 ◽

2021 ◽

Vol 21 (11) ◽

pp. 3661

Author(s):

Noman Khan ◽

Khan Muhammad ◽

Tanveer Hussain ◽

Mansoor Nasir ◽

Muhammad Munsif ◽

...

Keyword(s):

Real World ◽

Road Safety ◽

Learning Strategy ◽

Real Life ◽

Safety Training ◽

Virtual Space ◽

Safety Education ◽

Average Output ◽

Training Environment ◽

The Road

Virtual reality (VR) has been widely used as a tool to assist people by letting them learn and simulate situations that are too dangerous and risky to practice in real life, and one of these is road safety training for children. Traditional video- and presentation-based road safety training has average output results as it lacks physical practice and the involvement of children during training, without any practical testing examination to check the learned abilities of a child before their exposure to real-world environments. Therefore, in this paper, we propose a 3D realistic open-ended VR and Kinect sensor-based training setup using the Unity game engine, wherein children are educated and involved in road safety exercises. The proposed system applies the concepts of VR in a game-like setting to let the children learn about traffic rules and practice them in their homes without any risk of being exposed to the outside environment. Thus, with our interactive and immersive training environment, we aim to minimize road accidents involving children and contribute to the generic domain of healthcare. Furthermore, the proposed framework evaluates the overall performance of the students in a virtual environment (VE) to develop their road-awareness skills. To ensure safety, the proposed system has an extra examination layer for children’s abilities evaluation, whereby a child is considered fit for real-world practice in cases where they fulfil certain criteria by achieving set scores. To show the robustness and stability of the proposed system, we conduct four types of subjective activities by involving a group of ten students with average grades in their classes. The experimental results show the positive effect of the proposed system in improving the road crossing behavior of the children.

Download Full-text

Prioritizing and Analyzing the Role of Climate and Urban Parameters in the Confirmed Cases of COVID-19 Based on Artificial Intelligence Applications

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph17103730 ◽

2020 ◽

Vol 17 (10) ◽

pp. 3730 ◽

Cited By ~ 9

Author(s):

Sina Shaffiee Haghshenas ◽

Behrouz Pirouz ◽

Sami Shaffiee Haghshenas ◽

Behzad Pirouz ◽

Patrizia Piro ◽

...

Keyword(s):

Artificial Intelligence ◽

Infectious Disease ◽

Relative Humidity ◽

Cost Function ◽

Selection Process ◽

Environmental Parameters ◽

Pso Algorithm ◽

Data Set ◽

De Algorithm

Nowadays, an infectious disease outbreak is considered one of the most destructive effects in the sustainable development process. The outbreak of new coronavirus (COVID-19) as an infectious disease showed that it has undesirable social, environmental, and economic impacts, and leads to serious challenges and threats. Additionally, investigating the prioritization parameters is of vital importance to reducing the negative impacts of this global crisis. Hence, the main aim of this study is to prioritize and analyze the role of certain environmental parameters. For this purpose, four cities in Italy were selected as a case study and some notable climate parameters—such as daily average temperature, relative humidity, wind speed—and an urban parameter, population density, were considered as input data set, with confirmed cases of COVID-19 being the output dataset. In this paper, two artificial intelligence techniques, including an artificial neural network (ANN) based on particle swarm optimization (PSO) algorithm and differential evolution (DE) algorithm, were used for prioritizing climate and urban parameters. The analysis is based on the feature selection process and then the obtained results from the proposed models compared to select the best one. Finally, the difference in cost function was about 0.0001 between the performances of the two models, hence, the two methods were not different in cost function, however, ANN-PSO was found to be better, because it reached to the desired precision level in lesser iterations than ANN-DE. In addition, the priority of two variables, urban parameter, and relative humidity, were the highest to predict the confirmed cases of COVID-19.

Download Full-text