R&D Department Case Study

Northern System Service is involved in the development of AI-based pest diagnosis technology as part of a project commissioned by the Ministry of Agriculture, Forestry and Fisheries called the "Artificial Intelligence Future Agriculture Creation Project.

The background of this project is twofold: the aging of farmers and the increase in the number of new farmers make it difficult to pass on pest control techniques, while Japan's diverse climate and farming patterns make it difficult to diagnose and control new pests and diseases caused by diseases and alien organisms that follow the seeds and seedlings imported from other countries. The increase in imports of seeds and seedlings from other countries has led to the outbreak of new diseases and pests caused by diseases and alien organisms attached to those seeds and seedlings, making it difficult to diagnose and control pests. Therefore, there is a need for a system that allows farmers to easily diagnose and control pests and diseases. This project aims to develop a highly accurate pest diagnosis system using Deep Learning, and will collect data on tomatoes, eggplants, cucumbers, and strawberries, which are four major crops, in 24 prefectures. The National Institute, public research institutes, extension agencies, and private companies work together to collect data, develop, verify, disseminate, and commercialize the technology.

We are in charge of developing insect and pest diagnostic AI for pest and disease diagnosis AI development, contributing to the revitalization of agriculture using AI and ICT.

Background

1. Diverse natural environments and their changes

Agriculture in Japan is practiced not only in the agricultural areas of the plains, but also in a wide variety of locations, from mountainous and mountainous areas to residential suburbs and coastal suburbs. In addition, the country stretches from north to south and is influenced by ocean currents, resulting in a variety of climates, as well as a variety of pests and diseases.

2. Increased difficulties in pest diagnosis and control

As Japan's birthrate declines and the population ages, the depopulation of farming villages and the aging of experienced farmers have become serious problems due to the outflow of young people to cities and other factors. As a result, although the number of new farmers is increasing, it is difficult to pass on pest control techniques. To add to this predicament, the importation of seeds and seedlings for vegetable production has increased, resulting in an increase in damage caused by new diseases and pests, and there is an urgent need to improve the efficiency of pest diagnosis.

Proposed Problem Development of AI-based pest diagnosis technology

1. Pest Diagnosis by Deep Learning

1.1. Differences between traditional machine learning and deep learning

While in general machine learning, a human manually sets the features, deep learning allows the AI to set the features.

1.2. Challenges in developing AI-based pest diagnosis technology

Challenges of AI diagnosis (1) Need image data for training

• Applications developed overseas and not targeted for Japan.
• Little data exists for Japanese crops and pests.
• Difficult to use due to copyright restrictions even if they exist.

Challenges of AI diagnosis (2) Development of high-precision AI

• Difficult to achieve high precision in general-purpose web services such as Google.

Challenges of AI diagnosis (3) Unclear IP treatment of AI

• It is important to have it at your disposal.
• Public knowledge is not sufficient.
• Publicly funded results are highly sought after.

Based on the above three issues, we have structured three research projects.

1. Acquisition of electronic images and database construction of time-series damage by important pests occurring in major vegetables
• Published as open data
2. Development of high-precision artificial intelligence for pest diagnosis based on databased electronic images
• Open source software for algorithms
3. Development of a diagnostic application for wearable terminals that can be used at production sites, based on artificial intelligence for high-precision pest and disease diagnosis
• Government, aiming for practical application in agricultural field.

Research task

1. Promote "open innovation"

Open innovation is a new form of innovation in which the results of research and development are made public in a reusable form and innovation is promoted by utilizing internal and external resources. It is also called for in the Revitalization Strategy of Japan 2016, the Comprehensive Strategy for Science, Technology and Innovation 2016, and other documents. We believe that AI will be used as an infrastructure for AI application in agriculture, and will contribute to innovation and growth in agriculture as a whole.

1.1　Database construction of pest images

To verify pest damage to four major crops, file electronic image data, and build and publish a database of image information on important pests occurring in Japan, images of pests occurring on target crops will be taken for 10 pest species in each of 24 prefectures, from cold regions to warm regions. The images will be used to identify the status of the pests from early to late stages, and ancillary information such as the date and time of occurrence and weather conditions will also be collected to create a database. High reliability is assured by using expert judgment and specimen information in constructing this database. The data collected here will be used for training in Deep Learning.

2. Development of artificial intelligence to diagnose pests and diseases

AI to diagnose pests and diseases is developed separately for diseases, insects, and pests, since the conditions of these two types of diseases are different. The data collected in the first project will be used as training data to create a diagnostic AI using Deep Learning, and a server will be built to enable cross-utilization of the diagnostic AIs for diseases, insects, and pests. The diagnostic AI will be released as open source. We are in charge of the development of diagnostic AI for insects and pests for this disease and pest diagnostic AI development.

2.1 Our Research, "Creating Datasets."

【Purpose】Two images (leaf and fruit) were extracted from insect damage images provided by the Research Center for Agricultural Environmental Change, National Institute of Agro-Environmental Sciences.

【method】Extracting target areas for identification from images using the object detection network Shapemask [1].

【result】Background removal confirms the effect of making the discrimination model focus on the characteristics of the insect damage itself (suppression of over-learning).

2.2 Our research, "Learning Discriminative Models."

【Purpose】Learning Discriminative Models

【method】Convolutional Neural Network EfficientNet-B6[2] is used to train a dataset to build an AI for insect damage diagnosis.

【result】Average accuracy at diagnosis 86.7%
※ The results were verified using images taken in a completely different region than the location of the images used for training, and are at a level that can withstand real-world operation.

In the future, we aim to improve the accuracy of classification of initial damage and microfeatures.

Example of extraction of focus area by Grad-CAM[3] (tomato fruit, thrips)

[1] Weicheng Kuo, Anelia Angelova, et al. “ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors” arxiv:1904.03239, 2019.

[2] Mingxing Tan and Quoc V. Le. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. ICML 2019. arxiv:1905.11946. 2019.

[3] R. R.Selvaraju, A. Das, R. Vedantam, M. Cogswell, D. Parikh, and D. Batra. Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization. arXiv:1611.01646, 2016.

This research was supported by the Ministry of Agriculture, Forestry and Fisheries (MAFF) under the project "Development of Pest Identification Technology Using AI" JP17935051, and by the Public-Private R&D Investment Scaling Up Program (PRISM). We would also like to express our sincere appreciation to the public organizations in the participating prefectures that contributed to the collection of pest damage images.

3. Development of applications for identification of pests and diseases by smart phones and other devices

The third issue is the development of an application that identifies pests using smartphones and other devices. Pests found during farming are photographed with a smartphone, and the images and ancillary information are sent to a server for identification by the pest diagnosis AI, and the identified results are displayed on the smartphone. We are also developing a system that can be implemented with AR glasses. A video comparing farming with AR Glass and conventional farming is shown at the bottom of this page. Please take a look at the video (4:08 min.).

In the past, the user would have to take out a smartphone and look up the pest, but we believe that the use of AR glasses will enable automatic detection of pests that come into view during work, as shown in the video, and will make work more efficient.

Automatic detection of pests by AR Glass prevents delays in response due to missed early symptoms, promotes accurate early response to pests, minimizes damage losses, and contributes to increased agricultural production.

Revitalizing Agriculture with AI and ICT

We will contribute to the revitalization of agriculture through the use of AI and ICT, enabling the participation of a diverse range of people.