Smart Agriculture and Integrated Weed Management

Team name

Principal Investigator

Research Objectives

With the transition of crop planting methods towards mechanization and large-scale operations, there has been a significant reliance on chemical herbicides as the primary means forweed management. However, the prolonged and repeated use of a limited number of herbicides has resulted in the development of herbicide resistance among weed populations. Insome agricultural areas, the resistant weed populations have undergone rapid evolution, leading to frequent outbreaks of resistant weeds. At the same time, the irrational use of herbicides has led to a series of issues, such as environmental pollution, pesticide residues, and crop injury.

To address above problems, we strive to develop weed management tactics based on artificial intelligence. This involves utilizing computer vision and machine learning algorithms to identify crops and weeds, create weed maps, and combine them with intelligent control technology to achieve precise weed control. Our highly creative research team has successfully developed several smart weeders. Additionally, we have conducted research on integrated weed management by combining chemical control, crop cultivation, and AI-based weed control solutions to minimize chemical usage while effectively managing pests.

Through cross-disciplinary collaboration, our mission-driven and hardworking team is eager to pioneer the new frontier of AI-based crop protection. While our applied research focuses on solving real-world problems, we are dedicated to contributing to the scientific community through publications. The PI also serves as the Guest Editor and Associate Editor for the international journal of Crop Protection.

Team members

Xin Wang, Associate Research Professor, Senior Engineer, M.S. in Computer Science, Tsinghua University. Proficiency in embedded software and hardware design.

Xiaojun Jin, Associate Research Professor, Senior Engineer, Ph.D. in Software Engineering, Nanjing Forestry University. Front-end and back-end developers.

Research Projects

Artificial intelligence-based weed detection and precision weed control in turfgrasses and forage crops (2021-2024), National Science Foundation of China, 32072498. Project Objectives:Synthetic herbicide is the most important tool for weed management and is the top used pesticide in terms of total consumption and expenditure in China. Precision herbicide application can effectively decrease herbicide usage, weed control cost, and reduce the negative impact on environment and public health. A prerequisite of precision herbicide application is an autonomous system of weed detection. In recent years, artificial intelligence and machine vision, particularly deep learning convolutional neural network (DLCNN), has shown remarkable results in object detection and image classification. In this research project, the DLCNN will be trained according to the herbicide weed control spectrum with the ultimate goal of autonomous spot-spraying herbicides; various DLCNN architectures will be evaluated for detecting grassy weeds while growing in bermudagrass turfgrass and detecting broadleaf weeds while growing in alfalfa forage to provide the scientific basis of using DLCNN for weed detection. This research project will also investigate the impacts of environmental factors, weather, light, crop shading, weed growth stages on the performance of DLCNN for weed detection. The DLCNN models will be used in conjunction with a prototype of smart sprayer in field experiments to evaluate the effectiveness of weed detection, herbicide saving, and precision weed control. The results of this study are to provide the scientific basis for automatic weed detection and precision herbicide application.

Research Achievements

Intelligent Digital Greenhouse Management Software: The current smart greenhouse on the market has achieved real-time monitoring of various environmental factors, such as light, temperature, CO₂ concentration, moisture, and humidity. Our digital greenhouse management software system includes the smart screen software (Web), mobile app, and WeChat mini-program. It provides users with detailed Extension knowledge tailored to specific crops besides the functionalities mentioned above. Users can customize alarm configuration parameters and message notifications based on their own experiences and agronomic knowledge posted on the software, and thereby build automated intelligent greenhouses to provide optimized environments for crop growth. Through the software we have developed, the latest planting experience and research results from research institutions can be shared with growers and directly applied in production. If planting environments do not meet threshold values, farmers will be reminded to make timely corrections through mobile notifications. We will gradually enrich the technical knowledge in this software to include as many crop types as possible.The multi-terminal system has also implemented real-time video monitoring. Our research team is currently modeling the detection of diseases and pests for greenhouse crops such as strawberries and watermelons. The detection models willbe deployed in the cloud to achieve video monitoring of diseases and pests. Once diseases and pestsare detected, timely alerts will be sent to the user's mobile phone. Our smart greenhouse software combines the Internet of Things and artificial intelligence technology. In addition to conventional IoT sensor data display, it has also pioneered the approach of Extension along with cloud-based disease and pest monitoring functions, making the software leading in the industry.

Crop Phenotyping Software:The plant phenotyping instrumentson the current market, such as seed testing and crop phenotyping instruments, are designed with fixed cameras and physically connected to computers. During the usage, these devices found to be inconvenient to operate, slow in startup and processing speed, and inconvenient for mobile use across locations. Additionally, these instruments can measure a limited number of phenotyping parameters, and the measurement data is in offline mode. To address these issues, we have developed a multifunctional phenotyping software. The software can be installed on any cell phone, utilizing a mobile app for capturing photos, cloud-based models for efficient result calculation, and deploying models in the cloud. All measurement data are saved in the cloud, supporting data download or transfer via WeChat. The software can test various plant leaf phenotypes, seed germination, peanut shell shape, automate the measurement of peanut shell waist depth, accurately count seed numbers (regardless of plant species), and provide seed phenotype information such as circumference, color, length-width ratio. In addition, we are developing an algorithm to identify the infected area of strawberry fruits. We will continue to customize and develop other phenotype testing algorithms based on the actual needs of researchers to continuously enrich the functionality of this phenotyping software, making relevant scientific research work more efficient. The crop phenotyping software developed by our research group integrates the functions of seed testing, leaf phenotyping, peanut phenotyping, and seed phenotyping. It has richer functionalities, more accurate measurements, and lower costs compared to those on the current market.

Smart Sprayers: Computer vision-based crop and weed detection serves as the cornerstone of precision pesticide spraying. By leveraging technologies such as deep learning and neural networks, we can accurately identify and classify plants, thereby achieving precise weed control. However, there are still limitations to the existing detection methods. Weed recognition methods are often specific to particular regions and fields, lacking generalizability and posing challenges in their applications for smart weeders. Therefore, our laboratory has explored and developed various algorithms, such as semi-supervised learning, attention mechanism algorithms, segmentation models, etc., for application in different types of crops. The goal is to establish weed prescription maps, realize real-time weed recognition, and provide a foundation for precise automated weed control. Through our research, we have discovered differences and advantages/disadvantages among different algorithms, and thus it is important to select the most suitable algorithm model based on different intelligent machines and application scenarios.

Selected Publications

Liu T, Zhai D, He F, Yu J(2024) Semi-supervised learning methods for weed detection in turf. Pest Management Science (Accepted)

Sun H, Liu T, Zhai D, Yu J (2024) Evaluation of two deep learning-based approaches for detecting weeds growing in cabbage. Pest Management Science (Minor revision)

Liu T, Jin XJ, Wang J, Chen Y, Hu CS, Yu J (2023) Semi-supervised learning and attention mechanism for weed detection in wheat. Crop Protection 174:106389.

Jin X, Liu T, Mccullough PE, Chen Y, Yu J (2023) Evaluation of convolutional neural networks for herbicide susceptibility-based weed detection in turf. Frontiers in Plant Science 14:289

Zhuang J, Jin X, Chen Y, Meng W, Yu J, Bagavathiannan M (2023) Drought stress impact on the performance of deep convolutional neural networks for weed detection in bahiagrass. Grass and Forage Science. https://doi.org/10.1111/gfs.12583

Jin X, Bagavathiannan M, Maity A, Chen Y, Yu J (2022) Deep learning for detecting herbicide weed control spectrum in turfgrass. Plant Method 18:94 https://doi.org/10.1186/s13007-022-00929-4

Jin X, Bagavathiannan M, McCullough PE, Chen Y, Yu J (2022) A deep learning-based method for classification, detection, and localization of weeds in turfgrass. Pest Management Science 78: 4809-4821