Research on recognition and localization of unordered stacked watermelons based on machine vision

doi:10.6048/j.issn.1001-4330.2024.07.029

Xinjiang Agricultural Sciences ›› 2024, Vol. 61 ›› Issue (7): 1805-1813.DOI: 10.6048/j.issn.1001-4330.2024.07.029

• Prataculture·Agricultural Equipment Engineering and Mechanization·Animal Husbandry Veteri-narian • Previous Articles Next Articles

Research on recognition and localization of unordered stacked watermelons based on machine vision

WU Fengyang¹^,²(), HUANG Wenqian²(), TIAN Xi², YANG Yulin²

1. College of Mechanical and Electrical Engineering, Xinjiang Agricultural University, Urumqi 830052, China
2. Intelligent Equipment Technology Research Center of Beijing Academy of Agricultural and Forestry Sciences/National Agricultural Intelligent Equipment Engineering Technology Research Center, Beijing 100097, China

Received:2023-12-11 Online:2024-07-20 Published:2024-09-04
Correspondence author: HUANG Wenqian
Supported by:
The national key research and development plan project "Research and development of mobile fruit quality intelligent perception equipment and key link robots"(2022YFD2002202);Special Fund Project for National Watermelon Industry Technology System(CARS-25-07);Beijing Academy of Agricultural and Forestry Sciences Innovation Special Emerging Discipline Intersection Project"esearch and Development of Non destructive Testing Technology and Equipment for Watermelon Quality"(KJCX20230206)

基于机器视觉的无序堆放西瓜识别与定位技术

吴风扬¹^,²(), 黄文倩²(), 田喜², 杨育霖²

1.新疆农业大学机电工程学院,乌鲁木齐 830052
2.北京市农林科学院智能装备技术研究中心/国家农业智能装备工程技术研究中心,北京 100097

通讯作者: 黄文倩
作者简介:吴风扬(1999-),男,湖北黄石人,硕士研究生,研究方向为机器视觉,(E-mail)1519100570@qq.com
基金资助:
国家重点研发计划项目“移动式果品品质智能感知设备与关键环节机器人研发”(2022YFD2002202);国家西甜瓜产业技术体系专项资金项目(CARS-25-07);北京市农林科学院创新专项新兴学科交叉项目“西甜瓜品质无损检测技术与装备研发”(KJCX20230206)

Abstract

Abstract:

【Objective】To study a watermelon recognition and positioning technology based on machine vision to meet the automation requirements of watermelon handling equipment. 【Methods】2D image data and 3D point cloud data of watermelon were collected and the OpenCV function library was called to segment the two-dimensional image of watermelon and extract the outer contour of the watermelon; The 3D point cloud data was preprocessed using the PCL function and perform graphic feature matching with the 2D image to extract the centroid points of the top layer watermelon.【Results】The validation test results showed that the total recognition rate of the number of watermelons in a single round was 97.62%, the correct recognition rate was 95.58%, and the centroid recognition rate was 92.72%. The total recognition rate of 20 rounds of testing was 98.02%, the total correct recognition rate was 96.53%, the total centroid recognition rate was 94.17%, and the error rate between the total centroid number and the total number of watermelons was 2.36%; The total time for watermelon image processing was 101.8 seconds, which improved the efficiency by 77.8%.【Conclusion】The watermelon recognition and positioning method proposed in this article has high recognition rate and low error rate, which can provide technical support and algorithm reference for the visual part of automatic handling equipment.

Key words: watermelon; identification and positioning; centroid extraction; image processing; point cloud; OpenCV; PCL

摘要：

【目的】研究一种基于机器视觉的西瓜识别与定位技术,满足西瓜搬运设备的自动化要求,为自动搬运设备的视觉部分提供技术支持。【方法】采集西瓜的二维图像数据与三维点云数据。调用OpenCV函数库分割西瓜二维图像,提取出西瓜的外轮廓;采用PCL函数对三维点云数据进行预处理,并与二维图像作图形特征匹配,提取出顶层西瓜的质心点。【结果】单轮次的西瓜轮廓总识别率为97.62%,正确识别率为95.58%,质心识别率为92.72%。20轮测试的总轮廓识别率为98.02%,总正确识别率为96.53%,总质心识别率为94.17%,总质心个数和总西瓜个数之间的误差率为2.36%;西瓜图像处理的总用时为101.8s,效率提高了77.8%。【结论】基于机器视觉的西瓜识别与定位技术识别率较高且误差率较低。

关键词: 西瓜, 识别与定位, 质心提取, 图像处理, 点云, OpenCV, PCL

CLC Number:

S126
S651

WU Fengyang, HUANG Wenqian, TIAN Xi, YANG Yulin. Research on recognition and localization of unordered stacked watermelons based on machine vision[J]. Xinjiang Agricultural Sciences, 2024, 61(7): 1805-1813.

吴风扬, 黄文倩, 田喜, 杨育霖. 基于机器视觉的无序堆放西瓜识别与定位技术[J]. 新疆农业科学, 2024, 61(7): 1805-1813.

Figures/Tables 13

References 29

[1]	钱曼, 黄文倩, 王庆艳, 等. 西瓜检测部位差异对近红外光谱可溶性固形物预测模型的影响[J]. 光谱学与光谱分析, 2016, 36(6): 1700-1705, 1711.
	QIAN Man, HUANG Wenqian, WANG Qingyan, et al. Assessment of influence detective position variability on precision of near infrared models for soluble solid content of watermelon[J]. Spectroscopy and Spectral Analysis, 2016, 36(6): 1700-1705, 1711.
[2]	周艺璇. 西瓜翠衣的营养保健成分及药用价值研究进展[J]. 现代食品, 2022, 28(8): 51-53, 57.
	ZHOU Yixuan. Research progress on nutritional and health components and medicinal value of watermelon peel[J]. Modern Food, 2022, 28(8): 51-53, 57.
[3]	李干琼, 王志丹. 我国西瓜产业发展现状及趋势分析[J]. 中国瓜菜, 2019, 32(12): 79-83.
	LI Ganqiong, WANG Zhidan. Development status and trend analysis of watermelon industry in China[J]. China Cucurbits and Vegetables, 2019, 32(12): 79-83.
[4]	宋展, 胡宝贵. 后疫情时期消费者视角的北京市西瓜产业发展[J]. 北方园艺, 2022, (23): 134-139.
	SONG Zhan, HU Baogui. Development of watermelon industry in Beijing city from the perspective of consumers in the post epidemic period[J]. Northern Horticulture, 2022, (23): 134-139.
[5]	高丽娜. 西瓜采收技术[J]. 河南农业, 2015, (9): 46.
	GAO Lina. Watermelon Harvesting Techniques[J]. Henan Agriculture, 2015, (9): 46.
[6]	官大文, 王春源, 王骏发. 基于无人机机器视觉的西瓜侦测创新农场管理模式[J]. 海峡科学, 2020, (11): 62-68.
	GUAN Dawen, WANG Chunyuan, WANG Junfa. Innovative farm management mode of watermelon detection based on UAV machine vision[J]. Straits Science, 2020, (11): 62-68. [万方]
[7]	Kamaldeen Mohammed, Evans Batung, Ansumah Saaka-Sulemana, et al. Determinants of mechanized technology adoption in smallholder agriculture: Implications for agricultural policy[J]. Neurocomputing, 2023, 18(3): 545-557.
[8]	唐芳. 我国农机购置补贴与农业机械化发展现状与提升建议[J]. 农业经济, 2023, (4): 9-12.
	TANG Fang. China's agricultural machinery purchase subsidies and Mechanised agriculture development status and improvement suggestions[J]. Agricultural Economy, 2023, (4): 9-12.
[9]	罗辉, 崔亚飞. 基于机器视觉的工件识别与定位系统设计与实现[J]. 组合机床与自动化加工技术, 2021, (1): 101-105, 109. DOI
	LUO Hui, CUI Yafei. Design and implementation of workpiece recognition and positioning system based on machine vision[J]. Modular Machine Tool & Automatic Manufacturing Technique, 2021, (1): 101-105, 109.
[10]	崔旭东, 谭欢, 王平江, 等. 基于3D视觉的散乱堆放工件机器人抓取技术研究[J]. 制造技术与机床, 2021, (2): 36-41. DOI
	CUI Xudong, TAN Huan, WANG Pingjiang, et al. Research on vision-based robot’s grabbing attitude for scattered stacking workpieces[J]. Manufacturing Technology & Machine Tool, 2021, (2): 36-41.
[11]	Xu B R, Chai L, Zhang C L. Research and application on corn crop identification and positioning method based on Machine vision[J]. Precision Agriculture, 2023.
[12]	柴宏红, 邵科, 于超, 等. 基于三维点云的甜菜根表型参数提取与根型判别[J]. 农业工程学报, 2020, 36(10): 181-188.
	CHAI Honghong, SHAO Ke, YU Chao, et al. Extraction of phenotypic parameters and discrimination of beet root types based on 3D point cloud[J]. Transactions of the Chinese Society of Agricultural Engineering, 2020, 36(10): 181-188.
[13]	宋晨旭, 于翀宇, 邢永超, 等. 基于OpenCV的大豆籽粒多表型参数获取算法[J]. 农业工程学报, 2022, 38(20): 156-163.
	SONG Chenxu, YU Chongyu, XING Yongchao, et al. Algorith for acquiring multi-phenotype parameters of soybean seed based on OpenCV[J]. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38(20): 156-163.
[14]	刘万辉, 杨晔. 温室场景下成熟西瓜的图像识别研究[J]. 福建电脑, 2013, 29(5): 29-31.
	LIU Wanhui, YANG Ye. Study on image recognition of mature watermelon in greenhouse scene[J]. Journal of Fujian Computer, 2013, 29(5): 29-31.
[15]	吴明清, 王丽. 基于机器视觉技术的西瓜瓜瓤的识别方法[J]. 塔里木大学学报, 2014, 26(3): 88-91.
	WU Mingqing, WANG Li. Identification of the machine vision technology based on watermelon flesh[J]. Journal of Tarim University, 2014, 26(3): 88-91.
[16]	刘德儿, 朱磊, 冀炜臻, 等. 基于RGB-D相机的脐橙实时识别定位与分级方法[J]. 农业工程学报, 2022, 38(14):154-165.
	LIU Deer, ZHU Lei, JI Weizhen, et al. Real time recognition, localization, and grading method for navel oranges based on RGB-D cameras[J]. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38(14):154-165.
[17]	Liu K, Jiang K, Mu Y. Development of a vision guided robot for grafting watermelon based on hole insertion method[J]. Computers and Electronics in Agriculture, 2020.
[18]	Mohd Ali M, Hashim N, Bejo S K, et al. Comparison of laser backscattering imaging and computer vision system for grading of seedless watermelons[J]. Journal of Food Measurement and Characterization, 2020, 14(1): 69-77.
[19]	邹宏伟. 基于OpenCV的数字图像处理技术研究与实现[J]. 无线互联科技, 2019, 16(22): 118-119.
	ZOU Hongwei. Research and implementation of digital image processing technology based on OpenCV[J]. Wireless Internet Technology, 2019, 16(22): 118-119.
[20]	龙嫣然, 徐昱琳, 费心怡. 基于PCL的Jaco机械臂的目标抓取[J]. 工业控制计算机, 2019, 32(4): 65-67.
	LONG Yanran, XU Yulin, FEI Xinyi. PCL-based object grasping by robot arm hand system[J]. Industrial Control Computer, 2019, 32(4): 65-67.
[21]	刘星星, 王烁烁, 徐丽明, 等. 基于OpenCV的动态葡萄干色泽实时识别[J]. 农业工程学报, 2019, 35(23): 177-184.
	LIU Xingxing, WANG Shuoshuo, XU Liming, et al. Real time color recognition of moving raisin based on OpenCV[J]. Transactions of the Chinese Society of Agricultural Engineering, 2019, 35(23): 177-184.
[22]	汤朋文, 陶华敏, 肖山竹, 等. 几种常用图像分割算法自适应性的分析比较[J]. 数字技术与应用, 2016, (5): 140, 143.
	TANG Pengwen, TAO Huamin, XIAO Shanzhu, et al. Analysis and comparison of adaptability of several commonly used image segmentation algorithms[J]. Digital Technology & Application, 2016, (5): 140, 143.
[23]	何晓军, 徐爱功, 李玉. 利用HSI空间相似性的彩色形态学图像处理方法[J]. 计算机科学, 2019, 46(4): 285-292.
	HE Xiaojun, XU Aigong, LI Yu. Color morphology image processing method using similarity in HSI space[J]. Computer Science, 2019, 46(4): 285-292. DOI
[24]	Codaro E N, Nakazato R Z, Horovistiz A L, et al. An image processing method for morphology characterization and pitting corrosion evaluation[J]. Materials Science and Engineering: A, 2002, 334(1/2): 298-306.
[25]	明家辉, 朱静. 基于熵加权图像梯度的单目视觉SLAM研究[J], 2022, 29(1): 91-95.
	MING Jiahui, ZHU Jing. A Study on Monocular Vision SLAM Based on Entropy Weighted Image Gradient[J]. Dongguan Journal of Technology, 2022, 29(1): 91-95.
[26]	舒丽君, 陆玲. 基于PCL库三维点云树枝重建[J]. 电脑与信息技术, 2020, 28(2): 1-4.
	SHU Lijun, LU Ling. The point cloud of branch reconstruction based on PCL library[J]. Computer and Information Technology, 2020, 28(2): 1-4.
[27]	宫德宁, 吴琳娜, 李荣华, 等. 改进直通滤波SAC-IA粗配准算法[J], 2022, 43(2): 78-84.
	GONG Dening, WU Linna, LI Ronghua, et al. Improved pass through filter SAC-IA coarse registration algorithm[J]. Journal of Dalian Jiaotong University, 2022, 43(2): 78-84.
[28]	冯书庆, 王志勇, 谌志新, 等. 基于图像特征的模板匹配成鱼多目标识别算法研究与程序设计[J], 2023, 50(2): 74-84.
	FENG Shuqing, WANG Zhiyong, CHEN Zhixin, et al. Research and Program Design on Template Matching for Adult Fish Multi target Recognition Algorithm Based on Image Features[J]. Fisheries Modernization, 2023, 50(2): 74-84.
[29]	Shao C C, Li X L, Li F, et al. Large Mask Image Completion with Conditional GAN[J]. Frontiers in Plant Science, 2022, 7:745.

Research on recognition and localization of unordered stacked watermelons based on machine vision

基于机器视觉的无序堆放西瓜识别与定位技术

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 29

Related Articles 15

Recommended Articles 0

Metrics

[1]	ZHANG Wei, YANG Guohui, YU Hui. Effects of 2,4-epibrassinolide on growth and related genes expression of watermelon seedlings under drought Stress [J]. Xinjiang Agricultural Sciences, 2024, 61(3): 615-622.
[2]	QI Yacong, CHEN Yifei, YANG Huimin, WANG Xuenong. Research and test of target detection system based on 3D laser point cloud [J]. Xinjiang Agricultural Sciences, 2023, 60(11): 2833-2841.
[3]	Zhenghe YANG, Chen YU, Huimin YANG, Yifei CHEN, Xin ZHOU, Yan MA, Xuenong WANG. Geometric Parameters Extraction of Tomato Canopy in Greenhouse Based on LiDAR [J]. Xinjiang Agricultural Sciences, 2021, 58(10): 1909-1917.
[4]	HU Guozhi, XIONG Tao, WU Haibo, FENG Jiongxin, XU Yongyang. Effects of Chemical Fertilizer and NAM Long-Acting Agent on Nitrogen and Phosphorus Uptake and Utilization and Yield and Quality of Small Fruit Watermelonii [J]. Xinjiang Agricultural Sciences, 2020, 57(3): 470-476.
[5]	LI Meihua, YANG Yong, MA Xinli, ZHANG Xuejun, ZHANG Hong, ZHANG Yongbing, YI Hongping. The Identification of Purity of Hybrid Watermelon Cultivar Zaojia 8424 Using High-Throughput Screening Based on SSR Markers [J]. Xinjiang Agricultural Sciences, 2020, 57(3): 464-469.
[6]	HE Wei-zhong, WEI Kai-li, HUA Zhen-yu, SUN Tao, CHEN He, WANG Cheng. Preliminary Study on Nutritional Quality Differences and Evaluation Indexes of Different Watermelon Varieties [J]. Xinjiang Agricultural Sciences, 2019, 56(9): 1652-1658.
[7]	LIN Ming, DENG Chao-hong, PAN Jing-hai, CHENG You-qiang, LIU Hua-jun, BAI Xiao-shan, XIAO Li, LI Cheng-ye. Diversity of Seed-Watermelon Germplasms in Xinjiang Based on ISSR Markers Analysis [J]. Xinjiang Agricultural Sciences, 2018, 55(9): 1608-1615.
[8]	LIU Hua-jun, CHEN You-qiang, LIN Ming, BAI Xiao-shan, DENG Chao-hong, PAN Jing-hai, LI Cheng-ye. Selection of Superior Varieties (Lines) of Seed-watermelon Based on Growth Pperiod and Economic Yield [J]. Xinjiang Agricultural Sciences, 2018, 55(3): 430-438.
[9]	HAN Hong-wei, WANG Hao, SHE Jian-hua, WANG Qiang, ZHUANG Hong-mei, LIU Hui-fang, LI Ning. Using SRAP Markers for Rapid Detection of Zaojia(84-24) Watermelon Hybrid Seed Purity [J]. Xinjiang Agricultural Sciences, 2017, 54(9): 1621-1626.
[10]	GUO Zheng;SHI Jian-xin;ZHOU Jun;CAI Jian;LIU Hang. Software Design of Machine Vision On-Line Detection of Walnut Size [J]. , 2017, 54(3): 538-545.
[11]	LIN Ming;LI Chun-yan;BAI Xiao-shan;PAN Jing-hai;CHEN You-qiang;LIU Hua-jun;DENG Chao-hong;WANG Juan;LI Cheng-ye. Study on the Optimal Cultural Conditions for Pollen Viability of Seed-Watermelon [J]. , 2017, 54(3): 429-433.
[12]	HAN Shen;YANG Du;Yushanjiang Maimaiti;LI Cheng-ye;PAN Jing-hai;LI Yan-e;LIU Tong-ye. Molecular Identification of Acidovorax avenae Subsp.Citrulli of Edible Seed Watermelon and Melon in Xinjiang [J]. , 2015, 52(9): 1631-1639.
[13]	YAN Guo-rong;WANG Wei;BAI Yu-ting;LIU Zhi-yong;MA Yan-ming;XIAO Jing. Study on Constitution of Database of Watermelon Varieties of Common Knowledge [J]. , 2014, 51(8): 1548-1555.
[14]	MIAO Li;GONG Biao;WANG Xiu-feng;WEI Min;YANG Feng-juan;LI Yan;SHI Qing-hua. Different Physiological Response of Watermelon Cultivars to NaHCO3 Stress [J]. , 2014, 51(6): 1097-1105.
[15]	GAO Qiang;WANG Hui-lin;ZHENG Jian;XIANG Yin-chun. Effects of Fruiting Node and Diploid Pollinators on Colored Aborted Seed and Single Fruit Weight of Triploid Watermelon [J]. , 2014, 51(10): 1797-1802.