基于机器视觉的无序堆放西瓜识别与定位技术

doi:10.6048/j.issn.1001-4330.2024.07.029

新疆农业科学 ›› 2024, Vol. 61 ›› Issue (7): 1805-1813.DOI: 10.6048/j.issn.1001-4330.2024.07.029

• 草业·农业装备工程与机械化·畜牧兽医 • 上一篇下一篇

基于机器视觉的无序堆放西瓜识别与定位技术

吴风扬¹^,²(), 黄文倩²(), 田喜², 杨育霖²

1.新疆农业大学机电工程学院,乌鲁木齐 830052
2.北京市农林科学院智能装备技术研究中心/国家农业智能装备工程技术研究中心,北京 100097

收稿日期:2023-12-11 出版日期:2024-07-20 发布日期:2024-09-04
通信作者: 黄文倩(1980-),男,福建平和人,研究员,博士,硕士生导师,研究方向为农产品质量与安全快速无损检测,(E-mail)huangwq@nercita.org.cn
作者简介:吴风扬(1999-),男,湖北黄石人,硕士研究生,研究方向为机器视觉,(E-mail)1519100570@qq.com
基金资助:
国家重点研发计划项目“移动式果品品质智能感知设备与关键环节机器人研发”(2022YFD2002202);国家西甜瓜产业技术体系专项资金项目(CARS-25-07);北京市农林科学院创新专项新兴学科交叉项目“西甜瓜品质无损检测技术与装备研发”(KJCX20230206)

Research on recognition and localization of unordered stacked watermelons based on machine vision

WU Fengyang¹^,²(), HUANG Wenqian²(), TIAN Xi², YANG Yulin²

1. College of Mechanical and Electrical Engineering, Xinjiang Agricultural University, Urumqi 830052, China
2. Intelligent Equipment Technology Research Center of Beijing Academy of Agricultural and Forestry Sciences/National Agricultural Intelligent Equipment Engineering Technology Research Center, Beijing 100097, China

Received:2023-12-11 Published:2024-07-20 Online:2024-09-04
Supported by:
The national key research and development plan project "Research and development of mobile fruit quality intelligent perception equipment and key link robots"(2022YFD2002202);Special Fund Project for National Watermelon Industry Technology System(CARS-25-07);Beijing Academy of Agricultural and Forestry Sciences Innovation Special Emerging Discipline Intersection Project"esearch and Development of Non destructive Testing Technology and Equipment for Watermelon Quality"(KJCX20230206)

摘要/Abstract

摘要：

【目的】研究一种基于机器视觉的西瓜识别与定位技术,满足西瓜搬运设备的自动化要求,为自动搬运设备的视觉部分提供技术支持。【方法】采集西瓜的二维图像数据与三维点云数据。调用OpenCV函数库分割西瓜二维图像,提取出西瓜的外轮廓;采用PCL函数对三维点云数据进行预处理,并与二维图像作图形特征匹配,提取出顶层西瓜的质心点。【结果】单轮次的西瓜轮廓总识别率为97.62%,正确识别率为95.58%,质心识别率为92.72%。20轮测试的总轮廓识别率为98.02%,总正确识别率为96.53%,总质心识别率为94.17%,总质心个数和总西瓜个数之间的误差率为2.36%;西瓜图像处理的总用时为101.8s,效率提高了77.8%。【结论】基于机器视觉的西瓜识别与定位技术识别率较高且误差率较低。

关键词: 西瓜; 识别与定位; 质心提取; 图像处理; 点云; OpenCV; PCL

Abstract:

【Objective】To study a watermelon recognition and positioning technology based on machine vision to meet the automation requirements of watermelon handling equipment. 【Methods】2D image data and 3D point cloud data of watermelon were collected and the OpenCV function library was called to segment the two-dimensional image of watermelon and extract the outer contour of the watermelon; The 3D point cloud data was preprocessed using the PCL function and perform graphic feature matching with the 2D image to extract the centroid points of the top layer watermelon.【Results】The validation test results showed that the total recognition rate of the number of watermelons in a single round was 97.62%, the correct recognition rate was 95.58%, and the centroid recognition rate was 92.72%. The total recognition rate of 20 rounds of testing was 98.02%, the total correct recognition rate was 96.53%, the total centroid recognition rate was 94.17%, and the error rate between the total centroid number and the total number of watermelons was 2.36%; The total time for watermelon image processing was 101.8 seconds, which improved the efficiency by 77.8%.【Conclusion】The watermelon recognition and positioning method proposed in this article has high recognition rate and low error rate, which can provide technical support and algorithm reference for the visual part of automatic handling equipment.

Key words: watermelon; identification and positioning; centroid extraction; image processing; point cloud; OpenCV; PCL

中图分类号:

S126
S651

吴风扬, 黄文倩, 田喜, 杨育霖. 基于机器视觉的无序堆放西瓜识别与定位技术[J]. 新疆农业科学, 2024, 61(7): 1805-1813.

WU Fengyang, HUANG Wenqian, TIAN Xi, YANG Yulin. Research on recognition and localization of unordered stacked watermelons based on machine vision[J]. Xinjiang Agricultural Sciences, 2024, 61(7): 1805-1813.

图/表 13

图1 试验平台

Fig.1 Test platform diagram

图2 图像处理流程

Fig.2 Image processing process

图3 滤波比对

Fig.3 Filter comparison chart

图4 二值化比对

Fig.4 Binary comparison chart

图5 形态学处理流程

Fig.5 Morphological processing flow

图6 西瓜外轮廓提取过程

Fig.6 Watermelon contour extraction process

图7 点云精简前后对比

Fig.7 Comparison of point cloud simplification before and after

图8 顶层西瓜的位置确认

Fig.8 Confirmation of the position of the top watermelon

图9 特征匹配流程

Fig.9 Feature matching flowchart

图10 三维图像信息融合

Fig.10 3D image information fusion

图11 顶层西瓜质心提取

Fig.11 Top Watermelon Centroid Extraction

表1 单轮次顶层西瓜识别率统计

Tab.1 Single round top watermelon recognition rate statistics

拍照次数 Number of photos taken	顶层西瓜总数 Total number of top watermelon	西瓜轮廓数 Watermelon outline number	西瓜轮廓识别率 Watermelon contour recognition rate (%)	顶层西瓜数 Number of top watermelons	顶层西瓜识别率 Top watermelon recognition rate (%)
第1次 1st 第2次 2nd 第3次 3rd 第4次 4th 第5次 5th 第6次 6th 第7次 7th 合计 Total	7 6 5 4 3 2 1 28	7 5 5 4 3 2 1 27	100 83.33 100 100 100 100 100 97.62	6 5 5 4 3 2 1 26	85.71 83.33 100 100 100 100 100 95.58

表2 单轮次质心识别率统计

Tab.2 Statistics of single round centroid recognition rate

拍照次数 Number of photos taken	顶层西瓜总数 Total number of top watermelon	质心个数 Number of centroicls	质心识别率 Centroid recognition rate (%)	人工识别率 Artificial recognition rate (%)
第1次1st 第2次2nd 第3次3rd 第4次4th 第5次5th 第6次6th 第7次7th 合计Total	7 6 5 4 3 2 1 28	6 5 4 4 3 2 1 25	85.71 83.33 80 100 100 100 100 92.72	100

参考文献 29

[1]	钱曼, 黄文倩, 王庆艳, 等. 西瓜检测部位差异对近红外光谱可溶性固形物预测模型的影响[J]. 光谱学与光谱分析, 2016, 36(6): 1700-1705, 1711.
	QIAN Man, HUANG Wenqian, WANG Qingyan, et al. Assessment of influence detective position variability on precision of near infrared models for soluble solid content of watermelon[J]. Spectroscopy and Spectral Analysis, 2016, 36(6): 1700-1705, 1711.
[2]	周艺璇. 西瓜翠衣的营养保健成分及药用价值研究进展[J]. 现代食品, 2022, 28(8): 51-53, 57.
	ZHOU Yixuan. Research progress on nutritional and health components and medicinal value of watermelon peel[J]. Modern Food, 2022, 28(8): 51-53, 57.
[3]	李干琼, 王志丹. 我国西瓜产业发展现状及趋势分析[J]. 中国瓜菜, 2019, 32(12): 79-83.
	LI Ganqiong, WANG Zhidan. Development status and trend analysis of watermelon industry in China[J]. China Cucurbits and Vegetables, 2019, 32(12): 79-83.
[4]	宋展, 胡宝贵. 后疫情时期消费者视角的北京市西瓜产业发展[J]. 北方园艺, 2022, (23): 134-139.
	SONG Zhan, HU Baogui. Development of watermelon industry in Beijing city from the perspective of consumers in the post epidemic period[J]. Northern Horticulture, 2022, (23): 134-139.
[5]	高丽娜. 西瓜采收技术[J]. 河南农业, 2015, (9): 46.
	GAO Lina. Watermelon Harvesting Techniques[J]. Henan Agriculture, 2015, (9): 46.
[6]	官大文, 王春源, 王骏发. 基于无人机机器视觉的西瓜侦测创新农场管理模式[J]. 海峡科学, 2020, (11): 62-68.
	GUAN Dawen, WANG Chunyuan, WANG Junfa. Innovative farm management mode of watermelon detection based on UAV machine vision[J]. Straits Science, 2020, (11): 62-68. [万方]
[7]	Kamaldeen Mohammed, Evans Batung, Ansumah Saaka-Sulemana, et al. Determinants of mechanized technology adoption in smallholder agriculture: Implications for agricultural policy[J]. Neurocomputing, 2023, 18(3): 545-557.
[8]	唐芳. 我国农机购置补贴与农业机械化发展现状与提升建议[J]. 农业经济, 2023, (4): 9-12.
	TANG Fang. China's agricultural machinery purchase subsidies and Mechanised agriculture development status and improvement suggestions[J]. Agricultural Economy, 2023, (4): 9-12.
[9]	罗辉, 崔亚飞. 基于机器视觉的工件识别与定位系统设计与实现[J]. 组合机床与自动化加工技术, 2021, (1): 101-105, 109. DOI
	LUO Hui, CUI Yafei. Design and implementation of workpiece recognition and positioning system based on machine vision[J]. Modular Machine Tool & Automatic Manufacturing Technique, 2021, (1): 101-105, 109.
[10]	崔旭东, 谭欢, 王平江, 等. 基于3D视觉的散乱堆放工件机器人抓取技术研究[J]. 制造技术与机床, 2021, (2): 36-41. DOI
	CUI Xudong, TAN Huan, WANG Pingjiang, et al. Research on vision-based robot’s grabbing attitude for scattered stacking workpieces[J]. Manufacturing Technology & Machine Tool, 2021, (2): 36-41.
[11]	Xu B R, Chai L, Zhang C L. Research and application on corn crop identification and positioning method based on Machine vision[J]. Precision Agriculture, 2023.
[12]	柴宏红, 邵科, 于超, 等. 基于三维点云的甜菜根表型参数提取与根型判别[J]. 农业工程学报, 2020, 36(10): 181-188.
	CHAI Honghong, SHAO Ke, YU Chao, et al. Extraction of phenotypic parameters and discrimination of beet root types based on 3D point cloud[J]. Transactions of the Chinese Society of Agricultural Engineering, 2020, 36(10): 181-188.
[13]	宋晨旭, 于翀宇, 邢永超, 等. 基于OpenCV的大豆籽粒多表型参数获取算法[J]. 农业工程学报, 2022, 38(20): 156-163.
	SONG Chenxu, YU Chongyu, XING Yongchao, et al. Algorith for acquiring multi-phenotype parameters of soybean seed based on OpenCV[J]. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38(20): 156-163.
[14]	刘万辉, 杨晔. 温室场景下成熟西瓜的图像识别研究[J]. 福建电脑, 2013, 29(5): 29-31.
	LIU Wanhui, YANG Ye. Study on image recognition of mature watermelon in greenhouse scene[J]. Journal of Fujian Computer, 2013, 29(5): 29-31.
[15]	吴明清, 王丽. 基于机器视觉技术的西瓜瓜瓤的识别方法[J]. 塔里木大学学报, 2014, 26(3): 88-91.
	WU Mingqing, WANG Li. Identification of the machine vision technology based on watermelon flesh[J]. Journal of Tarim University, 2014, 26(3): 88-91.
[16]	刘德儿, 朱磊, 冀炜臻, 等. 基于RGB-D相机的脐橙实时识别定位与分级方法[J]. 农业工程学报, 2022, 38(14):154-165.
	LIU Deer, ZHU Lei, JI Weizhen, et al. Real time recognition, localization, and grading method for navel oranges based on RGB-D cameras[J]. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38(14):154-165.
[17]	Liu K, Jiang K, Mu Y. Development of a vision guided robot for grafting watermelon based on hole insertion method[J]. Computers and Electronics in Agriculture, 2020.
[18]	Mohd Ali M, Hashim N, Bejo S K, et al. Comparison of laser backscattering imaging and computer vision system for grading of seedless watermelons[J]. Journal of Food Measurement and Characterization, 2020, 14(1): 69-77.
[19]	邹宏伟. 基于OpenCV的数字图像处理技术研究与实现[J]. 无线互联科技, 2019, 16(22): 118-119.
	ZOU Hongwei. Research and implementation of digital image processing technology based on OpenCV[J]. Wireless Internet Technology, 2019, 16(22): 118-119.
[20]	龙嫣然, 徐昱琳, 费心怡. 基于PCL的Jaco机械臂的目标抓取[J]. 工业控制计算机, 2019, 32(4): 65-67.
	LONG Yanran, XU Yulin, FEI Xinyi. PCL-based object grasping by robot arm hand system[J]. Industrial Control Computer, 2019, 32(4): 65-67.
[21]	刘星星, 王烁烁, 徐丽明, 等. 基于OpenCV的动态葡萄干色泽实时识别[J]. 农业工程学报, 2019, 35(23): 177-184.
	LIU Xingxing, WANG Shuoshuo, XU Liming, et al. Real time color recognition of moving raisin based on OpenCV[J]. Transactions of the Chinese Society of Agricultural Engineering, 2019, 35(23): 177-184.
[22]	汤朋文, 陶华敏, 肖山竹, 等. 几种常用图像分割算法自适应性的分析比较[J]. 数字技术与应用, 2016, (5): 140, 143.
	TANG Pengwen, TAO Huamin, XIAO Shanzhu, et al. Analysis and comparison of adaptability of several commonly used image segmentation algorithms[J]. Digital Technology & Application, 2016, (5): 140, 143.
[23]	何晓军, 徐爱功, 李玉. 利用HSI空间相似性的彩色形态学图像处理方法[J]. 计算机科学, 2019, 46(4): 285-292.
	HE Xiaojun, XU Aigong, LI Yu. Color morphology image processing method using similarity in HSI space[J]. Computer Science, 2019, 46(4): 285-292. DOI
[24]	Codaro E N, Nakazato R Z, Horovistiz A L, et al. An image processing method for morphology characterization and pitting corrosion evaluation[J]. Materials Science and Engineering: A, 2002, 334(1/2): 298-306.
[25]	明家辉, 朱静. 基于熵加权图像梯度的单目视觉SLAM研究[J], 2022, 29(1): 91-95.
	MING Jiahui, ZHU Jing. A Study on Monocular Vision SLAM Based on Entropy Weighted Image Gradient[J]. Dongguan Journal of Technology, 2022, 29(1): 91-95.
[26]	舒丽君, 陆玲. 基于PCL库三维点云树枝重建[J]. 电脑与信息技术, 2020, 28(2): 1-4.
	SHU Lijun, LU Ling. The point cloud of branch reconstruction based on PCL library[J]. Computer and Information Technology, 2020, 28(2): 1-4.
[27]	宫德宁, 吴琳娜, 李荣华, 等. 改进直通滤波SAC-IA粗配准算法[J], 2022, 43(2): 78-84.
	GONG Dening, WU Linna, LI Ronghua, et al. Improved pass through filter SAC-IA coarse registration algorithm[J]. Journal of Dalian Jiaotong University, 2022, 43(2): 78-84.
[28]	冯书庆, 王志勇, 谌志新, 等. 基于图像特征的模板匹配成鱼多目标识别算法研究与程序设计[J], 2023, 50(2): 74-84.
	FENG Shuqing, WANG Zhiyong, CHEN Zhixin, et al. Research and Program Design on Template Matching for Adult Fish Multi target Recognition Algorithm Based on Image Features[J]. Fisheries Modernization, 2023, 50(2): 74-84.
[29]	Shao C C, Li X L, Li F, et al. Large Mask Image Completion with Conditional GAN[J]. Frontiers in Plant Science, 2022, 7:745.

编辑推荐 0

Metrics

阅读次数

全文

HTML			PDF

最新录用	在线预览	正式出版	最新录用	在线预览	正式出版
0	0	6	0	0	29

来源	本网站	其他网站

次数	21	14
比例	60%	40%

摘要

107

最新录用	在线预览	正式出版

0	0	107

来源	本网站	其他网站

次数	32	75
比例	30%	70%

基于机器视觉的无序堆放西瓜识别与定位技术

Research on recognition and localization of unordered stacked watermelons based on machine vision

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 29

相关文章 15

编辑推荐 0

Metrics

[1]	张伟, 杨国慧, 于辉. 2,4-表油菜素内酯对干旱胁迫下西瓜幼苗生长及相关基因表达的影响[J]. 新疆农业科学, 2024, 61(3): 615-622.
[2]	宋金迪, 刘君, 孙玉芳, 优丽图孜·乃比, 陈宝强, 颉兵兵. 铜胁迫下的西瓜食酸菌转录组分析[J]. 新疆农业科学, 2023, 60(2): 389-398.
[3]	齐亚聪, 陈毅飞, 杨会民, 王学农. 基于三维激光点云的靶标探测系统研究与试验[J]. 新疆农业科学, 2023, 60(11): 2833-2841.
[4]	杨征鹤, 喻晨, 杨会民, 陈毅飞, 周欣, 马艳, 王学农. 基于LiDAR的温室番茄冠层几何参数提取[J]. 新疆农业科学, 2021, 58(10): 1909-1917.
[5]	李寐华, 杨永, 马新力, 张学军, 张红, 张永兵, 伊鸿平. 基于SSR分子标记对西瓜杂交种早佳8424纯度的高通量鉴定[J]. 新疆农业科学, 2020, 57(3): 464-469.
[6]	胡国智, 熊韬, 吴海波, 冯炯鑫, 徐永阳. 化肥配施NAM长效剂对小果型西瓜氮、磷吸收利用及产量品质的影响[J]. 新疆农业科学, 2020, 57(3): 470-476.
[7]	何伟忠, 韦凯丽, 华震宇, 孙涛, 陈贺, 王成. 不同品种西瓜营养质量的差异及评价指标[J]. 新疆农业科学, 2019, 56(9): 1652-1658.
[8]	林明, 邓超宏, 潘竟海, 陈友强, 刘华君, 白晓山, 肖丽, 李承业. 新疆籽用西瓜ISSR反应体系的建立及其遗传多样性研究[J]. 新疆农业科学, 2018, 55(9): 1608-1615.
[9]	黄成文,王希东,刘君,杨东晟,古妮萨罕·麦托合提, 热孜宛·奥布力. 两种表达载体对西瓜食酸菌生物学特性的影响[J]. 新疆农业科学, 2018, 55(3): 457-467.
[10]	韩宏伟, 王浩, 佘建华, 王强, 庄红梅, 刘会芳, 李宁. 利用SRAP标记快速检测西瓜杂交种子纯度[J]. 新疆农业科学, 2017, 54(9): 1621-1626.
[11]	郭政;史建新;周军;蔡建;刘航. 机器视觉在线检测核桃大小软件设计[J]. , 2017, 54(3): 538-545.
[12]	胡金凤;黄成文;刘君. 西瓜食酸菌TA基因酵母双杂交诱饵载体的构建及其自激活作用的检测[J]. , 2017, 54(2): 289-296.
[13]	韩盛;韩成贵;玉山江·麦麦提;杨军;杨渡. 新疆吐鲁番地区三种甜瓜病毒病的发生与分子鉴定[J]. , 2016, 53(10): 1829-1842.
[14]	何丹;海利力·库尔班;玉山江·麦麦提;李继阳;杨渡;韩盛. 西瓜花叶病毒(WMV)外壳蛋白基因克隆及序列分析[J]. , 2015, 52(10): 1849-1858.
[15]	颜国荣;王威;白玉亭;刘志勇;马艳明;肖菁. 西瓜已知品种DUS性状数据库构建与应用[J]. , 2014, 51(8): 1548-1555.