Maize Seedling and Core Detection Method Based on Mask R-CNN

doi:10.6048/j.issn.1001-4330.2021.10.020

Abstract

Abstract:

【Objective】 In view of the problem of low recognition accuracy of corn seedlings and plant cores in the actual field environment, a corn seedling canopy segmentation algorithm based on improved Mask R-CNN is proposed in the hope of satisfying the identification requirements for target fertilization in precision operations, thus improving the use efficiency of chemical fertilizers, and reducing environmental pollution. 【Methods】 Firstly, the field corn seedling pictures were collected and the data were enhanced to generate the field data set. Secondly, the segmentation algorithm was trained by using ResNeXt50/101-FPN as feature extraction network, and compared with the training accuracy results of the original ResNet50/101-FPN. Finally, the canopy recognition algorithm was compared and verified by pictures with different light intensities and different degrees of occlusion. 【Results】 Under different light intensities, the average recognition accuracy of the target without accompanying weeds was higher than 95.5%, and the segmentation accuracy was 98.1%; when there was an overlap between the associated weeds and corn seedlings, the average recognition accuracy of the target was higher than 94.7%, and the segmentation accuracy reached 97.9%. After image testing, the average time to detect one frame of image was 0.11 s. 【Conclusion】 The results show that the maize seedling and plant core detection algorithm of Mask R-CNN in this paper has higher accuracy and segmentation accuracy, which is more suitable for target detection in different light intensities and crossover overlap of seedlings and grass with associated weeds.

Key words: maize seedling; precise fertilization; Residual Network; seedling stage; target recognition; image segmentation

摘要：

【目的】研究基于改进Mask R-CNN的玉米苗冠层分割算法,满足精准作业中对靶施肥的识别要求,提高化肥的使用效率,减少环境污染。【方法】采集田间玉米苗图片并增强数据,生成田间数据集;使用ResNeXt50/101-FPN作为特征提取网络对分割算法进行训练,并与原始ResNet50/101-FPN的训练精度结果作对比;采用不同光照强度及有伴生杂草的玉米苗图片对比验证冠层识别算法效果。【结果】在不同光照强度下,无伴生杂草的目标平均识别精度高于95.5%,分割精度达98.1%;在有伴生杂草与玉米苗有交叉重合情况下,目标平均识别精度高于94.7%,分割精度达97.9%。检测一帧图像的平均时间为0.11 s。【结论】Mask R-CNN的玉米苗及株芯检测算法有更高的准确率和分割精度,更能适应不同光照强度及有伴生杂草的苗草交叉重合情况的目标检测。

关键词: 玉米苗, 精准施肥, 残差网络, 苗期, 目标识别, 图像分割

CLC Number:

S513

Weirong ZHANG, Haojun WEN, Chaofan QIAO, Guangyan WANG. Maize Seedling and Core Detection Method Based on Mask R-CNN[J]. Xinjiang Agricultural Sciences, 2021, 58(10): 1918-1928.

张伟荣, 温浩军, 谯超凡, 汪光岩. 基于Mask R-CNN的玉米苗与株芯检测方法[J]. 新疆农业科学, 2021, 58(10): 1918-1928.

Figures/Tables 13

References 27

[1]	李昌新. 长期施肥对红壤旱地玉米生产力和土壤肥力的影响及其机制研究[D]. 南京:南京农业大学, 2009.
	LI Changxin. Effects of long-term fertilization on corn productivity and soil fertility as well as its mechanisms under upland red soil in subtropical China[D]. Nanjing: Nanjing Agricultural University, 2009.
[2]	陈学庚, 温浩军, 张伟荣, 等. 农业机械与信息技术融合发展现状与方向[J]. 智慧农业(中英文), 2020, 2(4):1-16.
	CHEN Xuegeng, WEN Haojun, ZHANG Weirong, et al. Advances and progress of agricultural machinery and sensing technology fusion[J]. Smart Agriculture, 2020, 2(4):1-16.
[3]	张继成. 基于处方图的变量施肥系统关键技术研究[D]. 哈尔滨:东北农业大学, 2013.
	ZHNG Jicheng. Research on key technologies of variable fertilizer system based on the prescription[D]. Harbin: Northeast Agricultural University, 2013.
[4]	傅隆生, 冯亚利, ElkamilTola, 等. 基于卷积神经网络的田间多簇猕猴桃图像识别方法[J]. 农业工程学报, 2018, 34(2):205-211.
	FU Longsheng, FENG Yali, Eikamil T, et al. Image recognition method of multi-cluster kiwifruit in field based on convolutional neural networks[J]. Transactions of the CSAE, 2018, 34(2):205-211.
[5]	闫建伟, 赵源, 张乐伟, 等. 基于残差网络的自然环境中刺梨果实的识别[J]. 中国农机化学报, 2020, 41(10):191-196.
	YAN Jianwei, ZHAO Yuan, ZHANG Lewei, et al. Recognition of Rosa roxbunghii fruit in natural environment based on residual network[J]. Journal of Chinese Agricultural Mechanization, 2020, 41(10):191-196.
[6]	张磊, 姜军生, 李昕昱, 等. 基于快速卷积神经网络的果园果实检测试验研究[J]. 中国农机化学报, 2020, 41(10):183-190, 210.
	ZHANG Lei, JIANG Junsheng, LI Xinyu, et al. Experimental research on orchard fruit detection based on fast convolutional neural network[J]. Journal of Chinese Agricultural Mechanization, 2020, 41(10):183-190, 210.
[7]	王振, 师韵, 李玉彬. 基于改进全卷积神经网络的玉米叶片病斑分割[J]. 计算机工程与应用, 2019, 55(22):127-132.
	WANG Zhen, SHI Yun, LI Yubin. Segmentation of corn leaf diseases based on improved fully convolutional neural network[J]. Computer Engineering and Applications, 2019, 55(22):127-132.
[8]	李就好, 林乐坚, 田凯, 等. 改进Faster R-CNN的田间苦瓜叶部病害检测[J]. 农业工程学报, 2020, 36(12):179-185.
	LI Jiuhao, LIN Lejian, TIAN Kai, et al. Detection of leaf diseases of balsam pear in the field based on improved Faster R-CNN[J]. Transactions of the Chinese Society for Agricultural Machinery, 2020, 36(12):179-185.
[9]	许景辉, 邵明烨, 王一琛, 等. 基于迁移学习的卷积神经网络玉米病害图像识别[J]. 农业机械学报, 2020, 51(2):230-236,253.
	XU Jinghui, SHAO Mingye, WANG Yichen, et al. Recognition of corn leaf spot and rust based on transfer learning with convolutional neural network[J]. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(2):230-236,253.
[10]	张春龙, 黄小龙, 刘卫东, 等. 苗间锄草机器人信息获取方法的研究[J]. 农业工程学报, 2012, 28(9):142-146.
	ZHANG Chunlong, HUANG Xiaolong, LIU Weidong, et al. Information acquisition method for mechanical intra-row weeding robot[J]. Transactions of the CSAE, 2012, 28(9):142-146.
[12]	赵鹏, 韦兴竹. 基于多特征融合的田间杂草分类识别[J]. 农业机械学报, 2014, 45(3):275-281.
	ZHAO Peng, WEI Xingzhu. Weed recognition in agricultural field using multiple feature fusions[J]. Transactions of the Chinese Society for Agricultural Machinery, 2014, 45(3):275-281.
	孙俊, 谭文军, 武小红, 等. 多通道深度可分离卷积模型实时识别复杂背景下甜菜与杂草[J]. 农业工程学报, 2019, 35(12):184-190.
	SUN Jun, TAN Wenjun, WU Xiaohong, et al. Real-time recognition of sugar beet and weeds in complex backgrounds using multi-channel depth-wise separable convolution model[J]. Transactions of the CSAE, 2019, 35(12):184-190.
[13]	苗荣慧, 杨华, 武锦龙, 等. 基于图像分块及重构的菠菜重叠叶片与杂草识别[J]. 农业工程学报, 2020, 36(4):178-184.
	MIAO Ronghui, YANG Hua, WU Jinlong, et al. Weed identification of overlapping spinach leaves based on image sub-block and reconstruction[J]. Transactions of the CSAE, 2020, 36(4):178-184.
[14]	胡炼, 罗锡文, 曾山, 等. 基于机器视觉的株间机械除草装置的作物识别与定位方法[J]. 农业工程学报, 2013, 29(10):12-18.
	HU Lian, LUO Xiwen, ZENG Shan, et al. Plant recognition and localization for intra-row mechanical weeding device based on machine vision[J]. Transactions of the CSAE, 2013, 29(10):12-18.
[15]	魏帅均, 张彦娥, 梅树立. 基于俯视图像的玉米株心快速识别方法研究[J]. 农业机械学报, 2017, 48(S1):136-141.
	WEI Shuaijun, ZHANG Yan'e, MEI Shuli. Fast recognition method of maize core based on top view image[J]. Transactions of the Chinese Society for Agricultural Machinery, 2017, 48(S1) :136-141.
[16]	孙哲, 张春龙, 葛鲁镇, 等. 基于Faster R-CNN的田间西兰花幼苗图像检测方法[J]. 农业机械学报, 2019, 50(7):216-221.
	SUN Zhe, ZHANG Chunlong, GE Luzhen, et al. Image detection method for broccoli seedlings in the field based on Faster R-CNN[J]. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(7):216-221.
[17]	宗泽, 赵硕, 刘刚. 苗期玉米冠层识别与质心定位方法研究[J]. 农业机械学报, 2019, 50(S1):27-33.
	ZONG Ze, ZHAO Shuo, LIU Gang. Coronal identification and centroid location of maize seedling stage[J]. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(S1):27-33.
[18]	Russell B C, Torralba A, Murphy K P, et al. LabelMe: A database and web-based tool for image annotation[J]. International Journal of Computer Vision, 2008, 77(1-3):157-173. DOI URL
[19]	HE K, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017(99):2961-2969.
[20]	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(6):1137-1149. DOI URL
[21]	LONG J, SHELHAMER E, DARRELL T. Fully Convolutional Networks for Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014, 39(4):640-651.
[22]	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]. Computer Vision and Pattern Recognition. IEEE, 2016: 770-778.
[23]	Xie S, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks[C]. Computer Vision and Pattern Recognition. IEEE, 2017: 1492-1500.
[24]	Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[J]. Computer Science, 2014.
[25]	Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the Inception Architecture for Computer Vision[J]. Computer Science, 2016:2818-2826.
[26]	孟欣欣, 阿里甫·库尔班, 吕情深, 等. 基于迁移学习的自然环境下香梨目标识别研究[J]. 新疆大学学报(自然科学版), 2019, 36(4):461-467.
	MENG Xinxin, Alifu Kuerban, LÜ Qingshen, et al. Research on Fragrant Pear Target Recognition in Natural Environment Based on Transfer Learning[J]. Journal of Xinjiang Normal University (Natural Sciences Ed.), 2019, 36(4):461-467.
[27]	ZongZ, Liu G, Zhao S. Real-Time Localization Approach for Maize Cores at Seedling Stage Based on Machine Vision[J]. 2020.

卷积主干 Convolution backbone	平均精度 AP		平均精度均值 mAP	训练时间 Training time (min)	平均检测时间 Average detection time (ms)
卷积主干 Convolution backbone	玉米叶片 The maize leaf	玉米芯 The maize core	平均精度均值 mAP	训练时间 Training time (min)	平均检测时间 Average detection time (ms)
ResNet50-FPN	0.801	0.255	0.528	170	53
ResNeXt50-FPN	0.806	0.255	0.531	217	55
ResNet101-FPN	0.816	0.266	0.541	224	61
ResNeXt101-FPN	0.812	0.281	0.547	349	67

卷积主干 Convolution backbone	平均精度 AP		平均精度均值 mAP	训练时间 Training time (min)	平均检测时间 Average detection time (ms)
卷积主干 Convolution backbone	玉米叶片 The maize leaf	玉米芯 The maize core	平均精度均值 mAP	训练时间 Training time (min)	平均检测时间 Average detection time (ms)
ResNet50-FPN	0.801	0.255	0.528	170	53
ResNeXt50-FPN	0.806	0.255	0.531	217	55
ResNet101-FPN	0.816	0.266	0.541	224	61
ResNeXt101-FPN	0.812	0.281	0.547	349	67

光照条件 Lighting conditions	图片数/ 玉米苗株数 Number of pictures/ maize seedlings	成功检测到的玉米苗株/玉米芯数量 Number of maize seedlings/ maize cores successfully detected		玉米苗/玉米芯识别成功率 Maize seedling/maize core identification success rate
光照条件 Lighting conditions	图片数/ 玉米苗株数 Number of pictures/ maize seedlings	ResNet101-FPN	ResNeXt101-FPN	ResNet101-FPN	ResNeXt101-FPN
阴天较弱光照 Weaker light on cloudy days	25/30	28/28	30/28	93.3%/93.3%	100%/93.3%
晴天较强光照 Stronger light on a sunny day	25/36	34/33	34/33	94.4%/91.7%	94.4%/94.4%

光照条件 Lighting conditions	图片数/ 玉米苗株数 Number of pictures/ maize seedlings	成功检测到的玉米苗株/玉米芯数量 Number of maize seedlings/ maize cores successfully detected		玉米苗/玉米芯识别成功率 Maize seedling/maize core identification success rate
光照条件 Lighting conditions	图片数/ 玉米苗株数 Number of pictures/ maize seedlings	ResNet101-FPN	ResNeXt101-FPN	ResNet101-FPN	ResNeXt101-FPN
阴天较弱光照 Weaker light on cloudy days	25/30	28/28	30/28	93.3%/93.3%	100%/93.3%
晴天较强光照 Stronger light on a sunny day	25/36	34/33	34/33	94.4%/91.7%	94.4%/94.4%

光照条件 Lighting conditions	图像序号 Image number	真实像素值 True pixel value	算法分割像素值 Pixel value obtained by algorithm segmentation		分割精度 SA
光照条件 Lighting conditions	图像序号 Image number	真实像素值 True pixel value	ResNet101-FPN	ResNeXt101-FPN	ResNet101-FPN	ResNeXt101-FPN
较弱光照 Weaker light	1	24 839	23 924	25 499	0.963	0.973
	2	17 041	16 972	17 031	0.996	0.999
	3	41 691	40 198	41 324	0.964	0.991
	4	41 337	42 952	40 336	0.961	0.976
	5	33 402	33 433	33 371	0.999	0.999
	mSA				0.977	0.988
较强光照 Stronger light	6	13 691	13 177	13 495	0.962	0.986
	7	20 764	21 801	21 649	0.95	0.957
	8	15 085	14 533	15 368	0.963	0.981
	9	20 196	20 664	20 910	0.965	0.977
	10	17 994	17 384	17 571	0.966	0.976
	mSA				0.961	0.975