Abstract

Surface defect detection is very important to ensure product quality, but most of the surface defects of industrial products are characterized by low contrast, large variation in size and category similarity, which brings challenges to the automatic detection of defects. To solve these problems, this paper proposes a defect detection method based on convolutional neural network. In this method, a backbone network with semantic supervision is applied to extract the features of different levels. While a multi-level feature fusion module is proposed to fuse adjacent feature maps into high-resolution feature maps successively, it significantly improves the prediction accuracy of the network. Finally, an encoding module is used to obtain the global context information of the high-resolution feature map, which further improves the pixel classification accuracy. Experiments show that the mean intersection of union (mIoU) of the proposed method is superior to other methods on a standardized defect detection dataset of steel strip (NEU_SEG, mIoU of 85.27%) and a magnetic-tile defect dataset (mIoU of 77.82%).

References

1.
Tsang
,
C. S. C.
,
Ngan
,
H. Y. T.
, and
Pang
,
G. K. H.
,
2016
, “
Fabric Inspection Based on the Elo Rating Method
,”
Pattern Recognit.
,
51
(
4
), pp.
378
394
.
2.
Qiwu
,
L.
,
Yichuang
,
S.
,
Pengcheng
,
L.
,
Oluyomi
,
S.
,
Lu
,
T.
, and
Yigang
,
H.
,
2018
, “
Generalized Completed Local Binary Patterns for Time-Efficient Steel Surface Defect Classification
,”
IEEE Trans. Instrum. Meas.
,
68
(
4
), pp.
667
679
.
3.
Gan
,
J.
,
Li
,
Q.
,
Wang
,
J.
, and
Yu
,
H.
,
2017
, “
A Hierarchical Extractor-Based Visual Rail Surface Inspection System
,”
IEEE Sens. J.
,
17
(
23
), pp.
7935
7944
.
4.
Dong
,
Y.
,
Tao
,
D.
,
Li
,
X.
,
Ma
,
J.
, and
Pu
,
J.
,
2015
, “
Texture Classification and Retrieval Using Shearlets and Linear Regression
,”
IEEE Trans. Cybern.
,
45
(
3
), pp.
358
369
.
5.
Zhou
,
J.
,
Huang
,
P. S.
, and
Chiang
,
F.-P.
,
2006
, “
Wavelet-Based Pavement Distress Detection and Evaluation
,”
Opt. Eng.
,
45
(
2
), pp.
409
411
.
6.
Simonyan
,
K.
, and
Zisserman
,
A.
,
2015
, “
Very Deep Convolutional Networks for Large-Scale Image Recognition
,”
International Conference on Learning Representations (ICLR) 2015
,
San Diego, CA
,
May 7–9
, pp.
1
14
.
7.
He
,
K.
,
Zhang
,
X.
,
Ren
,
S.
, and
Sun
,
J.
,
2016
, “
Deep Residual Learning for ImageRecognition
,”
IEEE Conference on Computer Vision and Pattern Recognition
,
Las Vegas, NV
,
June 26–July 1
, pp.
770
778
.
8.
Szegedy
,
C.
,
Liu
,
W.
,
Jia
,
Y.
,
Sermanet
,
P.
,
Reed
,
S.
,
Anguelov
,
D.
, and
Erhan
,
D.
,
2015
, “
Going Deeper With Convolutions
,”
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Boston, MA
,
June 7–12
, pp.
1
9
.
9.
Long
,
J.
,
Shelhamer
,
E.
, and
Darrell
,
T.
,
2015
, “
Fully Convolutional Networks for Semantic Segmentation
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
39
(
4
), pp.
640
651
.
10.
Badrinarayanan
,
V.
,
Kendall
,
A.
, and
Cipolla
,
R.
,
2017
, “
SegNet: A Deep Convolutional Encoder–Decoder Architecture for Image Segmentation
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
39
(
12
), pp.
2481
2495
.
11.
Ronneberger
,
O.
,
Fischer
,
P.
, and
Brox
,
T.
,
2015
, “
U-Net: Convolutional Networks for Biomedical Image Segmentation
,”
International Conference on Medical Image Computing and Computer-Assisted Intervention
,
Munich, Germany
,
Oct. 5–9
, pp.
234
241
.
12.
Noh
,
H.
,
Hong
,
S.
, and
Han
,
B.
,
2015
, “
Learning Deconvolution Network for Semantic Segmentation
,”
Proceedings of the IEEE International Conference on Computer Vision
,
Santiago, Chile
,
Dec. 11–18
, pp.
1520
1528
.
13.
Zhang
,
Z.
,
Zhang
,
X.
,
Peng
,
C.
,
Cheng
,
D.
, and
Sun
,
J.
,
2018
, “
ExFuse: Enhancing Feature Fusion for Semantic Segmentation
,”
Proceedings of the European Conference on Computer Vision (ECCV)
,
Munich, Germany
,
Sept. 8–14
, pp.
269
284
.
14.
Lin
,
G.
,
Milan
,
A.
,
Shen
,
C.
, and
Reid
,
I.
,
2017
, “
RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation
,”
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Honolulu, HI
,
July 21–26
, pp.
5168
5177
.
15.
Chen
,
L.-C.
,
Papandreou
,
G.
,
Schroff
,
F.
, and
Adam
,
H.
,
2017
, “
Rethinking Atrous Convolution for Semantic Image Segmentation
,”
CoRR abs/1706.05587v1, 06/17
.
16.
Chen
,
L. C.
,
Zhu
,
Y.
,
Papandreou
,
G.
,
Schroff
,
F.
, and
Adam
,
H.
,
2018
, “
Encoder–Decoder With Atrous Separable Convolution for Semantic Image Segmentation
,”
Presented at the Computer Vision and Pattern Recognition (CVPR)
,
Munich, Germany
,
Sept. 8–14
, pp.
801
818
.
17.
Guo
,
J.
,
Li
,
Y.
,
Lin
,
W.
,
Chen
,
Y.
, and
Li
,
J.
,
2018
, “
Network Decoupling: From Regular to Depthwise Separable Convolutions
,”
29th British Machine Vision Conference (BMVC 2018)
,
Newcastle, UK
,
Sept. 2–6
, https://arxiv.org/abs/1808.05517
18.
Chollet
,
F.
,
2016
, “
Xception: Deep Learning With Depthwise Separable Convolutions
,”
CoRR abs/1610.02357
.
19.
Wu
,
H.
,
Zhang
,
J.
,
Huang
,
K.
,
Liang
,
K.
, and
Yu
,
Y.
,
2019
, “
FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation
,”
CoRR abs/1903.11816
.
20.
Yibin
,
H.
,
Congying
,
Q.
, and
Kui
,
Y.
,
2018
, “
Surface Defect Saliency of Magnetic Tile
,”
Vis. Comput.
,
36
(
1
), pp.
85
96
.
21.
Zhang
,
L.
,
Yang
,
F.
,
Daniel Zhang
,
Y.
, and
Zhu
,
Y. J.
,
2016
, “
Road Crack Detection Using Deep Convolutional Neural Network
,”
Presented at the 2016 IEEE International Conference on Image Processing (ICIP)
,
Phoenix, AZ
,
Sept. 25–28
, pp.
3708
3712
.
22.
He
,
Y.
,
Song
,
K.
,
Meng
,
Q.
, and
Yan
,
Y.
,
2019
, “
An End-to-End Steel Surface Defect Detection Approach Via Fusing Multiple Hierarchical Features
,”
IEEE Trans. Instrum. Meas.
,
69
(
4
), pp.
1493
1504
.
23.
Zhang
,
D.
,
Song
,
K.
,
Xu
,
J.
,
He
,
Y.
, and
Yan
,
Y.
,
2020
, “
Unified Detection Method of Aluminium Profile Surface Defects: Common and Rare Defect Categories
,”
Opt. Lasers Eng.
,
126
, p.
105936
.
24.
Chen
,
J.
,
Liu
,
Z.
,
Wang
,
H.
,
Núñez
,
A.
, and
Han
,
Z.
,
2017
, “
Automatic Defect Detection of Fasteners on the Catenary Support Device Using Deep Convolutional Neural Network
,”
IEEE Trans. Instrum. Meas.
,
67
(
2
), pp.
257
269
.
25.
Huang
,
H. W.
,
Li
,
Q. T.
, and
Zhang
,
D. M.
,
2018
, “
Deep Learning Based Image Recognition for Crack and Leakage Defects of Metro Shield Tunnel
,”
Tunn. Undergr. Space Technol.
,
77
, pp.
166
176
.
26.
Xiong
,
L.
,
Zhang
,
D.
, and
Zhang
,
Y.
,
2019
, “
Water Leakage Image Recognition of Shield Tunnel Via Learning Deep Feature Representation
,”
J. Vis. Commun. Image Represent.
,
71
, p.
102708
.
27.
Yous
,
H.
,
Serir
,
A.
, and
Yous
,
S.
,
2019
, “
CNN-Based Method for Blotches and Scratches Detection in Archived Videos
,”
J. Vis. Commun. Image Represent.
,
59
(
2
), pp.
486
500
.
28.
Luo
,
Q.
,
Gao
,
B.
,
Woo
,
W. L.
, and
Yang
,
Y.
,
2019
, “
Temporal and Spatial Deep Learning Network for Infrared Thermal Defect Detection
,”
NDT E Int.
,
108
, p.
102164
.
29.
Qiu
,
L.
,
Wu
,
X.
, and
Yu
,
Z.
,
2019
, “
A High-Efficiency Fully Convolutional Networks for Pixel-Wise Surface Defect Detection
,”
IEEE Access.
,
7
, pp.
15884
15893
.
30.
Yang
,
H.
,
Chen
,
Y.
,
Song
,
K.
, and
Yin
,
Z.
,
2019
, “
Multiscale Feature-Clustering-Based Fully Convolutional Autoencoder for Fast Accurate Visual Inspection of Texture Surface Defects
,”
IEEE Trans. Autom. Sci. Eng.
,
16
(
3
), pp.
1450
1467
.
31.
Dong
,
H.
,
Song
,
K.
,
He
,
Y.
,
Xu
,
J.
, and
Meng
,
Q.
,
2019
, “
PGA-Net: Pyramid Feature Fusion and Global Context Attention Network for Automated Surface Defect Detection
,”
IEEE Trans. Ind. Inform.
,
99
, pp.
1
1
.
32.
Pohlen
,
T.
,
Hermans
,
A.
,
Mathias
,
M.
, and
Leibe
,
B.
,
2016
, “
Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes
,”
CoRR abs/1611.08323v2
.
33.
Hu
,
P.
,
Perazzi
,
F.
,
Heilbron
,
F. C.
,
Wang
,
O.
,
Lin
,
Z.
,
Saenko
,
K.
, and
Sclaroff
,
S.
,
2020
, “
Real-Time Semantic Segmentation With Fast Attention
,”
CoRR abs/2007.03815
.
34.
Mehta
,
S.
,
Rastegari
,
M.
,
Caspi
,
A.
,
Shapiro
,
L.
, and
Hajishirzi
,
H.
,
2018
, “
ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation
,”
European Conference on Computer Vision
,
Munich, Germany
,
Sept. 8–14
, pp.
561
580
.
35.
Liu
,
M.
, and
Yin
,
H.
,
2019
, “
Cross Attention Network for Semantic Segmentation
,”
2019 IEEE International Conference on Image Processing (ICIP)
,
Taipei, Taiwan
,
Sept. 22–25
, pp.
2434
2438
.
36.
Yu
,
C.
,
Wang
,
J.
,
Peng
,
C.
,
Gao
,
C.
,
Yu
,
G.
, and
Sang
,
N.
,
2018
, “
BiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation
,”
European Conference on Computer Vision
,
Munich, Germany
,
Sept. 8–14
, pp.
334
349
.
37.
Lee
,
C. Y.
,
Xie
,
S.
,
Gallagher
,
P.
,
Zhang
,
Z.
, and
Tu
,
Z.
,
2014
, “
Deeply-Supervised Nets
,”
CoRR abs/1409.5185
.
38.
Zhao
,
H.
,
Shi
,
J.
,
Qi
,
X.
,
Wang
,
X.
, and
Jia
,
J.
,
2017
, “
Pyramid Scene Parsing Network
,”
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Honolulu, HI
,
July 21–26
, pp.
6230
6239
.
39.
Wei
,
S.-E.
,
Ramakrishna
,
V.
,
Kanade
,
T.
, and
Sheikh
,
Y.
,
2016
, “
Convolutional Pose Machines
,”
CoRR abs/1602.00134 01/30
.
40.
Zhou
,
Z.
,
Rahman Siddiquee
,
M. M.
,
Tajbakhsh
,
N.
, and
Liang
,
J.
,
2018
,
UNet++: A Nested U-Net Architecture for Medical Image Segmentation
,
Springer International Publishing
,
Cham
, pp.
3
11
.
41.
Zeiler
,
M. D.
, and
Fergus
,
R.
,
2014
, “
Visualizing and Understanding Convolutional Networks
,”
European Conference on Computer Vision
,
Zurich, Switzerland
,
Sept. 6–12
, pp.
818
833
.
42.
Deng
,
J.
,
Dong
,
W.
,
Socher
,
R.
,
Li
,
L. J.
, and
Li
,
F. F.
,
2009
, “
ImageNet: A Large-Scale Hierarchical Image Database
,”
IEEE Conference on Computer Vision and Pattern Recognition
,
Miami Beach, FL
,
June 20–25
, pp.
248
255
.
43.
Tian
,
Z.
,
Shen
,
C.
,
Tong
,
H.
, and
Yan
,
Y.
,
2019
, “
Decoders Matter for Semantic Segmentation: Data-Dependent Decoding Enables Flexible Feature Aggregation
,”
CoRR abs/1903.02120
.
44.
Chen
,
L.-C.
,
Zhu
,
Y.
,
Papandreou
,
G.
,
Schroff
,
F.
, and
Adam
,
H.
,
2018
, “
Encoder–Decoder With Atrous Separable Convolution for Semantic Image Segmentation
,”
CoRR abs/1802.02611, 02/07
.
45.
Hinton
,
G. E.
,
Srivastava
,
N.
,
Krizhevsky
,
A.
,
Sutskever
,
I.
, and
Salakhutdinov
,
R. R.
,
2012
, “
Improving Neural Networks by Preventing Co-Adaptation of Feature Detectors
,”
Comput. Sci.
,
3
(
4
), pp.
212
223
.
46.
Zhang
,
H.
,
Dana
,
K.
,
Shi
,
J.
,
Zhang
,
Z.
,
Wang
,
X.
,
Tyagi
,
A.
, and
Agrawal
,
A.
,
2018
, “
Context Encoding for Semantic Segmentation
,”
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT
,
June 18–23
, pp.
7151
7160
.
47.
Song
,
G.
,
Song
,
K.
, and
Yan
,
Y.
,
2020
, “
EDRNet: Encoder–Decoder Residual Network for Salient Object Detection of Strip Steel Surface Defects
,”
IEEE Trans. Instrum. Meas.
,
69
(
12
), pp.
9709
9719
.
You do not currently have access to this content.