Abstract

Additive manufacturing (AM) is a new paradigm in design-driven build of customized products. Nonetheless, mass customization and low-volume production make the AM quality assurance extremely challenging. Advanced imaging provides an unprecedented opportunity to increase information visibility, cope with the product complexity, and enable on-the-fly quality control in AM. However, in situ images of a customized AM build show a high level of layer-to-layer geometry variation, which hampers the use of powerful image-based learning methods such as deep neural networks (DNNs) for flaw detection. Very little has been done on deep learning of variant geometry for image-guided process monitoring and control. The proposed research is aimed at filling this gap by developing a novel machine learning approach that is focused on variant geometry in each layer of the AM build, namely region of interests, for the characterization and detection of layerwise flaws. Specifically, we leverage the computer-aided design (CAD) file to perform shape-to-image registration and to delineate the regions of interest in layerwise images. Next, a hierarchical dyadic partitioning methodology is developed to split layer-to-layer regions of interest into subregions with the same number of pixels to provide freeform geometry analysis. Then, we propose a semiparametric model to characterize the complex spatial patterns in each customized subregion and boost the computational speed. Finally, a DNN model is designed to learn variant geometry in layerwise imaging profiles and detect fine-grained information of flaws. Experimental results show that the proposed deep learning methodology is highly effective to detect flaws in each layer with an accuracy of 92.50 ± 1.03%. This provides a significant opportunity to reduce interlayer variation in AM prior to completion of a build. The proposed methodology can also be generally applicable in a variety of engineering and medical domains that entail customized design, variant geometry, and image-guided process control.

References

1.
ASTM
,
2012
,
Standard Terminology for Additive Manufacturing Technologies
,
ASTM International
,
West Conshohocken, PA
, Standard No. ASTM 52900-15.
2.
Bourell
,
D. L.
,
2016
, “
Perspectives on Additive Manufacturing
,”
Annu. Rev. Mater. Res.
,
46
(
1
), pp.
1
18
. 10.1146/annurev-matsci-070115-031606
3.
Foster
,
B.
,
Reutzel
,
E.
,
Nassar
,
A.
,
Hall
,
B.
,
Brown
,
S.
, and
Dickman
,
C.
,
2015
, “
Optical, Layerwise Monitoring of Powder Bed Fusion
,”
Solid Freeform Fabrication Symposium
,
Austin, TX
,
Aug. 10–12
, pp.
295
307
.
4.
Yao
,
B.
,
Imani
,
F.
, and
Yang
,
H.
,
2018
, “
Markov Decision Process for Image-Guided Additive Manufacturing
,”
IEEE Robot. Autom. Lett.
,
3
(
4
), pp.
2792
2798
. 10.1109/LRA.2018.2839973
5.
Yao
,
B.
,
Imani
,
F.
,
Sakpal
,
A. S.
,
Reutzel
,
E.
, and
Yang
,
H.
,
2018
, “
Multifractal Analysis of Image Profiles for the Characterization and Detection of Defects in Additive Manufacturing
,”
ASME J. Manuf. Sci. Eng.
,
140
(
3
), p.
031014
. 10.1115/1.4037891
6.
Chen
,
R.
,
Imani
,
F.
,
Reutzel
,
E.
, and
Yang
,
H.
,
2019
, “
From Design Complexity to Build Quality in Additive Manufacturing—A Sensor-Based Perspective
,”
IEEE Sensors Lett.
,
3
(
1
), pp.
1
4
. 10.1109/lsens.2018.2880747
7.
Imani
,
F.
,
Yao
,
B.
,
Chen
,
R.
,
Rao
,
P.
, and
Yang
,
H.
,
2019
, “
Joint Multifractal and Lacunarity Analysis of Image Profiles for Manufacturing Quality Control
,”
ASME J. Manuf. Sci. Eng.
,
141
(
4
), p.
044501
. 10.1115/1.4042579
8.
Abdelrahman
,
M.
,
Reutzel
,
E. W.
,
Nassar
,
A. R.
, and
Starr
,
T. L.
,
2017
, “
Flaw Detection in Powder Bed Fusion Using Optical Imaging
,”
Addit. Manuf.
,
15
, pp.
1
11
. 10.1016/j.addma.2017.02.001
9.
Malekipour
,
E.
, and
El-Mounayri
,
H.
,
2018
, “Defects, Process Parameters and Signatures for Online Monitoring and Control in Powder-Based Additive Manufacturing,”
Mechanics of Additive and Advanced Manufacturing
, Vol.
9
,
Springer
,
New York
, pp.
83
90
.
10.
Amini
,
M.
, and
Chang
,
S.
,
2017
, “
Assessing data veracity for data-rich manufacturing
,”
IIE Annual Conference
,
Pittsburgh, PA
,
May 20–23
, pp.
1661
1666
.
11.
Imani
,
F.
,
Gaikwad
,
A.
,
Montazeri
,
M.
,
Rao
,
P.
,
Yang
,
H.
, and
Reutzel
,
E.
,
2018
, “
Process Mapping and In-Process Monitoring of Porosity in Laser Powder Bed Fusion Using Layerwise Optical Imaging
,”
ASME J. Manuf. Sci. Eng.
,
140
(
10
), p.
101009
. 10.1115/1.4040615
12.
Imani
,
F.
,
Yao
,
B.
,
Chen
,
R.
,
Rao
,
P.
, and
Yang
,
H.
,
2018
, “
Fractal Pattern Recognition of Image Profiles for Manufacturing Process Monitoring and Control
,”
ASME 2018 13th International Manufacturing Science and Engineering Conference
,
College Station, TX
,
June 18–22
, p.
V003T02A003
.
13.
Kan
,
C.
,
Chen
,
R.
, and
Yang
,
H.
,
2017
, “
Image-Guided Quality Control of Biomanufacturing Process
,”
Procedia CIRP
,
65
, pp.
168
174
. 10.1016/j.procir.2017.04.034
14.
Imani
,
F.
,
Gaikwad
,
A.
,
Montazeri
,
M.
,
Rao
,
P.
,
Yang
,
H.
, and
Reutzel
,
E.
,
2018
, “
Layerwise In-Process Quality Monitoring in Laser Powder Bed Fusion
,”
ASME 2018 13th International Manufacturing Science and Engineering Conference
,
College Station, TX
,
June 18–22
, p.
V001T01A038
.
15.
Tapia
,
G.
, and
Elwany
,
A.
,
2014
, “
A Review on Process Monitoring and Control in Metal-Based Additive Manufacturing
,”
ASME J. Manuf. Sci. Eng.
,
136
(
6
), p.
060801
. 10.1115/1.4028540
16.
Foster
,
B. K.
,
Reutzel
,
E. W.
,
Nassar
,
A. R.
,
Dickman
,
C. J.
, and
Hall
,
B. T.
,
2015
, “
A Brief Survey of Sensing for Metal-Based Powder Bed Fusion Additive Manufacturing
,”
Proceedings Volume 9489, Dimensional Optical Metrology and Inspection for Practical Applications IV
,
Baltimore, MD
,
Apr. 20–21
, p.
94890B
.
17.
Everton
,
S. K.
,
Hirsch
,
M.
,
Stravroulakis
,
P.
,
Leach
,
R. K.
, and
Clare
,
A. T.
,
2016
, “
Review of In-Situ Process Monitoring and In-Situ Metrology for Metal Additive Manufacturing
,”
Mater. Des.
,
95
, pp.
431
445
. 10.1016/j.matdes.2016.01.099
18.
Mani
,
M.
,
Lane
,
B. M.
,
Donmez
,
M. A.
,
Feng
,
S. C.
, and
Moylan
,
S. P.
,
2017
, “
A Review on Measurement Science Needs for Real-Time Control of Additive Manufacturing Metal Powder Bed Fusion Processes
,”
Int. J. Prod. Res.
,
55
(
5
), pp.
1400
1418
. 10.1080/00207543.2016.1223378
19.
Grasso
,
M.
, and
Colosimo
,
B. M.
,
2017
, “
Process Defects and In Situ Monitoring Methods in Metal Powder Bed Fusion: A Review
,”
Meas. Sci. Technol.
,
28
(
4
), p.
044005
. 10.1088/1361-6501/aa5c4f
20.
Hirsch
,
M.
,
Patel
,
R.
,
Li
,
W.
,
Guan
,
G.
,
Leach
,
R. K.
,
Sharples
,
S. D.
, and
Clare
,
A. T.
,
2017
, “
Assessing the Capability of In-Situ Nondestructive Analysis During Layer Based Additive Manufacture
,”
Addit. Manuf.
,
13
, pp.
135
142
. 10.1016/j.addma.2016.10.004
21.
Tang
,
M.
,
Pistorius
,
P. C.
, and
Beuth
,
J. L.
,
2017
, “
Prediction of Lack-of-Fusion Porosity for Powder Bed Fusion
,”
Addit. Manuf.
,
14
, pp.
39
48
. 10.1016/j.addma.2016.12.001
22.
Momenzadeh
,
N.
,
Nath
,
S.
,
Berfield
,
T.
, and
Atre
,
S.
,
2019
, “
In Situ Measurement of Thermal Strain Development in 420 Stainless Steel Additive Manufactured Metals
,”
Exp. Mech.
,
59
(
4
), pp.
1
9
. 10.1007/s11340-019-00513-3
23.
Montazeri
,
M.
,
Yavari
,
R.
,
Rao
,
P.
, and
Boulware
,
P.
,
2018
, “
In-Process Monitoring of Material Cross-Contamination Defects in Laser Powder Bed Fusion
,”
ASME J. Manuf. Sci. Eng.
,
140
(
11
), p.
111001
. 10.1115/1.4040543
24.
Kim
,
W.
,
Mechitov
,
K.
,
Choi
,
J.-Y.
, and
Ham
,
S.
,
2005
, “
On Target Tracking With Binary Proximity Sensors
,”
Fourth International Symposium on Information Processing in Sensor Networks
,
Los Angeles, CA
,
Apr. 25–27
,
IEEE
, pp.
301
308
.
25.
Cheng
,
B.
,
Lane
,
B.
,
Whiting
,
J.
, and
Chou
,
K.
,
2018
, “
A Combined Experimental-Numerical Method to Evaluate Powder Thermal Properties in Laser Powder Bed Fusion
,”
ASME J. Manuf. Sci. Eng.
,
140
(
11
), p.
111008
. 10.1115/1.4040877
26.
Cerniglia
,
D.
,
Scafidi
,
M.
,
Pantano
,
A.
, and
Rudlin
,
J.
,
2015
, “
Inspection of Additive-Manufactured Layered Components
,”
Ultrasonics
,
62
, pp.
292
298
. 10.1016/j.ultras.2015.06.001
27.
Leach
,
R.
,
2011
,
Optical Measurement of Surface Topography
, Vol.
14
,
Springer
,
New York
.
28.
Chivel
,
Y.
, and
Smurov
,
I.
,
2010
, “
On-Line Temperature Monitoring in Selective Laser Sintering/Melting
,”
Phys. Procedia
,
5
, pp.
515
521
. 10.1016/j.phpro.2010.08.079
29.
Bayle
,
F.
, and
Doubenskaia
,
M.
,
2008
, “
Selective Laser Melting Process Monitoring With High Speed Infra-Red Camera and Pyrometer
,”
Proceedings of SPIE, Fundamentals of Laser Assisted Micro- and Nanotechnologies
,
Petersburg, Russian Federation
,
June 25–28
, Vol.
6985
,
International Society for Optics and Photonics
, p.
698505
.
30.
Kleszczynski
,
S.
,
Zur Jacobsmühlen
,
J.
,
Sehrt
,
J.
, and
Witt
,
G.
,
2012
, “
Error Detection in Laser Beam Melting Systems by High Resolution Imaging
,”
Proceedings of the Twenty Third Annual International Solid Freeform Fabrication Symposium
,
Austin, TX
,
Aug. 6–8
, pp.
975
987
.
31.
Heigel
,
J. C.
, and
Lane
,
B. M.
,
2018
, “
Measurement of the Melt Pool Length During Single Scan Tracks in a Commercial Laser Powder Bed Fusion Process
,”
ASME J. Manuf. Sci. Eng.
,
140
(
5
), p.
051012
. 10.1115/1.4037571
32.
Mahmoudi
,
M.
,
Ezzat
,
A. A.
, and
Elwany
,
A.
,
2019
, “
Layerwise Anomaly Detection in Laser Powder-Bed Fusion Metal Additive Manufacturing
,”
ASME J. Manuf. Sci. Eng.
,
141
(
3
), p.
031002
. 10.1115/1.4042108
33.
Kruth
,
J.-P.
,
Mercelis
,
P.
,
Van Vaerenbergh
,
J.
, and
Craeghs
,
T.
,
2007
, “
Feedback Control of Selective Laser Melting
,”
Proceedings of the 3rd International Conference on Advanced Research in Virtual and Rapid Prototyping
,
Leiria, Portugal
,
Sept. 24–29
, pp.
521
527
.
34.
Kruth
,
J.-P.
,
Duflou
,
J.
,
Mercelis
,
P.
,
Van Vaerenbergh
,
J.
,
Craeghs
,
T.
, and
De Keuster
,
J.
,
2007
, “
On-Line Monitoring and Process Control in Selective Laser Melting and Laser Cutting
,”
Proceedings of the 5th Lane Conference
,
Erlangen, Germany
,
Sept. 25–28
,
Laser Assisted Net Shape Engineering
, Vol.
1
, pp.
23
37
.
35.
Clijsters
,
S.
,
Craeghs
,
T.
,
Buls
,
S.
,
Kempen
,
K.
, and
Kruth
,
J.-P.
,
2014
, “
In Situ Quality Control of the Selective Laser Melting Process Using a High-Speed, Real-Time Melt Pool Monitoring System
,”
Int. J. Adv. Manuf. Technol.
,
75
(
5–8
), pp.
1089
1101
. 10.1007/s00170-014-6214-8
36.
Seifi
,
S. H.
,
Tian
,
W.
,
Doude
,
H.
,
Tschopp
,
M. A.
, and
Bian
,
L.
,
2019
, “
Layer-Wise Modeling and Anomaly Detection for Laser-Based Additive Manufacturing
,”
ASME J. Manuf. Sci. Eng.
,
141
(
8
), p.
081013
. 10.1115/1.4043898
37.
Craeghs
,
T.
,
Clijsters
,
S.
,
Yasa
,
E.
, and
Kruth
,
J.-P.
,
2011
, “
Online Quality Control of Selective Laser Melting
,”
Proceedings of the Solid Freeform Fabrication Symposium
,
Austin, TX
,
Aug. 8–10
, pp.
212
226
.
38.
Craeghs
,
T.
,
Bechmann
,
F.
,
Berumen
,
S.
, and
Kruth
,
J.-P.
,
2010
, “
Feedback Control of Layerwise Laser Melting Using Optical Sensors
,”
Phys. Procedia
,
5
, pp.
505
514
. 10.1016/j.phpro.2010.08.078
39.
Malekipour
,
E.
, and
El-Mounayri
,
H.
,
2018
, “
Common Defects and Contributing Parameters in Powder Bed Fusion AM Process and Their Classification for Online Monitoring and Control: A Review
,”
Int. J. Adv. Manuf. Technol.
,
95
(
1–4
), pp.
527
550
. 10.1007/s00170-017-1172-6
40.
Yavari
,
M. R.
,
Cole
,
K. D.
, and
Rao
,
P.
,
2019
, “
Thermal Modeling in Metal Additive Manufacturing Using Graph Theory
,”
ASME J. Manuf. Sci. Eng.
,
141
(
7
), p.
071007
. 10.1115/1.4043648
41.
Sun
,
T.-H.
,
Tien
,
F.-C.
,
Tien
,
F.-C.
, and
Kuo
,
R.-J.
,
2016
, “
Automated Thermal Fuse Inspection Using Machine Vision and Artificial Neural Networks
,”
J. Intell. Manuf.
,
27
(
3
), pp.
639
651
. 10.1007/s10845-014-0902-y
42.
Librantz
,
A. F.
,
de Araújo
,
S. A.
,
Alves
,
W. A.
,
Belan
,
P. A.
,
Mesquita
,
R. A.
, and
Selvatici
,
A. H.
,
2017
, “
Artificial Intelligence Based System to Improve the Inspection of Plastic Mould Surfaces
,”
J. Intell. Manuf.
,
28
(
1
), pp.
181
190
. 10.1007/s10845-014-0969-5
43.
Zhang
,
Y.
,
Hong
,
G. S.
,
Ye
,
D.
,
Zhu
,
K.
, and
Fuh
,
J. Y.
,
2018
, “
Extraction and Evaluation of Melt Pool, Plume and Spatter Information for Powder-Bed Fusion AM Process Monitoring
,”
Mater. Des.
,
156
, pp.
458
469
. 10.1016/j.matdes.2018.07.002
44.
Kwon
,
O.
,
Kim
,
H. G.
,
Ham
,
M. J.
,
Kim
,
W.
,
Kim
,
G.-H.
,
Cho
,
J.-H.
,
Kim
,
N. I.
, and
Kim
,
K.
,
2018
, “
A Deep Neural Network for Classification of Melt-Pool Images in Metal Additive Manufacturing
,”
J. Intell. Manuf.
,
29
(
7
), pp.
1
12
. 10.1007/s10845-018-1451-6
45.
Scime
,
L.
, and
Beuth
,
J.
,
2018
, “
A Multi-scale Convolutional Neural Network for Autonomous Anomaly Detection and Classification in a Laser Powder Bed Fusion Additive Manufacturing Process
,”
Addit. Manuf.
,
24
, pp.
273
286
. 10.1016/j.addma.2018.09.034
46.
Lloyd
,
S.
,
1982
, “
Least Squares Quantization in PCM
,”
IEEE Trans. Inf. Pheory
,
28
(
2
), pp.
129
137
. 10.1109/TIT.1982.1056489
47.
Oliver
,
M. A.
, and
Webster
,
R.
,
1990
, “
Kriging: A Method of Interpolation for Geographical Information Systems
,”
Int. J. Geogr. Inf. Syst.
,
4
(
3
), pp.
313
332
. 10.1080/02693799008941549
48.
Krizhevsky
,
A.
,
Sutskever
,
I.
, and
Hinton
,
G. E.
,
2012
,
“ImageNet Classification With Deep Convolutional Neural Networks
,”
Proceedings of Advances in Neural Information Processing Systems 25 (NIPS 2012)
,
Lake Tahoe, NV
, pp.
1097
1105
. 10.1145/3065386
49.
Han
,
X.
,
Zhong
,
Y.
,
Cao
,
L.
, and
Zhang
,
L.
,
2017
, “
Pre-trained Alexnet Architecture With Pyramid Pooling and Supervision for High Spatial Resolution Remote Sensing Image Scene Classification
,”
Remote Sens.
,
9
(
8
), p.
848
. 10.3390/rs9080848
50.
Nair
,
V.
, and
Hinton
,
G. E.
,
2010
, “
Rectified Linear Units Improve Restricted Boltzmann Machines
,”
Proceedings of the 27th International Conference on Machine Learning (ICML-10)
,
Haifa, Israel
,
June 21–24
, pp.
807
814
.
You do not currently have access to this content.