Abstract
In this paper, we consider the optimal control of material microstructures. Such material microstructures are modeled by the so-called phase-field model. We study the underlying physical structure of the model and propose a data-based approach for its optimal control, along with a comparison to the control using a state-of-the-art reinforcement learning (RL) algorithm. Simulation results show the feasibility of optimally controlling such microstructures to attain desired material properties and complex target microstructures.
References
1.
Nikolas Provatas
, K. E.
, 2010
, Phase Field Methods in Materials Science and Engineering
, Wiley
, Hoboken, NJ.2.
Boettinger
, W. J.
,
Warren
, J. A.
,
Beckermann
, C.
, and
Karma
, A.
, 2002
, “Phase-Field Simulation of Solidification
,” Annu. Rev. Mater. Res.
, 32
(1
), pp. 163
–194
.10.1146/annurev.matsci.32.101901.1558033.
Moelans
, N.
,
Blanpain
, B.
, and
Wollants
, P.
, 2008
, “An Introduction to Phase-Field Modeling of Microstructure Evolution
,” Calphad
, 32
(2
), pp. 268
–294
.10.1016/j.calphad.2007.11.0034.
Nguyen
, L.
,
Shi
, R.
,
Wang
, Y.
, and
Graef
, M. D.
, 2016
, “Quantification of Rafting of Precipitates in Ni-Based Superalloys
,” Acta Mater.
, 103
, pp. 322
–333
.10.1016/j.actamat.2015.09.0605.
Travasso
, R. D.
,
Castro
, M.
, and
Oliveira
, J. C.
, 2011
, “The Phase-Field Model in Tumor Growth
,” Philos. Mag.
, 91
(1
), pp. 183
–206
.10.1080/14786435.2010.5017716.
Flint
, T.
,
Sun
, Y.
,
Xiong
, Q.
,
Smith
, M.
, and
Francis
, J.
, 2019
, “Phase-Field Simulation of Grain Boundary Evolution in Microstructures Containing Second-Phase Particles With Heterogeneous Thermal Properties
,” Sci. Rep.
, 9
(1
), p. 12
.10.1038/s41598-019-54883-87.
Cheniour
, A.
,
Tonks
, M.
,
Gong
, B.
,
Yao
, T.
,
He
, L.
,
Harp
, J.
,
Beeler
, B.
,
Zhang
, Y.
, and
Lian
, J.
, 2020
, “Development of a Grain Growth Model for U3Si2 Using Experimental Data, Phase Field Simulation and Molecular Dynamics
,” J. Nucl. Mater.
, 532
, p. 152069
.10.1016/j.jnucmat.2020.1520698.
Khaderi
, S.
,
Murali
, P.
, and
Ahluwalia
, R.
, 2014
, “Failure and Toughness of Bio-Inspired Composites: Insights From Phase Field Modelling
,” Comput. Mater. Sci.
, 95
, pp. 1
–7
.10.1016/j.commatsci.2014.07.0019.
Hansen-Dörr
, A. C.
,
de Borst
, R.
,
Hennig
, P.
, and
Kästner
, M.
, 2019
, “Phase-Field Modelling of Interface Failure in Brittle Materials
,” Comput. Methods Appl. Mech. Eng.
, 346
, pp. 25
–42
.10.1016/j.cma.2018.11.02010.
Allen
, S. M.
, and
Cahn
, J. W.
, 1979
, “A Microscopic Theory for Antiphase Boundary Motion and Its Application to Antiphase Domain Coarsening
,” Acta Metall.
, 27
(6
), pp. 1085
–1095
.10.1016/0001-6160(79)90196-211.
Cahn
, J. W.
, and
Hilliard
, J. E.
, 1958
, “Free Energy of a Nonuniform System. I. Interfacial Free Energy
,” J. Chem. Phys
, 28
(2
), pp. 258
–267
.10.1063/1.174410212.
Benner
, P.
, and
Stoll
, M.
, 2013
, “Optimal Control for Allen-Cahn Equations Enhanced by Model Predictive Control
,” IFAC Proc. Vol.
, 46
(26
), pp. 139
–143
.10.3182/20130925-3-FR-4043.0006213.
Blank
, L.
,
Farshbaf Shaker
, M.
,
Hecht
, C.
,
Michl
, J.
, and
Rupprecht
, C.
, 2013
, “Optimal Control of Allen-Cahn Systems
,” Birkhäuser Cham, Vol. 165
, Basel, Switzerland, p. 12
.14.
Zhang
, X.
,
Li
, H.
, and
Liu
, C.
, 2020
, “Optimal Control Problem for the Cahn–Hilliard/Allen–Cahn Equation With State Constraint
,” Appl. Math. Optim.
, 82
(2
), pp. 721
–754
.10.1007/s00245-018-9546-115.
Scarpa
, L.
, 2019
, “Optimal Distributed Control of a Stochastic Cahn–Hilliard Equation
,” SIAM J. Control Optim.
, 57
(5
), pp. 3571
–3602
.10.1137/18M122222316.
Kadiri
, M.
,
Louaked
, M.
, and
Trabelsi
, S.
, 2023
, “Optimal Control and Parameters Identification for the Cahn–Hilliard Equations Modeling Tumor Growth
,” Mathematics
, 11
(7
), p. 1607
.10.3390/math1107160717.
Wang
, R.
,
Parunandi
, K.
,
Yu
, D.
,
Kalathil
, D.
, and
Chakravorty
, S.
, 2019
, “Decoupled Data Based Approach for Learning to Control Nonlinear Dynamical Systems
,” IEEE Trans. Autom. Control
, 67(7), pp. 3582
–3589
.10.1109/TAC.2021.310855218.
Sutton
, R. S.
, and
Barto
, A. G.
, 2018
, Reinforcement Learning: An Introduction
, MIT Press
, Cambridge, MA.19.
Silver
, D.
,
Huang
, A.
,
Maddison
, C. J.
,
Guez
, A.
,
Sifre
, L.
,
van den Driessche
, G.
, and
Schrittwieser
, J.
, et al., 2016
, “Mastering the Game of Go With Deep Neural Networks and Tree Search
,” Nature
, 529
(7587
), pp. 484
–489
.10.1038/nature1696120.
Lillicrap
, T. P.
,
Hunt
, J. J.
,
Pritzel
, A.
,
Heess
, N.
,
Erez
, T.
,
Tassa
, Y.
,
Silver
, D.
, and
Wierstra
, D.
, 2015
, “Continuous Control With Deep Reinforcement Learning,” preprint arXiv:1509.02971
.10.48550/arXiv.1509.0297121.
Levine
, S.
,
Finn
, C.
,
Darrell
, T.
, and
Abbeel
, P.
, 2016
, “End-to-End Training of Deep Visuomotor Policies
,” J. Mach. Learn. Res.
, 17
(1
), pp. 1334
–1373
.https://www.jmlr.org/papers/volume17/15-522/15-522.pdf22.
Yuhuai
, W.
,
Elman
, M.
,
Shun
, L.
,
Roger
, G.
, and
Jimmy
, B.
, 2017
, “Scalable Trust-Region Method for Deep Reinforcement Learning Using Kronecker-Factored Approximation
,” 31st International Conference on Neural Information Processing Systems (NIPS
), Long Beach, CA, Dec. 4–9, pp. 5285
–5294
.https://proceedings.neurips.cc/paper_files/paper/2017/file/361440528766bbaaaa1901845cf4152b-Paper.pdf23.
Schulman
, J.
,
Levine
, S.
,
Moritz
, P.
,
Jordan
, M. I.
, and
Abbeel
, P.
, 2015
, “Trust Region Policy Optimization
,” Proceedings of the 32nd International Conference on Machine Learning
, Lille, France, July 6–11, pp. 1
–16
.https://arxiv.org/pdf/1502.0547724.
Schulman
, J.
,
Wolski
, F.
,
Dhariwal
, P.
,
Radford
, A.
, and
Klimov
, O.
, 2017
, “Proximal Policy Optimization Algorithms
,” preprint arXiv:1707.06347
.10.48550/arXiv.1707.0634725.
Henderson
, P.
,
Islam
, R.
,
Bachman
, P.
,
Pineau
, J.
,
Precup
, D.
, and
Meger
, D.
, 2018
, “Deep Reinforcement Learning That Matters
,” 32nd AAAI Conference on Artificial Intelligence
, New Orleans, LA, Feb. 2–7, pp. 3207
–3214
.https://dl.acm.org/doi/abs/10.5555/3504035.350442726.
Pan
, Y.
,
Farahmand
, A.-M.
,
White
, M.
,
Nabi
, S.
,
Grover
, P.
, and
Nikovski
, D.
, 2018
, “Reinforcement Learning With Function-Valued Action Spaces for Partial Differential Equation Control
,” Proceedings of the 35th International Conference on Machine Learning
, Stockholm, Sweden, July 10–15, pp. 3986
–3995
.https://merl.com/publications/docs/TR2018-028.pdf27.
Bensoussan
, A.
,
Da Prato
, G.
,
Delfour
, M. C.
, and
Mitter
, S. K.
, 1992
, Representation and Control of Infinite Dimensional Systems
, Vols. 1 and 2
, Birkhäuser Boston
, Boston, MA.28.
Lall
, S.
,
Marsden
, J. E.
, and
Glavaški
, S.
, 2002
, “A Subspace Approach to Balanced Truncation for Model Reduction of Nonlinear Control Systems
,” Int. J. Robust Nonlinear Control: IFAC-Affiliated J.
, 12
(6
), pp. 519
–535
.10.1002/rnc.65729.
Ravindran
, S.
, 2002
, “Adaptive Reduced-Order Controllers for a Thermal Flow System Using Proper Orthogonal Decomposition
,” SIAM J. Sci. Comput.
, 23
(6
), pp. 1924
–1942
.10.1137/S106482750037471630.
Kunisch
, K.
, and
Volkwein
, S.
, 2008
, “Proper Orthogonal Decomposition for Optimality Systems
,” ESAIM: Math. Modell. Numer. Anal.-Modél. Math. Anal. Numér.
, 42
(1
), pp. 1
–23
.10.1051/m2an:200705431.
Yu
, D.
,
Rafieisakhaei
, M.
, and
Chakravorty
, S.
, 2017
, “Stochastic Feedback Control of Systems With Unknown Nonlinear Dynamics
,” 56th IEEE Conference on Decision and Control (CDC
), Melbourne, VIC, Australia, Dec. 12–15, pp. 4309
–4314
.10.1109/CDC.2017.826429432.
Wang
, R.
,
Parunandi
, K. S.
,
Sharma
, A.
,
Chakravorty
, S.
, and
Kalathil
, D.
, 2020
, “On the Search for Feedback in Reinforcement Learning
,” 60th IEEE Conference on Decision and Control (CDC
), Austin, TX, Dec. 14–17, pp. 1560
–1567
.10.1109/CDC45484.2021.968335033.
Bryson
, A. E.
, and
Ho
, Y.-C.
, 1975
, Applied Optimal Control: Optimization, Estimation, and Control
, Routledge
, New York
.34.
Barrett
, J. W.
,
Nürnberg
, R.
, and
Styles
, V.
, 2004
, “Finite Element Approximation of a Phase Field Model for Void Electromigration
,” SIAM J. Numer. Anal.
, 42
(2
), pp. 738
–772
.10.1137/S003614290241342135.
Sharma
, A.
, and
Chakravorty
, S.
, 2023
, “A Reduced Order Iterative Linear Quadratic Regulator (ILQR) Technique for the Optimal Control of Nonlinear Partial Differential Equations
,” 2023 American Control Conference (ACC
), San Diego, CA, May 31–June 2, pp. 3389
–3394
.10.23919/ACC55779.2023.1015606236.
Goyal
, R.
,
Wang
, R.
,
Mohamed
, M. N. G.
,
Sharma
, A.
, and
Chakravorty
, S.
, 2023
, “An Information-State Based Approach to the Optimal Output Feedback Control of Nonlinear Systems
,” preprint arXiv:2107.08086
.10.48550/arXiv.2107.08086
