Abstract

In this work, we consider the problem of nonlinear system identification using data to learn multiple and often coupled parameters that allow a simulator to more accurately model a physical system or mechanism and close the so-called reality gap for more accurate robot control. Our approach uses iterative residual tuning (IRT), a recently developed derivative-free system identification technique that utilizes neural networks and visual observation to estimate parameter differences between a proposed model and a target model. We develop several modifications to the basic IRT approach and apply it to the system identification of a five-parameter model of a marble rolling in a robot-controlled labyrinth game mechanism. We validate our technique both in simulation—where we outperform two baselines—and on a real system, where we achieve marble tracking error of 4% after just five optimization iterations.

References

1.
Camacho
,
E. F.
, and
Alba
,
C. B.
,
2013
,
Model Predictive Control
,
Springer Science & Business Media
,
Berlin, Germany
.
2.
Beyer
,
H.-G.
, and
Schwefel
,
H.-P.
,
2002
, “
Evolution Strategies—A Comprehensive Introduction
,”
Nat. Comput.
,
1
(
1
), pp.
3
52
.
3.
Hansen
,
N.
,
2016
, “
The CMA Evolution Strategy: A Tutorial
,” CoRR.
4.
Sutton
,
R. S.
, and
Barto
,
A. G.
,
2018
,
Reinforcement Learning: An Introduction
, 2nd ed.,
The MIT Press
,
Cambridge, MA
.
5.
Boschert
,
S.
, and
Rosen
,
R.
,
2016
, “Digital Twin—The Simulation Aspect,”
Mechatronic Futures
,
P.
Hehenberger
, and
D.
Bradley
, eds.,
Springer
,
Berlin, Germany
, pp.
59
74
.
6.
Jakobi
,
N.
,
1997
, “
Evolutionary Robotics and the Radical Envelope-of-Noise Hypothesis
,”
Adaptive Behav.
,
6
(
2
), pp.
325
368
.
7.
Chebotar
,
Y.
,
Handa
,
A.
,
Makoviychuk
,
V.
,
Macklin
,
M.
,
Issac
,
J.
,
Ratliff
,
N.
, and
Fox
,
D.
,
2018
, “
Closing the Sim-to-Real Loop: Adapting Simulation Randomization With Real World Experience
,” CoRR, 10.
8.
Hanna
,
Josiah P.
, and
Stone
,
Peter
,
2017
, “
Grounded Action Transformation for Robot Learning in Simulation
,”
Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17)
,
Yokohama, Japan
,
Jan. 7–15
, pp.
3834
3840
.
9.
Golemo
,
F.
,
Taiga
,
A. A.
,
Courville
,
A.
, and
Oudeyer
,
P.-Y.
,
2018
, “
Sim-to-Real Transfer With Neural-Augmented Robot Simulation
,”
Conference on Robot Learning (CoRL)
,
Zürich, Switzerland
,
Oct. 29–31
, pp.
817
828
.
10.
James
,
S.
,
Wohlhart
,
P.
,
Kalakrishnan
,
M.
,
Kalashnikov
,
D.
,
Irpan
,
A.
,
Ibarz
,
J.
,
Levine
,
S.
,
Hadsell
,
R.
, and
Bousmalis
,
K.
,
2019
, “
Sim-to-Real Via Sim-to-Sim: Data-Efficient Robotic Grasping Via Randomized-to-Canonical Adaptation Networks
,”
Computer Vision and Pattern Recognition
,
Long Beach, CA
,
June 16–20
.
11.
Bergatt
,
C.
,
Metzen
,
J. H.
,
Kirchner
,
E. A.
, and
Kirchner
,
F.
,
2009
, “
Quantification and Minimization of the Simulation-Reality-Gap on a BRIO (R) Labyrinth Game
,”
Proceedings of the First International Workshop on Learning and Data Mining for Robotics
,
Bled, Slovenia
,
July 9
, pp.
26
38
.
12.
Åström
,
K.-J.
, and
Torsten
,
B.
,
1965
, “
Numerical Identification of Linear Dynamic Systems From Normal Operating Records
,”
IFAC Proc. Vol.
,
2
(
2
), pp.
96
111
.
13.
Åström
,
K. J.
, and
Eykhoff
,
P.
,
1971
, “
System Identification—A Survey
,”
Automatica
,
7
(
2
), pp.
123
162
.
14.
Zhu
,
S.
,
Kimmel
,
A.
,
Bekris
,
K. E.
, and
Boularias
,
A.
,
2018
, “
Fast Model Identification Via Physics Engines for Data-Efficient Policy Search
,”
Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI)
,
Stockholm, Sweden
,
July 13–19
, pp.
3249
3256
.
15.
Yu
,
W.
,
Tan
,
J.
,
Liu
,
C. K.
, and
Turk
,
G.
,
2017
, “
Preparing for the Unknown: Learning a Universal Policy With Online System Identification
,”
Robotics: Science and Systems
,
Cambridge, MA
,
July 12–16
.
16.
Allevato
,
A.
,
Schaertl Short
,
E.
,
Pryor
,
M.
, and
Thomaz
,
A. L.
,
2019
, “
TuneNet: One-Shot Residual Tuning for System Identification and Sim-to-Real Robot Task Planning
,”
Conference on Robot Learning (CoRL)
,
Osaka, Japan
,
Oct. 30–Nov. 1
.
17.
Allevato
,
A.
,
Pryor
,
M.
, and
Thomaz
,
A. L.
,
2020
, “
Multiparameter Real-World System Identification Using Iterative Residual Tuning
,”
International Design Engineering Technical Conferences & Computers and Information in Engineering Conference (IDETC/CIE)
,
Virtual
,
Aug. 17–19
.
18.
Ho
,
B. L.
, and
Kálmán
,
R. E.
,
1966
, “
Effective Construction of Linear State-Variable Models From Input/Output Functions
,”
Automatisierungstechnik
,
14
(
1–12
), pp.
545
548
.
19.
Chen
,
S.
,
Billings
,
S. A.
, and
Luo
,
W.
,
1989
, “
Orthogonal Least Squares Methods and Their Application to Non-Linear System Identification
,”
Int. J. Control
,
50
(
5
), pp.
1873
1896
.
20.
Verhaegen
,
M.
, and
Verdult
,
V.
,
2007
,
Filtering and System Identification: A Least Squares Approach
,
Cambridge University Press
,
Cambridge, UK
.
21.
Brockett
,
R. W.
,
1976
, “
Volterra Series and Geometric Control Theory
,”
Automatica
,
12
(
2
), pp.
167
176
.
22.
Giri
,
F.
, and
Bai
,
E.-W.
,
2010
,
Block-Oriented Nonlinear System Identification
, Vol.
1
,
Springer
,
Berlin, Germany
.
23.
Chen
,
S.
, and
Billings
,
S. A.
,
1989
, “
Representations of Non-Linear Systems: The NARMAX Model
,”
Int. J. Control
,
49
(
3
), pp.
1013
1032
.
24.
Babuška
,
R.
, and
Verbruggen
,
H.
,
2003
, “
Neuro-Fuzzy Methods for Nonlinear System Identification
,”
Ann. Rev. Control
,
27
(
1
), pp.
73
85
.
25.
Kaiser
,
E.
,
Kutz
,
J. N.
, and
Brunton
,
S. L.
,
2018
, “
Sparse Identification of Nonlinear Dynamics for Model Predictive Control in the Low-Data Limit
,”
Proc. R. Soc. A
,
474
(
2219
), p.
20180335
.
26.
Khosla
,
P.
, and
Kanade
,
T.
,
1985
, “
Parameter Identification of Robot Dynamics
,”
1985 24th IEEE Conference on Decision and Control
,
Fort Lauderdale, FL
,
Dec. 11–13
, IEEE, pp.
1754
1760
.
27.
Gautier
,
M.
, and
Khalil
,
W.
,
1988
, “
On the Identification of the Inertial Parameters of Robots
,”
Proceedings of the 27th IEEE Conference on Decision and Control
,
Austin, TX
,
Dec. 7–9
, IEEE, pp.
2264
2269
.
28.
Jong
,
M. T.
, and
Shanmugam
,
K. S.
,
1977
, “
Determination of a Transfer Function From Amplitude Frequency Response Data
,”
Int. J. Control
,
25
(
6
), pp.
941
948
.
29.
Sidman
,
M. D.
,
DeAngelis
,
F. E.
, and
Verghese
,
G. C.
,
1990
, “
Parametric System Identification on Logarithmic Frequency Response Data
,”
1990 American Control Conference
,
San Diego, CA
,
May 23–25
, IEEE, pp.
1888
1892
.
30.
Van Overschee
,
P.
, and
De Moor
,
B.
,
1996
, “
Continuous-Time Frequency Domain Subspace System Identification
,”
Signal Proc.
,
52
(
2
), pp.
179
194
.
31.
Abdalmoaty
,
M. R.
, and
Hjalmarsson
,
H.
,
2017
,
Simulated Pseudo Maximum Likelihood Identification of Nonlinear Models
, Vol.
50
,
Elsevier
,
Amsterdam, The Netherlands
, pp.
14058
14063
.
32.
Agrawal
,
P.
,
Nair
,
A. V.
,
Abbeel
,
P.
,
Malik
,
J.
, and
Levine
,
S.
,
2016
,
Advances in Neural Information Processing Systems
,
Morgan Kaufmann Publishers Inc.
,
Burlington, MA
, pp.
5074
5082
.
33.
Pinto
,
Lerrel
, and
Gupta
,
Abhinav
,
2017
, “
Learning to Push by Grasping: Using Multiple Tasks for Effective Learning
,”
International Conference on Robotics and Automation (ICRA)
,
Singapore
,
May 29–June 3
, IEEE, pp.
2161
2168
.
34.
Herman
,
M.
,
Gindele
,
T.
,
Wagner
,
J.
,
Schmitt
,
F.
, and
Burgard
,
W.
,
2016
, “
Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamics
,”
The 18th International Conference on Artificial Intelligence and Statistics (AISTATS)
,
Cadiz, Spain
,
May 9–11
, pp.
102
110
.
35.
Xu
,
Z.
,
Wu
,
J.
,
Zeng
,
A.
,
Tenenbaum
,
J.
, and
Song
,
S.
,
2019
, “
DensePhysNet: Learning Dense Physical Object Representations Via Multi-Step Dynamic Interactions
,”
Robotics: Science and Systems
,
Freiburg im Breisgau, Germany
,
June 22–26
.
36.
Bauza
,
M.
,
Hogan
,
F. R.
, and
Rodriguez
,
A.
,
2018
, “
A Data-Efficient Approach to Precise and Controlled Pushing
,” CoRR.
37.
Hermans
,
Tucker
,
Fuxin
,
L.
,
Rehg
,
J. M.
, and
Bobick
,
A. F.
,
2013
, “
Learning Contact Locations for Pushing and Orienting Unknown Objects
,”
2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids)
,
Atlanta, GA
,
Oct. 15–17
, IEEE, pp.
435
442
.
38.
Zeng
,
A.
,
Song
,
S.
,
Lee
,
J.
,
Rodriguez
,
A.
, and
Funkhouser
,
T. A.
,
2019
, “
TossingBot: Learning to Throw Arbitrary Objects With Residual Physics
,”
Robotics: Science and Systems
,
Freiburg im Breisgau, Germany
,
June 22–26
.
39.
Ajay
,
A.
,
Wu
,
J.
,
Fazeli
,
N.
,
Bauza
,
M.
,
Kaelbling
,
L. P.
,
Tenenbaum
,
J. B.
, and
Rodriguez
,
A.
,
2018
, “
Augmenting Physical Simulators With Stochastic Neural Networks: Case Study of Planar Pushing and Bouncing
,”
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
,
Madrid, Spain
,
Oct. 1–5
.
40.
Kloss
,
A.
,
Schaal
,
S.
, and
Bohg
,
J.
,
2017
, “
Combining Learned and Analytical Models for Predicting Action Effects
,” CoRR.
41.
Kirchner
,
E. A.
,
Woehrle
,
H.
,
Bergatt
,
C.
,
Kim
,
S. K.
,
Metzen
,
J. H.
,
Feess
,
D.
, and
Kirchner
,
F.
,
2010
, “
Towards Operator Monitoring Via Brain Reading—An EEG-Based Approach for Space Applications
,”
Proceedings of the 10th International Symposium on Artificial Intelligence, Robotics and Automation in Space
,
Sapporo, Japan
,
Aug. 29–Sept. 1
, pp.
448
455
.
42.
Allevato
,
A. D.
,
Short
,
E. S.
,
Pryor
,
M.
, and
Thomaz
,
A. L.
,
2020
, “
Iterative Residual Tuning for System Identification and Sim-to-Real Robot Learning
,”
Auto. Rob.
,
44
(
7
), pp.
1
16
.
43.
Erez
,
T.
,
Tassa
,
Y.
, and
Todorov
,
E.
,
2015
, “
Simulation Tools for Model-Based Robotics: Comparison of Bullet, Havok, Mujoco, Ode and Physx
,”
2015 IEEE International Conference on Robotics and Automation (ICRA)
,
Seattle, WA
,
May 25–30
, IEEE, pp.
4397
4404
.
44.
LeCun
,
Yann A.
,
Bottou
,
Léon
,
Orr
,
Genevieve B.
, and
Müller
,
Klaus-Robert
,
2012
, “Efficient Backprop,”
Neural Networks: Tricks of the Trade
,
Grégoire
Montavon
,
G. B.
Orr
, and
K.
Müller
, eds.,
Springer
,
Berlin, Germany
, pp.
9
48
.
45.
Kingma
,
D. P.
, and
Ba
,
J.
,
2014
, “
Adam: A Method for Stochastic Optimization
,” arXiv preprint arXiv:1412.6980.
46.
Quigley
,
M.
,
Conley
,
K.
,
Gerkey
,
B. P.
,
Faust
,
J.
,
Foote
,
T.
,
Leibs
,
J.
,
Wheeler
,
R.
, and
Ng
,
A. Y.
,
2009
, “
ROS: An Open-Source Robot Operating System
,”
International Conference on Robotics and Automation (ICRA)
,
Kobe, Japan
,
May 12–17
.
47.
Bradski
,
G.
,
2000
, “
The OpenCV Library
,”
Dr. Dobb’s Journal of Software Tools
,
25
(
11
), pp.
122
125
.
You do not currently have access to this content.