Abstract

In recent years, Augmented Reality (AR) has been successfully applied in various fields to assist in the execution of manual tasks. However, there is still a lack of a complete set of criteria for interface design for generating real-time interactive functions and effectively improving task efficiency through AR. In this study, subjects performed two kinds of trajectory tracking tasks in AR, the simple trajectory and complex trajectory. Their task performance under five different sensory feedbacks, namely, central vision, peripheral vision, auditory sensation, force feedback, and no feedback, were compared. The results show that in the trajectory tracking task in complex trajectories, the feedback information should not only provide prompts of error states but also provide suggestions for correcting the actions of the subjects. In addition, compared with visual sensation and auditory sensation, the force feedback has better adaptation. Furthermore, the subjects tend to rely on real-time force feedback to complete difficult tasks. It was found that in the manual trajectory tracking task, determining whether the trajectory tracking task is within the acceptable trajectory range will be affected by the postures subjects use for the tasks and will change the task performance. Therefore, it is suggested that when designing assisted functions, the limitations of the postures of the task should be considered. The experimental results and findings obtained in this study can provide a reference for the assisted interface design of manual tasks in AR.

References

1.
Chu
,
C. H.
,
Liu
,
Y. W.
,
Li
,
P. C.
,
Huang
,
L. C.
, and
Luh
,
Y. P.
,
2020
, “
Programming by Demonstration in Augmented Reality for the Motion Planning of a Three-Axis CNC Dispenser
,”
Int. J. Precis. Eng. Manuf. - Green Technol.
,
7
(
5
), pp.
987
995
.
2.
Müller
,
F.
,
Roner
,
S.
,
Liebmann
,
F.
,
Spirig
,
J. M.
,
Fürnstahl
,
P.
, and
Farshad
,
M.
,
2020
, “
Augmented Reality Navigation for Spinal Pedicle Screw Instrumentation Using Intraoperative 3D Imaging
,”
Spine J.
,
20
(
4
), pp.
621
628
.
3.
Aleotti
,
J.
,
Micconi
,
G.
,
Caselli
,
S.
,
Benassi
,
G.
,
Zambelli
,
N.
,
Bettelli
,
M.
, and
Zappettini
,
A.
,
2017
, “
Detection of Nuclear Sources by UAV Teleoperation Using a Visuo-Haptic Augmented Reality Interface
,”
Sensors
,
17
(
10
), p.
2234
.
4.
Milgram
,
P.
, and
Kishino
,
F.
,
1994
, “
A Taxonomy of Mixed Reality Visual Displays
,”
IEICE Trans. Inf. Syst.
,
77
(
12
), pp.
1321
1329
.
5.
Azuma
,
R. T.
,
1997
, “
A Survey of Augmented Reality
,”
Presence: Teleoperators Virtual Environ.
,
6
(
4
), pp.
355
385
.
6.
Siltanen
,
S.
,
Helle
,
S.
, and
Kähkönen
,
K.
,
2020
, “
Mixed Reality for Safety: A Review of the State-of-the-Art and Future Directions in Research
,”
IEEE Access
,
8
, pp.
9380
9405
.
7.
Ernst
,
M. O.
, and
Bülthoff
,
H. H.
,
2004
, “
Merging the Senses Into a Robust Percept
,”
Trends Cogn. Sci.
,
8
(
4
), pp.
162
169
.
8.
Newell
,
F. N.
,
Ernst
,
M. O.
,
Tjan
,
B. S.
, and
Bülthoff
,
H. H.
,
2001
, “
Viewpoint Dependence in Visual and Haptic Object Recognition
,”
Psychol. Sci.
,
12
(
1
), pp.
37
42
.
9.
Fujisaki
,
W.
, and
Nishida
,
S. Y.
,
2009
, “
Audio–Tactile Superiority Over Visuo–Tactile and Audio–Visual Combinations in the Temporal Resolution of Synchrony Perception
,”
Exp. Brain Res.
,
198
(
2
), pp.
245
259
.
10.
Zhu
,
B.
,
Kaber
,
D.
,
Zahabi
,
M.
, and
Ma
,
W.
,
2020
, “
Effects of Feedback Type and Modality on Motor Skill Learning and Retention
,”
Behav. Inf. Technol.
,
39
(
4
), pp.
431
442
.
11.
Zhang
,
S.
, and
Dai
,
S.
,
2018
, “
Real-Time Trajectory Generation for Haptic Feedback Manipulators in Virtual Cockpit Systems
,”
ASME J. Comput. Inf. Sci. Eng.
,
18
(
4
), p.
041014
.
12.
Mohanty
,
R. R.
,
Castillo
,
R. M.
,
Ragan
,
E. D.
, and
Krishnamurthy
,
V. R.
,
2020
, “
Investigating Force-Feedback in Mid-Air Sketching of Multi-Planar Three-Dimensional Curve-Soups
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
1
), p.
011010
.
13.
Mohanty
,
R. R.
, and
Krishnamurthy
,
V. R.
,
2021
, “
Kinesthetic Metaphors for Precise Spatial Manipulation: A Study of Object Rotation
,”
ASME J. Comput. Inf. Sci. Eng.
,
21
(
2
), p.
021010
.
14.
Mohanty
,
R. R.
,
Raina
,
A. S.
,
Chaudhuri
,
S.
,
Quek
,
F.
,
Sueda
,
S.
, and
Krishnamurthy
,
V. R.
,
2022
, “
Spatial Manipulation in Virtual Peripersonal Space: A Study of Motor Strategies
,”
ASME J. Comput. Inf. Sci. Eng.
,
23
(
2
), p.
021004
.
15.
Sun
,
M.
,
Ren
,
X.
, and
Cao
,
X.
,
2010
, “
Effects of Multimodal Error Feedback on Human Performance in Steering Tasks
,”
J. Inf. Process.
,
18
, pp.
284
292
.
16.
Williams
,
C. K.
,
Tremblay
,
L.
, and
Carnahan
,
H.
,
2016
, “
It Pays to go off-Track: Practicing with Error-Augmenting Haptic Feedback Facilitates Learning of a Curve-Tracing Task
,”
Front. Psychol.
,
7
, p.
2010
.
17.
Howard
,
T.
, and
Szewczyk
,
J.
,
2016
, “
Improving Precision in Navigating Laparoscopic Surgery Instruments Toward a Planar Target Using Haptic and Visual Feedback
,”
Front. Rob. AI
,
3
, p.
37
.
18.
Gao
,
B.
,
Kim
,
H.
,
Lee
,
H.
,
Lee
,
J.
, and
Kim
,
J. I.
,
2018
, “
Effects of Continuous Auditory Feedback on Drawing Trajectory-Based Finger Gestures
,”
IEEE Trans. Hum.-Mach. Syst.
,
48
(
6
), pp.
658
669
.
19.
Condino
,
S.
,
Fida
,
B.
,
Carbone
,
M.
,
Cercenelli
,
L.
,
Badiali
,
G.
,
Ferrari
,
V.
, and
Cutolo
,
F.
,
2020
, “
Wearable Augmented Reality Platform for Aiding Complex 3D Trajectory Tracing
,”
Sensors
,
20
(
6
), p.
1612
.
20.
Zabramski
,
S.
, and
Stuerzlinger
,
W.
,
2012
, “
The Effect of Shape Properties on Ad-Hoc Shape Replication With Mouse, Pen, and Touch Input
,”
Proceeding of the 16th International Academic MindTrek Conference
,
Tampere, Finland
,
Oct. 3–5
, pp.
275
278
.
21.
Kruijff
,
E.
,
Orlosky
,
J.
,
Kishishita
,
N.
,
Trepkowski
,
C.
, and
Kiyokawa
,
K.
,
2018
, “
The Influence of Label Design on Search Performance and Noticeability in Wide Field of View Augmented Reality Displays
,”
IEEE Trans. Vis. Comput. Graph.
,
25
(
9
), pp.
2821
2837
.
22.
Stern
,
J. A.
,
Brown
,
T. B.
,
Wang
,
L.
, and
Russo
,
M. B.
,
2005
, “
Eye and Head Movements in the Acquisition of Visual Information
,”
Psychologia
,
48
(
2
), pp.
133
145
.
23.
Zareinia
,
K.
,
Yang
,
X. D.
,
Irani
,
P.
, and
Sepehri
,
N.
,
2009
, “
Evaluating Factors That Influence Path Tracing With Passive Haptic Guidance
,”
Proceedings of the International Conference on Haptic and Audio Interaction Design
,
Dresden, Germany
,
Sept. 10–11
,
Springer
,
Berlin/Heidelberg
, pp.
21
30
.
24.
Bluteau
,
J.
,
Coquillart
,
S.
,
Payan
,
Y.
, and
Gentaz
,
E.
,
2008
, “
Haptic Guidance Improves the Visuo-Manual Tracking of Trajectories
,”
PLoS One
,
3
(
3
), p.
e1775
.
You do not currently have access to this content.