A number of cognitive skills relevant to conceptual design have been previously identified: divergent thinking, visual thinking, spatial reasoning, qualitative reasoning, and problem formulation. A battery of standardized test has been developed for each of these skills. This is the second paper in a series of papers on testing individual skill level differences in engineers and engineering students. In the first paper, we reported on the theoretical and empirical basis for divergent thinking test, as well as, on test formulation, data collection, norming studies, and statistical validation of that test. This paper focuses similarly on the efforts related to the visual thinking and spatial reasoning in engineering context. We have decomposed visual thinking into six categories: visual comprehension including perceptual speed, visual memory (that is, the visual memory system), visual synthesis mental image manipulation/transformation, spatial reasoning, and graphical expression/elaboration. We discuss the theoretical basis of a comprehensive test for engineers, test composition, trial runs, and computation of reliability measures. The alpha version was given to a small set of subjects to determine clarity of the questions and gauge difficulty level. The beta version was used for norming and test validation from over 500 samples that included engineering students and a smaller number of practicing engineers. Construct validation was achieved through basing the construction of our instrument off other well-known measures of visual thinking, while content validity was assured through thoroughly sampling the domain of visual thinking and including a variety of items both pertinent and specific to the engineering design process. The factor analysis reveals that there are possibly two eigenvalues above 1.0, an indication that it is a stable and accurate instrument. We emphasize that these tests are not just dependent on native abilities, but on education and experience; design skills are teachable and learnable.

References

1.
Shah
,
J.
,
2005
, “
Identification, Measurement & Development of Design Skills in Engineering Education
,”
International Conference on Engineering Design (ICED05)
,
Melbourne, Australia
, Aug.
2.
Khorshidi
,
M.
,
Woodward
,
J.
, and
Shah
,
J.
,
2012
, “
Towards a Comprehensive Test of Qualitative Reasoning Skill in Design
,”
ASME Design Theory and Methodology Conference, DETC2012-70657
,
Chicago
.
3.
Shah
,
J.
,
Millsap
,
R.
,
Woodward
,
J.
, and
Smith
,
S. M.
,
2012
, “
Applied Tests of Design Skills—Part I: Divergent Thinking
,”
ASME J. Mech. Des.
,
V134
, p.
021005
.10.1115/1.4005594
4.
Anderson
,
J. R.
,
2005
,
Cognitive Psychology and Its Implications
,
Worth Publishers
,
New York
.
5.
Matlin
,
M. W.
,
2009
,
Cognition
, 7th ed.,
John Wiley & Sons, Inc.
,
Hoboken, NJ
.
6.
Schacter
,
D. L.
,
Gilbert
,
D. T.
, and
Wegner
,
D. M.
,
2009
,
Psychology
,
Worth Publishers
,
New York, NY
.
7.
Finke
,
R. A.
,
1996
, “
Imagery, Creativity, and Emergent Structure
,”
Conscious Cogn.
,
5
, pp.
381
393
.10.1006/ccog.1996.0024
8.
Finke
,
R. A.
,
1985
,
Theories Relating Mental Imagery to Perception
,”
Psychol. Bull.
,
98
, pp.
236
259
.10.1037/0033-2909.98.2.236
9.
Finke
,
R. A.
,
Pinker
,
S.
, and
Farah
,
M.
,
1989
, “
Reinterpreting Visual Patterns in Mental Imagery
,”
Cogn. Sci.
,
13
, pp.
51
78
.10.1207/s15516709cog1301_2
10.
Ester
,
E. F.
,
Serences
,
J. T.
, and
Awh
,
E.
,
2009
, “
Spatially Global Representations in Human Primary Visual Cortex During Working Memory Maintenance
,”
J. Neurosci.
,
29
(
48
), pp.
15258
15265
.10.1523/JNEUROSCI.4388-09.2009
11.
Kosslyn
,
S. M.
,
Thompson
,
W. L.
,
Gitelman
,
D. R.
, and
Alpert
,
N. M.
,
1998
, “
Neural Systems That Encode Categorical vs. Coordinate Spatial Relations: PET Investigations
,”
Psychobiology
,
26
, pp.
333
347
.
12.
Baddeley
,
A. D.
, and
Wilson
,
B. A.
,
2002
, “
Prose Recall and Amnesia: Implications for the Structure of Working Memory
,”
Neuropsychologia
,
40
, pp.
1737
1743
.10.1016/S0028-3932(01)00146-4
13.
Brown
,
H. D.
,
Kosslyn
,
S. M.
, and
Dror
, I
. E.
,
1998
, “
Aging and Scanning of Imagined and Perceived Visual Images
,”
Exp. Aging Res.
,
24
, pp.
181
194
.10.1080/036107398244319
14.
Paivio
,
A.
, and
Csapo
,
K.
,
1973
, “
Picture Superiority in Free Recall: Imagery or Dual Coding?
,”
Cogn. Psychol.
,
5
(
2
), pp.
176
206
.10.1016/0010-0285(73)90032-7
15.
Unsworth
,
N.
, and
Engle
,
R. W.
,
2008
, “
Speed and Accuracy of Accessing Information in Working Memory: An Individual Differences Investigation of Focus Switching
,”
J. Exp. Psychol.: Learn., Mem., Cognit.
,
34
(
3
), pp.
616
630
.10.1037/0278-7393.34.3.616
16.
Baddeley
,
A. D.
,
2007
,
Working Memory, Thought, and Action
,
Oxford University
, Oxford, UK.
17.
Brady
,
T. F.
,
Konkle
,
T.
, and
Alvarez
,
G. A.
,
2009
, “
Compression in Visual Working Memory: Using Statistical Regularities to Form More Efficient Memory Representations
,”
J. Exp. Psychol. Gen.
,
138
(
4
), pp.
487
502
.10.1037/a0016797
18.
Shepard
,
R. N.
, and
Metzler
,
J.
,
1971
, “
Mental Rotation of Three-Dimensional Objects
,”
Science
,
171
, pp.
701
703
.10.1126/science.171.3972.701
19.
Cooper
,
L.
, and
Lang
,
J. M.
,
1996
, “
Imagery and Visual–Spatial Representations. Imagery and Visual–Spatial Representations
,”
Memory: Handbook of Perception and Cognition
,
E. L.
Bjork
and
R. A.
Bjork
, eds.,
Academic Press
, London, UK.
20.
Howes
,
M. B.
,
2007
,
Human Memory: Structures and Images
,
Sage Publications, Inc.
, Thousand Oaks, CA.
21.
De Beni
,
R.
,
Pazzaglia
,
F.
, and
Gardini
,
S.
,
2006
, “
The Role of Mental Rotation and Age in Spatial Perspective-Taking Tasks
,”
Appl. Cogn. Psychol.
,
20
(
6
), pp.
807
821
.10.1002/acp.1229
22.
Dror
,
I. E.
, and
Kosslyn
,
S. M.
,
1994
, “
Mental Imagery and Aging
,” Psychol. Aging,
9
(
1
), pp.
90
102
.10.1037/0882-7974.9.1.90
23.
Paivio
,
A.
,
1971
,
Imagery and Verbal Processes
,
Holt, Rinehart, and Winston
,
New York
.
24.
Bower
,
G. H.
,
1970
, “
Analysis of the Mnemonic Device
,”
Am. Sci.
,
58
, pp.
496
501
.
25.
Bellezza
,
F. S.
,
1981
, “
Mnemonic Devices: Classification, Characteristics, and Criteria
,”
Rev. Educ. Res.
,
51
(
2
), pp.
247
275
.10.3102/00346543051002247
26.
Chambers
,
D.
, and
Reisberg
,
D.
,
1985
, “
Can Mental Images be Ambiguous?
,”
J. Exp. Psychol. Hum. Percept. Perform.
,
11
(
3
), pp.
317
328
.10.1037/0096-1523.11.3.317
27.
Reed
,
S.
,
1974
, “
Structural Descriptions and the Limitations of Visual Images
,”
Mem. Cognit.
,
V2
(
2
), pp.
329
336
.
28.
Tversky
,
B.
, and
Hard
,
B. M.
,
2009
, “
Embodied and Disembodied Cognition: Spatial Perspective-Taking
,”
Cognition
,
110
, pp.
124
129
.10.1016/j.cognition.2008.10.008
29.
Tversky
,
B.
,
2005
, “
Functional Significance of Visuospatial Representations
,”
Handbook of Higher-Level Visuospatial Thinking
,
P.
Shah
and
A.
Miyake
, eds.,
Cambridge University Press
,
New York
, pp.
1
34
.
30.
Goel
,
V.
,
1995
,
Sketches of Thought
,
MIT
,
Cambridge, MA
.
31.
Goldschmidt
,
G.
,
1994
, “
On Visual Design Thinking: The Vis Kids of Architecture
,”
Des. Stud.
,
15
(
2
), pp.
158
174
.10.1016/0142-694X(94)90022-1
32.
McKim
,
R.
,
1980
,
Experiences in Visual Thinking
,
Wadsworth, Inc.
,
Belmont, CA
.
33.
Ullman
,
D.
,
Wood
,
S.
, and
Craig
,
D.
,
1990
, “
The Importance of Drawing in the Mechanical Design Process
,”
Comput. Graphics
,
V14
(
2
), pp.
263
274
.
34.
Goldschmidt
,
G.
,
1992
, “
Serial Sketching: Visual Problem Solving in Designing
,”
Cybern. Syst.
,
23
(
2
), pp.
191
219
.10.1080/01969729208927457
35.
Larkin
,
J.
, and
Simon
,
H.
,
1987
, “
Why a Diagram is (Sometimes) Worth Ten Thousand Words
,”
Cogn. Sci.
,
11
, pp.
65
99
.10.1111/j.1551-6708.1987.tb00863.x
36.
Goldschmidt
,
G.
,
2000
, “
The Backtalk of Self-Generated Sketches
,”
Des. Issues
,
19
(
1
), pp.
72
88
.10.1162/074793603762667728
37.
Suwa
,
M.
, and
Tversky
,
B.
,
2001
, “
Constructive Perception in Design
,”
Computational and Cognitive Models of Creative Design
,
J. S.
Gero
and
M. L.
Maher
, eds.,
University of Sydney
,
Sydney, Australia
.
38.
Beittel
,
K.
,
1972
,
Mind and Context in the Art of Drawing: An Empirical and Speculative Account
,
Holt Rinehart and Winston
,
NY
.
39.
McKoy
,
F.
,
Vargas-Hernandez
,
N.
,
Summers
,
J.
, and
Shah
,
J.
,
2001
, “
Influence of Design Representation on Effectiveness of Idea Generation
,”
ASME DTM Conference
,
Pittsburgh, PA
, Sept. 10–13.
40.
Tversky
,
B.
,
Agrawala
,
M.
,
Heiser
,
J.
,
Lee
,
P.
,
Hanrahan
,
P.
,
Phan
,
D.
,
Stolte
,
C.
, and
Daniele
,
M.-P.
,
2007
, “
Cognitive Design Principles for Generating Visualizations
,”
Applied Spatial Cognition: From Research to Cognitive Technology
,
G.
Allen
, ed.,
Erlbaum
,
Mahwah, NJ
.
41.
McGown
,
A.
,
Green
,
G.
, and
Rodgers
,
P.
,
1998
, “
Visible Ideas: Information Patterns of Conceptual Sketch Activity
,”
Des. Stud.
,
19
(
4
), pp.
431
453
.10.1016/S0142-694X(98)00013-1
42.
Verstijnen
, I
. M.
,
Van Leeuwen
,
C.
,
Goldschmidt
,
G.
,
Hamel
,
R.
, and
Hennessey
,
J. M.
,
1998
, “
Sketching and Creative Discovery
,”
Des. Stud.
,
19
(
4
), pp.
519
546
.10.1016/S0142-694X(98)00017-9
43.
Hegarty
,
M.
, and
Kozhevnikov
,
M.
,
1999
, “
Types of Visual–Spatial Representations and Mathematical Problem Solving
,”
Educ. Psychol.
,
91
, pp.
684
689
.
44.
Kozhevnikov
,
M.
,
Hegarty
,
M.
, and
Mayer
,
R.
,
2002
, “
Revising the Visualizer-Verbalizer Dimension: Evidence for Two Types of Visualizers
,” Cogn. Instruct.,
20
(
1
), pp.
47
77
.10.1207/S1532690XCI2001_3
45.
Kozhevnikov
,
M.
,
Hegarty
,
M.
, and
Motes
,
R.
,
2007
, “
Spatial Visualization in Physics Problem Solving
,”
Cogn. Sci.
,
V31
, pp.
549
579
.
46.
Presmeg
,
N.
,
1986
, “
Visualization in High School Mathematics
,”
Learn. Math.
,
63
(
3
), pp.
42
46
.
47.
Reynolds
,
C. R.
,
Pearson
,
N. A.
, and
Voress
,
J. K.
,
2002
,
Developmental Test of Visual Perception—Adolescent and Adult
,
Pro-Ed
,
Austin, TX
.
48.
College Entrance Examination Board
,
1939
, CEEB Special Aptitude Test in Spatial Relations, New York.
49.
Fay
,
E.
, and
Quaiser-Pohl
,
C.
,
1999
,
Schnitte—Ein Test zur Erfassung des räumlichen Vorstellungsvermögens
,
Swets Test Services
,
Frankfurt, Germany
.
50.
Rey
,
A.
,
1941
, “
L'examen Psychologique dans les cas d'encephalopathie Traumatique
,”
Arch. de Psychol.
,
28
, pp.
215
285
.
51.
Ekstrom
,
R. B.
,
French
,
J. W.
,
Harman
,
H. H.
, and
Dermen
,
D.
,
1976
,
Kit of Factor-Referenced Cognitive Tests
,
Educational Testing Service
,
Princeton, NH
.
52.
Vandenberg
,
S. G.
, and
Kuse
,
A. R.
,
1978
, “
Mental Rotation, a Group Test of Three-Dimensional Spatial Visualization
,”
Percept. Mot. Skills
,
47
, pp.
599
604
.10.2466/pms.1978.47.2.599
53.
Guay
,
R. B.
,
1980
,
Purdue Spatial Visualization Test
,
Purdue Research Foundation
,
West Lafayette, IN
.
54.
Just
,
M. A.
, and
Carpenter
,
P. A.
,
1985
, “
Cognitive Coordinate Systems: Accounts of Mental Rotation and Individual Differences in Spatial Ability
,”
Psychol. Rev.
,
92
, pp.
137
172
.10.1037/0033-295X.92.2.137
55.
Seddon
,
G. M.
,
Eniaiyeju
,
P. A.
, and
Chia
,
L. H. L.
,
1985
, “
The Factor Structure for Mental Rotations of Three-Dimensional Structures Represented in Diagrams
,”
Res. Sci. Technol. Educ.
,
3
(
1
), pp.
29
42
.
56.
Shah
,
J.
,
Woodward
,
J.
, and
Smith
,
S.
,
2011
, “
Applied Tests of Engineering Design Skills: Visual Thinking Characterization, Test Development and Validation
,” ICED 2011, Copenhagen, Denmark.
You do not currently have access to this content.