The study contains the following purposes: (1) exploring the test-structure of Raven’s Advanced Progressive Matrics (APM), (2) examining the process of solving APM, (3) investigating possible factors that affect the construct validity of APM, (4) developing New Figure Reasoning Test (NFRT), using cognitive measurement approach test design model and computer technology, (5) contrasting APM and NFRT from the viewpoints of test designing and empirical analysis, and (6) developing an automatic item-generation and online testing system for NFRT.
The main findings of the research results can be summarized as follows.
1.Test-structure of APM: The one-factor model can better explain the response patterns of both the high-school samples and the college samples. Since the main factor can not account for the variance validly, some other factors are believed to affect the test-structure.
2.Problem-solving process of APM: Constant in low rule, quantitative pairwise progression rule, figure addition or subtraction rule, distribution of three values rule, and distribution of two values rules─five problem-solving rules proposed by Carpenter et al. (1990)─can explain clearly how the testers induct and deduct while working on APM.
3.Other factors that affect the construct validity of APM: In addition to ability of induction and deduction, abilities not related to measurement goals─such as abstract thinking ability at a higher level, goal-management ability, mental flexibility, strategies of problem-solving, and learning ability─are all required by the process of solving APM. APM henceforth has poor construct validity.
4.Construct validity of NFRT: The construct validity of NFRT can be revealed by the following: (a) the test-structure of NFRT meets one-factor model; (b) Constant in low rule, quantitative pairwise progression rule, figure addition or subtraction rule, and distribution of three values rule can account for how the testers induct or deduct in the problem-solving process; (c) NFRT has a good nomothetic span.
5.Contrast between NFRT and APM: Unlike APM, NFRT has drawn attention of the researcher when he designs the test not only to more carefully control the cognitive process related to the measurement goal, but also to exclude affective factors that are not related to the measurement goals. The contrast between the empirical results of NFRT and APM reveal that the construct validity of the former is better than the latter.
6.Functions of automatic item-generation and online testing system: Responding to a specific input, the system can produce a test item with specific content features and specific difficulty, and yet various surface features. Secondly, the system can control the testing process according to specific inputs. Moreover, the system can generate an item automatically and give it out immediately. Finally, the system can keep track of the information of testing process, item features and testers’ response.
The study concludes with suggestions for using the two tests and further research.