Schweizer / DiStefano Principles and Methods of Test Construction

Standards and Recent Advances
2016
ISBN: 978-1-61676-449-4
Verlag: Hogrefe Publishing
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 336 Seiten

Reihe: Psychological Assessment – Science and Practice

This latest volume in the series Psychological Assessment – Science
and Practice describes the current state-of-the-art in test development
and construction. The past 10–20 years have seen substantial
advances in the methods used to develop and administer tests. In this
volume many of the world’s leading authorities collate these advances
and provide information about current practices, thus equipping researchers
and students to successfully construct instruments using
the latest standards and techniques. The volume is organized into five
related sections. The first explains the benefits of considering the underlying
theory when designing tests, with a focus on factor analysis
and item response theory in construction. The second section looks at
item format and test presentation. The third discusses model testing
and selection, while the fourth goes into statistical methods to identify
group-specific biases. The final section discusses current topics of
special relevance, such as multi-trait multi-state analyses and development
of screening instruments.

Zielgruppe

For psychologists concernced with assessment, psychometricians, and students.

Autoren/Hrsg.

Schweizer, Karl

DiStefano, Christine

Fachgebiete

• Sozialwissenschaften Psychologie Allgemeine Psychologie Differentielle Psychologie, Persönlichkeitspsychologie Psychologische Diagnostik, Testpsychologie

Inhaltsverzeichnis

1;Principles and Methods of Test Construction;1
1.1;Table of Contents;6
2;Chapter 1: Introduction;10
3;Chapter 2: The Use of Standards in Test Development;14
4;Chapter 3: Using Factor Analysis in Test Construction;33
5;Chapter 4: Item Response Theory as a Framework for Test Construction;59
6;Chapter 5: Item Types, Response Formats, and Consequences for Statistical Investigations;90
7;Chapter 6: Adaptive Testing;111
8;Chapter 7: Online Assessment;127
9;Chapter 8: Overview of Estimation Methods and Preconditions for Their Application With Structural Equation Modeling;142
10;Chapter 9: Examining Fit With Structural Equation Models;173
11;Chapter 10: Detecting Differential Item Functioning;204
12;Chapter 11: Assessing Measurement Invariance of Scales Using Multiple-Group Structural Equation Modeling;225
13;Chapter 12: Bifactor Modeling in Construct Validation of Multifactored Tests: Implications for Understanding Multidimensional Constructs and Test Interpretation;254
14;Chapter 13: Creating Short Forms and Screening Measures;279
15;Chapter 14: Using Multitrait–Multimethod Analyses in Testing for Evidence of Construct Validity;295
16;Chapter 15: Method Effects in Psychological Assessment Due to Item Wording and Item Position;315
17;Contributors;330
18;Subject Index;332

Leseproben

Chapter 2 The Use of Standards in Test Development (p. 7-8)
Fons J. R. van de Vijver

Department of Culture Studies, Tilburg University, The Netherlands The present chapter deals with the question of the use of standards in test development. The best known example of such standards are the “Standards for Educational and Psychological Testing” published by the American Educational Research Association, the American Psychological Association, and the national Council on Measurement in Education (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014). The latest version, published in 2014, has just been released. This version updated earlier editions of the standards. For example, the 1999 publication of the standards was an update of the 1985 edition (http://www.teststandards.org/history. htm), which mainly covered the groundwork of test development, grounded in classical statistics, such as classical test theory (Lord & Novick, 1968). At that time it was the most comprehensive overview of do’s and don’ts in test development. The 1999 version was updated to recognize the following (American Psychological Association, 2013):

Changes in federal law and measurement trends affecting validity; testing individuals with disabilities or different linguistic backgrounds; and new types of tests as well as new uses of existing tests. The Standards is written for the professional and for the educated layperson and addresses professional and technical issues of test development and use in education, psychology and employment. Changes from 1999 to the current standards are discussed in this chapter. The Standards, as they are usually referred to, were originally meant for the American market of test users, test developers, and policy makers. However, since the Standards were so comprehensive and similar standards were not formulated in many other countries, the book became an authoritative source in the area of test development.

The aforementioned quotation reflects important characteristics of many standards. First, they are compiled on a voluntary basis. Also, they provide links with the recent developments and psychometrics so as to ensure their scientific soundness and up-to-date nature. Finally, standards are influenced by various developments in science and society. Psychology is not unique in its attempts to enhance the quality of its services by implementing standards. The ISO (International Organization for Standardization; http://www.iso.org/iso/home/about.htm) is the world’s largest developer of voluntary international standards. In this organization, “a standard is a document that provides requirements, specifications, guidelines or characteristics that can be used consistently to ensure that materials, products, processes and services are fit for their purpose” (http://www.iso.org/iso/home/standards.htm). Since 1947, almost 20,000 standards have been developed, covering almost all aspects of technology and business. ISO certification has become an important hallmark of quality. Psychology as a discipline does not have the highly formalized systems of service delivery and quality checks as implemented in the ISO Standards. Still, the same underlying reasoning of enhancing quality by agreeing on standardized procedures can be found in psychology.

Since the launch of the Standards for Educational and Psychological Testing in 1955, many more standards have been developed. The present chapter gives an overview of recent advances in the development of standards in the domain of psychological assessment. It is impossible to review all aspects of the standards presented here. Therefore, I present the contents of standards in tables and deal with other aspects in the main text. I focus on various sets of standards that have been proposed in the last 20 years so as to accommodate new target groups and new modes of administration. Furthermore, I move outside of the realm of psychological and educational testing where the standards were originally developed. More specifically, I describe guidelines that were designed for international testing, notably dealing with translations and adaptations, standards for computer-based and Internet testing, standards for test use, and standards for quality control. Conclusions are drawn in the last section.

Two caveats are needed on terminology. The first is the distinction between educational and psychological testing. This distinction is made more in the American literature than in the European literature, in which the two types of assessment are often considered together. I follow here the European tradition and refer to testing and assessment as involving both educational and psychological instruments. Second, the literature uses two related concepts to refer to desirable features of psychological assessment: standards and guidelines. There is a subtle, yet essential, difference between the two. Standards typically have a prescriptive meaning. Standards describe prerequisites of instruments and their administration needed to ensure valid outcomes of the assessment process. Guidelines, on the other hand, are typically less prescriptive and are formulated as aspired or best practices. The distinction between these aspects seems to be easy to make. In practice, the distinction can be fuzzy as the terms are not always used from the perspective of this difference. Some guidelines are prescriptions, while some standards describe recommendable practices.

The Standards for Educational and Psychological Testing

The Standards for Educational and Psychological Testing are an initiative of the American Educational Research Association (AERA), the American Psychological Association (APA), and the National Council on Measurement in Education (NCME). The Standards for Educational and Psychological Testing have been very influential in psychology and education; the latest version, the fifth revision, was launched in 2014 (a description of the changes in this version was made by Plake & Wise, 2014). The history of the standards has clearly shown that defining norms regarding development, administration, and interpretation of tests helps to advance the quality of the field of assessment. References in the literature to the Standards for Educational and Psychological Testing are numerous (see, e.g., http://teststandards.org/files/Standards_citations_Jan_2010.pdf) and to the best of my knowledge, their reception has not been controversial. The standards are meant to provide criteria for the evaluation of tests, testing practices, and the effects of test use (AERA, APA, & NCME, 2014). The standards are not meant to influence policy, but they can provide recommendations on how psychometrics can be used to underline policy decisions. For instance, rather than prescribing which minimum cutoff score should be established for an admission test, the standards can help to identify conditions that are critical for determining cutoff scores.