Dorans / Pommerich / Holland Linking and Aligning Scores and Scales

2007
ISBN: 978-0-387-49771-6
Verlag: Springer US
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 396 Seiten

Reihe: Statistics for Social and Behavioral Sciences

In their preface to the second edition of Test Equating, Scaling, and Linking, Mike Kolen and Bob Brennan (2004) made the following observation: “Prior to 1980, the subject of equating was ignored by most people in the measurement community except for psychometricians, who had responsibility for equating” (p. vii). The authors went on to say that considerably more attention is now paid to equating, indeed to all forms of linkages between tests, and that this increased attention can be attributed to several factors: 1. An increase in the number and variety of testing programs that use multiple forms and the recognition among professionals that these multiple forms need to be linked. 2. Test developers and publishers, in response to critics, often refer to the role of linking in reporting scores. 3. The accountability movement and fairness issues related to assessment have become much more visible. Those of us who work in this field know that ensuring comparability of scores is not an easy thing to do. Nonetheless, our customers—the te- takers and score users—either assume that scores on different forms of an assessment can be used interchangeably or, like the critics above, ask us to justify our comparability assumptions. And they are right to do this. After all, the test scores that we provide have an impact on decisions that affect people’s choices and their future plans. From an ethical point of view, we are obligated to get it right.

Autoren/Hrsg.

Dorans, Neil J.

Pommerich, Mary

Holland, Paul W.

Inhaltsverzeichnis

1;Dedication;6
2;Foreword;7
3;Preface;9
4;List of Contributors;19
5;1 Overview;21
6;Part 1: Foundations;23
6.1;2 A Framework and History for Score Linking;24
6.1.1;2.1. Introduction;24
6.1.2;2.2. Predicting;25
6.1.3;2.4. Equating: Same Construct and the Same Intended Difficulty and Reliability;39
6.1.4;2.5. A Brief Note on the Theory of Equating;48
6.2;3 Data Collection Designs and Linking Procedures;50
6.2.1;3.1. Introduction;50
6.2.2;3.2. Features of Testing Situations;51
6.2.3;3.3. Types of Linking Considered;52
6.2.4;3.4. Linking Functions and Features of Testing Situations;54
6.2.5;3.5. Linking Designs;56
6.2.6;3.6. Linking Procedures;68
6.2.7;3.7. Summary and Conclusions;73
7;Part 2: Equating;75
7.1;4 Equating: Best Practices and Challenges to Best Practices;76
7.1.1;4.1. Equating;76
7.1.2;4.2. Best Practices;78
7.1.3;4.3. Challenges to Best Practices;80
7.1.4;4.4. Discussion;87
7.1.5;4.5. Summary;88
7.2;5 Practical Problems in Equating Test Scores: A Practitioner’s Perspective;90
7.2.1;5.1. Introduction;90
7.2.2;5.2 The Nonequivalent Groups Anchor Test Design;92
7.2.3;5.3. Characteristics of the New and Old Forms;93
7.2.4;5.4. Characteristics of the Groups Used for Equating;96
7.2.5;5.5. Characteristics of the Anchor Test (Common Items);101
7.2.6;5.6. Conclusions;104
7.3;6 Potential Solutions to Practical Equating Issues;106
7.3.1;6.1. Introduction;106
7.3.2;6.2. Observed-Score Equating Methods;108
7.3.3;6.3. Addressing the Fairness Issue: Population Invariance of Equating Functions;109
7.3.4;6.4. Addressing the Small-Samples Issue: Synthetic Linking Functions;113
7.3.5;6.5. Addressing Differences in Ability in the Two Populations of the NEAT Design;116
7.3.6;6.6. Addressing the Stability of Equating Results: Kernel Equating and Applications;118
7.3.7;6.7. Discussion;122
8;Part 3: Tests in Transition;124
8.1;7 Score Linking Issues Related to Test Content Changes;125
8.1.1;7.1. Introduction;125
8.1.2;7.2. Major Linking Issues for Tests in Transition;127
8.1.3;7.3. Considerations for Data Collection Design;133
8.1.4;7.4. Equatability Analyses;138
8.1.5;7.5. Discussion;149
8.2;8 Linking Scores Derived Under Different Modes of Test Administration;151
8.2.1;8.1. Introduction;151
8.2.2;8.2. Background;152
8.2.3;8.3. Comparability Issues Involving Scores from Computer- Based and Paper- and- Pencil Tests;154
8.2.4;8.4. Mode of Presentation Linking Designs;157
8.2.5;8.5. Random Groups Design;158
8.2.6;8.6. Single Group Counterbalanced Test Design;163
8.2.7;8.7. Anchor Test: Nonequivalent Groups Design;170
8.2.8;8.8. Summary;174
8.3;9 Tests in Transition: Discussion and Synthesis;176
8.3.1;9.1. The Liu and Walker Chapter on Test Content Changes;177
8.3.2;9.2. Eignor Chapter on Mode of Administration;183
8.3.3;9.3. Additional Perspectives;186
9;Part 4: Concordance;191
9.1;10 Sizing Up Linkages;192
9.1.1;10.1. Introduction;192
9.1.2;10.2. What Makes a Linking an Equating?;194
9.1.3;10.3. Why Is Reliability Important?;196
9.1.4;10.4. Uncertainty Reduction;197
9.1.5;10.5. Population Invariance of Linking Functions;201
9.1.6;10.6. Sizing Up Linkages;204
9.1.7;10.7. Summary;210
9.1.8;10.8. Postscript;211
9.2;11 Concordance: The Good, the Bad, and the Ugly;212
9.2.1;11.1 Background;212
9.2.2;11.2. Definitions;214
9.2.3;11.3. The Concordance Process;215
9.2.4;11.4. The Concordance Dilemma;224
9.2.5;11.5. Conclusions;228
9.3;12 Some Further Thoughts on Concordance;230
9.3.1;12.1. Four Common Score Uses in College Admission;230
9.3.2;12.2. Other Thoughts on Concordance Tables;241
9.3.3;12.3. Coastal Aspirations and Midland Acquisitions;242
9.3.4;12.4. Conclusions;242
9.3.5;12.5. Postscript;243
10;Part 5: Vertical Scaling;244
10.1;13 Practical Issues in Vertical Scaling;245
10.1.1;13.1. Introduction;245
10.1.2;13.2. Conceptual Issues;247
10.1.3;13.3. Technical Issues;252
10.1.4;13.4. Implementation Issues;257
10.1.5;13.5. Scale Maintenance Issues;260
10.1.6;13.6. Other Issues;262
10.1.7;13.7. Summary;262
10.2;14 Methods and Models for Vertical Scaling;264
10.2.1;14.1. Introduction;264
10.2.2;14.2. General Modeling Issues in Vertical Scaling Contexts;265
10.2.3;14.3. A Multidimensional, Multigroup IRT Model for Vertical Scaling;271
10.2.4;14.4. Discussion;282
10.3;15 Vertical Scaling and No Child Left Behind;284
10.3.1;15.1. Comments on the Other Vertical Scaling Chapters;284
10.3.2;15.2. Vertical Scales: An Historical Perspective;285
10.3.3;15.3. The NCLB Era;289
10.3.4;15.4. Summary;293
11;Part 6: Linking Group Assessments to Individual Assessments ;295
11.1;16 Linking Assessments Based on Aggregate Reporting: Background and Issues;296
11.1.1;16.1. Linking Methods;298
11.1.2;16.2. The Seeds Are Sown: Linkages of NAEP with IAEP and the ASVAB ;300
11.1.3;16.3. State Linkages with NAEP;304
11.1.4;16.5. More Recent Linkages with NAEP;313
11.1.5;16.6. Problems Requiring Further Research;315
11.1.6;16.7. Conclusions;319
11.2;17 An Enhanced Method for Mapping State Standards onto the NAEP Scale;322
11.2.1;17.1. Introduction;322
11.2.2;17.2. Outline of the Methodology;325
11.2.3;17.3. Details of the Methodology;327
11.2.4;17.4. Estimation of Variances;333
11.2.5;17.5. Results;337
11.2.6;17.6. Mapping the NAEP Achievement Standards onto a State Test Scale;340
11.2.7;17.7. Conclusions and Recommendations;342
11.3;18 Using Aggregate-Level Linkages for Estimation and Validation: Comments on Thissen and Braun & Qian;348
11.3.1;18.1. Comments on the Thissen Chapter;349
11.3.2;18.2. Comments on Braun and Qian;351
11.3.3;18.3. More on Failures of Invariance over Time;357
11.3.4;18.4. Conclusion;362
11.4;19 Postscript;363
11.4.1;19.1. The Descent of Linking;363
11.4.2;19.2. Extreme Linkages;364
11.4.3;19.3. IRT: Tool Versus Theory;365
11.4.4;19.4. Future Trends;365
11.4.5;19.5. Closing Comments;366
12;References;368
13;Author Index;389
14;Subject Index;394