Russ-Eft / RussEft / Bober-Michel Fieldbook of ibstpi Evaluator Competencies

1. Auflage 2014
ISBN: 978-1-62396-444-3
Verlag: Information Age Publishing
Format: EPUB
Kopierschutz: Adobe DRM (»Systemvoraussetzungen)

E-Book, Englisch, 113 Seiten

Reihe: The Ibstpi Book Series

The book, Evaluator Competencies: Standards for the Practice of Evaluation, details the development and validation of evaluator competencies by the International Board of Standards for Training, Performance, and Instruction (ibstpi). Developing an understanding of the ibstpi Evaluator Competencies may not be sufficient for individuals to determine how to improve their, or their colleagues', competencies. This Fieldbook provides additional information, resources, and tools to assist those who want to improve their own competencies or those who want to help other individuals improve. Thus, the goals for the present companion volume are: • To provide additional practical information in each of the four evaluator domains (i.e., professional foundations, planning and designing the evaluation, implementing the evaluation plan, and managing the evaluation). • To present practical tools and resources that support specific evaluator competencies, whether as an internal or an external evaluator. • To offer practical insights on the evaluator competencies from experienced evaluators. • To provide practical evaluation exercises and resources that can be used with undergraduate and graduate courses.

Autoren/Hrsg.

Russ-Eft, Darlene F.

RussEft,

Bober-Michel, Marcie J.

F., Darlene

Koszalka,

A., Tiffany

BoberMichel,

J., Marcie

Sleezer,

M., Catherine

Inhaltsverzeichnis

1;Fieldbook of ibstpi Evaluator Competencies;2
1.1;A Volume in Ibstpi Book Series;2
2;CONTENTS;8
2.1;Domain: Professional Foundations;8
2.1.1;1. Communicate Effectively in Written, Oral, and Visual Form 3;8
2.1.2;2. Establish and Maintain Professional Credibility 7;8
2.1.3;3. Demonstrate Effective Interpersonal Skills 13;8
2.1.4;4. Observe Ethical and Legal Standards 19;8
2.1.5;5. Demonstrate Awareness of the Politics of Evaluation 23;8
2.2;domain: Planning and Designing the Evaluation;8
2.2.1;6. Develop an Effective Evaluation Plan 29;8
2.2.2;7. Develop a Management Plan for the Evaluation 29;8
2.2.3;8. Devise Data Collection Strategies to Support the Evaluation Questions and Design 39;8
2.2.4;9. Pilot Test the Data Collection Instruments and Procedures 43;8
2.3;Domain: Implementing the Evaluation Plan;8
2.3.1;10. Collect Data 49;8
2.3.2;11. Analyze and Interpret Data 53;8
2.3.3;12. Disseminate and Follow Up the Findings and Recommendations 65;9
2.4;Domain: Managing the Evaluation;9
2.4.1;13. Monitor the Evaluation Plan 85;9
2.4.2;14. Work Effectively With Personnel and Stakeholders 91;9
2.5;Ibstpi Book Series;3
2.6;Fieldbook of ibstpi Evaluator Competencies;6
2.6.1;by;6
2.6.2;Darlene F. Russ-Eft Oregon State University;6
2.6.3;Marcie J. Bober-Michel San Diego State University;6
2.6.4;Tiffany A. Koszalka Syracuse University;6
2.6.5;and;6
2.6.6;Catherine M. Sleezer Employee Training and Performance Improvement Specialists;6
2.6.6.1;Information Age Publishing, Inc.;6
2.6.6.2;Charlotte, North Carolina • www.infoagepub.com;6
2.7;Companion Volume Accompanying Evaluator Competencies: Standards for the Practice of Evaluation in Organizations;10
2.7.1;Overview and Purpose;10
2.7.2;Table 1. A Listing of the Domain and Competencies Associated With Each Section;11
2.7.2.1;1. Communicate effectively in written, oral, and visual form;11
2.7.2.2;2. Establish and maintain professional credibility;11
2.7.2.3;3. Demonstrate effective interpersonal skills;11
2.7.2.4;4. Observe ethical and legal standards;11
2.7.2.5;5. Demonstrate awareness of the politics of evaluation;11
2.7.3;Table 1. Continued;12
2.7.3.1;6. Develop an effective evaluation and;12
2.7.3.2;7. Develop a management plan for the evaluation;12
2.7.3.3;8. Devise data collection strategies to support the evaluation questions and design;12
2.7.3.4;9. Pilot test the data collection instruments and procedures;12
2.7.3.5;10. Collect data;12
2.7.3.6;11. Analyze and interpret data;12
2.7.3.7;12. Disseminate and follow up the findings and recommendations;12
2.7.4;Table 1. Continued;13
2.7.4.1;13. Monitor the evaluation plan;13
2.7.4.2;14. Work effectively with personnel and stakeholders;13
2.8;evaluator competency 1;16
2.8.1;Communicate Effectively In Written, Oral, And Visual Form;16
2.8.1.1;Communicating and Reporting;16
2.8.1.1.1;Purpose;16
2.8.1.1.2;Background;16
2.8.1.1.3;Introduction to the Resource or Tool for This Section;17
2.8.1.1.4;Resources for Adaptation to Your Evaluation Needs;17
2.8.1.2;ADDITIONAL RESOURCES;19
2.8.2;domain;14
2.8.2.1;Professional Foundations;14
2.9;evaluator competency 2;20
2.9.1;Establish and Maintain Professional Credibility;20
2.9.1.1;Choosing an Evaluator;20
2.9.1.1.1;Purpose;20
2.9.1.1.2;Background;20
2.9.1.1.3;Introduction to the Resources or Tool for This Section;21
2.9.1.1.4;Resources for Adaptation to Your Evaluation Needs;21
2.9.2;Table 2.1. Checklist for Matching Evaluator Qualifications to the Stages of Evaluatio;21
2.9.3;Table 2.1. Checklist for Matching Evaluator Qualifications to the Stages of Evaluation (Continued);22
2.9.4;Table 2.1. Checklist for Matching Evaluator Qualifications to the Stages of Evaluation (Continued);23
2.9.4.1;ADDITIONAL RESOURCES;24
2.10;evaluator competency 3;26
2.10.1;Demonstrate Effective Interpersonal Skills;26
2.10.1.1;Analyzing the Evaluation Context;26
2.10.1.1.1;Purpose;26
2.10.1.1.2;Background;26
2.10.1.1.3;Group or Organizational Influences;27
2.10.1.1.4;People Influences;27
2.10.1.1.4.1;Introduction to the Resources or Tool for This Section;28
2.10.1.1.4.2;Resources for Adaptation to Your Evaluations Needs;28
2.10.1.2;Exercise: The Implementation of the Performance Analysis for Training Model at MSC;29
2.10.1.3;FOOTNOTES;30
2.10.1.4;ADDITIONAL RESOURCES;31
2.11;evaluator competency 4;32
2.11.1;Observe Ethical and Legal Standards;32
2.11.1.1;Ethical Standards;32
2.11.1.1.1;Purpose;32
2.11.1.1.2;Background;32
2.11.1.1.3;Introduction to the Resources or Tool for This Section;33
2.11.1.1.4;Resources for Adaptation to Your Evaluation Needs;33
2.11.1.2;ADDITIONAL RESOURCES;34
2.12;evaluator competency 5;36
2.12.1;Demonstrate awareness of the politics of evaluation;36
2.12.1.1;Dealing With Politics in an Evaluation;36
2.12.1.1.1;Purpose;36
2.12.1.1.2;Background;36
2.12.1.1.3;Introduction to the Resources or Tool for This Section;37
2.12.1.1.4;Resources for Adaptation to Your Evaluation Needs;38
2.12.1.2;Additional Resources;38
2.12.2;Domain;40
2.12.2.1;Planning and Designing the Evaluation;40
2.13;Evaluator competencies 6 and 7;42
2.13.1;Develop an Effective Evaluation Plan and Develop a Management Plan for the Evaluation;42
2.13.1.1;An Evaluation Plan;42
2.13.1.1.1;Purpose;42
2.13.1.1.2;Background;42
2.13.1.1.3;Introduction to the Resources or Tool for This Section;43
2.13.1.1.4;Resources for Adaptation to Your Evaluation Needs;43
2.13.2;Table 6.1. Outline for an Evaluation Plan;43
2.13.3;Table 6.1. Outline for an Evaluation Plan (Continued);44
2.13.4;Table 6.1. Outline for an Evaluation Plan (Continued);45
2.13.4.1;Entrepreneurial Consulting Workshops Program Evaluation Plan1;45
2.13.4.1.1;Program Background and Rationale;46
2.13.4.1.2;Evaluation Purpose;47
2.13.4.1.3;Audience and Stakeholders;47
2.13.4.1.4;Key Questions;48
2.13.4.1.4.1;1. To what extent have program graduates established their training and development consulting corporations based on sound business, accounting, and marketing practices?;48
2.13.4.1.4.2;2. To what extent are program graduates able to communicate effectively with varied stakeholders in the areas of working relationships, professional expertise, and consulting theory?;48
2.13.4.1.4.3;3. How effectively are program graduates able to coach and facilitate clients in resolution of business, organizational management, and employee concerns?;48
2.13.4.1.4.4;4. To what extent do program graduates have the skills to perform consulting competencies including: conducting needs assessments, creating management plans for contractual services, and problem- solving in the context of training and development con...;48
2.13.4.2;Note;51
2.13.4.3;ADDITIONAL RESOURCES;51
2.13.5;Appendix A. The Entrepreneurial Consulting Workshops Program Logic Model;49
2.13.6;Appendix A. The Entrepreneurial Consulting Workshops Program Logic Model (Continued);50
2.14;evaluator competency 8;52
2.14.1;Devise Data Collection Strategies to Support the Evaluation Questions and Design;52
2.14.1.1;Data Collection Strategies;52
2.14.1.1.1;Purpose;52
2.14.1.1.2;Background;52
2.14.1.1.3;Introduction to the Resources or Tool for This Section;53
2.14.1.1.3.1;Resources for Adaptation to Your Evaluation Needs;53
2.14.2;Table 8.1. Data Collection Methods Listed in Ascending Order of Obtrusiveness;53
2.14.3;Table 8.1. Data Collection Methods Listed in Ascending Order of Obtrusiveness (Continued);54
2.14.4;Table 8.2. Checklist of Questions to Consider When Choosing Data Collection Methods;55
2.14.4.1;ADDITIONAL RESOURCES;55
2.15;evaluator competency 9;56
2.15.1;Pilot Test the Data Collection Instruments and Procedures;56
2.15.1.1;Pilot Testing;56
2.15.1.1.1;Purpose;56
2.15.1.1.2;Background;56
2.15.1.1.3;Introduction to the Resources or Tool for This Section;57
2.15.1.1.4;Resources for Adaptation to Your Evaluation Needs;58
2.15.1.2;ADDITIONAL RESOURCES;58
2.15.2;Domain;60
2.15.2.1;implementing The evaluation Plan;60
2.16;evaluator competency 10;62
2.16.1;Collect Data;62
2.16.1.1;Data Collection via Interviews;62
2.16.1.1.1;Purpose;62
2.16.1.1.2;Background;62
2.16.1.1.3;Introduction to the Resources or Tool for This Section;64
2.16.1.1.4;Resources for Adaptation to Your Evaluation Needs;64
2.16.1.2;Sample Interview Background, Script, and Questions;64
2.16.1.2.1;Background;64
2.16.1.2.2;Interview Script;64
2.16.1.2.3;Sample Questions;65
2.16.1.2.3.1;1. What do you think about the recent customer survey?;65
2.16.1.2.3.2;2. What changes in our production process do you think we should make to address our weaknesses and improve our strengths?;65
2.16.1.2.3.3;3. What other factors outside the organization could lead to changes in our production process? Follow-up: What could the changes look like?;65
2.16.1.2.3.4;4. Are there other factors inside the organization that could lead to changes in our production process? Follow-up: What could the changes look like?;65
2.16.1.2.3.5;5. What are the barriers to making efficient and effective changes in the production process?;65
2.16.1.2.3.6;6. What are the supports for making efficient and effective changes in the production process?;65
2.16.1.2.3.7;7. In thinking about our future production process, what additional information should be considered?;65
2.16.1.2.3.8;8. Thank you for meeting with me to discuss the strategic needs for the production area. If I think of additional questions later or need clarification on a question, can I call you?;65
2.16.1.3;ADDITIONAL RESOURCES;65
2.17;Evaluator COMPETENCY 11;66
2.17.1;Analyze and Interpret Data;66
2.17.1.1;Analyzing Qualitative Data;66
2.17.1.1.1;Purpose;66
2.17.1.1.2;Background;66
2.17.1.1.3;Introduction to the Resources or Tool for This Section;67
2.17.1.1.4;Resources for Adaptation to Your Evaluation Needs;67
2.17.2;Table 11.1. Example of a Spreadsheet or Table Developed to Analyze Qualitative Data;67
2.17.3;Table 11.1. Example of a Spreadsheet or Table Developed to Analyze Qualitative Data (Continued);68
2.17.4;Table 11.2. Example Showing Analysis of Qualitative Data (by Category);69
2.17.5;Table 11.3. Example Showing Presentation of Qualitative Data in a Report;69
2.18;Evaluator Competency 11 Continued;70
2.18.1;Using Numbers to Describe Data;70
2.18.1.1;Purpose;70
2.18.1.2;Background;70
2.18.1.3;Introduction to the Resources or Tool for This Section;70
2.18.1.4;Resources for Adaptation to Your Evaluation Needs;71
2.18.2;Make Inferences about Numeric Data;72
2.18.2.1;Purpose;72
2.18.2.2;Background;72
2.18.2.3;Introduction to the Resources or Tool for This Section;73
2.18.2.4;Resources for Adaptation to Your Evaluation Needs;73
2.18.2.5;Mean Differences Between Groups;73
2.18.2.6;Relationships Between Variables;74
2.18.2.6.1;Figure 11.1. Negative correlation.;76
2.18.3;Additional Resources;76
2.19;EVALUATOR COMPETENCY 12;78
2.19.1;DISSEMINATE AND FOLLOW-UP THE FINDINGS AND RECOMMENDATIONS;78
2.19.1.1;Writing for Different Audiences;78
2.19.1.1.1;Purpose;78
2.19.1.1.2;Background;79
2.19.1.1.3;Introduction to the Resources or Tool for This Section;79
2.19.1.1.4;Resources for Adaptation to Your Evaluation Needs;80
2.19.1.1.5;Title;80
2.19.1.1.6;Executive Summary;80
2.19.1.1.7;Overview;80
2.19.1.1.8;Impact of the Literature Review;80
2.19.1.1.9;Contextual Factors;81
2.19.1.1.10;Methods;81
2.19.1.1.11;Findings;81
2.19.1.1.12;Recommendations;81
2.19.1.2;Disseminate and Follow Up the Findings and Recommendations;82
2.19.1.3;Supporting Results with Tables;82
2.19.1.3.1;Purpose;82
2.19.1.3.2;Background;82
2.19.1.3.3;Introduction to the Resources or Tool for This Section;83
2.19.1.3.4;Resources for Adaptation to Your Evaluation Needs;83
2.19.2;Table 12.1. Instructor Facilitation Preferences;83
2.19.3;Table 12.2. Respondents Perceptions of the Training (n = 20);84
2.19.4;Table 12.3. User-Friendliness of the Website;84
2.19.4.1;Supporting Results With Charts and Graphs;85
2.19.4.1.1;Purpose;85
2.19.4.1.2;Background;85
2.19.4.1.3;Introduction to the Resources or Tool for This Section;85
2.19.4.1.4;Resources for Adaptation to Your Evaluation Needs;86
2.19.4.1.5;Examples of How to Display Data;86
2.19.4.1.5.1;1. The facilitator organized the session well.;86
2.19.4.1.5.2;2. The materials that the instructor provided during the course are useful on the job.;86
2.19.4.1.5.3;3. What is your level of proficiency in basic computer skills?;86
2.19.4.1.5.4;4. How important are the skills taught in the course to your job?;86
2.19.5;Table 12.4. Session Organization and Handout Quality;86
2.19.5.1;Figure 12.1. Histogram showing the survey ratings for course organization and materials.;87
2.19.6;Table 12.5. Basic Computer Functions (Skill Level/Skill Importance);88
2.19.6.1;Figure 12.2. Basic computer functions (skill level/skill importance shown as a line graph).;89
2.19.6.2;Figure 12.3. Basic computer functions (skill level/skill importance) shown as a stacked bar graph.;90
2.19.6.3;Tips for Displaying Data;90
2.19.6.4;Formatting Tables, Charts, and Graphs For Use in Electronic Form;90
2.19.6.4.1;Purpose;90
2.19.7;Table 12.6. Test Scores for Participants at Each Proficiency Level;91
2.19.7.1;Background;91
2.19.8;Table 12.7. Truncated Table of Test Scores for Participants at Each Proficiency Level;92
2.19.8.1;Introduction to the Resources or Tool for This Section;92
2.19.8.2;Resources for Adaptation to Your Evaluation Needs;92
2.19.8.2.1;Figure 12.3a. Fixed border.;93
2.19.8.2.2;Figure 12.3b. Moveable border.;93
2.19.8.2.3;Figure 12.4a. Three dimensional chart showing distribution of students according to units.;94
2.19.8.2.4;Figure 12.4b. Two dimensional chart showing distribution of students according to units.;94
2.19.8.2.5;Figure 12.5. Two dimensional chart showing distribution of students according to units.;95
2.19.8.3;ADDITIONAL RESOURCES;95
2.20;Evaluator Competency 13;98
2.20.1;Monitor the Evaluation Plan;98
2.20.1.1;Risk Analysis;98
2.20.1.1.1;Purpose;98
2.20.1.1.2;Background;98
2.20.1.1.3;Introduction to the Resources or Tool for This Section;99
2.20.1.1.4;Resources for Adaptation to Your Evaluation Needs;99
2.20.2;Table 13.1. Evaluation Risks;99
2.20.3;Table 13.1. Evaluation Risks (Continued);100
2.20.4;Table 13.1. Evaluation Risks (Continued);101
2.20.4.1;ADDITIONAL RESOURCES;102
2.20.5;Domain;96
2.20.5.1;Managing the Evaluation;96
2.21;Evaluator Competency 14;104
2.21.1;Work Effectively With Personnel and Stakeholders;104
2.21.1.1;Developing a Proposal;104
2.21.1.1.1;Purpose;104
2.21.1.1.2;Background;104
2.21.1.1.3;Introduction to the Resources or Tool for This Section;105
2.21.1.1.4;Resources for Adaptation to Your Evaluation Needs;106
2.21.2;Table 14.1. Projected Evaluation Phases, Outcomes, and Timelines;107
2.21.2.1;ADDITIONAL RESOURCES;107
2.21.2.2;About The Authors;108