Links and Downloads
Module 1
Module 2
Ideas for Item Writing
Module 3
Module 4
Module 5
Sample Score and Interpretation Reports
Course Library
Course Text General Resources Innovative Item Types Item Writing Job Analysis Passing Standards Evaluation and Return on Investment | Legal Considerations Americans with Disabilities Act Finding Tests Performance Assessment Test Security Statistics |
Course Text
Shrock, S. A., & Coscarelli, W. C. (2007). Criterion-referenced test development: Technical and legal guidelines for corporate training and certification (3rd ed.). Silver Spring, MD: International Society for Performance Improvement.
General Resources
American Psychological Association. (2006). Testing and assessment: FAQ/Finding information about psychological tests. The APA Science Directorate answers hundreds of calls and emails each year from persons trying to locate the right test or find more information about psychological tests. APA neither sells nor endorses testing instruments, but it does provide guidance in using available resources to find psychological tests. Answers to frequently asked questions are provided at http://www.apa.org/science/faq-findtests.html
Association of Test Publishers. (2006). Answers to frequently asked questions regarding testing in general are provided at http:// www.testpublishers.org/general. From the drop down menu on left hand side, click on Testing FAQs and select, General, Testing in Businesses, or Testing in Certification and Licensure Settings. Retrieved from http://www.testpublishers.org/questions.htm
Biddle, D. (2012). Adverse impact and test validation: A practitioner's handbook (3rd ed.). Scottsdale, AZ: Infinity Publishing.
Coscarelli, W. C., & Shrock, S.A. (1996). How to transform an organization through criterion-referenced testing. In M.Silberman (Ed.). The 1996 McGraw-Hill team and organization development sourcebook. (pp. 207–217). New York: McGraw-Hill.
Downing, S. M., & Haladyna, T. M. (Eds.). (2006). Handbook of test development. Mahwah, NJ: Lawrence Erlbaum.
Fein, M. (2012). Test Development: Fundamentals for certification and evaluation. Peoria, IL: Versa Press.
Gardner, J.R. (Ed.). (2012). Assessment and learning (2nd ed.). Sage Publications Ltd.
Guion, Robert M. (2011). Assessment, measurement, and prediction for personnel decisions(2nd ed). New York, NY: Routledge.
Hacker, D. G. (1998). Testing for learning outcomes. [Info-line No. 907]. Alexandria, VA: American Society for Training and Development. Good resource on how to write tests for training evaluation.
Hale, J. (2007). The performance consultant's fieldbook: Tools and techniques for improving organizations and people (2nd ed.). San Francisco: Jossey-Bass/Pfeiffer.
Hale, J. (2012). Performance-based certification: How to design a valid, defensible, cost-effective program (2nd ed.). San Francisco: Jossey-Bass/Pfeiffer.
Linn, R. L. (1993). Educational measurement (3rd ed.). Phoenix, AZ: American Council on Education and The Oryx Press.
Parshall, C. G., Spray, J., Kalohn, J. C., & Davey, T. (2001). Practical considerations in computer-based testing. New York: Springer-Verlag.
Shank, P. (December 2009). Develop valid assessments. [InfoLine No. 0912]. Alexandria, VA: American Society for Training and Development.
Society of Industrial and Organizational Psychology (SIOP). (2013). Welcome to SIOP’s FYI on workplace topics. SIOP provides information on workplace topics based on decades of research in the field of industrial and organizational psychology. One of the topics is employment testing. Retrieved from http://www.siop.org/Workplace/
Standards for Educational and Psychological Testing. (1999). Prepared by the Joint Committee for Educational and Psychological Testing of the American Educational Research Association, American Psychological Associations, and the National Council on Measurement in Education. Washington, DC: American Educational Research Association. Chapter 9 addresses testing individuals of diverse linguistic backgrounds. Chapter 10 addresses testing individuals with disabilities.
U.S. Department of Labor Employment and Training Administration. (2000). Testing and assessment: An employer’s guide to good practices. [O*NET Guide]. This guide can help managers and workforce development professionals understand and use employment testing and assessment practices to meet their organization's human resources goals. It includes a glossary of assessment terms. Retrieved from http://www.onetcenter.org/dl_files/empTestAsse.pdf
Van de Vijver, F. & Hambleton, R. (1996). Translating tests: Some practical guidelines. European Psychologist, 1, 89–99. Retrieved from http://arno.uvt.nl/show.cgi?fid=28531
Westgaard, O. (1999). Justifying the test. In Tests that work: Designing and delivering fair and practical measurement tools in the workplace. Pfeiffer.
Workforce Management. (2013). This website is available at http://www.workforce.com It contains relevant articles on many topics including workforce planning, assessment, succession planning, and talent management.
Innovative Item Types and Technology Enhanced Assessment
Jodoin, M. G. (2003). Measurement efficiency of innovative item formats in computer-based testing. Princeton, NJ: Educational Testing Service. Also Journal of Educational Measurement, 40(1) 1-15. Available March 14, 2006, at EBSCO Research Database: http://www.blackwellpublishing.com/journal.asp?ref=0022-0655&site=1
Johnson, R.L., Penny, J.A., Gordon, B. (2009). Assessing performance: Designing, scoring, and validating performance tasks. New York, NY: The Guilford Press.
Parshall, C.G., & Harmes, J.C. Improving the quality of innovative item types: Four tasks for design and development. Journal of Applied Testing Technology. Association of Test Publishers. Retrieved from http://www.testpublishers.org/assets/documents/Improving.pdf
Scalise, K., & Gifford, B. (2006). Computer-based assessment in e-learning: A framework for constructing “intermediate constraint” questions and tasks for technology platforms. Journal of Technology, Learning, and Assessment, 4(6).Scalise, K. (2009). Computer-based assessment: “Intermediate Constraint” questions and tasks for technology platforms. Retrieved from http://pages.uoregon.edu/kscalise/taxonomy/taxonomy.html
Tippins, N.T. & Adler, S. (2011). Technology-enhanced assessment of talent. San Francisco, CA: Jossey-Bass.
Zenisky, A. L., & Sireci, S. G. (2002). Technological innovations in large-scale assessment. Applied Measurement in Education, 15(4), 337–362.
International Test Commission. ITC guidelines on computer-based and Internet-delivered testing. Retrieved from http://www.intestcom.org/Downloads/ITC%20Guidelines%20on%20Computer%20-%20version%202005%20approved.pdf
Item Writing
Anderson, L.W., Krathwohl, D.R., Airasian, P.W., & Cruikshank, K.A. (2000). A taxonomy for learning, teaching, and assessing: a revision of bloom’s taxonomy of educational objectives, abridged (2nd ed.). Pearson.
Haladyna, T. M. (2004). Developing and validating multiple-choice test items (3rd ed.). New Jersey: Lawrence Erlbaum. Chapter 8 addresses validity evidence coming from item development procedures and, specifically, elicitation techniques during a pilot.
International Personality Item Pool. A scientific collaboratory for the development of advanced measures of personality and other individual differences. Retrieved from http://ipip.ori.org/ipip/
International Test Commission. ITC guidelines for translating and adapting tests. Retrieved from http://www.intestcom.org/upload/sitefiles/40.pdf
International Test Commission. ITC guidelines for test use. Retrieved from http://www.intestcom.org/upload/sitefiles/41.pdf
International Test Commission. ITC guidelines for quality control in scoring, test analysis, and reporting of test scores. Retrieved from http://www.intestcom.org/upload/sitefiles/qcguidelines_2012-07-13.pdf
Osterlind, S. J. (1998). Constructing test items: Multiple-choice, constructed-response, performance, and other formats (2nd ed.). In Evaluation in education and human services[Series]. Madaus, G. F., & Stufflebeam, D. L. (Series Eds.). Boston: Kluwer Academic Publishers.
Shank, P. (September 2010). Create better multiple-choice questions. [InfoLine No. 1009]. Alexandria, VA: American Society for Training and Development.
Job Analysis
American Society for Training and Development. (June 2005). Be a better job analyst. [InfoLine No. 8903]. Published by the American Society for Training and Development.
Franklin, M. (June 2005). A guide to job analysis. [InfoLine No. 0506]. Alexandria, VA: American Society for Training and Development.
Job-Analysis.Net work. This website is a resource for job analysis methods, legal information, forms, and tips. However, some of the links it provides are to websites that have not been updated. Retrieved from http://www.job-analysis.net/
Lucia, A., & Lepsinger, R. (1999). The art and science of competency models: Pinpointing critical success factors in organizations. San Francisco: Jossey-Bass/Pfeiffer.
Norton, R. (1997). DACUM Handbook. Center on Education and Training for Employment, College of Education, The Ohio State University.
Occupational Information Network. (2006). The National O*NET™ Consortium website offers news and information about the O*NET program and offers O*NET products, including reports, career exploration tools, and the O*NET database. Retrieved from http://www.onetcenter.org/whatsnew.html
Office of Personnel Management Job Analysis. This website is published by the federal government. The site covers job analysis and provides sample forms. Retrieved from http://www.opm.gov/policy-data-oversight/assessment-and-selection/job-analysis/
Rothwell, W.J. & Graber, J.M. (2010). Competency-Based Training. Alexandria, VA: American Society for Training and Development.
Shippmann, J., Ash, R., Battista, M., Carr, L., Eyde, L., Hesketh, B., Kehoe, J., Pearlman, K., & Prien, E. (2000). The practice of competency modeling. Personnel Psychology, 53.
Whetzel, D., & Wheaton, G. (2007). Applied measurement methods in industrial psychology and human resources management. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Passing Standards
Cizek, G. J. (Ed). (2012). Setting performance standards: Foundations, methods, and innovations (2nd ed.). New York, NY: Routledge.
Gushta, M. M. (2003). Standard-setting issues in computerized-adaptive testing. Paper presented at a conference. Retrieved from the Centre for Research in Applied Measurement and Evaluation (CRAME) website: http://www2.education.ualberta.ca/educ/psych/crame/research.html
Lin, Jie. (2003). Bookmark standard setting procedure: Strengths and weaknesses. Paper presented at a conference. Retrieved from the Centre for Research in Applied Measurement and Evaluation (CRAME) website: http://www2.education.ualberta.ca/educ/psych/crame/research.html
Ricker, K. L. (2003). Setting cut scores: Critical review of Angoff and Modified-Angoff methods.Paper presented at a conference. Retrieved from the Centre for Research in Applied Measurement and Evaluation (CRAME) website: http://www2.education.ualberta.ca/educ/psych/crame/research.html
Sadesky, G. (2004) Standard setting using the attribute hierarchy model. Paper presented at a conference. Retrieved from the Centre for Research in Applied Measurement and Evaluation (CRAME) website: http://www2.education.ualberta.ca/educ/psych/crame/research.html
Zieky, M., & Perie, M. (2006). A primer on setting cut scores on tests of educational achievement. Retrieved from the ETS website: http://www.ets.org/research/policy_research_reports/publications/publication/2006/dbkw
Evaluation and Return on InvestmentBarksdale, S., & Lund, T. (2001). Rapid evaluation. Alexandria, VA: American Society for Training and Development. Excellent resource with lots of tools to assist you.
Grafinger Hacker D. (July 1989). Testing for learning outcomes. [InfoLine No. 907]. Alexandria, VA: American Society for Training and Development.
Hale, J. (2002). Performance-based evaluation: Tools and techniques to measure the impact of training. San Francisco: Jossey-Bass/Pfeiffer.
Kirkpatrick, D.L. (January 2007). The four levels of evaluation. [InfoLine No. 0701]. Alexandria, VA: American Society for Training and Development.
Kirkpatrick, D. L. and Kirkpatrick, J.D. (2006) (3rd ed.). Evaluating training programs: The four levels. San Francisco: Berrett-Koehler.
Kirkpatrick, D. L. and Kirkpatrick, J.D. (2007) (3rd ed.). Implementing the four levels: A practical guide for effective Evaluation of Training Programs. San Francisco: Berrett-Koehler.
Phillips, J.J. (1999). Level 2 Evaluation: Learning. [InfoLine No. 9814]. Alexandria, VA: American Society for Training and Development.
Phillips, J. J. (2011). Return on investment in training and performance improvement programs(2nd ed.). Burlington, MA: Butterworth-Heinemann.
Phillips, J. J., Phillips, P. P., & Hodges, T. K. (2004). Make training evaluation work. American Society for Training and Development.
Phillips, P. P., & Phillips, J. J. (2006). Return on Investment Basics. Alexandria, VA: American Society for Training and Development.
Phillips, P. P. (2002). The bottom line on ROI: Basics, benefits, and barriers to measuring training and performance improvement. Atlanta: The Center for Effective Performance.
Shrock, S. (1999). Level 2 assessment may eliminate demands for ROI. In Performance Improvement, 38(6).
Stolovitch, H. D., & Keeps, E. J. (2011). Telling ain't training. Alexandria, VA: American Society for Training Development (2nd ed.). Chapter 9 discusses testing and examining.
Stolovitch, H. D., & Keeps, E. J. (2004). Training ain't performance. Alexandria, VA: American Society for Training Development. Chapter 10 is an excellent resource regarding ROI. The list of references for further reading is also very good.
Stolovitch, H. D., & Keeps, E. J. (2005). Beyond telling ain't training fieldbook. Alexandria, VA: American Society for Training Development. Chapter 11 discusses testing and examining.
Legal Considerations
Biddle Consulting Group. (2006). Uniform Guidelines. com. Retrieved from http://www.uniformguidelines.com
Biddle, D. (2012). Adverse impact and test validation: A practitioner's handbook (3rd ed.). Scottsdale, AZ: Infinity Publishing.
Outtz, J.L. (Ed.). (2010). Adverse impact: Implications for organizational staffing and high stakes selection (2010). New York, NY: Taylor and Freancis Group, LLC.
Rose, R. G. (1993). Practical issues in employment testing. Odessa, FL: Psychological Assessment Resources.
Ryan, A. M., & Tippins, N. T. (2004). Attracting and selecting: What psychological research tells us. Human Resource Management, 43(4), 305-318.
Society of Industrial and Organizational Psychology (SIOP). (2006). Principles for the validation and use of personnel selection procedures. Retrieved from http://www.siop.org/_Principles/principlesdefault.aspx
U.S. Department of Labor home page available at http://www.dol.gov/
U.S. Department of Labor Office of Federal Contract Compliance Programs home page available at http://www.dol.gov/ofccp
U.S. Department of Labor. Employment laws assistance for workers and small businesses (elaws). The elaws Advisors are interactive e-tools that provide easy-to-understand information about a number of federal employment laws. Each Advisor simulates the interaction you might have with an employment law expert. It asks questions and provides answers based on responses given. Retrieved from http://www.dol.gov/elaws/
U.S. Department of Labor. OASP/Office of Compliance Assistance Policy Employment Law Guide. This Guide describes the statutes and regulations administered by the Department of Labor (DOL) that affect businesses and workers. The Guide is designed mainly for those needing hands-on information to develop safety and health, wage, benefit, and nondiscrimination policies for businesses in general industry. Retrieved from http://www.dol.gov/compliance/guide/
U.S. Equal Employment Opportunity Commission website. Retrieved from http://www.eeoc.gov/
U.S. Equal Employment Opportunity Commission. (2002). EEOC compliance manual section 13: National origin discrimination. Retrieved from http://www.eeoc.gov/policy/docs/national-origin.html
Uniform guidelines on employee selection procedures (1978). Part 1607: U.S. Equal Employment Opportunity Commission. Federal Register, 43, 38290-38315. Retrieved from http://www.access.gpo.gov/nara/cfr/waisidx_03/29cfr1607_03.html
Americans with Disabilities Act
Cornell University. (2000). Pre-Employment Screening Considerations and the ADA.
Cornell University. (2000). Pre-Employment Testing and the ADA.
Cornell University. (2001). The ADA and Personnel Training.
Ekstrom, R. B., & Smith, D. K. (Eds.). (2002). Assessing individuals with disabilities in educational, employment, and counseling settings. Washington, DC: American Psychological Association. Chapters 12, 13, and 14 are especially relevant to testing in the employment setting.
Job Accommodation Network (JAN) Bulletin on ADAAA: http://www.jan.wvu.edu/bulletins/adaaa1.htm
U.S. Equal Employment Opportunity Commission. (2005). Americans with disabilities act: Disability discrimination. Retrieved from http://www.eeoc.gov/laws/types/disability.cfm
U.S. Equal Employment Opportunity Commission. The ADA: Your responsibilities as an employer. Retrieved from http://www.eeoc.gov/facts/ada17.html
U.S. Equal Employment Opportunity Commission. Notice Concerning The Americans with Disabilities ACT (ADA) Amendments ACT of 2008. Retrieved from http://www.eeoc.gov/laws/statutes/adaaa_notice.cfm
Finding Tests
Buros Institute of Mental Measurements. (1938- ). Mental Measurements Yearbook series and Tests in Print series. The Institute neither sells nor endorses testing instruments, but it does provide guidance in using available resources to find psychological tests. Information available at http://www.unl.edu/buros/
Keyser, D. J., & Sweetland, R. C. (Eds.). (1994). Test critiques (Vol. 10). [Index]. Austin, TX: Pro-Ed.
Maddox, T. (Ed.). (2008). Tests: A comprehensive reference for assessments in psychology, education, and business (6th ed.). Austin, TX: Pro-Ed.
Murphy, L., Geisinger, K, Carlson, J. & Spies, R. A. (Eds.). (2011). Tests in print VIII: An index to tests, test reviews, and the literature on specific tests. Lincoln, NE: University of Nebraska Press.
Spies, R. A., Geisinger, K., & Carlson, J. (Eds.). (2010). The eighteenth mental measurements yearbook. Lincoln, NE: University of Nebraska Press.
Performance Assessment
Johnson, R.L., Penny, J.A., Gordon, B. (2009). Assessing performance: Designing, scoring, and validating performance tasks. New York, NY: The Guilford Press.
Whetzel, D., & Wheaton, G. (2007). Applied measurement methods in industrial psychology and human resources management. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Test Security
Wollack, J.S. & Fremer, J.J. (Eds.). (2013). Handbook of test security. New York, NY: Routledge.
Assessment security options: Considerations by delivery channel and assessment model(2013). Association of Test Publishers Security Committee.
Association of test publishers security survey report 2013. Association of Test Publishers Security Committee. CreateSpace Independent Publishing Platform.
Statistics
Houston Shore, J. (June 2009). Basic statistics for trainers. [InfoLine No. 0906]. Alexandria, VA: American Society for Training and Development.