Measurement & Testing in I/O Psychology – PSYC 845

CG • Section 8WK • 11/08/2019 to 04/16/2020 • Modified 07/28/2020

Course Description

In this course, students will review and apply methodologies to assess the measurement qualities of tests pertinent to organizations. Specifically, students will demonstrate their ability to critique and improve existing measures using data with the goal of bolstering the validity arguments of the test scores. Students will learn to make data-driven decisions from classical and advanced measurement techniques that are readily used by the I/O psychologists of today.

Prerequisite

PSYC 545

Rationale

The purpose of this course is to further develop the student’s working knowledge of various issues and techniques in measurement. Research is the soul of science and its language is data. The appropriate analysis of data, its limitations, and the evidence for or against theory are further discussed in this course.

Measurable Learning Outcomes

Upon successful completion of this course, the student will be able to:

  1. Apply modern quantitative methods to evaluate the psychometric qualities of assessments relevant to I/O psychologists. Students will explain the basic tenets of Classical Test Theory and Item Response Theory and their applications to I/O assessments.
  2. Evaluate the validity evidence to support the use and interpretation of test scores. Students will articulate how empirical evidence and its claims are a central tenant of psychological practice.
  3. Demonstrate the skills of gathering information from a library including computerized information sources, bibliographic systems, computerized literacy skills in word processing APA format, conducting internet/electronic searches, and utilization of various statistical packages, as well as other sources from which one can present a persuasive argument.
  4. Critique the method, results, and conclusions of existing literature supporting the use of an assessment and suggest new studies where the current literature falls short.
  5. Analyze data using SPSS to determine several properties of an assessment including test score dimensionality, reliability, and validity evidence appropriate for the test use.
  6. Explain the process for standard setting and how it relates to validity.

Course Assignment

Readings and lecture presentations/notes

(MLOs A-F)

Course Requirements Checklist

After reading the Course Syllabus and Student Expectations, the student will complete the related checklist found in Module/Week 1.

Discussion Board Forums (2)

Discussion boards are collaborative learning experiences. Therefore, the student is required to provide a thread in response to the provided prompt for each forum. Each thread must be 350–400 words and demonstrate course-related knowledge. In addition to the thread, the student is required to reply to two other classmates’ threads. Each reply must be 200 words. (MLOs A and F)

Critique Paper

The student will write a 5-10 page paper in current APA format including title page and references. The student will critique the psychometric properties and validity evidence supporting claims about test scores. The number of references is expected to be appropriate to cover all areas where the original authors could improve. (MLOs B, C, and D)

SPSS or R Assignments (5)

In these assignments, the student will learn how to analyze raw test data and/or results to examine the quality of the items or test as a whole. The student will be expected to follow all steps to obtain accurate results, interpret those results, and suggests areas of further data collection and study. (MLOs A-E)