Clf alpha = 0

Examples of Psychometrics

research101chart_blogimgIn order for any scientific instrument to provide measurements that can be trusted, it must be both reliable and valid. These psychometrics are crucial for the interpretability and the generalizability of the constructs being measured.

Reliability is the degree to which an instrument consistently measures a construct - both across items (e.g., internal consistency, split-half reliability) and time points (e.g., test-retest reliability). One of the most common assessments of reliability is Cronbach’s Alpha, a statistical index of internal consistency that also provides an estimate of the ratio of true score to error in Classical Test Theory. A general rule of thumb is that solid scientific instruments should have a Cronbach’s Alpha of at least .7. There are exceptions to this rule in the case of brief measurements when breadth of content is of primary interest in recapturing a longer scale (see example here).

Below is an example of a reliability analysis for a Recreational Shopping scale. The analysis provides a summary of how the items within the scale perform together in measuring a person’s propensity for recreational shopping. In this example, the overall reliability statistic is .732. The analysis also elucidates the efficacy of each individual item by reporting information such as corrected item-total correlation and Cronbach’s Alpha if an item were deleted. As seen in the example below, we know that item #4 is a great item because it has a high item-total correlation (correlates strongly with the other items) and the overall reliability would drop significantly if the item were deleted from the scale. These, and other metrics all go into understanding the makings of a reliable survey.

As demonstrated in the video linked above, a measure can be reliable without being valid but it cannot be valid without being reliable. Building on reliability, validity is an index of whether or not a particular instrument measures what it purports to measure. For example, let’s say a researcher gave Samantha a paper-and-pencil survey of Extraversion. How would the researcher know that the computed score on that survey actually reflected Samantha’s true level of Extraversion?

There are several different forms of validity. Some of the most commonly assessed forms of validity include content validity, construct validity, and criterion validity. Content validity is an assessment of how well the breadth of the construct has been assessed. For example, have all the elements of Extraversion been captured in the survey (e.g., gregarious, outgoing, active)?

Construct validity is a measure of how well an instrument measures an operationalized or latent construct. Two important sub-components of construct validity include convergent (the degree to which two instruments which measure the same construct are correlated; generally the higher the better) and discriminant validity (the degree to which two unrelated measures are correlated; generally the lower the better). The validity analysis reported below represents the convergent validity of TipTap Lab’s Image Selection Task (IST) of Recreational Shopping with the original paper-and-pencil Recreational Shopping scale (a scale that has been previously scientifically validated and cited in numerous research studies).

This statistic can be interpreted like any correlation (the closer the number is to 1, the stronger the relationship). So, if the correlation is high (as we see below), convergence is strong. Convergent validity is a particularly important statistic at TipTap Labbecause we employ this methodology to convert long, paper-and-pencil measures (all previously validated in external research contexts) into short and engaging image based measurements. It is critical for us to recapture the psychometric properties of the original scales.

Lastly, criterion validity (including both predictive and concurrent validity) is an assessment of how well an instrument predicts known related behaviors or constructs. For instance, if Samantha scored high on the Extraversion scale, we know from previous research that she should be more likely (than an Introvert) to attend a party or talk to a stranger. In the course of our research, criterion validity is constantly being evaluated as more constructs and behavioral outcomes are being studied.

At TipTap Lab, we employ advanced psychometric techniques to build the most reliable and valid measurements possible. We are constantly iterating our process and improving our items as well as our methodology.

Share this article

Related Posts

Assessment jobs
Assessment jobs

Latest Posts
What is a Psychometric assessment?
What is a Psychometric…
This photo appeared in the Bulletin…
Montessori teacher interview questions
Montessori teacher…
The job of a preschool teacher is quite…
Interview questions for technical Support
Interview questions…
Providing great customer service sets…
Logical exam with answers
Logical exam…
The following questions will test your…
Internship phone interview questions
Internship phone…
Phone interviews are commonly used to…
Featured posts
  • Montessori teacher interview questions
  • Software Development Life Cycle Interview questions
  • Assessment jobs
  • Psychometric test Sample questions with Answers
  • Internship phone interview questions
  • Interview questions for technical Support
  • Top companies Recruitment
  • Personality profiling Questionnaire
  • 50 Behavioral interview questions
Copyright © 2017 l All rights reserved.