The quality parameters validity, reliability and objectivity - and the cassudy methodology.
When it comes to psychological tests and even social studies, some people turn up their noses - "it's all not real science and the results are dubious!" In general, this criticism is not really justified, although there have been some spectacular cases of dubious and refuted studies in the past.
The quality parameters of diagnostic studies
The following three quality parameters have therefore been established for psychological studies: validity, reliability and objectivity. But what exactly do they mean?
Validity: a study or procedure is considered valid if it actually measures what it is supposed to measure. A brief example: you have created a questionnaire whose evaluation is supposed to reveal, for example, whether a person is a narcissist. If an actual narcissist[1] fills out this questionnaire, then narcissist should also come out as a result and vice versa.
Reliability: a study or procedure is considered reliable if the result is stable when repeated. Meaning, if I use the above questionnaire with a test subject several times, the same result should always come out. In particular, the result should also be stable to changes in the boundary conditions, i.e., it should not depend, for example, on whether the questionnaire is completed in the morning or in the evening.
Objectivity: a study or a procedure is considered objective if no subjective influence is exerted by third parties when conducting the study. Subjective influence could occur, for example, if a questionnaire takes the form of a personal interview in which the answers may depend on how sympathetic the interviewer is perceived to be or on the speech melody and facial expression with which the questions are presented.
Cassudy and its particular advantages in validity
Looking at the approach taken with cassudy in terms of validity, there is clear evidence[2] that cassudy just "works". Cassudy is based on linking attention to importance/meaning. In short, this means that information that receives a high level of attention from the respondent is also information that is particularly important to the respondent and ultimately relevant to his/her decision. This correlation has been scientifically proven and is moreover not a surprise, but rather in line with common sense.
Defining criteria for professional success
Things get a bit more difficult when you want to establish a relationship between "what is important to a person" and "how professionally successful is the person". The first difficulty is how to measure professional success. Traditional studies measure professional success by criteria such as annual salary, rank in the organization, or number of subordinate employees. This measurement methodology defines success as moving up the hierarchy rather than success in a specific role. If the cassudy methodology is to be used in leadership development, these traditional criteria for professional success can certainly be used. However, if I want to determine the prospects of success for the role of a Senior Expert, for example, as part of a recruiting process, the traditional criteria tend not to fit at all. Here, cassudy also offers an elegant way out. With cassudy, companies can define their own benchmark. A logistics company looking to hire a new branch manager, for example, can first conduct the aptitude test with successful branch managers in the company and then use the results to derive target values for new branch managers. This is an immense advantage, because the success of an employee can thus be predicted "company-specifically" and one is not dependent on the mechanisms of traditional studies, which by definition are rather generic. In this way, companies can use the Cassudy methodology to create their own specific diagnostics, tailored to their own needs.
Reliability anchored in the methodology
Turning now to reliability, the cassudy methodology[3] has been studied for reliability and has been shown to be stable with respect to the parameters studied. Thus, results obtained in a laboratory setting (all subjects complete the test in a classroom under observation at the same time) have also been confirmed in less controlled settings (subjects complete the test unobserved at home with possible distractions and interference during the test). Similarly, the effects of stress or hunger were studied, and there too the results remained stable. A cassudy case study always includes instructions on how best to complete the study (quiet place, avoid disturbances and distractions, follow the time guidelines). Deviations from the guidelines can usually be detected on the basis of the data collected. In addition, it should be in the interest of the test person to respect the defined conditions of such a diagnostic.
Objectivity through online methodology and use of simple language
Objectivity! Since cassudy diagnostics is an online procedure without the intervention of a human person, objectivity per se is given. At most, it could be argued that perhaps complex formulations could disadvantage non-native speakers or that the extended reading time of texts could be misinterpreted as particular conscientiousness in the case of non-native speakers. In fact, cassudy chooses to use very simple language in the wording of the diagnostic case studies. In contrast, the use of technical terms that play a role in the subject's aptitude is deliberate.
In summary, the cassudy methodology fulfills the classic diagnostic quality parameters while at the same time offers additionally the option to build up company-internal validity scenarios, thus creating a completely independent and specific diagnostic landscape.
Questions? Remarks?
Then please contact me directly
Werner.Sohn@cassudy.com
[1] whose narcissism, for example, has been proven by other methods or is simply obvious
[2] Schulte-Mecklenbeck, M., Kühberger, A., & Ranjard, R. (Eds.). (2011). Society for judgment and decision making. A handbook of process tracing methods 164 for decision: A critical review and user's guide. Hove, New York, N.Y.: Psychology Press [3] even more generally the process tracing methodology