A brief journey into science -
so what are the most reliable ways to predict an employee's success in a particular role?
Science is fairly unanimous on this question (based on the results of meta-studies) - they are, in order of significance, the work sample, tests of general cognitive skills, and expertise tests. Or, to put it another way, anyone who handles a typical task or business case in the future role properly, quickly and "correctly" in compliance with the given margins and framework conditions, should be able to succeed in this role.
Let's construct an example for this: You want to check whether an employee can be successful as a future strategic buyer. As a work sample, you can select a typical business case of a strategic buyer, for example, the streamlining of the supplier portfolio. Specifically, this involves the employee selecting one supplier from a list of 3-4 suppliers to be removed from the portfolio.
For the expertise review, proceed as follows: You provide the employee with a variety of information about each supplier in the portfolio. Expertise is required to prioritize this information ("which information is relevant to the task, which is rather not"), to understand it ("what is a kick-back agreement, for example, and how should it ideally be structured") and to evaluate it ("how is a feature actually pronounced and how do I evaluate the sum of all relevant features"). In addition, you also provide the employee with a strategic framework for action as it exists in your company. This could be, for example, guidelines on topics such as quality, sustainability, customer orientation, good governance or similar.
You can test the employee's cognitive abilities by offering him or her complex information that is to be processed and correlated within a specified period of time. For the specific example of the strategic buyer, you can do this by showing trends for each supplier. How have purchasing volumes, prices, quality, innovation, exception management, etc. developed over the last few years and how does this development fit in with the strategic guidelines? This requires a quick grasp of the situation and the ability to aggregate and correlate complex information.
In our example, we have now outlined the technical framework for the work sample. In order to optimize the testing of expertise and cognitive skills, it makes sense not to keep the solution for the work sample too trivial. If there is a supplier in the portfolio of suppliers that offers, for example, the smallest sales volume, the worst prices, and the worst quality, then the solution is too obvious and the intellectual challenge of the work sample is too low. It is better to show in the work sample the dilemma, which is what prevails in reality, namely a rather ambiguous information and performance situation between the individual suppliers and thus the need for a more complex weighing of the decision to be made.
There are basically two criteria for the actual evaluation of the work sample:
1) Did the test subject figure out the "right solution" or plausibly demonstrate an alternative solution?
2) Did the test subject follow a stringent process to arrive at a solution to the task?
As mentioned above, evaluation in terms of the "right solution" is somewhat tricky, because ideally the work sample is designed in such a way that there is no clear "right solution" and that the chosen solution (in the given example, therefore, the supplier has to be removed from the portfolio) is a measure of the individual preferences of the process or rather than the solution that would be clearly preferable from a technical point of view.
The second evaluation path of the work sample is the process that the test subject goes through to reach a result. Many aspects can be relevant here:
Does the test subject first obtain an overview of the overall situation?
Does the test subject take into account contextual aspects such as corporate strategy or other guidelines and requirements?
Does the test subject proceed rather "linearly" in the work sample or does he/she jump back and forth between different activities?
Does the processor distinguish between more important and less important aspects in the processing?
Evaluating this process means first of all being able to observe it. In classic work samples, which are usually carried out as part of an assessment center, this requires an explicit observer who "looks over the shoulder" of the processor. This is a very elaborate procedure and the above mentioned aspects of the processing can only be observed very indirectly.
The situation is quite different if the work sample is processed online and the activities of the process or during processing can also be followed online. A clever design of the work sample then makes it possible, e.g. , to record the completeness and intensity of the use of the information provided and to draw conclusions about the prioritization of the test subject. Likewise, the basic approach (conscientiousness, balance, learning curve over the time axis) can be observed. The evaluation of the editing process can thus be based on a much broader data base than would otherwise be possible for a human observer.
Cassudy uses exactly this methodology in the design and evaluation of online work samples (case studies). In this way, it is possible to simultaneously capture and evaluate a processor's expertise, general cognitive abilities, and personality traits in a work sample. A multi-faced picture oft he processor emerges, which in turn can be matched with the expectations for covering the success factors. Such a case study can be completed in a reasonable period of time (25-45 minutes) and is usually perceived by the test subject as "entertaining" and not as a dull answering of questionnaires in the context of an aptitude diagnosis.
The advantages of such a procedure should now be clear. However, the detailed design of an online work sample (a.k.a. case study) is a non-trivial undertaking and is described in a bit more detail elsewhere.
If you have any further questions, please feel free to contact me directly. I am looking forward to discuss the cassudy approach with you.
Werner Sohn
0176 1022 2375