HOW TO: designing case studies best suited for tracking cognitive processes - part 2
In part 1, published in December 2021, the high level design criteria for a cassudy case study are presented. This part will now focus on the more detailed layout aspects how to presenting the options to the test subject and how to choose as well as order criteria to finally identify the test person´s traits.
Presenting the different decision options to the test subject
The core of a cassudy case study is always the selection of a preferred option out of a set of predefined other ones. The presentation of the different options can be done sequentially (option by option on different pages) or it can be done in a matrix format (see example below):
If the options are presented option by option on separate pages, the test subject needs to come to an „interim“ assessment per option and will then finally pick his/her preferred one in a „last step decision“. This approach is called „option wise decision making“ and is obviously the natural way of deciding under such a case design.
If the matrix presentation of options is used, the test person may use different approaches to identify the preferred option. He/she may either go for the same approach as above, meaning go through all criteria (attributes) of a single option or he/she may scan, for a given attribute, all different options. The latter is called attribute wise approach (horizontal scan), the former is called option wise approach (vertical scan). The matrix presentation of options is obviously easier to handle, offers more scan strategies and finally is of lower mental strain than the option wise presentation. Depending on the desired strain level and the business scenario under consideration the most adequate presentation of the options to the test subject shall be used.
Determining the attributes (criteria) that describe the different options
If the test subject´s willingness to cooperate shall be determined or evaluated, the options need to have attributes that clearly relate to cooperation type of behaviour. Typically, for a given option of say 25 attributes, at least 4 of these attributes shall distinctly relate to cooperative aspects. If the test subject shows a high level (or at least above average level) of interest in these 4 attributes, cooperation is considered important for the test subject (to do some expectation management: the example mentioned would tell if the test subject is interested in cooperation. If the test subject is at the same time „a smart cooperator“ cannot be determined).
In one single case study, it makes sense to analyze and evaluate not more than 4 different traits simultaneously. This means, out of the 25-30 attributes of an option, 16 attributes max. should relate to the traits under investigation and all other attributes should be neutral with respect to those traits.
There are cases where determining the level of interest of the test subject in a topic is not sufficient for characterizing the test subject in detail. Take risk propensity – a test subject´s interest level in risk-related attributes does not tell if the test subject is risk averse or risk prone. In order to do this, the different options to choose from need to have different risk levels, so that the chosen option will tell the test subject´s resulting risk propensity (alternatively one could also use a standard risk propensity questionnaire evaluation).
The sequence of criteria/attributes in the description of an option
In the western world we are all used to scan documents from left to right and from top to bottom. And when scanning a list we would typically assume that the elements on top of the list are more important than the elements at the end of that list. This may turn out as a systematic problem for a typical cassudy case study layout.
The key data captured in a cassudy study is the attention distribution across all attributes/criteria of an option. The level of attention is defined as a combination of the frequency of accessing the data for a given attribute and the total access time for the same attribute. So if we have to assume that the test subject per se focuses more on the upper part of a list than on the lower part, this effect should be counterbalanced by adjusting the sequence of the attributes accordingly. Concretely this means that if there are e.g. four attributes that relate to a specific personality trait these four attributes should be evenly spread across the total list of attributes that make up an option. The downside of such an attribute sequence is that the list looks a bit disordered or even chaotic.
One final remark on the sequence of attributes in an option. In a cassudy case study the test subject typically looks at several (two to three) options within this case study. So „shuffling“ the attributes around for the different options could also be a way to manage the above mentioned bias effect. The downside of this approach would be that a sequence of attributes that the test subject familiarized with while working on the first option would not be kept for the subsequent options, leading to an extra strain for the test subject. That´s why cassudy case studies do not make use of shuffling at all.
Too difficult?
One more remark – this blog (part 1 and 2) are meant to give you an idea of a „good case study design“. In fact there is a lot more details to be considered and traps to be avoided. The cassudy team has more than a decade of experience with these kind of case studies. And you can be sure, we ran into many traps and dead ends. So if you feel overwhelmed at first don´t give up. If you want to use cassudy for very specific (or exotic) purposes feel free to get in touch with us for a feasibility check or for training and guidance for the perfect case study design.