Simulation exercises are a well established concept in personnel selection. Based on the assumption that they will act similarly in everyday work situations, assessment centres (ACs) allow applicants to demonstrate the way they would behave in certain situations.

Situational Judgement Tests (SJTs) are based on a similar assumption. However, a new study challenges that belief. So are SJTs worth it at all? And if so, under what conditions?

Let’s get to the bottom of the study.

ACs are complex and require a lot of resources, so usually only a preselected candidate is invited to attend. Situational Judgement Tests (SJTs) are a popular and economical way to include simulation exercises earlier in the selection process. They don’t simulate a situation, but rather just describe it.

How do Situational Judgement Tests work?

Applicants are presented in text or video form, job-specific critical situations like those that can happen in everyday work. They then choose from various answers which action they consider the most effective in that situation. Various studies have shown that SJTs can be very good at predicting future job performance.

Are situational descriptions therefore an inescapable part of SJTs like a webcam is to a video interview? Far from it!

At least, this is what a recent study conducted by Stefan Krumm and his colleagues found out! Which raises the question: what do SJTs measure anyway?

Situational Judgement Tests without a situation?

The idea behind this study would be the same as telling a recruiter not to ask the candidate any questions, but to let them talk freely about whatever they want. If you want to see how people behave in situations, using SJTs, you would assume people wouldn’t be able to choose appropriate options without knowing the situation, right? In other words: are people able to say how they would react if there was no context or situation?

To examine this, Krumm and colleagues excluded in multiple studies the descriptions from SJTs. Candidates were only presented the behaviour options.

The result is amazing: in up to three-quarters of the cases, it made no difference whether the situation was simply omitted or not. So what does this mean?

Back to Basics: What do STJs measure anyway?

Although there is no doubt that SJTs are a very useful and valid tool, opinions are divided on the reasons why. Two different perspectives exist.

1. SJTs measure general knowledge domains on effective behaviour, which are actually independent of situation. Most people know roughly what behaviour works and is acceptable and which is not. Therefore including a specific situation would be completely unnecessary.

2. SJTs assess what is called practical intelligence or heuristic decision making. (A fascinating video by psychologist Gerd Gigerenzer about the topic here.) This means that you can respond appropriately to specific problems. Thus, the context here is important!

Conclusion: The situation is only partially important!

The results of the study do not question the general usefulness of SJTs. However, they raise the question of context-dependent and context-independent elements.

In SJTs that measure broader skill sets such as “integrity” or “teamwork”, it was largely irrelevant whether a situation was described or not. In more specific SJTs, it was important to describe the situation. For example, in the study, the capability of pilots to make good decisions in critical situations was assessed. As you can imagine, this requires specific knowledge of how to react appropriately.

Recommendation for Recruiting: For expert positions please use situations!

>>For entry level positions, specific contextual methods are often unnecessary. Keeping things simple is good a thing, as tests are expensive and complicated.

>>However, for selecting and developing specialised experts, strong context and use of specific situations is necessary, to test the extent of relevant knowledge domains and whether they can be transferred to practical decision making.