Critical Eye is a semi-regular feature where RealKM analyses and discusses the methodology and science behind claims made in publications.
In this column we are analysing the study The Myers-Briggs Type Indicator (MBTI) and Promotion at Work. Its abstract says:
Over 6000 adult managers attending an assessment centre completed the MBTI and also specified in detail how long it had been before they were promoted to a managerial role, and also, where appropriate, to a senior managerial role. Correlational analyses showed shorter times to promotion were associated with being Extraverted and a Sensing Type. Regressions showed that Feeling types took longer to be promoted to managerial roles, and that Perceiving types took longer to be promoted to senior positions. Implications for the selection and management of people were considered.
Let’s dig in straight away. A layperson reading this abstract is likely to take away four claims:
- You will be promoted to management faster if you are an Extravert
- You will be promoted to management faster if you are a Sensing Type
- You will be promoted to management slower if you are a Feeling Type
- You will be promoted to senior management slower if you are a Perceiving Type
But what did the study actually do? Can it live up to these claims?
Firstly, Myers-Briggs is a limited metric because it is self-assessed. This means that the possibility of dishonest or self-deceptive answers exists. In fact, at best all we can say that we are evaluating how people want others to think of them rather than their actual behaviours in practice.
Secondly, the sample size of 6000 is pretty big, so we can mostly rule out the possibility of aberrant results through random chance. However, by design the study only surveyed people who had been successful in becoming a manager. This is a weakness because we won’t be able to compare the attributes of those who aren’t managers. However, it is not a critical problem because establishing successful attributes of managers may still be valuable.
Thirdly, the actual statistical methods used bear closer analysis. Claims (1) and (2) are based on a correlational analysis. This compares the values of two independent variables across a sample and sees if they can be shown to move in similar directions. A correlation of 1 means that as variable A rises, variable B always rises by a proportional amount, whereas a score of -1 means that as A rises, B falls by a proportional amount. A score of 0 means that there is no discernable relationship.
In this study, the correlation between all 8 of the Myers-Briggs scores and the number of years to promotion is less than 0.1. In other words, less than 10% of the variance in how long it takes to be promoted can be explained by your Myers-Briggs test score on any of its dimensions.
A far stronger correlation was found between age and promotion time, but even the strongest correlation (between age and promotion to senior management) is just 0.33. Generally correlations of 0.3 or less are deemed to be so weak as to be insignificant. Claims (1) and (2) are very weak and should be discarded.
Claims (3) and (4) are based on regression analysis, which attempts to control for all other variables under study to see whether a dependent variable predicts an independent variable result. In layman’s terms, regression measures change where “all else is equal”. Here the study appears to be on stronger ground. Both claims have a significance of 0.00 (and in regression the lower the score, the higher the significance). On the other hand, the beta score is comparatively low, which means that even though there is significance, it will only produce a relatively small shift in change. Once again, age was a far stronger predictor than either of the Myers-Briggs factors identified. Despite acknowledging this in the discussion, the relevance of age to promotion is not mentioned at all in the abstract. Claims (3) and (4) have some justification but should have been better qualified.
My personal view of Myers-Briggs is that it isn’t worth using due to some of the limitations described earlier, but you still frequently see articles and organisations that reference it. Scientific studies that genuinely examine its usefulness as a predictor should be welcomed, but not if they try to draw conclusions that simply aren’t borne out by their own evidence.