International Test Users Conference: 2002

Sydney, NSW, Australia

Symposium Title:

Testing and selecting medical school students

and the validation of two measures of the traits of morality.

Part 1: Determining and measuring the personal qualities of competent, ethical medical practitioners: Selecting medical students.

DA Powis

The sole criterion for entrance to medical school has traditionally been that of prior scholastic achievement. However, awareness of the shortcomings of this single criterion emerged in the medical literature of the last thirty years, in the opinions of those who teach medical students, in the personal experiences of patients, and in the extreme instances that sometimes make headlines in the media. Since 1978, the University of Newcastle has used measures of abilities and personal qualities in conjunction with scholastic ability in their medical school selection procedures. Today, no Australian medical school, and a decreasing number of medical schools worldwide, select only on academic performance.

The purpose of this paper is to focus on the abilities and personal traits that research has indicated are relevant and appropriate to underpin competent and ethical medical practice. Traits such as logical reasoning ability, empathy, and sensitivity to the needs of the patient, profession and society, for example, are considered appropriate to the context of medical practice while prejudice, arrogance and impulsiveness would appear inappropriate. However, a number of ethical issues arise in the context of medical school selection such as the need for empirically derived evidence of the reliability and validity of selection instruments and the degree to which the abilities and traits measured might be influenced by the experience of medical education.

The validation research program and results of two new psychometric instruments, developed and tested as a project initiated in 1997 by the BMed Admissions Committee at the University of Newcastle, will be presented in the following two papers of the symposium. The project task was to develop psychometric measures that would indicate those applicants to medical school who would be likely to engage in ethically inappropriate behaviour within the practice of medicine. The research program has involved, to date, more than 9,000 applicants to medical school, and students attending medical school, in Australia, Israel, New Zealand, the United Kingdom and Fiji, plus a small number of psychology undergraduate students and practising medical clinicians from Australia. The research findings to date have provided evidence of the validity of the measures.

Part 2: The Involved-Detached dimension of moral behaviour:

development and validity of the NACE scale.

D Munro

The research program, introduced in Part 1, adopted two different approaches to the exploration of ethical behaviour and the selection of medical school students. One approach, the subject of this paper, was to consider the personality traits that might underlie behaviours in medical students that might be appropriate or inappropriate to the practice of medicine. Initial research resulted in the development of the NACE scale, based on analysis of data from several samples which consistently produced a first order four factor structure of Narcissism, Aloofness, Self-Confidence and Empathy. A second order analysis produced a dimension defined as Involved (Narcissism and Aloofness) versus Detached (Self-Confidence and Empathy). Additional research produced expected relationships between the NACE scale scores and scores from instruments such as the Emotional Intelligence Scale (Schutté et al., 1998), Cattell's 16 Personality Factors scale (16PF), and a measure of the Big Five personality traits (Goldberg, 2000). These results have been presented earlier (Munro & Bore, 2000, 2001)

In order to provide further evidence of the construct validity of the NACE scale a further study was recently completed by participating medical schools in Scotland and England. In this study, the results of which will be presented in detail in this paper, 600 applicants to medical schools completed a battery of tests consisting of NACE, a modified version of the Eysenck Personality Questionnaire, Goldberg's Big Five scale, the Horney Coolidge Type Indicator (2001), the Torrubia et al. (2001) Sensitivity to Punishment – Sensitivity to Reward Questionnaire, and the Mojac scale (Bore, 2001). The results were again found to support the validity of the NACE scale. Additionally, and as observed in previous studies, the NACE Involved-Detached dimension was found to be orthogonal to the Mojac scale Libertarian-Communitarian dimension of moral orientations. The Mojac scale and a model of six moral types derived from the orthogonality of the Mojac and NACE scores is the subject of the third paper of this symposium.

Part 3: The Libertarian-Communitarian dimension and the factor structure of morality: evidence for a model of six moral types.

MR Bore

A second approach to the project initiated in 1997 by the BMed Admissions Committee at The University of Newcastle was to examine moral behaviour from the cognitive-developmental perspective. However, research by Bore (1996) found that an individual's developmental stage of moral reasoning, as measured by Kohlbergian based instruments, was only weakly related to the individual's moral decisions. A review of the major theories and measures of the field of moral psychology suggested moral orientation, rather than reasoning stage, might be more predictive of moral behaviour and that individual differences in moral orientation could be conceptualised as a Libertarian (values individual freedom) versus Communitarian (values duty to the social group) dimension. The Mojac scale was developed as a measure of the Libertarian-Communitarian dimension and data from several samples indicated acceptable reliability (Alphas ranging from .88 and .90), acceptable test-retest reliability (r = .77, 12 month interval between test and retest), and only small differences in mean scores provide by samples from Australia, Fiji, New Zealand and Israel.

Highly significant correlations with scores from a number of established personality measures have provided evidence of the construct validity of the Mojac scale. Factor analysis of Mojac and NACE data suggested a personality structure of morality from which a model of six moral types emerged (Bore & Munro, 2000, 2001). Results from the large study in the United Kingdom, outlined in the previous paper, found that a Detached Libertarian moral type is likely to produce high scores on Eysenck's Psychoticism factor as measured by a modified version of the Eysenck Personality Questionnaire. More generally, the relationships observed in the UK data provided further evidence of the validity of the Mojac scale, the NACE scale and the model of six moral types. While the measures have a potentially broad range of applications, the original goal of developing psychometric measures that could indicate those applicants to medical school who are likely to engage in ethically inappropriate behaviour within the practice of medicine appears to be within reach.