DEFINING IMPACT

SCHOLARLY IMPACT

Section Contents:

Introduction

Defining Impact

Telling Impact Stories

Moving Forward

What is impact?

With this question, we are really asking, “What are the best ways to evaluate education scholarship?”

What we measure and how we define impact is important because “[i]ndicators change the system through the incentives they establish” (Hicks et al, 2015). What ‘counts’ as impact shape and constrain our focus and endeavours.

“Impact metrics can too often focus on what is measurable at the expense of what is important” (Buttliere quoting Tinkler, 2015). The types of metrics that ‘count’ affect and interact with knowledge production and can restrict the kinds of ES work that is encouraged, funded, and undertaken.

In contemplating what ‘counts’ as impact, we need to ensure that we are indeed measuring what we value and not just measuring what we can easily count, and thus end up valuing only what we can count (Biesta, 2009).

So how do we expand our definition of impact in medical and health professions education? Many other fields are engaging in this discussion and we should familiarize ourselves with current developments in metrics. But we must not use indicators from other fields unquestioningly. We should carefully consider what makes sense, is meaningful, and aligned with our own health professions education context. We should consider expanding the metrics currently used in academic medicine if we want to ensure that we capture all the types of impact that health professions educators and education researchers have.

To ensure we are measuring what we value, we need to expand our understanding of what indicators can provide evidence of impact. While presented as objective, “[m]etrics inevitably embody particular values” (Frodeman et al, 2012). The implicit assumptions embedded in some of our definitions of ES impact will be explored in the following sections.

References

  • Biesta G. Good education in an age of measurement: On the need to reconnect with the question of purpose in education. Educational Assessment, Evaluation and Accountability. 2009;21(1):33-46.
  • Buttliere B. We need informative metrics that will help, not hurt, the scientific endeavor – let’s work to make metrics better [Internet]. London School of Economics Impact Blog; 2015 Oct [cited 2020 Apr 22]. Available from: https://blogs.lse.ac.uk/impactofsocialsciences/2015/10/08/we-need-informative-metrics-how-to-make-metrics-better/
  • Frodeman R, Holbrook JB, Barr K. The university, metrics, and the good life. In Brey P, Briggle A, Spence E, editors.

    The Good Life in a Technological Age. New York: Routledge; 2012. p. 307-315.

  • Hicks D, Wouters P, Waltman L, de Rijcke S, Rafols I. Bibliometrics: The Leiden manifesto for research metrics. Nature. 2015;520(7548):429-31.

Previous

Next

Back To Top