Keeping "Generalization" Relevant

What are the implications of research for our wider understanding of the world, especially regarding causes and effects, and for “best practices” within professional and personal realms? While not fully addressing these questions, this week’s Research Methods Cafe Conversation took up the issue of generalizability in qualitative research. There were concerns raised that qualitative research was being labelled as “ungeneralizable,” even by the authors themselves, and that this was being presented as a common limitation of such research. Overlooking generalization may prevent valuable research from having its intended impact, but there are several possible reasons for this trend, as discussed in the conversation. Generalizability may be seen as constrained by the statistical logic of quantitative designs and therefore impossible to demonstrate in qualitative research. Another possibility mentioned is that authors simply would not or could not do the intellectual labor to show their research was generalizable. Some participants affirmed “transferability” as an alternative term/concept for qualitative research, which would perhaps have less baggage than “generalizability.” The article linked above provides a thoughtful approach to different forms of generalizability that may be relevant to qualitative research. Addressing generalizability may mean considering whether the findings resonate with community members, provide useful tools or theories, spur people to action, or address injustice. These are all worthy goals and should not be restricted to qualitative research. A full discussion and critique of the article is not possible here, but I would like to respond to several points from the article and from the group conversation.


First, there is the sense that qualitative research rests on different assumptions about the world and about knowledge, and therefore “generalizability” based on probabilistic sampling is inappropriate. As a pragmatist, I accept that there are numerous theories, tools, and interpretive frameworks available to meet different research purposes. A concept, such as “randomization” that carries weight in one research context may indeed be ill-suited to another. However, this does not exclude the possibility of conceptual cross-over, and I think sampling is a concept with usefulness for many forms of research. This article covers several types of sampling that can be used in qualitative research. The idea of sampling is to get a taste for the whole by taking just a few bites. Within quantitative research, probability-based selection may be part of the design because we do not have access to the whole, or because we want the sample to be just as large as needed to be informative about the phenomenon in question. The specific individuals/samples chosen are not as important as how they relate to the larger population. In qualitative research, probability sampling is not ruled out a priori, but space is not available here to defend this point in detail. More commonly, “purposive” or “judgement” samples may be the goal in qualitative research, and the individuals may be chosen for study based on their own characteristics, such as unique experiences and insights. Yet in the article above the sample chosen is still connected to a larger group, such as by showing the range of a phenomenon or of viewpoints about it. When researchers discuss participants’ characteristics and the reasons for selecting them, they allude to their identities and group memberships, and this forms part of a discussion about how generalizable the findings may be. Indeed, qualitative researchers, who often reach a deeper familiarity with their participants, may be especially well positioned to do this.


Next, there is an unhelpful assumption that generalizability, as sought by quantitative researchers, is something absolute and unquestionable. In education, one example is the idea of “what works” as demonstrating the best teaching and learning approaches in black and white terms. In fact, one researcher active in this area is quick to point out the context-dependent nature of the findings and that they are subject to revision by future evidence. The absolutist view of generalizability is part of the spectre of positivism, which is frequently referenced by those critical of quantitative and even mixed-methods approaches. In my experience, even researchers who work with numbers are rarely willing to adopt a “positivist” label and some consider it derogatory. Without diving into ontological/epistemological quicksand, it is important to note that “reality,” if it exists, is always developing, and our understanding of it expands as we acquire new tools to explore it. Likewise, research, as it is consumed, is always about the past, and what is considered generalizable today may not be so tomorrow. For this reason, generalizability is something that needs to be considered in both the design and analysis phases of research, as well as something research users need to judge for themselves. Rather than simply resting on probability, rich description is needed to support such judgements, and this is one area where qualitative research can set the standard. Researchers also need to provide evidence that findings remain relevant across chronological, geographic, and social contexts, and optimally this would be part of a cycle of research. Considering generalizability may aid in designing the research, making sense of the results, and choosing the next steps. Generalizability may not be best demonstrated by a one-off study, whether qualitative or quantitative.


There are myriad good ways to do research, but this does not imply there are no universals. In fact, I believe generalizability and similar concepts point to one common expectation for research of all designs and methods: it should illuminate something beyond the context in which it was done. Research deserving of the name should add to knowledge about the world, whether this is accomplished with narratives or numbers, with large or small samples. Conceiving of generalizability as something emergent and developing rather than proven and immutable may place additional responsibilities on both qualitative and quantitative researchers. On the one hand, qualitative researchers may need to consider how their studies frame and represent the world in microcosm, and quantitative researchers may need to highlight the roles of context and interpretation in their findings and pay attention to how research users engage with and make use of them. To move research forward, I believe that emphasizing the shared values and goals of researchers is as important as seeing where we differ. Along these lines, the conversation around generalizability should continue among academics, stakeholders, and policy makers. 


This blog expresses the author’s views and interpretation of comments made during the conversation. Any errors or omissions are the author’s. 


Loraine Hitt is a doctoral student in the School of Education at Durham University. Her interests are in metacognition/self-regulation, research synthesis, and research design.

Comments