Using Realist Interviews to Improve Theory on the Mechanisms and Outcomes of Sport for Development Programmes

The complex nature of Sport for Development (SfD) programmes makes impact evaluation challenging. Realist evaluation has been proposed as a new, theory-driven approach to evaluate complex programmes. The present study aimed to explore the value of conducting realist interviews to gain improved insight into the mechanisms and outcomes of three SfD programmes in the Netherlands: a programme that promotes sports participation among socially vulnerable youth; a combined lifestyle intervention for adults of low social economic status; and a sports-based programme for marginalised adults. In addition, the study aimed to investigate the applicability of a conceptual model from the field of social enterprise (Roy, Baker, & Kerr, 2017) as the preliminary programme theory for those interviews. First, for each programme, a realist interview was conducted with one researcher as the key informant. Thereafter, the findings from and experiences with the individual realist interviews were discussed among the informants in a group meeting. The results revealed that the conceptual model functioned well as preliminary programme theory for the SfD programmes. The realist interviews contributed to theoretical awareness and trustworthiness. Importantly, the interviews highlighted knowledge gaps and generated ideas for programme improvement. Hence, the realist interview technique is recommended as a methodological tool to generate, validate, and improve programme theory in the field of SfD. This study had, however, an explorative character, and more research is needed to confirm and generalize the findings and to learn how a greater number of stakeholders might contribute to this type of realist evaluation.


Introduction
Within the field of Sport for Development (SfD), there is broad agreement that we need greater insight and evidence on the effectiveness of SfD programmes as well as the mechanisms that bring these effects about (e.g., Coalter, 2007;Gould & Carson, 2008). These insights are needed to optimize the impact of these programmes and to convince programme funders to maintain their investment in successful programmes. SfD programmes are, however, complex interventions consisting of multiple components targeted towards multiple outcomes, which makes it extremely difficult to establish the effects and the underlying causes (Wold & Mittelmark, 2018). Moreover, SfD programmes take place in real-life, hence in changing contexts, which complicates the evaluation of their effect even further. A relatively new approach to programme evaluation, in response to these challenges, is realist evaluation (Pawson, 1996). Realist evaluation acknowledges the complexity of social programmes. Its main objective is not to provide an answer to the question 'Does the programme work?' but to answer the question 'What part of the programme works, for whom, and under what circumstances?' In other words, it aims to reveal the coherence between the context, the mechanisms, and the outcomes of a programme. Programme theory, which outlines the sequence of expected mechanisms and outcomes (Jolley, 2014;Weiss, 2000), can be very helpful for realist evaluation. Realist evaluation checks whether the assumptions of such theory hold (Manzano, 2016;Westhorp, 2014).
The aim of the present study was twofold. First, it aimed to explore the value of using a realist interview technique to gain greater insight into the mechanisms and outcomes of SfD programmes. The realist interview technique has been described by Mukumbang, Marchal, Van Belle, and van Wyk (2019) as a useful, but an underutilized, tool for realist evaluation. In a realist interview, respondents are asked to examine and comment on existing programme theory, with the aim of improving these theories. According to Pawson and Tilley (1997) placing programme theories for examination could inspire, validate, falsify, and/or modify hypotheses about how programmes work, which is an essential process for theory refinement. Second, the study aimed to explore the applicability of a conceptual model from the field of social enterprise as the preliminary programme theory for a realist inquiry in the field of SfD. This model developed by Roy et al. (2017) is explained in further detail in the next section.

A Conceptual Model from the Field of Social Enterprise
To explore the value of using a realist interview technique, the present study made use of an existing programme theory from the field of social enterprise. Although examples of programme theory can be found within the field of SfD (e.g., Coalter, 2012;Gould & Carson, 2008;Pawson, 2006;Witt & Crompton, 1997) and in the closely related field of health promotion (e.g., Herens, Wagemakers, Vaandrager, van Ophem, & Koelen, 2017;Van Koperen et al., 2013), we decided to borrow a theory from a different, yet adjacent, scientific field to allow for new, unexpected theoretical insights (Chalip, 2006;Sparkes & Smith, 2014). In addition, theories in the field of SfD tend to focus on youth (e.g., Coalter, 2012;Haudenhuyse, Theeboom, & Nols, 2013;Lyras & Welty Peachey, 2011), whereas we wanted to cover a broader spectrum of SfD programmes. The selected theory is the conceptual model by Roy and colleagues (2017), which describes the mechanisms (i.e., mediating variables) and outcomes of social enterprise as a health and wellbeing intervention (see Figure 1). The model has been empirically informed by research on 13 social enterprises in the city of Glasgow. In general, social enterprises are businesses with primarily social objectives (Kerlin, 2013), such as a community centre offering services and employment training to local people, or a coffee bar employing disadvantaged people at a fair wage. Although SfD programmes are rarely approached as social enterprises, they share important characteristics. Both SfD programmes and social enterprises aim to reduce social vulnerability and strive for a more equal society (Sepulveda, 2015). Also, both operate largely outside the health (care) sector despite their aim to improve the health and well-being of a disadvantaged population (Caló, Roy, Donaldson, Teasdale, & Bagioni, 2019). Finally, their focus is typically on local communities.
As can be seen in Figure 1, the model by Roy et al. (2017) identifies four different forms of social enterprise: personal care services; arts and creativity; work integration; and community development. Each of these forms of social enterprise has been linked to certain mediating variables, such as improving knowledge and skills and providing meaningful work. These mediating variables, in turn, have been found to produce various intermediate outcomes related to physical, mental, and social determinants of health. At last, the intermediate outcomes are believed to contribute to the ultimate goal of improved health and well-being among social enterprise participants. The model was selected as a preliminary programme theory for the current study because of its broad scope of mechanisms and outcomes, which deemed it suitable for capturing a broad range of SfD programmes. Notably, Roy et al. (2017) invited other researchers to test and refine the model for various types of social enterprises. In the current study, we used the model to gain greater insight into the mechanisms and outcomes of three, relatively diverse, SfD programmes in the Netherlands. These three programmes are (1) Youth, Care, and Sport, (2) X-Fittt 2.0, and (3) programmes by The Life Goals Foundation. A brief description of the three programmes follows below.

Brief Description of the Three SfD Programmes
Youth, Care and Sport refers to a broad set of initiatives developed to increase sports participation among socially vulnerable young people in community sports clubs in the Netherlands (Super, Hermens, Verkooijen, & Koelen, 2014). The emphasis of these initiatives is (1) on the inclusion of sports in the care trajectories of youth, by stimulating youth care workers to guide their young clients to community sports clubs where possible, and (2) on the professionalization of sports coaches when they are working with socially vulnerable young people. Participation is voluntary and includes participation in local sports clubs in regular teams, sometimes with additional pedagogical support when needed.
X-Fittt 2.0 is a combined lifestyle intervention for low-income, overweight people living in a deprived  neighbourhood (Wagemakers, Mulderij, Verkooijen, Groenewoud, & Koelen, 2018). It targets multiple lifestyle behaviours, but foremost physical activity and nutrition. The programme is funded by a healthcare insurance company and the municipality and is free of charge for participants. The programme lasts two years, the first 12 weeks of which consists of group sports sessions twice a week, an individual sports session once a week, advice from a dietician, and four hours of coaching by a lifestyle coach. Thereafter, participants are encouraged to remain physically active by receiving six hours of lifestyle coaching over the remaining two years. The Life Goals Foundation refers to sports programmes for socially vulnerable adults in the Netherlands, such as the homeless, drug addicts, exoffenders, and psychiatric patients (Society Impact, 2019). The programmes aim is to increase their participation in society. The Life Goals foundation builds collaborations between municipalities, social care institutions, and community sports clubs to start local Life Goals programmes. In addition, they train sports coaches in the Life Goals methodology, which teaches them how to interact with participants and how to create a supportive environment. The Life Goals programmes do not have fixed durations, and consist of one to five sports activities a week. An important aspect of the Life Goals methodology is the Life Goals sessions, in which participants learn how to use their newly learned skills, such as trust and collaboration, in their daily life.

Methods
To identify and reflect on the mechanisms and outcomes of SfD programmes we used the realist interview technique. In their recent article, Mukumbang and colleagues (2019) distinguish three types of realist interviews: theory gleaning interviews, theory testing interviews, and theory consolidation interviews. The interview technique applied in the current study fits best with the theory testing interviews since our purpose was to apply and refine initial programme theory, rather than to initiate (gleaning) or confirm (consolidation) theory. For the X-Fittt 2.0 programme, an initial programme theory, or logic model, was available (Wagemakers et al., 2018), and the Life Goals programme also had a theoretical basis. Nevertheless, for the purpose of this research, we asked the informants to reflect on a model that they were unfamiliar with, namely the model by Roy et al. (2017), to spark new and innovative insights. In addition, we wanted to test the model's fit as preliminary programme theory for different types of SfD programmes.
The interviews were held with one key informant for each of the three SfD programmes, as it was a first exploration of the value of the realist interview technique. The informants were researchers involved in the evaluation of the specific SfD programme and co-authors of this article (Sabina Super, Lisanne Sofie Mulderij, and Dico de Jager). The first author (Kirsten Thecla Verkooijen) conducted the interviews. The interviews started with an explanation of the purpose and procedure of the interview. Hereafter, the model by Roy et al. (2017) was introduced and shown to the interviewee on a large paper sheet. This sheet was then used to discuss the various elements of the model from right to left, thus, starting with 'the ultimate goal' and working towards the 'type of intervention.' The two overarching questions during the interviews were: (1) "Are the mechanisms and outcomes in the conceptual model applicable to the particular SfD programme?", and (2) "What mechanisms or outcomes are missing in the model?" The opening question was: "Do you agree that improved health and well-being is the ultimate aim of [name of the programme]?" Questions on mechanism and outcomes focused on the actual mechanism and outcomes in the programmes, rather than expected mechanism and outcomes. Also, informants were asked to elaborate on the observed mechanism and outcomes in relation to the context of each programme. As suggested by Pawson (1996), the interviewer adopted an active and explicit role in teaching the preliminary theory, while the interviewee was asked to confirm or falsify and, above all, to refine that theory. During the interview, the interviewer used a marker to write notes on the sheet and checked with the interviewees if these notes captured his or her reflections correctly.
All three interviews lasted approximately forty minutes, including an explanation of the model by the interviewer, and were audio-recorded and transcribed. Each transcript was checked by the corresponding informant for accuracy. Thereafter, the recording and transcripts were independently analysed by two researchers, the first author and for each programme a different coauthor. To support validity, this co-author was never the informant of that same programme. After the individual interviews, the results from the interviews and experiences with the interview technique were discussed among the researchers in a group meeting of approximately one hour. The central question of that meeting concerned the applicability of the realist interview technique as a methodological strategy to improve insight into the mechanisms and outcomes of SfD programmes. The paper sheets written on during the individual interviews were brought to the meeting as input. Observations from the group meeting were integrated into the findings.

Results
Although the aim of this research was, first and foremost, to explore the value of using realist interviews in the field of SfD, we start the result section with a reflection on the applicability of the conceptual model by Roy et al. (2017) as preliminary programme theory. The reflection on the value of the realist interviews will follow.

Applicability of the Conceptual Model as Preliminary Programme Theory
A summary of the respondents' reflections on the mechanisms and outcomes of the SfD programmes, guided by the conceptual model, can be found in Table 1. In general, the respondents found many of the model's components relevant and applicable to the SfD programmes. For all three SfD programmes, the ultimate goal was 'improved wellbeing.' Yet, for the lifestyle programme X-Fittt 2.0, improved physical health was most important, whereas for the other two programmes well-being was. Or as the informant of the Youth, Care and Sports programmes put it: "For young people, having a fun life is more important than being healthy." Especially the intermediate mental health outcomes, such as improved coping and self-confidence, were perceived as strongly applicable to all three informants. Also, some mediating variables, such as engendering a safe and supportive environment, appeared highly important to all three programmes. However, the respondents also pointed out components that were perceived as irrelevant or beyond the scope of their particular programme. For instance, employment and income, access to information and welfare, and improved awareness and understanding of social issues were, according to the informants, of little importance to the SfD programmes. Nevertheless, being confronted with the model triggered the respondents to think about these components and how they may be part of the programme in an alternative way. For example, the informant of the Youth, Care and Sports programmes proposed replacing 'providing meaningful work' with 'providing a meaningful activity,' and believed that the sports club could be a place for young people to disclose their problems and be directed towards professional help rather than directly providing information and welfare.
For some components, respondents indicated that there was currently too little evidence to conclude whether these were indeed mechanisms or outcomes of the programme. For example, the informant of the lifestyle programme X-Fittt 2.0 was not sure if feelings of self-worth among participants increased: "I have never heard from participants that their self-image improved….Participants mention improvements in selfconfidence, they don't mention self-worth, which makes it hard to conclude anything on that." Similarly, whether access to information and care served as a mediating variable in the Youth, Care, and Sports programmes was unknown to the informant, and the informant of the Life Goals programmes was not able to confirm with certainty that a reduction of stigma and marginalization, a core aim of these programmes, had been achieved. Also, for some intermediate outcomes, only temporary improvements were observed. For instance, a temporary effect on healthy eating was observed for the Life Goals programmes: "Participants often get inspired, at least in the beginning, to eat more healthily. However, this change in behaviour is often not sustained when sports coaches do not work towards this goal." Within the X-Fittt 2.0 programme, temporary effects were found for the sense of community: "Sense of community increases a lot during the group activities, but after 12 weeks, when the activities are over, participants usually don't see each other again, and the sense of community disappears." All three informants judged the model to be appropriate for realist evaluation in the field of SfD, and they could easily see how SfD programmes could be added as a separate form of social enterprise to the model. Nevertheless, they also identified elements that they believed were currently missing from the model, such as emotional health and the provision of small challenges (Youth, Care and Sports), reduced health care consumption, increased social participation (X-Fittt 2.0), as well as pleasure and socialization (The Life Goals Foundation). In addition, the informant of the Youth, Care and Sports programmes stressed that not all components of the model are necessarily affected positively. For instance, self-confidence among youth may decrease when sports activities are felt to be too challenging. Further, stigmatization may actually increase when socially vulnerable children stand out in a group.

The Value of Conducting Realist Interviews
Overall, the respondents agreed that participating in the realist interviews was a useful exercise as it initiated theoretical awareness and generated validation of existing assumptions. In other words, the informants felt that the interviews helped to improve theory on how an SfD programme works. In addition, they felt that the interviews helped them to identify knowledge gaps, such as the lack of insight on the effect of the lifestyle programme on participants' self-worth. Identification of these knowledge gaps was important to them because they give direction to future research. The interviews and group discussion also revealed opportunities to improve SfD programmes. For instance, the X-Fittt 2.0 programme may benefit from making social contacts between participants more lasting, while The Life Goals foundation might expand its focus to lifestyle behaviours, like nutrition, to increase its impact. Furthermore, for both X-Fittt 2.0 and The Life Goals Foundation, the interviews and group discussion sparked ideas on how to address employability in the programmes. Finally, while none of the respondents judged access to information and welfare to be a function of the current programmes, the group meeting triggered a discussion on the pros and cons of making sports coaches so-called 'case managers,' responsible for the provision of information and/or the referral of participants to the health and care domain. The group discussion seemed to be a valuable addition to the individual interviews. Despite, or because of, the differences between programmes, the group discussion helped to further reflect on each programme's specific mechanisms and outcomes. Also, the meeting facilitated the exchange of ideas and practices for programme improvement. An important observation from the individual interviews was that sometimes mechanisms and outcomes were reported according to what the programme sought to implement or achieve rather than what the programme was actually successful in doing. In those cases, the interviewer had to actively ask the informant for clarification. During the group meeting, the informants acknowledged that this helped them become more aware of existing discrepancies between the intended and the actual programme outcomes. Finally, while reflecting on the realist interviews, all three informants agreed that distinguishing between the mecha-nisms and the outcomes of a programme was not an easy task. Sometimes mechanisms could hardly be distinguished from outcomes, depending on the context of a programme. Expanding social networks was an example of a difficult to define programme element, since it may serve as a means of increasing information and skills (e.g., employability), but also affect social wellbeing directly.

Discussion
This research aimed to explore the value of using a realist interview technique as a methodological strategy to provide greater insight into the mechanisms and outcomes of SfD programmes. In addition, the study aimed to examine the suitability of a conceptual model from the field of social enterprise as a basis for these interviews. The interviews and group discussion with the three key informants proved itself to be a meaningful exercise that may contribute to science and practice in the field of SfD in three ways. First, reflecting on the proposed programme theory enhanced theoretical awareness and elicited new and more trustworthy insights into the mechanisms and outcomes of the programmes studied. Hence, realist interviews can help to disentangle what works within an SfD programme and why, which then can be used to further adjust and refine programme theory. Secondly, realist interviews may contribute to ideas for programme improvement. The individual interviews, and certainly the group discussion, in which the informants learnt from each other's expertise, generated ideas to increase the impact of SfD programmes by, for example, adding new elements such as employability or nutrition to the existing programmes. Finally, realist interviews may help to identify the intended mechanisms and outcomes for which proof of their actual presence is still lacking. Identification of these knowledge gaps may guide further research in which all intended mechanism and outcomes are captured and assessed. A complete evaluation contributes to accountability, which for stakeholders, including programme managers, policymakers, and funding agencies, is crucial for (adjusting) future programmes and policies (Jolley, 2014).
Notwithstanding the perceived value of the realist interviews, it was also perceived as a challenging exercise. Likewise in other studies, it appeared quite difficult to distinguish between outcomes, mechanisms, and context. Because programmes do not operate in a vacuum, mechanisms may work differently in different contexts (Jagosh et al., 2015). Over time, a previously defined mechanism may actually become an outcome, while an outcome may become context (Herens et al., 2017). Another challenge was to differentiate between actual and intended mechanisms and outcomes. At times, the informants needed reminding that they had been asked about the observed programme outcomes and mediating factors, rather than about the intended outcomes and mediators. Despite these reminders, assumptions, rather than ob-servations, may have influenced their input. Especially in the field of SfD, widespread assumptions about positive development through sports are often too easily taken for granted (Nols, Haudenhuyse, & Theeboom, 2017). Hence, a realist interviewer needs to be alert for possible false assumptions and not to be afraid to ask further questions. This requires a different, more critical, role as interviewer compared to more traditional interviews in which respondents are treated as indisputable sources of information (Mukumbang et al., 2019).
The second aim of this research was to explore the suitability of a conceptual model from the field of social enterprise as a preliminary programme theory for realist evaluation in the SfD field. Overall, the model by Roy et al. (2017) provided a good basis to reflect on the perceived mechanisms and outcomes of the three SfD programmes, despite the fact that these programmes differed substantially with respect to the target group, contents, objectives, and context. Hence, this model from an adjacent scientific field proved to generalise well across different SfD programmes and populations. On a critical note, one could say that the applicability of the model may even have been too good, in so much as it did not sufficiently challenge the respondents' dominant way of thinking (Sparkes & Smith, 2014), an important reason for choosing the model in the first place. Furthermore, we want to stress that our purpose was not to develop one definitive programme theory that should cover all SfD programmes in future research. As Haudenhuyse and colleagues (2013) well explain, participants are unlikely to be best served with top-down predefined programme outcomes. Hence, the model can be used as a template for realist interviews, but in the end, each programme would preferably have its own programme theory to reflect its unique context. In fact, the X-Fittt 2.0 programme had developed its own programme theory prior to its start in close collaboration with stakeholders and citizens (Wagemakers et al., 2018). Reflecting once again on the previously developed programme theory may help to detect and adapt to changes in the programme and its context, which is an important element of action research and benefits programme outcomes (Jolley, 2014;Mukumbang et al., 2019).
Some limitations of our study should be noted. First of all, interviews were conducted with only one key informant per programme, as our study was a first exploration of the value of realist interviews using a programme theory borrowed from another field. It is recommended, however, to include more informants to cross-validate the input for each programme. The study addressed three different SfD programmes to learn about the possible benefits of the realist interview technique. As such, the lessons learned from our exercise (i.e., theoretical awareness, programme improvement, and knowledge gaps) might contribute to the science and practice within the field of SfD, but we cannot extrapolate our findings to draw conclusions about SfD programmes in general. Also, the informants were all academic researchers and co-authors of this paper. This was decided because we believed researchers to be a relatively objective source of information and often responsible for programme evaluation. However, relying on the expertise of these academics obviously created a particular bias. We acknowledge that it would have been valuable to involve more stakeholders such as staff, coaches, and programme participants, since they could have provided other relevant contributions toward clarifying the programme theory (Manzano, 2016;Pawson & Tilley, 1997). Furthermore, we recommend that future research should involve more stakeholders, not only to complement programme theory but also to make it a joint reflection exercise. However, engaging more stakeholders requires time, as well as the need to learn together how to reflect on, discuss, and unravel mechanism, outcomes, and context in an interview or group discussion, as not all stakeholders are familiar with programme theories. Especially regarding youth SfD programmes, it might be challenging to have children, or young people, reflect on mechanisms and outcomes (Super, Wentink, Verkooijen, & Koelen, 2017). Nevertheless, regardless of who the informant is, time, place, and actor perspectives define the mechanisms, outcomes, and contexts addressed in the interviews, and therefore, these are always dynamic in nature (Herens et al., 2017).

Conclusion
While acknowledging its limitations, our 'reflection exercise' looks promising for SfD programmes, and probably for programmes in other fields too. The interviews and group discussion provided meaningful insights: Reflecting on the proposed programme theory enhanced theoretical awareness about mechanisms and outcomes, and related to this, revealed opportunities for programme improvement and facilitated the identification of knowledge gaps. The conceptual model provided a good basis to reflect on the perceived mechanisms and outcomes of the three SfD programmes and proved to generalise well across different SfD programmes and populations. We hope that this explorative study, illuminating the benefits of the realist interview technique, can inspire others to validate and refine existing programme theory, to improve programme design, and in a broader sense may contribute to the scientific advancement of the SfD field. We recommend engaging multiple stakeholders to make the realist inquiry more comprehensible, and thus, more worthwhile. Collecting the experiences from a greater number and variety of programmes will at the same time help further develop and refine the realist interview technique.

About the Authors
Kirsten Thecla Verkooijen is an Assistant Professor at the chair group Health and Society of Wageningen University and Research, in the Netherlands. She wrote her PhD dissertation about the role of self-and social identity in health-related risk behaviour among Danish adolescents. Her current research addresses health promotion and health inequalities among socially vulnerable groups, often with a focus on sports and exercise. Sabina Super is a Lecturer and Researcher at the chair group Health and Society. She obtained her PhD degree in 2017 on a research project focusing on the role of sports participation in the personal development of socially vulnerable young people. Currently she is working on several research projects that focus on people's capacity to cope with stressful or challenging life events (such as poverty or disease). Most of her projects include vulnerable groups such as children in youth care, people with a low socio-economic status, or multi-problem families. Her work is inspired by the salutogenic model of health.
Lisanne Sofie Mulderij (MSc) holds a Master's degree in Nutritional Epidemiology and in 2016 started a PhD program in the chair group Health and Society of Wageningen University & Research, the Netherlands. Her research focuses on care-physical activity (care-PA) initiatives for people with a lower socioeconomic status, including 1) the evaluation of the impact of care-PA initiatives on participants, 2) the unravelling of the effective elements of care-PA initiatives for people with a low SES, 3) the investigation of the societal costs and benefits of care-PA initiatives for people with a low SES, and 4) the exploration of alternative ways to fund care-PA initiatives, prevention, and healthcare.
Dico de Jager (MSc) holds a Master's degree in Sport Policy and Sport Management. In 2019 he started as a Researcher at the Wageningen University and Research in Wageningen, the Netherlands. His research is focused on the critical elements of sport programs for social vulnerable adults, especially the elements needed for successful life skills transfer to other social domains. In addition, he works for the Life Goals Foundation, which creates and implements sport programs for vulnerable adults.