The Practice of Urban Experimentation in Dutch City Labs

Urban planning by experiment’ can be seen as an approach that uses experimentation to innovate and improve urban planning instruments, approaches, and outcomes. Nowadays, urban experiments—interventions in the city with the aim to innovate, learn, or gain experience—are increasingly taking place in the context of Urban Living Labs. In the Netherlands, a certain type of Urban Living Lab, called city labs, is flourishing, and it has been suggested that these labs could make an important contribution to ‘urban planning by experiment.’ However, previous studies have indicated that this will depend on how experimentation is conducted in these labs. To obtain a more comprehensive picture of the practice of experimentation, we conducted a survey among Dutch city labs, supplemented by individual and group interviews with practitioners from a small subset of the 17 responding labs. We conclude that there is a poor match between the practice of experimentation in Dutch city labs and the characteristics that are considered to support effective ‘urban planning by experiment’ (i.e., a structured approach to experimentation, co-creation of experiments, active and targeted dissemination of lessons learned, and experiments as linking pins between municipal policy goals and the needs of urban society). This suggests that the current contribution of Dutch city labs to ‘urban planning by experiment’ is probably quite limited. Further research is needed to determine whether the typical practice of experimentation encountered in the Dutch city labs, i.e., action-oriented, resource-limited, and largely driven by opportunities, is also found in Urban Living Labs elsewhere.


Introduction
Urban experimentation is en vogue. Amongst policymakers, city officials, urban professionals, and scholars alike there is increasing attention for this concept (Evans, Karvonen, & Raven, 2016). Urban experiments can be seen as purposeful interventions in the city with the aim to innovate, learn, or gain experience (Bulkeley & Castán Broto, 2013). They are distinctive from strictly scientific experiments in that they are conducted in reallife settings, are solution-oriented, offer no full control, rely on the mobilization of lay knowledge, and are not easy to replicate . Due to their local scale and limited geographical scope-urban experiments usually focus on a small part of the city, e.g., a neighborhood, street, or even just a single buildingurban experiments confront place-based dynamics and deliver place-based knowledge and results (Karvonen & van Heur, 2014).
Various factors appear to have contributed to the current popularity of urban experimentation, including climate change and the associated attention for urban climate adaptation and carbon-neutral cities (Bulkeley & Castán Broto, 2013), as well as recent 'urban agendas' with experimental implementation approaches, for example by Habitat III Valencia et al., 2019) and the EU (Pukarthofer, 2019). Also, the recent economic crisis and its effects on urban development and housing and real-estate markets in cities should not be overlooked (Potjer, 2019), nor the already longer-standing neoliberal process of devolution of responsibilities to the local level of the state (Swyngedouw, Moulaert, & Rodriguez, 2002). Overall, urban experimentation can be seen as a way for cities to face profound uncertainty about new development trajectories in new coalitions with local actors (Harmes-Liedtke & Gianetti, 2019).
Whereas considerable attention has been paid in the scientific literature to (urban) experimentation in connection to new approaches in policy development (e.g., McFadgen & Huitema, 2017a, 2017bvan der Heijden, 2016), the specific connection with urban planning is still largely unexplored. To conceptualize the contribution of urban experimentation to urban planning, we therefore mainly draw on the literature on experimental governance, supplemented with findings from two case studies that explicitly consider urban planning (Bisschops & Beunen, 2019;Scholl & Kemp, 2016). Urban planning is here understood as the practices of public authoritiesincluding regulations, procedures, and spatial plansthat aim to steer spatial urban development into a desired direction (Wolfram, 2018).
Already in 1954, John Dewey insisted, in his work on 'democratic experimentalism,' on the importance of experimental knowledge for joint problem solving as a way for government institutions to innovate in response to dysfunctional routines (Sabel, 2012). 'Urban planning by experiment' can thus be seen as an approach that uses experimentation to innovate and improve urban planning instruments, approaches, and outcomes (Scholl & Kemp, 2016). The need for this emerges when the current repertoire can no longer offer effective, readymade solutions to major challenges. This can be due to new and complex or wicked problems or due to changed conditions, such as an economic recession, more assertive citizens, or a 'retreating' government. The experimental governance literature suggests four different types of contribution from experimentation in this respect (Kronsell & Mukhtar-Landgren, 2018;Laakso, Berg, & Annala, 2017). First, experiments can serve to test new and hopefully more effective approaches and solutions to new or wicked problems before adopting these at a larger scale. In urban planning, this could be experiments with responses to issues like climate change, air pollution, or social inequality. Second, experiments offer a way to deal with increased uncertainty and complexity by following a flexible, case-sensitive approach rather than a rigid, one-size-fits-all procedure. Examples in urban planning concern experiments with an organic development strategy instead of a masterplan or with flexible, alternative financing schemes. Third, when combined with a participatory, co-creative approach, employing urban experiments may produce more effective outcomes that are more widely accepted and more actively supported. With respect to urban planning, examples include the active involvement of local creative professionals, entrepreneurs, and residents in the design and execution of experiments. Finally, all of these contributions may even facilitate system-wide urban sustainability transitions when the lessons from multiple experiments at multiple locations are combined and disseminated. The various impacts on urban planning Scholl and Kemp (2016) identified in their case study of a series of urban experiments in Maastricht (the Netherlands) fit these four types of possible contributions. However, these types are not mutually exclusive and a single experiment may deliver multiple types of contribution, as is shown in the case studied by Bisschops and Beunen (2019). Here, experimentation focused on co-creation of an alternative form of urban planning to support organic redevelopment of a brownfield site; i.e., a combination of the first three types of contribution.
Urban experimentation is increasingly taking place in the context of Urban Living Labs (ULL), defined as geographically-embedded spaces that facilitate and direct explicit experimentation and learning, based on participation and user involvement (Voytenko, McCormick, Evans, & Schliwa, 2016). Bulkeley et al. (2016, p. 13) stress the solution-oriented focus of this experimentation when describing ULL as "purposefully intended to bring together multiple actors that seek to intervene in order to address contemporary urban challenges and foster learning through forms of open and engaged experimentation." Whereas the term ULL was initially reserved for initiatives with a focus on real-life testing of technological innovations, it has now come to encompass a wide variety of urban collaboratives that aim to test novel products, services, or approaches of a technological, social, or institutional nature (Bulkeley et al., 2019). In the Netherlands, a type of ULL is especially flourishing, called city labs ('stadslabs'), which bears features of what Bulkeley et al. (2019) labeled 'civic labs' as well as 'organic (grassroots) labs,' with the municipality or local citizen groups in the lead, respectively, and a strong focus on local urban challenges and solutions, usually with a spatial dimension (de Kraker, Scholl, & Rijkens, 2018).
Considering the role of ULL in urban experimentation and the possible contributions of urban experimentation to urban planning listed above, it should come as no surprise that these Dutch city labs have been attributed an important role in 'urban planning by experiment' (van Uum, 2018). The case study of M-LAB by Scholl and Kemp (2016) suggests that city labs indeed have considerable potential in this respect, but the authors conclude that realization of this potential depends on how experimentation is conducted in city labs. From the successful case of M-LAB, four characteristics can be derived that are important in this respect: (1) a structured approach to experimentation is followed, i.e., experiments are planned in broad outline, based on explicit learning questions; (2) experiments are co-created by a diversity of urban actors; (3) lessons learned from experiments are actively disseminated to target audiences, such as urban planning professionals, municipal departments, and aldermen; and (4) experiments are positioned at the boundary of municipality and urban society and connect major urban planning challenges with the concerns and ambitions of other urban actors. The extent to which the practice of experimentation in (Dutch) city labs matches these characteristics is not known, however. Surveys of city labs (or ULL more generally) have thus far not paid much attention to their experimentation practices (e.g., Bulkeley et al., 2019;Voytenko et al., 2016). Despite repeated calls for more research on the practices of urban experimentation (Bulkeley et al., 2019;Sengers et al., 2019;Voytenko et al., 2016), only a limited number of case studies has considered these practices, while focusing on certain aspects, such as co-creation (Menny, Palgan, & McCormick, 2018;Puerari et al., 2018), local embeddedness (Frantzeskaki, van Steenbergen, & Stedman, 2018), and dissemination (von Wirth et al., 2019). Moreover, it is not clear how representative these findings are.
To address this gap and to obtain a more comprehensive picture of the practice of experimentation, we conducted a survey among 37 Dutch city labs, supplemented by individual and group interviews with practitioners from a small subset of these city labs. We were interested in particular in how structured their approach is to experimentation, which urban actors are involved, how actively lessons are disseminated, and the nature of their relationship with the municipality with respect to experimentation. In the following sections, we describe our research methods in more detail and present our findings on the practice of urban experimentation in city labs. In the concluding discussion, we assess to what extent this practice matches the characteristics that are thought to support effective 'urban planning by experiment,' discuss possible reasons for the observed poor match and ways to address this, and end with an outlook on further research.

Methods
We investigated the practice of urban experimentation among the members of a network of Dutch city labs, which is described below in more detail. The methods of study included a survey addressed to all labs in the network, semi-structured interviews with three of the labs that responded, and two co-design workshops with four of the responding labs. Details of these methods and the analytical framework used are provided below, and short descriptions of the city labs that participated in the interviews and workshops are presented in Supplementary File 1.

The Network of Dutch City Labs
The network of Dutch city labs consists of all labs that have received a grant from the Creative Industries Fund NL ('Stimuleringsfonds voor de Creatieve Industrie,' from here on called CIF-NL), which is a national fund to support innovation in architecture and urbanism, design, and digital culture (CIF-NL, n.d.-a). With the network, CIF-NL aims to promote the exchange of knowledge and expertise among Dutch city labs. Between 2014 and 2018, CIF-NL launched a series of Open Calls directed at city labs and focusing on new ways of addressing urgent urban development issues. Successful applicants received grants from CIF-NL of around 25,000€ as temporary financial support for their activities (CIF-NL, n.d.-b). Our choice of this research population was based on having access to the network through collaboration with CIF-NL and on the intended role of the city labs in urban planning and development as evident from the Open Call texts (CIF-NL, n.d.-c).

Analytical Framework
Core concepts in our study of the practice of urban experimentation in Dutch city labs were: experiment; learning; structured approach; co-creation; and boundary position. 'Experiments' in the context of city labs have been defined as innovation projects where the goal is to learn something rather than to achieve a predetermined outcome (Scholl & Kemp, 2016). In our study, we used a less stringent condition, defining an experiment as 'a project or initiative which has learning as a major goal.' This enabled details of the conception of experimenting in the city labs to be studied by asking questions about examples, success, and failure. 'Learning' is the process of acquiring new (or modifying existing) knowledge, skills, or attitudes; learning may lead to behavioral change, but this is not necessarily included in the definition of learning (de Kraker, 2017). City labs involve a variety of actors carrying out a wide range of activities, and, in consequence, many different types of learning can be observed in city labs (Scholl et al., 2017). In our study, we focused on learning as drawing explicit lessons from an experiment with respect to the questions or objectives that drove the experiment, as well disseminating these lessons to target audiences outside the lab. With 'a structured approach to experimentation' we mean that an experiment is planned in broad outline, based on explicit learning questions, and with balanced attention for setting-up, conducting, and learning from experiments, as well as for the connections between these three steps (Scholl & Kemp, 2016;Scholl et al., 2017). It also includes the notion of reflection and wellconsidered choices with respect to the what, how, why, and with whom questions. 'Co-creation' in the context of city labs has been described as active engagement of diverse urban actors in the various stages of experimentation, including the decision-making processes (Menny et al., 2018). In this study, we operationalized it in terms of involvement of multiple urban actors in decisions and actions with regard to setting-up, conducting, and learning from experiments. Finally, with 'boundary position' we refer to the interface between the municipality and the urban society (Scholl & Kemp, 2016), which we operationalized in questions about the role of the municipality and other urban actors in decision-making, in particular concerning the focus of experimentation. Overall, these core concepts were operationalized in explicit questions in the survey and specific discussion topics in the interviews and co-design workshops, as detailed below. Processing of the primary data obtained was therefore straightforward without the need for a content analysis with inductive coding.

Survey, Interviews, and Co-Design Workshops
For the survey, we developed an online questionnaire (in Qualtrics) consisting of 55 open questions divided over five themes: the city lab itself; experimentation in general; setting up experiments; conducting experiments; and learning from experiments (see Supplementary File 2). In addition, 16 statements about experimentation in city labs were presented at the end of the questionnaire, with possible responses ranging from 'strongly agree' to 'strongly disagree' on a 5-point Likert scale, plus a 'not-applicable' option. The issues addressed in the questions and statements concerned details of the approaches taken, the actors involved in decisions and actions, the challenges encountered, and possible areas for support. An invitation e-mail with a link to the survey was sent by the network coordinator of CIF-NL to the 37 Dutch city labs in the network and a reminder e-mail was sent after 15 days. In total, 17 city labs completed or largely completed the survey questionnaire. This rate of response represents 46% of the city labs in the network. However, according to the CIF-NL network coordinator, 10 of the 20 non-respondent city labs were ones that were no longer active, bringing the response rate for active members of the network to 63%. The respondents were all 'lab officials,' usually a coordinator or initiator.
After processing the responses to the questionnaire, the coordinators of three city labs were contacted for a semi-structured telephone interview of about 30 minutes. These three labs were selected as being the most experienced and structured in their approach to experimentation. This concerned Stadslab Water in de Dordtse Ruimte (Dordrecht), Stadslab Nijmegen (Nijmegen), and Stadslab ZOHO (Rotterdam). The interviews aimed at obtaining a more 'narrative' account of experimentation in these labs and focused on the interactions between the lab and the municipality in this respect.
Two half-day co-design workshops were conducted with the coordinators of four city labs. These four had responded positively to an invitation to these workshops at the end of the questionnaire and were able to attend on the proposed dates. This concerned 't Lab van Weert (Weert), Maastricht-LAB (Maastricht), Stadslab Luchtkwaliteit (Rotterdam), and Stadslab Water in de Dordtse Ruimte (Dordrecht). The aim of the co-design workshops was two-fold: to achieve a better understand-ing of the practice of experimentation in Dutch city labs, and to design together with the participating labs a tool to support city labs in experimentation. In the first workshop, the focus was on identifying the support needs by reflecting on the strengths and weaknesses of an experiment (one 'focal experiment' per lab) and howin hindsight-a different approach could have improved this experiment. Based on the results, a prototype support tool was developed, which was tested and adapted in the second workshop.

Results
The results we present here are mainly based on the survey, supplemented where indicated with findings from the interviews and the co-design workshops.

Features of the Responding City Labs
Of the 17 responding city labs, five were located in Rotterdam and the other 12 were relatively evenly distributed across the rest of the Netherlands (see Supplementary File 3). The average 'age' of the labs at the time of the survey was 3-4 years and the 'age' range was from 1 to 7 years. About a third of the labs (six) received financial support from the local government and operated under the responsibility of the municipality, while the remaining two thirds (11) operated independently and received funding from a variety of sources or generated some income from activities. Half of the labs (nine) employed one or more paid lab officials, usually the coordinators, while in most of the other labs a financial compensation was paid for free-lance specified services for the lab. For events or projects, all these labs made use of unpaid volunteers. Only two labs relied completely on volunteers. The core team running the lab usually consisted of local creative professionals (e.g., designers, architects, urbanists) and active, engaged residents. Only three of the responding labs indicated that local government officials formed the core team or were part of it.
About half of the labs (eight) had a specific thematic focus, such as sustainability-related issues (renewable energy, climate adaptation, circular economy, air quality), or socio-economic issues (livability, local/social economy, culture). The other half (nine) had no single thematic focus and aimed more broadly at integrated and inclusive bottom-up urban development. Also, in nine of the 17 cases the lab had a geographic focus on a particular district or neighborhood within the city. In the case of the 'independent' city labs, the focus-thematic and/or geographic-appeared to be more the result of contingencies than based on much reflection and/or on research or policy considerations. Only two labs referred explicitly to municipal policy goals when asked about their focus. Overall, labs with a specific thematic focus tended to cover the entire city, whereas labs with a specific geographic focus within the city tended to aim more broadly at integrated urban development (Table 1).

Experimentation in City Labs
All the labs confirmed that they were conducting experiments, broadly defined in the questionnaire as 'activities that had learning as a major goal.' When asked for examples of activities that matched this definition, over half of the labs (nine) provided clear examples (e.g., see Table 1), whereas for five labs this was unclear and for the other three labs the emphasis in their examples was on different goals, such as agenda-setting, mobilizing citizens, or just implementing plans. The majority of labs (11) also used the term 'experiment' for these activities, while the remainder preferred for various reasons to use more neutral terms, like project or pilot. The overall goal of experimentation was similar for all city labs: to contribute to urban development-within the focal area of the lab-by following a more co-creative, innovative, activating, and participatory approach than usually employed by municipalities and project developers.
In about half of the cases (eight), experiments were initiated by the core team of the lab, and in the other cases by other urban actors, such as active residents, neighborhood networks, and 'city makers' (local creative professionals). The municipality was mentioned specifically as the initiator of experiments in only one case. In most cases, even when the initiative to experiment came from an external urban actor, the core lab team was usually involved in and in charge of all stages of experimentation: coordination; decisions on whom to involve; implementation; and evaluation. As urban actors commonly involved in the experiment in addition to the core team and the initiators, actively engaged residents and city makers were mentioned several times. When asked about urban actors whose involvement was desirable but also difficult to obtain, a large variety of actors was mentioned (residents, entrepreneurs, dissat-isfied citizens, banks, developers, municipal agents, etc.), but no clear pattern emerged. Lessons from the experiments were usually shared offline in various ways with the urban actors associated with the lab, and, also, commonly published online, but without a specific target audience in mind. Other city labs in the network constituted the only specifically-mentioned external audience for sharing lessons. The municipality was not mentioned in this respect.
Based on the survey, the city labs' approach to experimentation can be characterized as 'informal.' It is strongly action-oriented, but in a flexible and open way, focused on learning-by-doing. Only a few labs follow a structured approach to setting up, conducting, and learning from experiments. Several labs even labeled such a structured approach as 'bureaucratic.' Almost all labs act opportunistically in selecting the topic, location, approach, partners, duration, and/or timing of an experiment. This dependency on opportunities, including initiatives taken by other urban actors, is often associated with the (very) limited resources of most labs.

Challenges in Setting Up, Conducting, and Learning from Experiments
When asked about problems and challenges encountered in setting up, conducting, and learning from experiments, a large variety of issues was mentioned, relating to almost every aspect of experimentation (Table 2). Only two issues were clearly mentioned more frequently than other issues. The first issue concerned the funding of the experiments, and acquisition of sufficient funding was also the area that labs most often indicated as in need of support. The second issue concerned various aspects of engaging a wider group of people to support or participate in the experiments beyond the core partners of the lab.  From the responses to the 16 closed questions at the end of the questionnaire, a similar picture emerges (Supplementary File 4). None of the answers was unanimously supported by all labs, but overall the majority of the labs were positive about the various aspects of experimentation addressed in the questions. The two issues on which a majority of the labs held negative views concerned the financial resources for conducting experiments (Q7) and the ability to engage all relevant parties in the experiments, including the more difficult to reach parties (Q3 and Q4).
Despite the general absence of a structured approach to experimentation, several 'good practices' (as in elements of a structured approach) were reported by multiple labs, such as determining the learning agenda before starting the experiment, evaluating the experiment both during its course and at the end, accepting 'failure' as a learning opportunity, drawing lessons, and following these up. However, the overall approach, also to these activities can be qualified as informal and inconsistent, and this concerns also the various aspects of learning from the experiments: formulation of learning questions or objectives prior to the experiment; how evaluation is conducted; and if and how lessons are documented and shared, and with whom. When asked about the need for support in learning from experiments, the most frequent response concerned support in how to do this in a more structured manner.
The issues that emerged from the co-design workshops with four city labs confirm this. When reflecting on their focal experiments, the participants indicated that the mindset was usually very much 'on the action,' on getting the experiment done, and not in a very reflexive way. This strong action-orientation went at the cost of considering critically before the start whether the experiment was the most appropriate way of achieving the lab's goals, and of making plans for dissemination of lessons and follow-up. The key areas in which city labs needed support, as identified in the workshops, concerned the framing of the experiment, in particular in relation to municipal policy goals, co-designing the experiment with stakeholders, and learning from experiments in a structured way.
The interviews with the three city labs having the most structured approach to experimentation showed that having this structure was, in itself, not a guarantee for success. The interviewees indicated that the involvement of multiple parties and changing conditions resulted in inconsistent and shifting framing of the experiments in terms of the aims, learning questions, lessons learned, and need for follow-up. Adding to this was the complex relationship of these three city labs with the municipality, as they were independent in principle, but receiving municipal financial support for the experiments at the same time. For example, the lab in Dordrecht successfully conducted experiments on spatial adaptation to increased risks of flooding, but later, when the municipality started a campaign on the issue, the achievements and lessons from these experiments were ignored and the lab participants felt demotivated. The experiments of the city lab in Nijmegen were initially based on citizen initiatives, without connection to the municipality. Recently, however, the lab has sought to strengthen the links with the municipal policy agenda. To achieve this, the lab is now hired by the municipality to play a role as intermediary in a sustainable neighborhood project. Contrary to the city lab, the municipality does not seem to see or frame this new role as an experiment to learn from, and focuses on the substantive outcomes of the project.

Discussion
In this section, we first present our key findings and conclusion on the practice of experimentation in Dutch city labs with respect to their potential contribution to 'urban planning by experiment.' We also suggest possible reasons for the poor match between observed practices and the characteristics that are considered to support the contribution of city labs to 'urban planning by experiment,' as well as possible ways to strengthen this contribution. We end with an outlook on the wider implications of this study and future developments.

The Practice of Experimentation in Dutch City Labs: Key Findings and Conclusion
The studied city labs do see experimenting as a core activity, although it appears that for some city labs, at least, goals other than learning prevail in their activities, such as agenda-setting or mobilizing citizens. Only a few city labs follow a more formally structured approach to experimentation. In most cases, experimentation is in many respects driven by opportunities, which probably has to do with the generally very limited human and financial resources of city labs.
As part of the opportunity-driven approach of most city labs, starting an experiment frequently depends on initiatives of other urban actors, such as local creative professionals or activist residents. In addition to the lab officials, who (help to) set up and coordinate the experiments, other core partners of the lab are usually involved as well, such as engaged volunteers, other interested creative professionals, and-in a few cases-the municipality. City labs strive for wider and active engagement beyond this core group of 'usual suspects,' but find it difficult to involve a large group of residents, including the more critical ones, entrepreneurs, and the municipality when it is not a lab partner. Thus, co-creation of experiments is 'standard practice,' but the diversity of urban actors involved seems rather limited.
Most city labs are actively sharing news about new and running experiments and lessons learned, using various online and offline channels. However, a structured approach to determining the learning goals of the experiments, evaluation, drawing lessons, and disseminating these lessons to defined external target groups is lacking in most city labs. It is therefore uncertain whether all lessons that could be learned from an experiment are made explicit and reach those that could apply them. Notably, municipalities are not targeted when not directly involved in the lab or specific experiments.
All city labs focus in their experiments on the needs of urban society, either by responding to initiatives of citizens and other urban actors, actively listening to residents, or engaging with creative professionals, such as social designers who claim to address the needs of the local communities. Making the connection with the urban planning and development agendas of municipalities appears much less frequently in experimentation practices. Only in a few labs was the municipality an element of the core team, and the municipality was mentioned only once as an external actor initiating an experiment. Moreover, the municipality was considered difficult to involve in experiments and was not mentioned as a target audience for the lessons or learning achieved through experiments. Also, in the interviews, the relationship with the municipality was described as problematic.
Based on these findings, we conclude that there is a poor match between the practice of experimentation in Dutch city labs and the four characteristics that are considered to support effective 'urban planning by experiment' (a structured approach to experimentation, co-creation of experiments, active and targeted dissemination of lessons learned, and experiments as linking pins between municipal policy goals and the needs of urban society). This suggests that the current contribution of Dutch city labs to 'urban planning by experiment' is probably quite limited.

Possible Explanations
There appear to be three major reasons for the poor match we observed. The first is that many of the studied city labs originate from citizen initiatives (so-called "organic ULL" in the terminology of Bulkeley et al., 2019), and do not have well-established relationships with the municipality. The second reason is probably also associated with this origin in citizen initiatives and concerns the strong action-orientation in experimentation, which often goes at the cost of spending time on formulating learning questions before moving to action, and taking time for drawing and disseminating the lessons after the action. Moreover, for most city labs the experiments also serve other goals than learning, which are sometimes considered more important (e.g., agenda-setting, mobilization). Also Bulkeley et al. (2019) found that other aims may be more important, such as city branding. The third reason for the poor match is that these city labs are strongly resource-limited (in time, competencies, and funds) and rely on emerging opportunities to conduct experiments. To a large extent this also applies to city labs that are financially supported by municipalities (so-called "civic ULL" in the terminology of Bulkeley et al., 2019), as the financing is normally restricted to providing office space and paying for lab coordination. Whereas Bulkeley et al. (2019) also pointed at the limited resources and improvised funding as being typical of organic ULL, the better funding position they ascribed to civic ULL (and the associated capacity to focus on the priorities of municipal governments) appears to be too optimistic and is probably biased by an overrepresentation of (inter)nationally co-funded demonstration projects in their sample.

Options to Strengthen the Role of City Labs in 'Urban Planning by Experiment'
One possible avenue for strengthening their role in 'urban planning by experiment' is by paying explicit attention in city labs to a structured approach to setting up, conducting, and learning from experiments. The two codesign workshops described under 'Methods' focused on the development of a tool to support this. This tool, which will be described in detail in a subsequent publication, does not prescribe one particular approach, but aims to support reflection and discussion among the urban actors involved in experimentation in order to develop their own well-considered and agreed-upon plan. The tool does this by asking questions in a logical order about many aspects of experimentation and efficiently supporting reflection and discussion by providing possible answers based on good practices collected in our survey and from the literature. These questions and suggestions also cover issues associated with the other three characteristics: wider involvement of stakeholders; learning and targeted dissemination of lessons; and connecting citizens' needs and initiatives with the local policy agenda.
Whereas developing a more structured approach to experimentation is an option that can be taken up by city labs themselves, other issues, such as lack of resources and poor alignment with the local policy agenda, require action from the side of the local government. To promote this, CIF-NL has recently launched two calls for proposals that aim to strengthen collaborations between municipalities and local creative professionals in city labs (CIF-NL, n.d.-c). These city labs are expected to focus on more bottom-up, creative approaches to current sociospatial challenges, such as the energy transition, climate adaptation, urban mobility and health, and social inclusion. As municipalities had to be (co-)applicants, these calls have resulted in a series of new or renewed Dutch city labs, in which the municipality is now a core partner. The challenge for these labs will be to strike a balance between, on the one hand, maintaining a good connection with the local urban development agenda and, on the other hand, creating room for novel ideas, open-ended experimentation, and room for failure.

Outlook
Urban experimentation by city labs could make important contributions to urban planning and enable 'urban planning by experiment.' However, the practice of experimentation in the 17 Dutch city labs studied here generally does not match the characteristics that previous research identified as key for making this contribution. This suggests that the current contribution of Dutch city labs to 'urban planning by experiment' is limited. The studied city labs represent a mix of what Bulkeley et al. (2019) labeled as "organic ULL" (led by citizens) and "civic ULL" (led by municipalities). Both types of ULL are also common in other European countries, like Austria, Sweden, and the UK (Bulkeley et al., 2019). We expect that the typical practice of experimentation encountered in the Dutch city labs, i.e., action-oriented, resourcelimited, and largely driven by opportunities, is likely to be found there as well. Of course, further research is needed to ascertain this, but it would imply that with the current practices, the role of ULL in 'urban planning by experiment' will be modest at best in other European countries as well.
For the network of Dutch city labs, initiatives have recently been undertaken to address several issues, such as a more structured approach to experimentation and a more direct relationship with the municipal priorities, which are expected to strengthen the role of city labs in 'urban planning by experiment.' However, follow-up studies will be needed to ascertain this, as there are also risks involved in 'forcing' the current pluralist and rather fuzzy practice of city labs into the mold of 'urban planning by experiment.' As indicated before, the current practice of experimentation relies rather heavily on 'windows of opportunity,' in terms of combinations of creative ideas, motivated residents and engaged professionals, suitable locations, and lack of interest from big market players. Moreover, the lack of a structured approach leaves much room for different interpretations of the experiment and its outcomes, making it possibly interesting for a wider group of actors to be involved than when the experiment is defined more explicitly and narrowly. In short, a more structured approach with a tighter connection to the local or even national policy agenda might 'drain the energy' from city labs, and thus undermine the very goal it aims to achieve.