A Geodesign Decision Support Environment for Integrating Management of Resource Flows in Spatial Planning

Improving waste and resource management entails working on interrelations between different material flows, territories and groups of actors. This calls for new decision support tools for translating the complex information on flows into accessible knowledge usable by stakeholders in the spatial planning process. This article describes an open source tool based on the geodesign approach, which links the co-creation of design proposals together with stakeholders, impact simulations informed by geographic contexts, systems thinking, and digital technology—the Geodesign Decision Support Environment. Though already used for strategic spatial planning, the potential of geodesign for waste management and recycling is yet to be explored. This article draws on empirical evidence from the pioneering application of the tool to promote spatially explicit circular economy strategies in the Amsterdam Metropolitan Area.


Introduction
With circular economy (CE) becoming a new sustainability paradigm (Geissdoerfer, Savaget, Bocken, & Hultink, 2017), strategies to reduce waste generation through better resource management have been climbing up the policy and planning agendas in numerous cities and regions. Improving waste and resource management entails understanding the interrelations between different material flows (e.g., organic waste, construction and demolition waste, plastics), territories (cities, regions, func-tional territorial units) and groups of actors (industrial actors along the cycle of a given material flow, waste management companies, regional and local authorities, civil society groups, builders and developers). This entails an increased complexity of interdependencies, relations and impacts of new kinds of circular processes and interventions that need to be considered in the decision-making process. Such complexity calls for new Spatial Decision Support Systems (SDSS) for translating the intricate information on material flows and related actors into accessible knowledge that could be used by stakeholders in the spatial planning process. SDSS typically combines tools from participatory Geographic Information Systems (GIS) with decision support tools, which have the capacity to animate and clarify discussions between stakeholders rather than just representing optimal results (de Wit, Brink, Bregt, & Velde, 2009).
The geodesign approach is a widely used methodology for exploring and addressing complex territorial challenges in different geographical scales while cooperating with stakeholders in an iterative and bottom-up manner (Li & Milburn, 2016). Therefore, geodesign emerges as a suitable methodology for supporting planning for the CE. However, to date, it has hardly been applied in the development of territorial strategies for reducing the generation of waste and closing the loops of material flows. Given the above-mentioned complexity and the importance of material flows in this field, the application requires modifying the methodology in order to integrate methods and technologies suitable for exploring the volumes and geographies of material flows, life cycle of materials and governance analyses. Technological innovation and rapidly increasing computational power, new means of sharing data and information and digital literacy, have a great potential to be effectively deployed in the pursuit of sustainability (Retief, Bond, Pope, Morrison-Saunders, & King, 2016). The tool proposed in this article, along with its underlying methodology, addresses this challenge by integrating geodesign with the Urban Living Labs (ULLs) approach (e.g., Steen & van Bueren, 2017). ULLs are becoming increasingly popular for engaging citizens and key stakeholders in the process of knowledge co-creation and co-design of experimental solutions to urban challenges in a real-life context. While geodesign is already used for strategic spatial planning, its potential for waste management and CE is yet to be explored. This article explores whether and how geodesign can be used to improve waste and resource management. It also describes a web-based open source tool that adapts geodesign for the purpose of spatial diagnosis and elaborates on territorial and systemic ecoinnovative strategies toward a CE through the Geodesign Decision Support Environment (GDSE).
Section 2 outlines the theoretical background for the GDSE and builds on recent geodesign and living lab approaches and technology implementations in the field of spatial planning. Section 3 describes the geodesign-based GDSE methodology to support collaborative resource flow management. The methodology is applied within an ongoing living lab aimed at improving waste and recycling management in the Amsterdam Metropolitan Area (AMA; Section 4). Finally, conclusions on the usefulness and limitations of the GDSE are provided in Section 5.

Theoretical Background
CE is primarily driven by the agreements between multiple actors to share resources, materials and infrastruc-ture for as long as their physical properties allow. This increases the pool of stakeholders that could act together, which may create collective strategies to achieve higher benefits to everyone's interests. Mathematical models could theoretically be used to optimize the total sum of individual, environmental, social and economic benefits. However, in practice, modelling such a system accurately is too complicated. This type of modelling requires the integration of technology and analytical methods with new collaborative approaches for spatial decisionmaking. We propose an approach that builds on three elements: current technological advances and related analytical methods, the geodesign framework, and the ULL approach as a methodological environment for stakeholder involvement.

Technology and Analysis Methods
GIS are not only used for cartographic analysis but are increasingly being used for building narratives, qualitative storytelling and within synthesis approaches with the goal for equity and justice (Sui, 2015). Although the usefulness of GIS in all stages of impact assessments have already been recognized (e.g., Eedy, 1995), it is still seldomly applied in sustainability assessments (e.g., Sholarin & Awange, 2015). SDSS are used to help address similar ill-defined problems and are defined as interactive, computer-based systems designed to support a group of users in achieving higher effectiveness in decision-making on spatial issues (Malczewski, 1999). They are meant to support rather than to replace human judgements, and improve the effectiveness rather than the efficiency of a process (Uran & Janssen, 2003). Thus, they are intended to be advisory units that are more capable to digest large amounts of data and can perform quick computations. Decision-making tends to entail social and political conflicts while also relating to values that reflect cultural, historical and social norms that are deemed acceptable by a community (Jones & Morrison-Saunders, 2016). This is crucial for spatial planning and waste management, which are (1) connected to specific geographical contexts with intrinsic cultural, historical and social values, and (2) directly affect the environment and the society in a given territory.
Currently, the most common combination of methods for assessing the impacts of potential resource flow changes includes Material Flow Analysis (MFA) and Life Cycle Assessment (LCA; e.g., Guinée, 2002). MFA is a systematic assessment of the flows and stocks of materials within a system that is defined in a space and time (Brunner & Rechberger, 2016) and provides a system understanding of a particular state of resource flows. MFA is typically applied in the built environment (e.g., Crawford, 2011). Although MFA studies have always had explicit spatial and temporal boundaries (e.g., Stephan & Athanassiadis, 2017), what happens within those limits is rather considered as a black box, where materials flow from inputs to outputs through various stocks and processes. These flows and processes are not typically described in great detail spatially, except with a few attempted studies. For example, Roy, Curry and Ellis (2014) spatially allocated construction material flows within administrative units of Kildare County, Ireland. Wallsten (2015) used the context of hibernating the stock of subsurface urban infrastructure to demonstrate how social science approaches can provide hands-on advice for private and local actors involved in material recycling. Vivanco, Ventosa and Durany (2012) developed a model for material and spatial characterization of waste flows, which included indicators that were potentially useful for assessing key policy strategies for waste management and the minimization of transport by locating adequate facilities. Even though there have been existing attempts to introduce a spatial dimension into the MFA methodology, the spatial granularity is very coarse and its usefulness in decision-making has not been validated as of yet.
LCA is used to assess environmental, social and economic impacts of products or services through all the stages of their lifetime in comparison to a baseline scenario (Taelman, Tonini, Wandl, & Dewulf, 2018). LCA intends to support decision-making and therefore, the involvement of decision-makers throughout the entire study is crucial in order to avoid issues addressed by the study that may differ from those that the decisionmakers deem as important. Depending on the situation, it may be relevant to include other stakeholders that may be affected by or can influence the consequences of the decision (Weidema, 2000). Failure to involve stakeholders may result in controversies or may hamper the implementation of the suggested environmental improvements. Hence, decision-making in spatial planning and resource management should not be top-down and should include local stakeholders, especially if they are the ones most affected by the decisions made. Although LCA is mostly used for environmental impacts, it may also include several impact categories, such as social or economic impacts (Jeswani, Azapagic, Schepelmann, & Ritthoff, 2010). LCA also aims to include as many substances and compounds, which is required to provide a full impact assessment. The method is widely accepted and standardized in ISO 14040 (Technical Committee ISO, 2019). However, conducting an LCA requires an extensive amount of time and data that is not often available. Moreover, communicating the results usually requires an expert audience (Elia, Gnoni, & Tornese, 2017). This is not in line with typical geodesign workshops that would last only a few days. Thus, the integration of geodesign with living labs prolongs the study period and allows the use of more advanced impact assessment methods.

Geodesign
Geodesign has emerged as a relevant concept for furthering the development of enhanced SDSS. The use of SDSS for policymaking has changed over the last decades, which can be reflected by an increased role of pub-lic participation in combination with collaborative approaches (Keenan & Jankowski, 2019). The increasingly apparent multi-stakeholder nature of policymaking has led to the recent development of SDSS that aim to address group decision-making (Jankowski, 2009). In parallel, many participatory approaches for spatial decisionmaking emerged, which require more collaborative tools and methodologies (Li & Milburn, 2016). Geodesign is a leading methodology to support spatial planning as it tightly couples the creation of design proposals with impact simulations informed by geographical context (Steinitz, 2012), and ensures a close collaboration between the stakeholders and decision-makers throughout the entire process that starts at problem identification and finishes at proposed interventions. Specifically, geodesign offers a framework that facilitates collaboration in iterative spatial decision processes involving future spatial interventions in a geographic study area. Figure 1 illustrates the structure of this framework.
The process involves three iterative feedback loops, which aim to (1) understand, scope, and model a geographic study area, (2) specify methods to operationalize the process, and (3) carry out the geodesign process tasks. Each iteration addresses a set of six questions, each of which is answered by specific models. The framework represents the collaboration as the interaction required between four types of stakeholders: the people of the place, geography-oriented natural and social sciences experts, design and planning professionals, and their IT technologists.

Urban Living Labs
There are multiple ways to involve the affected people into the planning process. The International Association of Public Participation (IAP2) has devised a spectrum that explains the different levels of public participation ( Figure 2). As seen from this spectrum, merely involving the public into the planning process does not mean that their tacit knowledge and community preferences are used to improve the planning process. SDSS are being used on the full range of the spectrum-from acting as information systems to empowering the stakeholders to become the decision-makers. Living labs constitute an effective method for incorporating innovation and technology into participatory and multidisciplinary planning processes.
According to the European Network of Living Labs (ENoLL), living labs can be regarded as "user-centered, open innovation ecosystems based on a systematic user co-creation approach in public-private-people partnerships, integrating research and innovation processes in real-life communities and settings" (ENoLL, 2019). ULLs are comprised of physical and virtual environments, in which public-private-people partnerships experiment with an iterative method to jointly develop innovations (i.e., co-creation) that include the involvement of end-users and aim at identifying and addressing ur-  (Steinitz, 2012). Graphic by author Libera Amenta. ban sustainability challenges. Main characteristics of an ULL are geographical embeddedness, experimentation and learning, participation and user involvement, leadership and ownership, and evaluation and refinement (Voytenko, Mccormick, Evans, & Schliwa, 2016). The ENoLL approach is based on the quadruple helix model of partnership, which categorizes actors as the government, industry, the public and academia, who work together to generate innovative solutions in a process involving five phases, namely co-exploring, co-design, co-production, co-decision, and co-governance (ENoLL, 2019).

Integrating Geodesign, Living Labs and Technology
This article argues that collaboration between actors within an iterative geodesign process with feedback loops plays a central role alongside innovation and the implementation of new technology, which can be facilitated through a living lab approach. The integration between geodesign, the living lab approach, GIS, MFA and LCA into a single support environment ( Figure 3) allows for the following innovations: (1) MFA in a geographical context: via a new method of Activity-Based Spatial Material Flow Analysis (AS-MFA; Resource Management in Peri-Urban Areas [REPAiR], 2017) by geo-locating activities and actors involved in resource flows; (2) Visualization of resource flows: via AS-MFA data analysis and visualization tools in order to gain insights into the current status quo at early stages of the solution creation process rather than only at the stage of evaluation; (3) Simulation of proposed changes: applying the solutions as simulations of changes in the overall mapped resource flow network; (4) LCA for impact assessment: using the AS-MFA data to describe the LCA baseline scenario and the simulated resource flow network of proposed strategies.
The GDSE provides an environment to support the collaborative efforts towards improving resource management and thus enhancing the transition towards CE. It incorporates all the relevant methodologies identified in the theoretical framework and provides both the researchers and the stakeholders with an overall structure and tools. The environment consists of software, hardware and processware.

Software
The GDSE is a core product of an ongoing EU-funded research project called REPAiR. It features an open source prototype web application that supports both the decision-making process and the research that is required for each of the five steps to guide the living lab process for a study area (Figure 4), available on the project's website. REPAiR aims to implement the GDSE in living labs in six European metropolitan areas to develop place-based eco-innovative spatial development strategies that aims to have a quantitative reduction of waste flows in the peri-urban areas (REPAiR, 2019b). Within REPAiR, a GDSE-related eco-innovative strategy is understood as: An alternative course of actions aimed at addressing the objectives identified within a Peri-Urban Living Lab (PULL) for developing a more CE in peri-urban areas, which can be composed of a systemic integration of two or more elementary actions, namely ecoinnovative solutions (EIS). (REPAiR, 2018a) To facilitate the ease of reading, from this point forward, "eco-innovative solutions" will also be referred to as either "solutions" or "EIS", while eco-innovative strategies will also be referred to as "strategies". While designed and tested for the specific purposes of the REPAiR case studies, the GDSE is meant to be easily reusable, which is one of the guiding principles of the software development process. Thus, the GDSE is built with free and open source components and has an open license. All versions of the source code are available on a public GitHub repository (https://github.com/ MaxBo/REPAiR-Web). Figure 5 shows the current backend integration of various components into a single platform that supports a range of functions: data management and storage, data visualization, stakeholder input, simulation and assessment of alternatives, and connection to an external LCA assessment.
Data storage and management is done via the Open Science Framework (https://osf.io). GeoServer

Technology and analysis
Living Lab  (http://geoserver.org) is used to publish and host spatial data layers, as web feature services, incorporated and visualized in the GDSE, which are externally prepared using QGIS (https://qgis.org). All the AS-MFA data used for the analysis and assessment are stored in a PostgreSQL object-relational database (https://www.postgresql.org). LCA is conducted externally. All outputs are displayed in the GDSE. Vagrant (https://www.vagrantup.com) is used for providing a reproducible, operating system which is independent of the software environment setup. Two main roles that are supported by the GDSE are the researcher and the stakeholder (Table 1). A researcher (or a group of researchers) is responsible for organizing the geodesign process, finding and involving the relevant stakeholders, collecting, preparing, uploading and selecting relevant data, performing impact assessment, preparing and holding the interactive workshop sessions, collecting stakeholder input from those sessions for use in subsequent ones. A stakeholder (or a group of stakeholders) uses the system at workshop sessions, which are facilitated and moderated by re-searchers. The GDSE provides different functions within two separate environments for the previously described roles: the setup mode and the workshop mode.

Hardware
The GDSE hardware component features interactive touch-enabled screens to facilitate workshop communication in two ways: (1) between users and the GDSE software (tools and support information), and (2) dialogue between the users. The touch tables ( Figure 6) can easily be switched between horizontal or vertical mode, depending on the purpose (group discussions or presentations).

Processware
The processware involves a series of interconnected workshops and the guidelines on how to organize these workshops. These are part of the REPAiR's PULLs (REPAiR, 2019a). A PULL workshop is a meeting in which stakeholders from the field of waste and resource management Read the generated summary of the whole included into the evaluation of the conclusions geodesign process gather to discuss waste management issues related to the future use of an area or region. Stakeholders work together in small groups of 2 to 6 participants, with each group using the GDSE on a touch table in a co-design process of solutions that together make up CE strategies. PULL workshops typically follow the Charrette System's five-part format (Lennertz & Lutzenhiser, 2006): (1) Pre-workshop survey + introduction and goals; (2) Support information + GDSE demonstration; (3) Division in small groups and (cross-group) touch table assignment using the GDSE; (4) Presentation of results; (5) Plenary session and discussion/post-workshop survey.
A REPAiR PULL features four types of workshops, which are categorized according to the first four phases of the REPAiR co-creation process in living labs: coexploring, co-design, co-production, and co-decision (REPAiR, 2018a). The fifth phase, 'co-governance' does not involve PULL workshops.

Co-Exploration Workshop
This workshop takes place at the end of the coexploration PULL phase and aims at: (1) Developing a common understanding of the territory, including the mapping of wasted landscapes, or wastescapes (Amenta & van Timmeren, 2018), and stakeholders; (2) Categorizing and defining the main CE challenges and objectives. Table 2 shows the process leading to the workshop. The first two geodesign questions are addressed with the help of GIS and MFA. This involves mapping the region, defining the stakeholders and experts, and selecting and mapping key material flows. The GDSE is used to show and interactively discuss the study area and its status quo (maps, charts, stakeholders and key flows), and thereby help to construct a common knowledge among local research teams and other participants of the PULL. Moreover, the GDSE supports groups of stakeholders to jointly define challenges and objectives as well as think about paths for developing eco-innovative strategies. Concretely, spatial and social analyses, as well as material flows and stocks are displayed and discussed using interactive maps and Sankey diagrams linked to these maps.
The process model relates to the dynamics of the system and is meant to represent the material flows within the chosen temporal and spatial scope. Therefore, the first task is identifying a key flow (e.g., organic waste, construction and demolition waste, electronic waste) for further investigation. The key flow is chosen during a collaborative process according to the criteria defined by the stakeholders. As explained in Section 2.1, MFA is typically used for detailed analyses of resource flows. The GDSE does not only incorporate a standard MFA method but also connects it with a geographical context. By introducing a new AS-MFA method (REPAiR, 2017) while geographically locating activities and actors involved in the resource flows, this enables further (iterative) identification of stakeholders and experts for potential strategies.

Co-Design Workshop
This workshop takes place at the end of the PULL phase co-design. Its main aims are: (1) Identifying, mapping and visualizing key activities and actors in the value chains that should be included in the discussion and development of ecoinnovative solutions; (2) Identifying specific CE challenges in the study area; (3) Identifying and mapping actor networks for each individual eco-innovative solutions development.

Co-Exploration Representation Model
How should the study area be described?

GIS Definition and mapping of Region-Focus, and Sample Areas
Definition and mapping of Wastescapes

Process Model
How does the study area operate?

MFA & GIS Selection of key resource flows
Definition and mapping of material flows and waste management system Table 3. Addressing geodesign questions at PULL phase co-design.

Co-Design Evaluation Model
Is the current study area working well?
GIS & LCA Sustainability assessment of the status quo Assessment of the status quo's resource flow circularity

Change Model
How might the study area be modified?

MFA Definition and common understanding of what constitutes an EIS
Characteristics and effect of EIS on the process model terial flows and actors (e.g., companies) in the area based on their commercial activity. The GDSE stores the developed solutions, their descriptions and also the selection of the potential actors involved. The third geodesign question ("is the current study area working well?") refers to an assessment of the status quo or baseline scenario that allows for future comparisons with the proposed strategies (alternative future scenarios). The GDSE evaluates the status quo in terms of flow indicators based on the MFA data and a sustainability assessment. Flow indicators are first identified using existing literature (Zhang, Yang, & Yu, 2009) and then are selected through a collaborative process by the stakeholders during a co-design workshop. REPAiR defines an initial list of flow indicators, which includes flow amounts (for each material or their combination, e.g., vegetal waste vs. separate vegetables and fruits), flow structure (e.g., percentage of renewable material in each flow), flow intensity (e.g., amount of flow consumed/conducted per person), flow efficiency (relationship between economic factors and each material flow), and flow density (material consumption/conduction to sustain urban development) (REPAiR, 2019a). To undertake the sustainability assessment of the status quo for the study area, the REPAiR team has developed a framework for conducting a sustainability assessment on four impact categories (Taelman et al., 2018). This framework will be used to assess the impacts of developed ecoinnovative strategies at later stages of the PULL.

Co-Production Workshop
This workshop takes place at the end of the PULL phase co-production and aims to attain: (1) The ranking of objectives per decision-maker group; (2) A set of flow targets the group wants to achieve; (3) One strategy per small group and key flow. Table 4 illustrates how GDSE addresses geodesign questions 4 and 5 with the help of GIS and MFA. The third phase aims to develop one eco-innovative strategy per small group and key flow to address the objectives previously defined in earlier workshops. Each small group will select several solutions, which will together make up their eco-innovative strategy.
Co-production workshops focus mainly on the development of eco-innovative strategies, expert knowledge on specific eco-innovative solutions that make up the strategies, and relative importance of sustainability indicators, which are based on the LCA methodology and which measure the various impacts of the strategies developed. Main outcomes of this workshop are ranked CE objectives, weights of the sustainability indicators, selected eco-innovative solutions and developed eco-innovative strategies. Multicriteria (MCA) methods support the comparisons of impacts of the strategies on sustainability.

Co-Decision Workshop
This workshop takes place at the end of the PULL phase co-decision and aims to reach a common understanding of: (1) The differences and similarities between the ranked objectives per stakeholder small groups; (2) The flow indicators that were used for setting targets for specific objectives; (3) The differences and similarities between the strategies implemented in terms of the related solutions, across stakeholder groups, and locations of EIS implementations; (4) How the specific processes in the value chain of the key flows contribute to the different impacts, in particular to the extent to which the developed strategies modify the key flows and meet the various target sets; (5) Potential sustainability assessments of the strategies developed by individual small groups; (6) Agreements and disagreements (i.e. consensus level) on objectives, targets, related strategies and where the selected EIS have been implemented for all key flows. Table 4. Addressing geodesign questions at the PULL phase of co-production.

Co-Production Change Model
How might the study area be modified?

Decision Model
How should the study area be changed?

MCA Relating EIS to objectives
Ranking of objectives Pairwise comparison of the relative importance of sustainability indicators Defining the targets Table 5 shows how the GDSE supports the co-decision phase. The last two geodesign questions are addressed with the help of LCA, and flow assessment calculations. The main outcomes are a concrete plan with detailed implementation actions for each eco-innovative strategy, a list of actors and stakeholders to collaborate in the implementation of each specific strategy, a timeline for actual implementation of each strategy and the corresponding EIS. The assessment of proposed strategies is done using two methodologies: LCA and the assessment of flow changes. While the flow changes are assessed in real time during the workshop, the LCA is performed after the workshop by LCA practitioners. This is due to the complexity of the LCA as well as the current lack of software interoperability.
Assessing flow changes is done by comparing the status quo flow indicator set during the co-design phase with the anticipated changes introduced by the strategies in the co-production phase. Once a combination of solutions and their implementation areas are chosen by the workshop participants, a flow calculation algorithm redistributes the flows in between the economic activities, keeping the overall mass balance of the affected flows consistent. The algorithm hypothetically distributes the total surplus or shortfalls within an economic activity in between all the actors present in a chosen geographical area of implementation. That way, the flow changes are reflected in the chosen indicators and their values can be compared with the targets that were set up in the co-production phase.
At the time of writing this article, some modules of the GDSE are not yet fully operational. However, the GDSE has already been used in the workshops described in this article, which have been held in parallel to the GDSE development process. The GDSE is designed with help of intended end-users, in line with the living lab approach, in which end-users test and provide constant feedback on the support tools. This is also in line with the recommendations of Uran and Janssen (2003) that SDSS should be developed to serve their intended purpose instead of those of the study team. The next section presents the application of the GDSE methodology to an Amsterdam case study.

The Amsterdam Peri-Urban Living Lab
The GDSE methodology is tested and applied as part of the ongoing living lab of the AMA, which encompasses the city of Amsterdam, the provinces North Holland and Flevoland. This is comprised of 32 municipalities, and a total population of over 2.4 million inhabitants. With an area of 539 km 2 , the AMA focus area (Figure 7) is located in the peri-urban areas in the west and south west of the AMA and constitutes a pilot case study of REPAiR.
Yearly household waste data was gathered for the AMA. The datasets came from the CBS, Statistics Netherlands. Waste data for companies was retrieved via the Dutch register for electronic waste notifications and communication of the National Contact Point for Waste (Dutch acronym: LMA), which describes the supply, composition and processing of company/industrial waste in the Netherlands. Both datasets describe waste flows for the year 2016. This data is entered by the collectors and managed by the government and contains information on the type of waste (Eural code), waste generator (e.g., name and location of the company), and waste collector (name and location of waste treatment), and the type of waste treatment.

Using a GDSE for Co-Developing Eco-Innovative CE Strategies in Amsterdam
The first four phases of the PULL process in the AMA involved four types of workshops, namely coexploration, co-design, co-production, and co-decision (REPAiR, 2018a). At the time of writing this article, the GDSE had been used at the first three phases of the ongoing PULL process in the AMA. Three PULL workshops have thus been organized with local governments and Table 5. Addressing geodesign questions at the PULL phase co-decision.

Co-Decision Impact Model
What differences might the change cause?

LCA, Flow assessment calculation
Sustainability and flow assessment of Eco-Innovative Strategies

Decision Model
How should the study area be changed?

Designing rules of system
Establishing and documenting the agreements and conflicts between differente interests and groups of decision makes Triggering future local development and supporting decision-making processes policy makers, local business representatives, international partners of the REPAiR project consortium, and the PULL hosting team. This section presents results from these workshops.

Results: Co-Exploration
The workshop aimed to define key waste and resource management challenges in the study area by the means of: (1) Verifying challenges already identified in previous interviews with stakeholders and literature review; (2) Adding new challenges if required or needed; (3) Developing challenges to a detailed level along with suggested solution paths.
The first step was to share with the participants relevant information on the AMA, which was collected, categorized and uploaded to the GDSE by the PULL team using the GDSE setup mode. Then the stakeholders were required to discuss and modify (i.e., validate, correct, remove, complement) all the information where deemed necessary. This information included (1) maps of the focus area (topographic and related to resource and waste management), (2) relevant charts with the first list of circularity challenges of the area, and (3) the first list of main stakeholders of the PULL process. "Challenge trees" were used as the main materials to present CE challenges in the AMA to stakeholders both in an A3 article format and digitally in the GDSE. Each branch on a challenge tree (Figure 8, right panel) represents one main challenge for the AMA, and each sub-branch represents specific challenges for a particular main challenge branch. Above each challenge branch, there are two fringes, each containing a question for the participants. The questions were: "what if we do this? (where and who should be involved?)" and "what should be assessed?". Participants were asked to provide feedback on each challenge tree by suggesting modifications and inserting sticky notes for each fringe. The results were directly fed into the GDSE (Figure 8). The main workshop outcomes included a categorized list of CE AMA challenges along with possible solution paths.

Results: Co-Design
The main objective of this workshop was to develop initial sketches of eco-innovative solutions towards CE in the AMA, based on the CE objectives identified in a previous workshop. The specific workshop aims were to: (1) Verify and rank the identified objectives with the selected stakeholders; (2) Develop initial sketches for how to meet the objectives, developing preliminary sets of EIS that follow a common GDSE-friendly template.
The output from the previous co-exploration workshop (CE challenges) was used as input for this workshop. "Solution sheets" were used as main materials to communicate eco-innovative solutions to participants, and to describe solutions using a common template. A solution sheet (Figure 9) was an A3-formatted sheet that contained specific information about a solution. A sheet contained three panels, namely solution card (containing main characteristics, category and description), CE dia- gram of the solution, system diagram with activities and flows in the solutions. Participants were asked to review, complete the sheet and suggest how to modify the solution ( Figure 9). The main workshop outcome was a catalogue of solutions that addressed the ranked CE objectives in the AMA. The solutions in this catalogue were digitized and directly fed into the GDSE to make the solutions available for ensuing PULL workshops.

Results: Co-Design/Co-Production
The third PULL workshop was the most recent and was categorized as part of both the co-design and coproduction phase. It aimed at further developing the solutions discussed in the previous workshop. The workshop included three parallel sessions, each focusing on one key flow category: food waste, wastescapes, and construction and demolition waste. A GDSE-enabled touch table was available for each session (Figure 10). The GDSE was used to provide support information on flows, solutions, activities, and actors. The participants were asked to work on one session table at a time and to select solutions for further development. Specific main goals of the workshop were to: (1) Co-develop EIS, following a GDSE-friendly template, based on an EIS initial set; (2) Match EIS with CE objectives.
New solution sheets were used as materials. The GDSE was used as main software tool on three touch tables to help users retrieve information concerning the solutions they were discussing and working on.
Stakeholders used the GDSE to analyze possible actors and existing waste streams related to the ecoinnovative solutions they worked on. Figures 11 and 12 illustrate how the stakeholders used the GDSE to map actors relevant to a food waste EIS, and to visualize waste streams connected to this EIS.
The main outcome of this workshop was the updated EIS catalogue for the AMA. Through a research by design approach, together with local stakeholders, young designers and students of industrial ecology, architecture, urbanism, and with the help of the GDSE, 27 ecoinnovative solutions were developed on the basis of aspects, such as relevance for practice, possible areas for further EIS implementation, actors to be involved, business model to implement, and potential policy changes. Figure 13 shows an example of one eco-innovative solution: mycelium blocks for wastescapes modelled in the GDSE. The CE diagrams are displayed for this solution: at the current linear state (on the left) and a new proposed, more circular, value chain (on the right). Actors involved in this solution can also be retrieved in an interactive map. The EIS catalogue has been uploaded to the GDSE and will be used by participants of subsequent co-production and co-design workshops to support the process of combining EIS into strategies.

Effectiveness of PULL workshops
Surveys were conducted before and after the workshops. Pre-workshop surveys contained questions about the participants' workshop expectations, general expertise and interest in eco-innovative solutions. The surveys were completed by an average of 19 workshop participants, whose backgrounds included human geography, urban design, architecture, and MSc students in Architecture, Industrial Ecology and Urbanism. They rated their own expertise/interest in EIS as 6.4 on a 1-10 scale. Post-workshop surveys contained questions on their experience and specific aspects of workshop effectiveness (Table 6). In general, participants gave good ratings to all workshops, and in particular, the third workshop had the highest rating for average effectiveness and for specific workshop features.
The next steps for the PULL in the AMA will involve further operational EIS development that resulted from this workshop towards more detailed solutions that can be represented, assessed and compared iteratively in the GDSE. Dedicated PULL meetings will be held separately for each material flow investigated, and will host smaller groups of stakeholders who are experts in the different material flows in order to further detail the EIS in the GDSE. Stakeholders will be asked to jointly define, and interactively modify strategies for specific key flows by combining one or more implementations of solutions ( Figure 14). The GDSE will provide real-time feedback on the impacts of strategies on flow changes and sustainability indicators (Figures 15 and 16).

Conclusions
To address the question of whether and how geodesign can be used to improve waste and resource management, this article proposes a geodesign-based tool for supporting a collaborative process of developing ecoinnovative strategies to advance CE in peri-urban areas. Geodesign can provide a helpful framework for improving waste and resource management, which is evident by the observations and outcomes of the PULL workshops, and the positive reactions of the participants in the surveys. In fact, geodesign allows for a structured and comprehensive organization of the process and diagnosis of challenges, design and selection of solutions, and  14. GDSE screenshot showing a strategy for key flow "food waste", which is composed of three solutions, each with their own area of application and list of actors implementing them. Any solution in the strategy can be edited in a separate pop-up window (e.g., EIS "from bread to beer" in the strategy shown here).  decision-making on strategies for a given territory with close stakeholder involvement. In addition, the GDSE integrates spatial data on material flows and related actors, which are presented in a visual and accessible way and ensures a sound and accessible evidence base in the participatory process. In order to address several limitations of geodesign, the GDSE integrates human creativity into a digital interface with complex spatial and metabolic analysis methods in the participatory context of living labs. This allows for informed coordination of waste management activities in space and evidence-based co-design of innovative and spatial solutions with stakeholders. This integration anchors the geodesign process in ongoing experimentations in study areas and enables a continuous engagement of stakeholders in the analysis, building on relatively simple visualization of complex data on material flows in space, and in the co-design of innovative circular solutions. Geodesign thinking enables the process of adding a spatial dimension to typically non-spatial analysis methods (e.g., MFA). Moreover, as the stakeholders argued, the GDSE's key advantage is the ability to make the exploration, design and decision-making process transparent to the participants.
Naturally, there are limitations to the GDSE approach. Firstly, even though the potential of the GDSE to support participatory development of spatial waste and resource management strategies has been demonstrated and validated by the stakeholders involved, the tool is still work in progress. The strategies developed so far with the GDSE have not yet been taken up and implemented by the Amsterdam region stakeholders.
Secondly, the GDSE's capacity to assist in the analysis phase and spatial visualization of material flows depends on the availability of data. Likewise, the quality of data is a critical concern for the GDSE's ability to model the impacts of the strategies co-created with stakeholders. While a robust dataset on material flows was available in the Amsterdam pilot case study, considerable efforts were needed to collect and feed the data into the GDSE and the availability of such data cannot be taken for granted in all regional contexts.
Thirdly, given the complexity and uncertainty involved in enacting CE strategies, a successful GDSE application in the living labs critically depends on the ability to attract and retain the engagement of not only key territorial stakeholders along the entire value chain, but also experts with specific technical knowledge on both the processes and technologies envisaged in the co-designed strategies. Considering the busy agendas of some stakeholders, this proves challenging in practice, as they need to commit and allocate precious time to repeated interactions in the living lab over several months, which cannot be taken for granted. Thus, future GDSE applications require developing robust procedures for identifying the most relevant and knowledgeable stakehold-ers and keeping them involved in the process. Successful implementations in living lab workshop requires the involvement of an experienced moderator.
Fourth, while the GDSE allows for the estimation of the impacts of strategies co-designed in the living lab, there is a considerable amount of uncertainty about their actual real-life effects. This highlights the need for monitoring the outcomes of the decision-making process facilitated by the GDSE and the implementation of the strategies developed. Integrating monitoring measures within the proposed approach would allow for validation and the creation of a scope for an iterative learning process among the stakeholders. Overcoming these limitations will require further development and testing of the tool as well as scrutiny of the implementation of the strategies developed using the GDSE in a longer temporal perspective.
To conclude, the GDSE-urban-living-lab combination provides a relational space including stakeholders in a structured process in a specific location, spanning over a longer time period, allowing for a more sustained process of co-exploration of the status quo, cocreation of knowledge, and co-production of solutions and strategies. This long-term iterative engagement between stakeholders not only empowers them but also enables a more in-depth analysis for a better integration of various strands of knowledge, while building on inputs from research at each iteration. An open source GDSE facilitates the implementation of innovation in a living lab. The GDSE is developed in cooperation with end-users, which facilitates not only continuous tailoring of the tool based on end-user feedback, but also a smoother adaptation of this open source tool to other case studies, or in a different living lab setting. Future work will focus on a comparative analysis of GDSE applications in different regional settings.