‘Sensor’ship and Spatial Data Quality

This article describes a Los Angeles-based website that collects volunteered geographic information (VGI) on outdoor advertising using the Google Street View interface. The Billboard Map website was designed to help the city regulate signage. The Los Angeles landscape is thick with advertising, and the city efforts to count total of signs has been stymied by litigation and political pressure. Because outdoor advertising is designed to be seen, the community collectively knows how many and where signs exist. As such, outdoor advertising is a perfect subject for VGI. This paper analyzes the Los Angeles community's entries in the Billboard Map website both quantitatively and qualitatively. I find that members of the public are well able to map outdoor advertisements, successfully employing the Google Street View interface to pinpoint sign locations. However, the community proved unaware of the regulatory distinctions between types of signs, mapping many more signs than those the city technically designates as billboards. Though these findings might suggest spatial data quality issues in the use of VGI for municipal record-keeping, I argue that the Billboard Map teaches an important lesson about how the public's conceptualization of the urban landscape differs from that envisioned by city planners. In particular, I argue that community members see the landscape of advertising holistically, while city agents treat the landscape as a collection of individual categories. This is important because, while Los Angeles recently banned new off-site signs, it continues to approve similar signs under new planning categories, with more in the works.


Introduction: Outdoor Advertising and the Los Angeles Landscape
To outdoor advertisers, Los Angeles is "the largest outdoor advertising market in the United States" (JCDecaux, 2007).To community activists, the city is "ground zero of billboard blight" (M.Ashburn, personal communication, 2011).Los Angeles has a landscape of suburban sprawl designed for automobile travel, and it is the home of the entertainment industry-factors that led to a density of billboards greater than other cities (Gudis, 2004).In the 1920s, the city began requiring permits for off-site signs, signs that advertise a product or service not available on the same site and commonly called "billboards" (1 L. A.M.C. 4.4 § 14.4.4(B)(11)).Yet the city enacted nearly no regulations as to the location and amount of billboards that could be erected.Years later, the Department of City Planning (2009) opined that the city's lax regulations: "have shaped the way signage has been incorporated into our streetscapes, in a way that can now in retrospect be described as excessive.A proliferation of signage adds significantly to the visual clutter for which Los Angeles has become well-known, and points to the need for stricter sign regulations." In 2002, the City of Los Angeles made two important steps toward regulating signage.It banned new off-site signs, and it created the Off-Site Sign Periodic Inspection Program, which charged the Department of Building and Safety (LADBS) with creating a compre-hensive inventory of existing off-site signs.Almost immediately, outdoor advertising companies challenged these laws in court.The three corporations that own the lion's share of billboards in the city held up the inventory program for years, and even after the city was legally cleared to restart the program, political pressures kept the city from commencing work and, later, releasing its results (Sedano, 2016).This article describes a project to employ volunteered geographic information (VGI) to map off-site signs in Los Angeles.The project was begun during the years that the city's inventory program was stymied and was designed to aid regulation by gathering data that the city was legally and politically unable to.Off-site signs are a perfect subject for VGI in urban settings because signs are made to be seen by the public at large, and there are a lot of them.While governing bodies may not have specific knowledge of the changing landscape of signs, collectively, residents do.
This article studies how urban residents mapped the landscape of off-site signs in Los Angeles and, as such, is positioned within the emerging field of VGI and its concern with the quality of spatial data created by non-professionals.Yet the article arrives at its key finding-that residents understand signage far differently than the professionals who make and enforce the zoning code-by way of landscape theory.Landscape is the field of social life, the land that we develop or choose not to develop, and the structures we build, mold, and maneuver around.The urban landscape is a "palimpsest", a concatenation of the old and the new rather than layer upon layer (Schein, 1997, p. 662).The guiding principle of modern landscape theory is that landscape is both a material and a cultural construct (Olwig, 1996).The landscape is primarily a visual field, but not an objective one (Cosgrove, 2003).Following art historian Berger (1972), who identifies the power inherent in the gaze and employs the phrase "ways of seeing" to capture the idea that perception is a learned skill, Cosgrove (1984) posits that the landscape is seen differently by different viewers.He studies the role of 18th and19th C. British landscape painting in remaking both the cultural conception of landscape and the material landscape to match this ideal of aristocratic property owners.The relationship between the material and the cultural is thus dialectical, and the movements of this dialectic as they play out on the landscape are deeply political (Mitchell, 2003).In urban settings, scholars note the often wide disparity between the landscape conceived by planning and that of lived reality (Mustafa, 2005;Scott, 1998).Expert ontologies of particular landscapes are a key site of contestation; the power to create the categories by which landscape is defined and regulated underwrites the making and remaking of the material landscape (Robbins, 2001).
In the following section I discuss recent examples of VGI in urban settings, highlighting municipal govern-ments' tendency to engage residents as sensors for simple data and issues in spatial data quality that arise with VGI.Next, I describe the design of the Billboard Map website, how volunteers used the site, and the results of a field audit of the first 326 entries on the site.I analyze these results using traditional spatial data quality factors, and I find that the data was spatially accurate but that over-completeness of the dataset was an issue as users entered more types of signs than the city inventory enumerates.I then turn to the landscape of Los Angeles to consider why residents might have mapped more signs than city agents, and I find that the landscape is suffused with off-site signs that the city permits under a variety of new categories beyond the categories of traditional billboards.I argue that city agents see the landscape as a composite of individual items, distinctly categorized.Residents, however, see the landscape as a unified, cohesive whole.I argue that the VGI map of signage pursuant to this vision of the landscape shows fidelity to the landscape and to the law, and I suggest that the limited inventory created by the city is a tactic to obscure the true number of offsite signs in the city.

Literature Review: VGI in Municipal Settings and Spatial Data Quality
Together, the Internet, global positioning systems, mobile devices equipped with spatial locators, and apps for capturing and sharing spatial data now let persons untrained in GIS or cartography easily create and share spatial data and maps (Haklay, Singleton, & Parker 2008).The public has responded enthusiastically, and the result is vast amounts of VGI-locationally referenced data created by non-professionals.Much of the data is spawned as the unplanned, individual moments of sharing that are ubiquitous to Facebook and Instagram, but some data is borne of civic and communityminded projects, such as Cyclopath, a website for the biking community of Minneapolis, MN, USA, to share routes and road conditions (Priedhorsky, Jordan, & Terveen, 2007).Viewing urban residents as "citizen sensors" (Goodchild, 2009), local governments are interested in VGI as a fount of community data.Years of neoliberalization have left local bodies with decreased funding for service provision, making VGI an attractive option as residents become potential sources of free labor (Johnson & Sieber, 2013).Ganapati (2011) identifies this type of citizen engagement in three areas: transportation information sharing, service management, and community mapping.This speaks to a broader use of social media by governments for data sharing with citizens.Linders (2012) offers a typology for citizen participation using social media by: "Citizen Sourcing (Citizen to Government)", "Government as a Platform (Government to Citizen)" and "Do it Yourself Government (Citizen to Citizen)".Though the examples and types are not strictly VGI, many rely on spatial data sharing, such as Chicago's Snow Portal for sharing and accessing road conditions.
A key impediment to government use of VGI is mistrust of non-expert data (Johnson & Sieber, 2013).This topic encompasses both the traditional issues of spatial data quality as well as the more ontological questions of whether the quality of VGI should be judged differently than professionally created geographic information.In the last few decades, the judgment of spatial data quality advanced along with the methods of data creation (Devillers et al., 2010).Traditionally, spatial data quality was considered solely by positional accuracy, how closely the placement of a data point on a map matches its actual location on the face of the Earth (Van Oort, 2005).Spatial data quality assessments now judge attribute accuracy, the validity of all information associated with a data point besides its position, such as the name of a river; temporal quality, the data quality over time, with an assessment of the rate of change of the source material and the rate at which the dataset is updated; and completeness, the exhaustiveness of a dataset, considering both whether data is missing and excess data is included (Van Oort, 2005).Spatial data may now be easy to create, but these many factors of quality are not easy to assure, especially in formal institutional settings (Johnson & Sieber, 2013).Metadata is also an issue: the quality of professionally created datasets is well tested and documented, while the quality of VGI is generally not (Mooney, Corcoran, & Winstanley, 2010).Further, VGI often involves the mashing up of varying data types from varying sources, undermining quality and making it more complicated to judge (Hall, Chipeniuk, Feick, Leahy, & Deparday, 2010).Due to these reasons for mistrust, the reliability of VGI is a major concern (Delavar & Devillers, 2010).
The largest and most comprehensive dataset of VGI is OpenStreetMap (OSM); correspondingly, it is also the most studied (Koukoletsos, Haklay, & Ellul, 2012).The spatial data quality factors of positional accuracy, attribute accuracy, and completeness OSM data of England (Haklay, 2010), France (Girres & Touya, 2010), and Germany (Zielstra & Zipf, 2010) have been analyzed.In each case, researchers found positional accuracy was very good, attribute data was incomplete, and the completeness of the data varied widely, with nearly complete datasets in urban areas but broad swaths of unmapped areas outside cities.Girres and Touya (2010) and Haklay (2010) suggest that OSM should issue more stringent specifications in place of its current informal rules for data collection and tagging, which they note are often more suggestions for data collection and tagging than rules.However, they caution that OSM should still allow for contributor freedom, in order to maintain its volunteer base.Similarly, Van Exel, Dias and Fruijtier (2010) argue that even for a seeming-ly traditional type of dataset, such as OSM, traditional spatial data quality indicators may need to be retooled.For example, semantic accuracy may be hard to judge: predefined schema for attribute data is uncommon in crowdsourced datasets to allow volunteers' "creative input" but has a negative effect on spatial data quality.These scholars argue that the use of the dataset be considered before judging the quality of VGI.For instance, creative and personal data should not be judged by the same rigorous accuracy standards as a traditional spatial dataset such as OSM.
Johnson and Sieber ( 2013) also find that local governments use VGI as a participation platform to dialogue with residents rather than simply gain or share information.In this vein, the field of VGI aligns with public participation GIS (PPGIS) and its concern with democratizing the tool of GIS.PPGIS is a broad field, incorporating a wide variety of peoples, contexts, and methods to achieve the goal of community empowerment (Elwood, 2008;Sieber, 2006)."At its heart, the overlap between PPGIS and VGI relies on the investigation by individuals of locations that are important to them" (Tulloch, 2008, p. 164).The fields diverge, he argues, in that, "VGI is more about applications and information while PPGIS seems more concerned processes and outcome" (Tulloch, 2008, p. 170).The critical work of PPGIS is directly relevant to the analysis of VGI and in many ways is the necessary precursor and backdrop to its analysis (Elwood, 2008).Key in PPGIS is to "conceptualize data as socially produced and embedded" (Elwood, 2008, p. 177) and acknowledge the "difficulty of integrating spatial data that originate from different epistemologies, as 'local knowledge' and 'official knowledge' often do" (Elwood, 2008, p. 180).Still, implementations of VGI systems by planning and other government agencies to engage with the public in what might be deemed participation rather than simply information sharing are "sparse" (Rinner, Kumari, & Mavedati, 2011), if not "few and far between" (Ganapati, 2011).In the vast majority of VGI literature, urban residents are understood as "sensors", whose unique experience of the urban landscape is only recognized for making them "expert sensors" of the landscape than as potential partners in planning deliberation (see, e.g., Karimipour & Azari, 2015).Governments' failure to use VGI for community participation reflects the failure of official planners and decision-makers to sustain community participation generally.Brown (2012) argues that improvement in PPGIS technologies and techniques have not resulted in meaningful participation, because government agencies do not accept it.In an evaluation of ten years of PPGIS projects, Brown "has yet to observe any tangible evidence that PPGIS data has been used in agency decision making, let alone influence and improve the substantive quality of decisions in planning outcomes" (Brown, 2012, p. 14).

The Billboard Map Website: Description
Weeks after the billboard inventory ordinance was passed in 2002, the largest outdoor advertising companies in the area brought actions in state and federal court to halt the program.The cases settled in early 2007, but the city did not restart the program.When asked by the media why the program was stalled, LADBS personnel stated that litigation prevented the program from being restarted; however, the City Attorney's office admitted that no current litigation was preventing the program (Pelisek, 2008).The Billboard Map website was envisioned to fill this data vacuum.The goal was to create a map that might match the inventory of off-site signs that that the city had planned but, at that time, had neither completed nor released due to political and legal pressure.The design imperative was to collect data that the city could use in its regulatory effort.
Google is a common basis for VGI projects due to the ubiquity of Google Maps and the availability the Google Maps API.Similar to this project's goal, Johnson, Belblidia and Campbell (2011) create a publicly accessible urban dataset using Google mapping tools.They employ Google Earth's satellite imagery to map vacant lots in Detroit.This project employed the Google Maps API for the site's base map and the Google Street View (GSV) interface rather than satellite imagery to locate signs.Billboards have a relatively small footprint compared to their sign faces, and they are difficult to identify from above, making satellite imagery ineffective for locating signs.GSV is a feature of Google Maps that shows street-level photographic imagery of streetscapes within the context of a map (Anguelov et al., 2010).The GSV interface provides full pan, tilt, and zoom capabilities from a user's perspective.A user can rotate the current view to turn the view a complete 360 degrees, zoom the camera in and out at a particular location in front of the camera, and increase or decrease the pitch of the view to move the field of view up or down and towards or away from the horizon.The user can proceed down a street by clicking to the next available camera position or by clicking on a location in the distance.Google updates its GSV dataset at specific locations approximately every eighteen months (Badland, Opit, Witten, Kearns, & Mavoa, 2010).
Since its inception, scientists have tested GSV for usability and relied on its growing dataset as a basis of research.On point for this project are studies that use GSV as source for streetscape audits.Badland et al. (2010) find GSV audits to be faster and less expensive than physical site visits and that efficiency improves rapidly with user experience.Curtis, Duval-Diop and Novak (2010) use GSV to audit New Orleans to identify neighborhood patterns of return and rebuilding after Hurricane Katrina.Although video was available from local community groups, the authors find that GSV is just as effective and chose to rely on GSV as their data source so that their methods could easily be replicated.The design of this project was inspired their work.The design of the Billboard Map pushed beyond the existing literature on GSV streetscape audits by relying on non-experts.
As with other community-minded VGI websites, the desire for broad-based participation was countered by the desire for accurate data.I followed the lead of the Cyclopath designers (Priedhorsky et al., 2007) by favoring open access over site control that might enhance spatial data quality.The site thus operated as a Wiki: users were responsible for creating the data and maintaining the quality of the entries through edits and deletions of errors.Steps to promote more accurate data collection, such as in-person training, online training, and mandatory online instructions will invariably discourage some potential users from participating.I opted to make instructions available on the site but not to require them for participation.Requiring users to register with the site prior to usage was also seen as a way to increase data quality, on the assumption that if one cares enough to register with the site then one will tend to be more careful in entering data than an anonymous visitor.Differing from Cyclopath here, I opted for open access and chose to allow users to add map points without registering.However, registration was required to edit and delete existing billboard entries.
The home page featured a map frame that opened on the extent of all current sign entries, above a table listing the entries (Figure 1).The user could scroll and zoom with the standard Google Maps controls.To the left of the map was a bar with instructions on using the system, which a user could click to hide for a larger map view.
To begin the process of recording signs, the user clicked on a location in the main mapping interface.This action launched a pop-up window with three main features: (1) a window with the GSV viewshed directed northwards from the point selected by the user; (2) a map window centered on the point; and (3) attribute information fields including the approximate address of the point, estimated using employed using the latitude/longitude supplied by the Google Maps API and a reverse geocoding process described by Goldberg and Cockburn (2010) (Figure 2).In the viewshed window, a red rectangular box overlain on the image was used to identify the location of a sign in 3D space.Users panned and zoomed the GSV image until the red rectangle surrounded the sign of interest.When the user saved the entry, the program computed the 3D spatial location of the billboard, and the map updated in realtime to show the new entry.The pop-up window also provided fields for users to record the sign attributes that were collected by city inspectors during their fieldwork.These attributes, identified by the head of the city's inspection program, included: (1) number of sign faces (many signs are double-faced, with sign faces on the front and back); (2) lighted or unlighted; (3) digital or non-digital; and (4) type of sign: pole, wall, roof (L.Zamperini, personal communication, April 13, 2010) (Figure 3).Finally, the window provided a "Notes" space for users to provide free-form comments on the sign.
I reached potential users by notifying online media venues of the project, and a variety of these venues publicized the site.The Coalition to Ban Billboard Blight (CBBB) described the project in a blog post.Their subscriber list is relatively small compared to the other venues, but the audience is directly interested in the topic.Curbed LA, a website covering local real estate development, and the website of the Los Angeles Times, the main regional newspaper, both covered the project.According to Google Analytics, the majority of visitors to the Billboard Map who entered at least one sign on the map linked to the page from either CBBB or Curbed LA.This finding was not surprising as the project relied solely on user interest in the topic to generate engagement: unlike VGI studies that offer gifts (Brown & Kyttä, 2016), the Billboard Map offered users no monetary or material reward.From February through April, 2011, 31 users registered with the site, and many used the site without registering.In this time period, 326 entries were added to the map.326 entries provided a sizeable enough collection to test the usability of the initial incarnation of the site for its intended purpose-supplementing the official inventory of signs.In the following section, I describe the spatial data quality of these entries.

The Billboard Map Website: Results and Analysis of Spatial Data Quality
The Billboard Map, like most existing VGI systems currently employed in municipal governance (Ganapati, 2011), envisioned the residents of Los Angeles as sensors rather than as partners or participants in planning or policy deliberations.The site was intended to collect data useful for regulation, specifically to help the LADBS catalog signs.City agents charged with the dayto-day tasks of regulation, for example with maintaining permits and enforcing municipal codes, require accurate data for their work.To consider the utility of the Billboard Map for this purpose, testing the traditional spatial data quality parameters of the VGI data against the expectations of the city inspectors is key.
The spatial data quality of the 326 volunteered entries in the Billboard Map was based on groundtruthing of the data rather than a comparison against a reference data set as used in other tests of VGI accuracy (see, e.g.Haklay, 2010).Because the city had not yet released its inventory in 2011, and no other public dataset of existing signs existed, there was no reference data set against which to compare the VGI entries.A field test of the 326 entries was therefore required, and it was completed using a Trimble GeoXH GPS receiver to record location and attribute data.After the city's inventory was released in late 2012, I was able to confirm my findings against the city's dataset (Los Angeles Department of Building and Safety, 2014).
The first result sought was the positional accuracy of the web entries.Using ArcMap, I calculated the distance between the location of web entries with the location of corresponding field entries using the "XY to Line" tool, chosen because it yields the desired distance calculation, as well as a visual confirmation of the process (Figure 4).Prior to running the process, I corrected the location of the field points using 4" pixel resolution natural color orthophotography from the 2012 Los Angeles Regional Imagery Acquisition Consortium dataset.This test showed that 43% of web entries were within 20 feet of the intended sign, 75% were located within 50 feet and 91% were located with 100 feet.The city's inventory was not used to confirm these findings because, though it provided coordinate information for each sign, the coordinates are to a point randomly sited within the parcel containing each sign, not to the sign's exact location within the parcel.Hence, my field location points were more accurate.
Given that urban planning is focused on individual parcels, the second result sought was whether web entries were sited within the correct parcels.ArcMap is capable of determining if a point is within the boundary of an areal feature or an adjacent areal feature, but parcels are often separated by streets and sidewalks.To assure the findings were accurate, a manual analysis was necessary.For this, I used Los Angeles County's parcel dataset, visually comparing the parcel that contained a web entry and the parcel that contained the corresponding field point (Figure 5).Even with 91% of web entries within 100 ft. of the correct location, this test revealed that only 50% of web points were sited within the correct parcel, 88% were located in the correct parcel or within one parcel of the correct one, and 98% were located in the correct parcel or within two parcels of the correct one.The disparity between positional accuracy and correct parcel placement is explained by the urban setting of Los Angeles, as the commercial corridors which host outdoor advertising are often lined with narrow parcels.This level of accuracy is likely not good enough to be considered viable for LADBS's purposes in regulating signage, as signage is permitted based on parcel.The Billboard Map relied on users to manually enter attribute information including the number of sign faces, and whether a sign was lighted or digital.In the majority of cases, these fields were left empty.The finding is in line with existing studies of VGI.OSM's positional quality far exceeds the quality of other attributes (Girres & Touya, 2010;Haklay, 2010;Zielstra & Zipf, 2010).
The temporal quality of the website's data was in large part dependent upon the temporal quality of the GSV data.Although the website allowed a user to enter a sign whether or not it was actually shown in GSV, no users during the test period did so.Although GSV data is updated, on average, every 18 months, a review of GSV in Los Angeles shows that the data has been updated every three to six months for the last three years.The inventory of billboards created by the City of Los Angeles, on the other hand, is conducted every two years.
The spatial data quality assessment was run on 326 signs, and, given that the estimated number of off-site signs in Los Angeles at the time was 10,000, it was clear that the Billboard Map data was incomplete.However, the entry rate of the data suggests that completeness would be an on-going issue.According to the city's 2014 inventory, there are 8,814 off-site sign faces within the municipal boundaries.In the first three months of operation, the Billboard Map contained 431 sign faces (105 of the 326 signs were double-faced signs).At a rate of 431 in three months, it would take years for the public to map the nearly 9,000 sign faces in the city.
As in OSM datasets, the Billboard Map's completeness varies over space.Whereas OSM coverage drops from urban to rural areas, the Billboard Map dataset varies across the urban setting.Most users entered only one sign, but a few users entered many signs along one stretch of roadway.Accordingly, most parts of the city were unmapped while a few corridors have near complete coverage.For instance, one registered user entered 56 sign faces along a 2.2mile stretch of Melrose Boulevard, making the coverage in this area much more complete than in other parts of the city.
The spatial data quality factor that separates the Billboard Map most noticeably from other VGI studies is over-completeness.Unlike OSM users, the users of the Billboard Map entered excess data points.First, they mapped signs that were outside the municipal boundary of Los Angeles.The city of Los Angeles comprises a large, awkwardly shaped area, with numerous smaller municipalities within its bounds or adjacent to it.Some of the cities, such as West Hollywood, which contains the famed Sunset Strip, and City of Industry, have much more lax regulation of signage than the city of Los Angeles.With the proximity of the cities and the size of signage, outdoor advertising that sits in, and is therefore regulated by, one city can easily be seen from other cities.
The aspect of over-completeness that is most striking is the type of signs mapped by volunteers.Billboard Map users mapped signs that were not of the type identified in the city inventory.City inspectors recorded only the traditional style of off-site signs, the pole, wall, and roof signs known in the vernacular as "billboards" (Figure 3).The VGI dataset of the Billboard Map includes many types of signs beyond these traditional billboards.These include massive signs integrated within new architecture (Figure 6a), supergraphics wrapped around older structures (Figure 6b), signs posted on fences (Figure 6c), and wall signs of larger dimensions that older wall billboards (Figure 6d).Judged against the city's inventory, these signs are excess data that undermine the spatial data quality of the dataset.

The Ontology of Off-Site Signage and the Ontology of the Landscape
Though LADBS limited its inventory to traditional pole, wall, and roof billboards, the legislation creating the program does not so limit its scope.The inventory ordinance states, "All off-site sign structures as defined in Section 14.4.2 of the LAMC and subject to the provisions of Chapter I of the LAMC are subject to regular inspection" (Off-Site Sign Periodic Inspection Program, 2014).Chapter I of the LAMC regulates development on private property but not government-owned property, hence the inventory is only of signs on private property.Section 14.4.4defines an off-site sign structure as, "A structure of any kind or character, erected, used or maintained for an off-site sign or signs, upon which any poster, bill, printing, painting, projected image or other advertisement may be placed" (Sign Regulations, 2008).Notably, this language does not limit an off-site sign to traditional pole, wall, and roof signs.According to the language of the statute, therefore, the inventory should include any structure used for an off-site sign.
On closer inspection, the "excess" Billboard Map data appear to fit within this broad definition.These signs show ads for banking services draped down the sides of an office tower (Figure 6a) and wrapped around the top of a touristic gift shop in the heart of Hollywood (Figure 6b); they show ads for phone service on the fence around a car repair shop (Figure 6c) and liquor on the exterior of a butcher shop (Figure 6d).Therefore, these signs are "off-site signs" according to the city's definition and thus within the mandate of the city's inventory.
The discrepancy between the city residents' and inspectors' compilations of off-site signs appears to be based on differing ontologies of the two projects.The users of the Billboard Map website had a different conception as to the scope of signs to be mapped than that of the expert field inspectors working for the city.Somewhat ironically, inspectors for the city's "Off-Site Sign Periodic Inspection Inventory" only mapped a limited collection of off-site signs.For the Billboard Map, city residents mapped all kinds of off-site signs.
In fact, many types of off-site signs exist throughout the Los Angeles landscape in addition to traditional billboards.As noted, the city banned new off-site signs in 2002.Also in 2002, the city signed a contract with global outdoor advertising company JCDecaux for street furniture adorned with off-site signage (JCDecaux, 2002).Soon after, hundreds and then thousands of new off-site signs appeared across the city pursuant to this program.Because the inventory ordinance limits its scope to private property, these signs are not technically within its scope as they sit on public sidewalks.In addition to the ban on new off-site signs and the inventory program, the 2002 sign laws enacted a new zoning mechanism entitled Sign Districts, and the city soon enacted the first such district, the Hollywood Signage Supplemental Use District (2004).In the following years, the city permitted more than fifty offsite signs, most of which were supergraphic signs, spanning whole building walls.Billboard Map users mapped many of these signs, including those shown in Figure 6a and 6b.In 2007, the city allowed off-site signs on walls placed around construction sites and undeveloped lots, under the deceptively entitled Graffiti Abatement Program (2007).These walls are intended to be temporary but without enforcement by the city and because they give a financial incentive to property owners to keep their parcels undeveloped, these often become permanent fixtures of the cityscape.Figure 6c shows one such signed mapped by a resident.Not long after the temporary construction wall sign ordinance was passed, the owner of the company that lobbied for the signs began erecting off-site signs under the cover of a law that permits temporary promotional signs.These signs feature small text along the sign frame stating "Come into (name of on-site business) and enter our sweepstakes for a chance to win these or related prizes".The sign faces, however, always display offsite advertisements, and are never images related to an on-site sweepstakes or other promotion.A number of these were also identified in the Billboard Map, such as that shown in Figure 6d.In 2011, the city entered into a contract with Martin Outdoor Media for bus benches adorned with off-site signs, and thousands of these signs now fill the city.The image in Figure 6d shows two such benches in the foreground.Like the JCDecaux street furniture signs, these sit on public property and are thus beyond the scope of the inventory mandate.
This list reveals the vast difference between the scope of the mapping project as understood by the Billboard Map users and the scope the city inspectors were tasked with.The discrepancy is due not just to different ontologies between the two mapping projects but to different ontologies as to the landscape itself.The Billboard Map data reveals that city residents view the urban landscape very differently than do city agents: urban residents view the landscape as a cohesive whole, while city agents view a landscape of categories.Landscape scholars argue that landscape is a "way of seeing" (Cosgrove, 1984), as much cultural as material.Landscapes are therefore open to interpretation as well as contestation.The disparity between the lived experience of landscape and the expert, planned conceptions of landscape (Mitchell, 2003;Mustafa, 2005) might explain why urban residents view the landscape differently than the city agents who regulate it.LADBS inspectors are tasked with enforcing specific code provisions, granting permits for individual projects, counting and cataloguing each of the thousand of off-site signs.They labor in the minutiae of the municipal code, and, in their working lives, the landscape is a categorical one.
The broader issue, though, is with the setting of the categories themselves."Where competing accounts of what constitutes the categories of landscape exist, the fixing of those categories is an inherently political exercise" (Robbins, 2001, p. 162).The power inherent in the setting of landscape categories derives from the dialectical relationship between the cultural and material.Those who set the categories can remake the landscape accordingly.This dialectic, and the power to remake the landscape according to a changing ontology, is evidenced in the Los Angeles landscape.In 2002, Los Angeles banned new off-site signs following years of community protestation against the landscape of advertising (Pelisek, 2008).In the years following the ban, the city approved new categories of signage including street furniture, temporary construction wall, and bus bench signs, and then outdoor advertisers added thousands of new signs to the urban landscape pursuant to these categories.The city created the Hollywood Sign District in 2004, and it thereby permitted fifty massive new off-site signs to adorn development projects that are visible for miles outside of the sign district itself.By remaking the categories of landscape, the city decisionmakers have remade the landscape itself.
In 2010, the Hollywood Sign District was effectively cancelled due to public backlash against the changes to the landscape.Yet the city's creation of new categories of signs continues.In 2011, the city approved a new sign district in the heart of its downtown that will allow a massive new development project adorned with offsite signage.At a public hearing before the City Council's Planning and Land Use Committee (2011), then-City Council member Jan Perry stated in support of the project, "What is being proposed by the developers is not a billboard", thereby distancing the proposed signs from the cultural baggage associated with the traditional categories of off-site signs.However, the Billboard Map shows that the residents of Los Angeles understand the landscape as a cohesive field of advertising, not as the collection of regulatory categories under which these signs are permitted.Whether the City Council deems these new signs "billboards" or not, residents see them as more off-site signs.

Conclusions. VGI for Improving Expert Data and Community Knowledge
The Billboard Map website was envisioned as a method of collecting data for the city's off-site inventory in the years that the program was politically stalled.The head of the inventory program was skeptical at the outset about the utility of the data for the city's purposes (L.Zamperini, personal communication, June 22, 2011), a common governmental response to VGI (Johnson & Sieber, 2013).On first look, Mr. Zamperini's skepticism is borne out by the results of the spatial data quality analysis, as the non-expert mappers did not understand the scope of the city's inventory project and mapped far more types of signs, yielding a dataset with excess data.To make the Billboard Map's data match the scope of the city's inventory, a future iteration of the project could employ a filter on sign type to limit the types of signs mapped by users or to require volunteers to complete a tutorial prior to mapping.Scholars note that stricter rules for collection of VGI can im-prove the quality of spatial data (Girres & Touya, 2010;Haklay, 2010), yet scholars also suggest caution in applying rules so as not to stifle creative and unconventional map-making (Van Exel et al., 2010).The Billboard Map VGI shows that the creativity of non-professionals reveals itself in unlikely ways.The spatial data sought here-off-site signs-is straightforward.Signs are large, material structures, not ephemeral happenings.The project was not designed to collect opinions or ideas about signs (Rinner et al., 2011); it was not designed to map emotions about signs (Kwan, 2007); it did not ask residents to envision future spaces with or without signs (Seeger, 2008).Yet from the simple mapping task undertaken by so-called sensors, we learn an unexpected insight about how residents experience the urban landscape.This finding supports the argument for caution in applying filters or other rules for data collection in VGI projects.
Further, by inadvertently ignoring the city's categorical distinctions, the users of the Billboard Map have pointed out that the so-called "Off-Site Sign Periodic Inspection Program" is a vastly incomplete record of off-site signs in the city.It may be that the discrepancy between the city's actual landscape of off-site signs and the city's inventory of off-site signs is an inadvertent error.Just as we can imagine technical rules for improving the quality of the Billboard Map dataset, we can imagine instructing LADBS inspectors to correct the scope of the project to match the language of the ordinance.Yet the scope of the inventory appears quite intentionally limited; in fact, a number of off-site signs beyond pole, wall, and roof signs were listed in the 2012 inventory but removed in 2014, including the massive off-site signs on the "Hollywood and Highland" development that is now home to the Academy Awards show and an upscale shopping mall.The outdoor advertising industry has great influence in Los Angeles City Hall, and legislators repeatedly push for growth of signs and lack of transparency at the behest of the industry (Pelisek, 2008;Smith, 2012).The creation of new sign categories to avoid the ban on off-site signs has been a tactic of urban and industry decisionmakers.The inventory itself obscures the fact that many of the pole, wall, and roof signs in the city are unpermitted and illegal (Sedano, 2016).The limited scope of the inventory to only a handful of the many types of off-site signs that now adorn cityspace appears to be another tactic in this overall strategy.
The political backdrop of outdoor advertising inspires a different viewpoint on the question of whether to include spatial data quality rules to improve the volunteers' mapping of signs.Asking residents to map signs according to the city's limited ontology is to engage them in the Sisyphean task of helping a city appear to regulate without actually regulating.Much more useful is for residents to continue to map signs according to their experience of cityspace.A map of the actual extent of off-site signage in the city could be incredibly useful in countering the conjoined efforts of capital and state to grow the advertising landscape, offering a rebuttal to the city's categorically limited yet politically acceptable inventory.
As outdoor advertising grows in Los Angeles cityspace, it spreads in cities around the world.Indeed, companies such as JCDecaux, which have remade the Los Angeles landscape, are remaking the urban landscape globally by coordination with local agencies (Iveson, 2012).Los Angeles residents have alerted us to the fullness of signage that is obscured by categorization.The global nature of the industry requires future study to discover whether these tactics are employed throughout the world to spread signage, perhaps uncovering the growth of advertising obscured by the local nature of sign regulation.Engaging urbanites to map the full extent of signage is a counter-tactic available when officials lack the political will to regulate the advertising landscape.

Figure 1 .
Figure 1.The home page of the Billboard Map website, featuring a Google Maps base map and points of signs entered by users.

Figure 2 .
Figure 2. The Billboard Map's pop-up window for data collection, featuring a Google Street View window, a map window, and fields for attribute data.

Figure 3 .
Figure 3. Examples of types of signs and sign attributes collected by the city inventory and the Billboard Map, including (a) a double-sided pole sign; (b) a lighted roof sign; and (c) two digital pole signs.

Figure 4 .
Figure 4. Sample image of ArcMap with line measurements between web and field data points.

Figure 5 .
Figure 5. Sample image from ArcMap with web point separated from corresponding field point by one parcel.

Figure 6 .
Figure 6.Sample Google Screen View images of Billboard Map entries, showing "excess" data points of signage not collected by city inspectors.