--- title: "r311: An R interface to the open311 standard" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{r311: An R interface to the open311 standard} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>" ) httptest2::start_vignette("../man/figures/r311") library(r311) ``` # Introduction open311 is an international open-access standard for civic issue management and public service communication. The standard allows administrations to better manage citizen requests, citizens to more easily participate in administrative work, and researchers and data scientists to access data regarding public service communication. As an open standard, open311 is not a centralized API, but a framework implemented by various cities (e.g. San Francisco, CA, Chicago, IL, Cologne, DE, Turku, FI, Zurich, CH) and services (e.g. SeeClickFix, FixMyStreet). It is way past the golden age of open311 APIs and much of development in civic issue tracking has shifted to less open-access and less standardized alternatives. Many former prime examples have abandoned or severely limited their open311 endpoints. Nonetheless, open311 still constitutes a valuable source for open government data. Many cities and services still maintain an open311 service. `r311` allows the seamless management and selection of endpoints and retrieval of service and request data. It supports (but does not depend on) many popular R frameworks such as the tidyverse, `sf` and `xml2` for response formatting. `r311` is designed to be slim, both in content and dependencies. It imports only two import-less packages used for HTTP response handling. The functionality is limited to two main features: - Endpoint management - Sending requests This vignette will briefly cover these two features. # Endpoints Since open311 is an open and decentralized standard, there is not one but many endpoints that one can access. An endpoint is commonly implemented by a city administration, but can also be managed by a service provider such as FixMyStreet. Each endpoint can define multiple jurisdiction IDs, although, in practice, most endpoints only define a single jurisdiction. It can thus be difficult to manage the multitude of endpoints and jurisdictions. Efforts have been made to list open311 servers, but most of them are incomplete or outdated. `r311` offers an updated and modifiable endpoint list that defines a number of open311 implementations that are defined for use in the package. The list can be read using `o311_endpoints`. ```{r} o311_endpoints() ``` The list does neither claim comprehensiveness nor up-to-dateness. It arguably provides some of the most important and easily accessible endpoints as of `r format(Sys.Date(), "%Y")`. However, `r311` also offers the ability to add new endpoints to `o311_endpoints` using `o311_add_endpoint`. You need to provide a name (for lookup) and a root URL (the URL used to send requests). The following code adds the open311 test server of Rostock, Germany. ```{r} o311_add_endpoint( name = "Rostock Test", root = "https://demo.klarschiff-hro.de/backoffice/citysdk/", jurisdiction = "rostock.de" ) ``` Retrieving the endpoints list again confirms that you successfully added a new row to the endpoints dataframe. ```{r} nrow(o311_endpoints()) ``` You can now select the Rostock test API to the session using `o311_api`. This function matches an API using endpoint name and jurisdiction ID and attaches it to the active session. Query functions automatically detect the attached API. ```{r} o311_api("Rostock Test") ``` After attaching an API, `o311_ok` confirms that the selected endpoint is able to handle request queries. ```{r} o311_ok() ``` As the result is `TRUE`, you can safely start receiving real request data. # Making requests After selecting an API and attaching it to the session, all \code{o311_*} functions can access it. You can now make requests. ## Services To get an overview of the available services in a jurisdiction, you can use `o311_services`, which returns a list of Rostock's administrative services. ```{r} o311_services() ``` ## Requests To get data about civic issues in the city area, run `o311_requests`. ```{r} o311_requests() ``` Using the output of `o311_services`, you can further narrow down the output of requests. Open311 defines a set of standard parameters which are implemented by all endpoints. Using the `service_code` parameter with one of the previously returned service codes, only complaints about businesses illegally using shared spaces are returned. ```{r} o311_requests(service_code = "16") ``` Similarly, using a `service_request_id` from the output, you can retrieve a single service request from the API. ```{r} o311_request("4") ``` ## Bulk queries Many endpoints define a page limit meaning that responses are divided into pages. A query without parameters returns the first page. Pagination can be controlled with the `page` argument. To control pagination, the `o311_request_all` function can come in handy. It automatically iterates through pages and heuristically decides when to stop. The following example retrieves data from the first two pages, resulting in a tibble with 100 service requests. ```{r} o311_api("Rostock, DE") o311_request_all(max_pages = 2) ``` ## Non-standard parameters `r311` implicitly supports API extensions introducing custom paths and parameters. One such API is Klarschiff Rostock which is based on CitySDK. Klarschiff CitySDK defines a number of non-default paths and parameters which extend the filtering abilities of open311 requests. Available parameters can usually be found in the respective documentation (e.g. on GitHub for [Klarschiff CitySDK](https://github.com/bfpi/klarschiff-citysdk)). The following query returns the last 50 requests tagged with the "idea" keyword. ```{r} tickets <- o311_requests(keyword = "idea", max_requests = 50) ``` Some custom parameters can also alter the shape of the output. In the following example, we query just the count of total requests using the `just_count` parameter. The result is a 1×1 tibble containing a count value. ```{r} o311_requests(just_count = TRUE) ``` The CitySDK extensions also offers additional URL paths which can be queried using the generic `o311_query` function. ```{r} poly <- o311_query("areas") plot(poly$grenze) plot(tickets$geometry, add = TRUE, pch = 16) ``` # Cleanup Endpoint data is stored persistently between sessions so that you can create your own database of open311 endpoints. This database is stored in the system's user directory as returned by `tools::R_user_dir("r311")`. To reset the database, run ```{r} o311_reset_endpoints() ``` This will default back to the endpoints defined by `r311` and remove all endpoints manually added by `o311_add_endpoints`.