--- title: "How to retrieve data from the Meetnetten database" author: "Toon Westra, Frederic Piesschaert" date: 2024-06-25 output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{How to retrieve data from the Meetnetten database} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include = FALSE} tryCatch( con <- inbodb::connect_inbo_dbase("S0008_00_Meetnetten"), error = function(e) e, finally = database_access <- exists("con") ) knitr::opts_chunk$set( collapse = TRUE, comment = "#>", eval = database_access, cache = FALSE ) if (!database_access) { load("get_data_meetnetten_results.Rda") } ``` # Introduction The Flemish species monitoring programme [Meetnetten.be](https://www.meetnetten.be) consists of a set of monitoring schemes for priority species in Flanders, the northern part of Belgium. The monitoring programme is a collaboration between the Nature and Forest Research Institute (INBO), the Nature and Forest Agency (ANB), the NGO Natuurpunt and many volunteers. Each monitoring scheme consists of a number of monitoring sites at which target species are counted using a standardized protocol. The count data is stored in the Meetnetten database which is maintained by INBO. We present some functions to easily access the data from the Meetnetten database. # Aim We make functions available to query data directly from the Meetnetten database. This avoids writing your own queries. We have provided functions to query + monitoring schemes included in the species monitoring programme + locations and sublocations (for example different sections of a transect) of the monitoring schemes + visits (count events) + observations # Packages and connection The main functions that we will use in this tutorial all start with `get_meetnetten_*`. These functions are made available by loading the `inbodb` package. ```{r setup, message = FALSE, eval = TRUE} library(inbodb) library(dplyr) library(sf) library(ggplot2) ``` These functions will only work for people with access to the INBO network. As an INBO employee, you should make sure you have read rights for the INBO Data Warehouse server, otherwise place an ICT-call. The following R-code can be used to establish a connection to the Meetnetten database: ```{r connection_inbo} con <- connect_inbo_dbase("S0008_00_Meetnetten") ``` # Functionality ## Information on monitoring schemes The function `get_meetnetten_schemes()` gives an overview per species group of all monitoring schemes included in Meetnetten and the protocols that are applied in the monitoring schemes. In most cases, a monitoring scheme is dedicated to one target species and the monitoring scheme is named after the target species. In some cases, a monitoring scheme has several target species, such as the `Algemene Broedvogelmonitoring Vlaanderen`. ```{r get_scheme} scheme_info <- get_meetnetten_schemes(con) %>% as_tibble() ``` ```{r, eval=TRUE} scheme_info ``` ## Locations and sublocation of monitoring schemes Each monitoring scheme consists of a fixed set of locations where the target species is/are counted on a regular basis. Some locations are subdivided in sublocations. This is, for example, the case for most butterfly monitoring schemes, where the sublocations represent the 50 meter sections of a transect. The function `get_meetnetten_locations()` returns the locations and the sublocations for one or several monitoring scheme. It returns a list of two `sf` objects: `main_locations` and `sublocations`. In the following example we select the locations and sublocations of the Argusvlinder (*Lasiommata megera*) monitoring scheme. ```{r example_locations} locations <- get_meetnetten_locations(con, scheme_name = "Argusvlinder") main_locations <- locations$main_locations sublocations <- locations$sublocations ``` Let's have a look at the `main_locations`. ```{r, eval = TRUE} main_locations %>% head(5) ``` Two variables need some further explanation: + `is_sample`. For more common species, a random sample is drawn from all locations in Flanders where the target species occurs. For location that are selected `is_sample` = `TRUE`. Locations that are not selected, indicated by `is_sample` = `FALSE`, can be counted optionally. + `is_active`. When `is_active` = `FALSE`, the location is no longer counted. This is the case when the location appears to be inaccessible or when the target species does not occur at the location any more. However, most often these inactive locations where counted in previous years. So the the database also contains observations for (currently) inactive locations. Next, let's have a look at the sublocations of the location Arenbergpolder. The locations consists of 14 sections. Note that we only show the active (`is_active` = `TURE`) sublocations. ```{r, eval = TRUE} sublocations_show <- sublocations %>% filter(location == "Arenbergpolder") %>% filter(is_active) sublocations_show ``` Below we show a basic map of the locations and sublocations. The location is the wider area where the target species occurs. The sublocations indicate where the target species is counted. ```{r, fig.width = 7, eval = TRUE} main_location_show <- main_locations %>% filter(location == "Arenbergpolder") ggplot(sublocations_show) + geom_sf(data = main_location_show) + geom_sf(aes(colour = sublocation)) ``` We can also select and plot all locations from the dragonfly (in Dutch *libellen*) monitoring schemes. ```{r} # get locations for a specific species_group locations_dragonflies <- get_meetnetten_locations(con, species_group = "libellen") ``` ```{r, fig.width = 7, eval = TRUE} locations_dragonflies$main_locations %>% ggplot() + geom_sf(aes(fill = scheme, colour = scheme), alpha = 0.5) ``` ## Visits The function `get_meetnetten_visits()` returns all visits (count events) for one or more monitoring schemes. Below we select all visits for the location Arenbergpolder of the Argusvlinder monitoring scheme. We see that in most years the location is counted 6 times per year as demanded by the [monitoring protocol](https://purews.inbo.be/ws/portalfiles/portal/17657883/MaesEtal_2019_MonitoringsprotocolDagvlindersVersie2.pdf). ```{r example_visits} visits <- get_meetnetten_visits(con, scheme_name = "Argusvlinder", collect = TRUE) ``` ```{r, eval = TRUE} visits %>% filter(location == "Arenbergpolder") %>% select(location, visit_id, start_date, start_time, visit_status) ``` Use `?get_meetnetten_visits` to get more information on the variables that are returned by the function. ## Observations The function `get_meetnetten_observations()` provides all observations for one or more monitoring scheme. Let's select all observations from one visit at the Arenbergpolder location of the Argusvlinder monitoring scheme, by applying a filter based on the variable `visit_id`. ```{r example_observations} observations <- get_meetnetten_observations(con, scheme_name = "Argusvlinder", collect = FALSE) %>% filter(visit_id == 1080) %>% collect() ``` First we take a look at the observation of the target species. ```{r, eval = TRUE} observations %>% filter(target_species) %>% select(location, sublocation, start_date, scientific_name, count) ``` Optionally observers can also record non target species that can be counted using the same protocol. The code below provides the total number of individuals per species, including the non target species, that were counted in one visit at the Arenbergpolder location. ```{r, eval = TRUE} observations %>% group_by(start_date, target_species, name_nl, scientific_name) %>% summarise(count_total = sum(count)) %>% ungroup() ``` Use `?get_meetnetten_observations` to get more information on the variables that are returned by the function. An important note: before you start analysing the data from a certain monitoring scheme, it is important to know how the monitoring schemes were designed and how the data is organised. So don't forget to check the monitoring protocol (available on the [INBO website](https://www.vlaanderen.be/inbo/zoeken/?q=monitoringsprotocol)). ## Closing the connection Close the connection when done ```{r closing_conn} dbDisconnect(con) rm(con) ``` ```{r, echo=FALSE, include=FALSE} rm(database_access) save.image(file = "get_data_meetnetten_results.Rda") ``` # Data policy Since many of the target species are sensitive species, we do not publicly make available the exact location of the observations. Location information is generalized to 1, 5 or 10 km Universal Transverse Mercator (UTM) grid cells, when publishing the data on [GBIF](https://www.gbif.org). The generalisation rules are the same as in [waarnemingen.be](https://waarnemingen.be/). Please respect this policy when you publish results based on the Meetnetten data. Also please contact [info@meetnetten.be](info@meetnetten.be) before publishing results. This way we can check if the publication complies with the data policy as agreed by the different project parters of [Meetnetten](https://www.meetnetten.be).