Using the iNaturalist API to detect novel records

This time, I’m interested in detecting if any of the records uploaded to iNaturalist in Uruguay during the past year, 2023, belong to new species for NaturalistaUY (our national site) or for the iNaturalist global platform. To answer this, I once again explored the iNaturalist API. If you want to know more about my first experiment with the API, see my previous post How many users in NaturalistaUY are Uruguayan?.

So, I need to know if any record is:

• new for iNaturalist, i.e., a species recorded in Uruguay in 2023 that has no previous records in the platform.
• new for NaturalistaUY: i.e., a species recorded in Uruguay in 2023 that has been recorded before in the platform, but has no previous records in Uruguay.

The API has a call for taxa which takes a list of species or ids (taxon_id) and returns information about them, such as taxonomic name and, lucky for us, the number of observations the taxa has (observations_count). We will use this to get the numbers for ‘species’ (as taxon rank) we had in Uruguay during 2023.

To do it, we first create the function getiNatTaxonRank which takes a taxon_id as an argument, and returns the taxon_name, taxon_rank (e.g., species, family, subspecies), and observations_count for the taxon.

library(httr)
library(jsonlite)
library(knitr)
library(tidyverse)

getiNatTaxonRank <- function(taxon_id){

  taxaRanks <- tibble(taxon_name = character(),
                      taxon_id = numeric(),
                      taxon_rank = character(),
                      observations_count = numeric())

  num_results = 0 # used to put the API to sleep and print on the console the num

  for (taxon_id_i in taxon_id) {  

    if ((num_results %% 10) + 10 == 10) {
      Sys.sleep(10) # every 10 calls, the code stops for 10
    }

    call_url <- str_glue('https://api.inaturalist.org/v1/taxa/',
                         '{taxon_id_i}')

    get_json_call <- GET(url = call_url) %>%
      content(as = "text") %>% fromJSON(flatten = TRUE)

    results <- as_tibble(get_json_call$results)

    taxaRanks_i <- tibble(taxon_name = results$name,
                          taxon_id = taxon_id_i,
                          taxon_rank = results$rank,
                          observations_count = results$observations_count)

    taxaRanks <- rbind(taxaRanks, taxaRanks_i)
    num_results <- num_results + 1
    cat(num_results, '\n')
  }
  return(taxaRanks)
}

I will download all the records on 2023 from here naturalista.uy/observations/export, using the following URL query: quality_grade=any&identifications=any&place_id=7259&verifiable=true&d1=2023-01-01&d2=2023-12-31. And then read the file.

observations_2023 <-read_csv('datos/observations-2023-UY.csv')

To use the function we will get a unique list of IDs (taxa_list).

taxa_list <- observations_2023 %>%
  filter(!is.na(taxon_id)) %>%
  distinct(taxon_id) %>% pull(taxon_id)

We then run the function.

taxonRank_obs_may_2023 <- getiNatTaxonRank(taxa_list)

Now, let’s reply our questions.

1. new for iNaturalist, i.e., a species recorded in Uruguay in 2023 that has no previous records in the platform.

To answer this we will assess the field observations_count. If the taxon has only 1 record, then we have a new species!

taxonRank_observations_2023 %>%
  filter(observations_count==1) %>%
  kable()

There are 12 species!¹ We could also try if we have any second records.

taxonRank_observations_2023 %>%
  filter(observations_count==2) %>%
  kable()

There are 7 species that were recorded for the second time in the platform on 2023.

2. new for NaturalistaUY: i.e., a species recorded in Uruguay in 2023 that has been recorded before in the platform, but has no previous records in Uruguay.

To find out if we have any new records in Uruguay, we cannot use the observations_count because this value is relative to the entire platform. Thus, we will need to get all the records uploaded in Uruguay (yes, all the recods!). This is a huge database (more than 90.000 records) but can be downloaded from here naturalista.uy/observations/export, using the following URL query: quality_grade=any&identifications=any&place_id=7259&verifiable=true

obs_NatUY_before2023 <-read_csv('datos/observations-UY-before2023.csv')

Now, we will check all the taxon_id (to the species level taxon_rank) to see how many records they have in Uruguay before 2023.

speciesList_2023 <- taxonRank_observations_2023 %>%
  filter(taxon_rank=='species')

speciesList_2023 %>%
  filter(!taxon_id %in% obs_NatUY_before2023$taxon_id) %>%
  filter(observations_count>1) %>%
  arrange(observations_count) %>% head(n=25) %>% kable()

In total we have 484 species. Here’s a sample of the 25 species with 2 or more observations on iNaturalist (see observations_count).

And, that’s all !

Hope you find this useful too ✨