How to {magick}ally Visualize Historical Google Maps Traffic Data

Visualise historical google maps using {magick}

Motivation

Many African countries have implemented a full lockdown strategy to curb the spread of CoVID-19. I thought it was interesting to see how compliance would vary across cities without relying solely on news media. Unfortunately, GoogleMaps does not provide historical traffic data beyond traffic information on a Typical (specified) day and time. I figured out I could take an independent screenshot of the Google maps at regular intervals and animate the same in the future to evaluate compliance and compare across countries.

Outline

• Requirements

• Setting up work directory

• Google Maps Traffic Data Retrieval

• Task Scheduling

• Animate Retrieved Traffic Data

Requirements

• Have a Google account. If you don’t have one. You could click here to create an account.

• Setup and obtain a google API Key - The Google Developers website have a short tutorial on how to set up and obtain a Google API key. Note that all new users are required to sign up and create a billing account. Once you are done creating a google API key, copy the key and assign it to an object map_key in r

Code

map_key <- 'YOUR-APIKEY-HERE_R'

• Install and load the following R packages[libraries] :

  • googleway- for plotting Google Map and overlaying it with shapes and markers such as traffic data

  • htmlwidgets - for creating standalone googlemap web pages

  • magick - for animating images

  • tidyverse - a collection of packages for data wrangling and string manipulation

  • webshot - for taking screenshot of saved standalone googlemap web pages

Code

## Hint

install.packages("package.name")
library(package.name)

• Internet connection

• A basic knowledge of data analysis using R

Setting up work directory

You will need to set a directory on your local machine where all files will be saved using the setwd() function. Inside the working directory, you also need to create a new folder every day where each hourly photographic imagery will be saved. This can be achieved using an if-else statement that checks if a folder with the day’s date dat_traffic already exists in the working directory and creates one if otherwise. Note that the date on your local machine needs to be correct.

Code

setwd("YOUR/DESIRED/WORKING/DIRECTORY/HERE/")

dat_traffic <- Sys.Date()

    if (file.exists(paste0(getwd(), "/", `dat_traffic`))){
      print("Directory already exists")
      
    } else {
    
      dir.create(paste0(getwd(), "/", `dat_traffic`))
      print("Directory has now been Created")
      
    }

Also, create an object that will hold information on the date-time that maps are downloaded. This object will be needed when saving the imagery as it will be attached to file names when saving on your local machine. The information can be extracted from the current date and time on your local machine using the Sys.time() function. In addition, you can use the str_sub() function in {string} part of {tidyverse} to extract all characters starting from the 6th character. Since Windows PC does not accept special characters like “:” in file names, you could use the str_replace() function to replace all instances of “:” with “-” and ” ” with “_”.

Code

dat_time <- Sys.time() %>% 
            ## Format on my local machine is: "2020-04-01 20:21:30"
            str_sub(start = 6) %>% 
            str_replace(pattern = ":",
                        replacement = "-") %>% 
  
            str_replace(pattern = ":",
                        replacement = "-") %>% 
            
            str_replace(pattern = " ",
                        replacement = "_")

Google Maps Traffic Data Retrieval - Johannesburg

To retrieve GoogleMaps in R, you can use the google_map function from {googleway}. The function takes the following arguments:

• key: your Google API key obtained in stage one above map_key

• location: the location (using longitude and latitudes) that you want to visualize. You can obtain the longitude and latitude from GoogleMaps website and adjust until the desired areas has been covered.

• width & height: your desired width and height of the map. You could resize to fit in as many areas as desired.

• zoom: the zoom level of the map. Higher levels signify zoom in level.

Once the map has been succesfully retrieved, you could overlay with “live” traffic information using the add_traffic() function. You could also temporarily save the traffic object as a “.html” file using saveWidget().

Finally, a screenshot of the temporary webpage can be taken using the webshot() function taking the arguments vwidth and vweight both of which should correspond to the initial weight and heigh values from the google_map(). The delay argument is set to 4 to allow the webpage to be fully loaded before a screenshot is taken.

The file should be save inside the day’s folder with the name of the city/street/municipality being downloaded and the date and time. Example: “2020-03-26/JohannesburgCBD 03-26_19-55-04.png”.

Code

Johannesburg_traffic <- google_map(key = map_key, 
                        location = c(-26.166747,
                                     28.041337),
                        width = 1280,
                        height = 980,
                        zoom = 13) %>%
                        add_traffic()
        
                        saveWidget(Johannesburg_traffic,
                                   "temp.html",
                                   selfcontained = FALSE)
                                   
                        webshot("temp.html", 
                                vwidth = 1280,
                                vheight = 980,
                                file = paste0(`dat_traffic`, "/",
                                              "Johannesburg ", dat_time,
                                              ".png"),
                                cliprect = "viewport",
                                delay = 4)

Everything above until here should be saved as a standalone “Script.R” that can be scheduled to run at intervals.

Schedule Subsequent Download of Google Maps Traffic Data

Please see this StackOverflow website for guidance on how to schedule R scripts in Windows. The information on the webpage can be summarized as follows:

• Open the scheduler: START -> Search for –> Task Scheduler

• Create a new Task

• Under tab General

  • Input the name of the task “GoogleMaps Traffic”

  • Input a description “Automate GoogleMaps Traffic”

  • Run only when user is logged on and ensure that your PC is ON at all schedule times or select otherwise.

• Under the Triggers tab

  • Create a new trigger

  • Begin the task: “On a schedule”

  • Settings: “Daily”

  • –> Start date: The date you want to start the schedule.

  • –> Start time: The start time for downloads: “07:00”.

  • Repeat task every: “1 hour” for a duration of “12 hours” - to capture only traffic data every hour for 12 hours in a day.

• Under the Actions tab

  • Create a new action

  • –> Choose Start Program

  • Under program/script, browse to Rscript.exe which could be placed here:

    “C:Files.2.exe”

  • Under Add arguments (optional), input the path where the script is to be found.

    E.g.: -e “source(‘C:/LOCATION_OF_YOUR_R_SCRIPT/Script.R’)”

    Take not of the -e and the /

Click OK when you are done and ensure that your PC is always connected to the internet.

Animate Downloaded GoogleMaps Traffic Data

After the Traffic data has been downloaded, e.g. for a day, a week or a month, you could animate the images using the {magick} package. To achieve this, you’ll need to create an object with the folder names in which all images are saved. Remember that previously, you had created an object dat_traffic with the date of each first run of the code on a day and created a folder using the retained value. Assuming that you are creating an animation for five days, you could list all the dates as is below. You will then create a function to list all the files within each of the folders. You have to set a working directory setwd() for the getwd() function to look in the right directory.

Note that you will need to create a new “Animation-Script.R” with all the codes below.

Code

dates <- c("2020-03-30",
           "2020-03-29",
           "2020-03-28",
           "2020-03-27",
           "2020-03-26")

setwd("YOUR/DESIRED/WORKING/DIRECTORY/HERE/")

files <- NULL
for (i in dates) {
  
  sub_files <- list.files(paste0(getwd(), "/", i))
  sub_files <- paste0(i, "/", sub_files)
  files <- c(files, sub_files)
}

You could print (files) to see what the object looks like.

Next, you will use image_read() to read each of the imageries into R. Using a for-loop statement, each successfully read image will be assigned to object png.dat and annotated using the features embedded in each file name. You could define three annotations for the image

• Title: City Name {Day-Month}

  You could annotate each one of the imageries with a Title that contains information about the name of the city, as well as the date to distinguish among city imageries (if there are more than one cities) and the dates for each image (for imageries collected over days). The str_extract function provides an opportunity to extract characters from an object. FOr the city name, you will need to extract the name of the city using the pattern in which the file names were saved. Since the City names contain only numeric variables, you could specify the “[[:alpha:]]+” pattern to capture all alphabetic characters in the file names. You could also use the same function to extract the dates. The pattern “[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}” was specified for the function to look, first for any four digits numeric character and then a “-”, followed by a two-digit numeric character, followed by “-” and then another two-digit numeric character. This would match a structure like “2020-03-26”. You could then format the date extracted with as.Date() using a Day-Month format. Since you are unlikely to download imageries for more than a year. You should consider a different format if you are looking to download imageries for more than a year. Other arguments in image_annotate() are boxcolour for the fill colour for the annotation box, size for font size of the annotation text, colour for the annotation text colour and location for the position of the annotation.

• Caption: Animation: @YourName

  You could include your name if you would like to be contacted to provide additional information about the map. gradity is also for the positiion of the annotation.

• Time: Time: HH:MM:SS

  You could also annotate each one of the imageries with the time to be able to distinguish each imagery in the animation successfully. str_extract will be very useful for this annotation since the time of download is also embedded in the file name for each imagery. As is the case with date in the title annotation, you could specify the pattern *“_[[:digit:]]{2}-[[:digit:]]+-[[:digit:]]+”* to start from where there is *“_”* and then a two-digit numeric character, followed by “-” and then another two-digit numeric character, followed by”-” and then another two-digit numeric character. This would match a structure like *“_09-44-02”*. The str_replace() function could be used to replace all instances of “-” with “:” and “_” with ““. Other arguments are the same as the title annotation.

  assign() assigns the name in str_replace(f, pattern = ".png", replacement = "") to the updated “png.dat” object.

  Code

  for(f in files){
      png.dat <- image_read(f)
      png.dat <- image_annotate(png.dat, paste0(str_extract(f, pattern = "[[:alpha:]]+"), " {",
                                format(as.Date(str_extract(f, pattern = "[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}")),
                                                "%d-%b"), "}"),
                                boxcolor = "white", size = 50,
                                color = "Black", location = "+0+0")
      png.dat <- image_annotate(png.dat, "Animation: @YourName",
                                boxcolor = "#000000", size = 30,
                                color = "#DBBB3D", gravity = "southwest")
      png.dat <- image_annotate(png.dat, paste0("Time: ", str_extract(f, pattern = "_[[:digit:]]{2}-[[:digit:]]{2}-[[:digit:]]{2}") %>%
                                                          str_replace("_", "") %>% str_replace("-", ":") %>% str_replace("-", ":")),
                                boxcolor = "white", size = 30,
                                color = "Black", gravity = "northeast")
      assign_name <- str_replace(f, pattern = ".png",
                                 replacement = "")
      assign(assign_name, png.dat)
    }

Assuming you are making animations for more than one city, you could write a for-loop statement to get all objects in the workspace for one city, merge them, animate using the image_animate() function and save to local disk as a .gif file using image_write().

Code

city_names <- c("Johannesburg")


for (i in city_names) {
  
  a.c_city <- mget(ls(pattern = paste0(i, " [0-9]+-[0-9]+_[0-9]+-[0-9]+-[0-9]+")))
  
  a <- NULL
  a <- a.c_city[[1]]
  
  for (j in 2:(length(a.c_city))) {
    a <- c(a, a.c_city[[j]])
  }
  
  A.img <- image_scale(a)
  A.ani <- image_animate(A.img, fps = 1, dispose = "previous")
  image_write(A.ani, paste0(getwd(), "/Animations/", i, ".gif"))
}

Was the tutorial helpful? You could like, and/or share with the people in your network.

If you notice any error in the codes or omission in the tutorial, please reach me via email: emmanuel[{at}]olamijuwon[{dot}]com

For additional guidance on other cool things that can be done with {magick}, please see the package website here.