--- title: "Checking Availability with rdfp" author: "Steven M. Mortimer" date: "2018-03-29" output: rmarkdown::html_vignette: toc: true toc_depth: 4 keep_md: true vignette: > %\VignetteIndexEntry{Checking Availability with rdfp} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, echo = FALSE} NOT_CRAN <- identical(tolower(Sys.getenv("NOT_CRAN")), "true") knitr::opts_chunk$set( collapse = TRUE, comment = "#>", purl = NOT_CRAN, eval = NOT_CRAN ) ``` First, we load **rdfp** and specify the DFP network we would like to connect to. Then we authenticate by using `dfp_auth()`. Any existing cached token would be used or we will be prompted to authenticate via the browser. ```{r load-package, eval=FALSE} suppressWarnings(suppressMessages(library(dplyr))) suppressWarnings(suppressMessages(library(lubridate))) library(rdfp) options(rdfp.network_code = 123456789) dfp_auth() ``` ```{r auth, include = FALSE} suppressWarnings(suppressMessages(library(dplyr))) suppressWarnings(suppressMessages(library(lubridate))) library(here) library(rdfp) token_path <- here::here("tests", "testthat", "rdfp_token.rds") suppressMessages(dfp_auth(token = token_path, verbose = FALSE)) options_path <- here::here("tests", "testthat", "rdfp_options.rds") rdfp_options <- readRDS(options_path) options(rdfp.network_code = rdfp_options$network_code) ``` ### Availability by Month Ad traffickers or display advertising reporters might get requests to determine the availability of a line item by month for a multi-month contract proposal. A LineItem has one InventoryTargeting object that describes which AdUnit and Placement objects it can target, and optional additional Targeting subclass objects that represent geographical, custom, or other criteria. You can either: 1. Create your own line item from scratch 2. Use `dfp_getLineItemsByStatement()` to pull down details on a line item and modify any fields that need to be different for your line item. #### A LineItem from Scratch Creating your own line item is easy. Some line item fields are absolutely required in order to forecast so you may need to try different lines to see how they forecast. Personally, I've found that most line items need the fields we create in the example below (e.g `primaryGoal`, `targeting`, etc). **One quirk with the DFP API is that the fields on your line item must be provided in the same order as the reference documentation.** You can review the documentation for this object at https://developers.google.com/ad-manager/api/reference/v201905/LineItemService.LineItem. ```{r line-item-from-scratch} sample_line <- list() sample_line$startDateTime <- '' sample_line$endDateTime <- '' sample_line$deliveryRateType <- 'EVENLY' sample_line$lineItemType <- 'STANDARD' sample_line$priority <- 8 sample_line$costType <- 'CPM' sample_line$creativePlaceholders$size <- list(width=666, height=176, isAspectRatio='false') sample_line$creativePlaceholders$expectedCreativeCount <- 1 sample_line$creativePlaceholders$creativeSizeType <- 'PIXEL' sample_line$primaryGoal <- list(goalType='LIFETIME', unitType='IMPRESSIONS', units=1000) sample_line$targeting <- list(inventoryTargeting=list(targetedAdUnits=list(adUnitId=133765936, includeDescendants="true"))) ``` You'll notice that this example line item does not have targeting, which you can add. It also does not have a `startDateTime` and `endDateTime`. We'll write a loop and pass those in each time to get availability for each month across multiple months. #### Modifying an Existing LineItem If you want to use an existing line item because it's already got so many details, then here is an example of how you would pull that item down and use it. You will need to change out the `filterStatement` to pick the line item you want. ```{r modifying-existing-line-item} my_filter <- "WHERE LineItemType='STANDARD' and Status='DELIVERING' LIMIT 10" line_item_details <- dfp_getLineItemsByStatement(list(filterStatement=list(query=my_filter))) # pull out the 1st line item in the list of returned results # we'll use this as a template for creating the hypothetical line items single_item <- line_item_details[[1]] ``` #### Creating Datetimes in DFP DFP keeps track of dates and times by separating things like the year, month, and day into 3 separate integers. The special DFP format requires a function to convert `Date` objects in R into the corresponding DFP list. The following function is helpful in doing just that. ```{r date-converter-func, warning=FALSE} # supply a datetime dfp_date_to_list(Sys.time()) # supply a date and assume the beginning of the day for hours, mins, secs dfp_date_to_list(Sys.Date()+1, "beginning") ``` #### Generating Forecasts Here is an example of how to loop through individual months and determine availability 6 months out for the sample line item created previously. It is fairly straightforward to take the month start and end dates and substitute them into the hypothetical line item. The one quirk is that, in order to follow the API, we must create lists of lists that are redundant at times. For example, the forecast request has a `lineItem` field that takes a `ProspectiveLineItem` that contains its own `lineItem` field: `list(lineItem=list(lineItem=this_sample_line) ...`. As long as you follow the object fields according to the Google reference documentation, then everything should work. Most everything needs to be formed as a list of lists with field names. ```{r generating-forecasts} all_forecasts <- NULL month_start_dates <- as.Date(format(Sys.Date() %m+% months(1:6), '%Y-%m-01')) month_end_dates <- ceiling_date(month_start_dates, 'months') - 1 for(i in 1:length(month_start_dates)){ this_sample_line <- sample_line this_sample_line$startDateTime <- dfp_date_to_list(month_start_dates[i], daytime='beginning') this_sample_line$endDateTime <- dfp_date_to_list(month_end_dates[i], daytime='end') forecast_request <- list(lineItem=list(lineItem=this_sample_line), forecastOptions=list(includeTargetingCriteriaBreakdown='false', includeContendingLineItems='false')) this_result <- dfp_getAvailabilityForecast(forecast_request) this_result <- this_result[,c('unitType', 'availableUnits', 'reservedUnits')] this_result$forecast_month <- format(month_start_dates[i], '%Y-%m') all_forecasts <- rbind(all_forecasts, this_result) } all_forecasts ``` ### Availability by Targeting Criteria The previous example shows how to determine availability across multiple months for a single line item. It is also possible to determine the availability for each targeting segment of your forecast. The `dfp_getAvailabilityForecast()` function takes a second or two, so it is very inefficient to submit each county individually. Instead, you can submit a line item with targeting that says County A or County B or County C ... until you've covered each county that you need a forecast for. If you ask for targeting criteria breakdowns in the `forecastOptions`, then you can retrieve the availability contribution from each county while only using one call to `dfp_getAvailabilityForecast()`. #### Determining Geo Ids In this example, we'll determine availability for all counties in Texas. First, you will need to determine the geography codes for all of the counties in Texas. Luckily, DFP provides a table of geographic codes for the entire world. Codes can be retrieved like so: ```{r getting-geo-codes} # get codes for US states and counties request_data <- list(selectStatement= list(query="SELECT Id , Name , CanonicalParentId , CountryCode , Type FROM Geo_Target WHERE CountryCode='US' AND (Type='State' OR Type='County')")) us_geos <- dfp_select(request_data) texas_id <- us_geos %>% filter(type == 'STATE', name == 'Texas') %>% select(id, state = name) us_counties <- us_geos %>% filter(type == 'COUNTY') %>% select(id, canonicalparentid, county = name) texas_counties <- inner_join(us_counties, texas_id, by=c('canonicalparentid'='id')) ``` Next, we need to format the geographies into objects of type `Location` that at least contain an id denoting the area. ```{r format-geos} # format the county ids into a list so they can be passed # over to the geoTargeting field of the ProspectiveLineItem geo_targets <- as.list(texas_counties$id) geo_targets <- lapply(geo_targets, FUN=function(x){list(id=x)}) names(geo_targets) <- rep('targetedLocations', length(geo_targets)) ``` The sample line we created earlier does not contain the start and end datetimes, so we'll consider a 90-day period in the future. ```{r prep-sample-line} this_sample_line <- sample_line # look at availability for the next 90 days this_sample_line$startDateTime <- dfp_date_to_list(Sys.Date() + 1, daytime='beginning') this_sample_line$endDateTime <- dfp_date_to_list(Sys.Date() + 91, daytime='end') ``` #### Setting up The Targeting Specifying the targeting can be a little tricky. We need to NULL the original targeting field we created in the sample line and put geoTargeting first because the documentation and the API needs elements in the order that they are specified. For example, the ForecastService needs the targeting fields to be in the order listed at https://developers.google.com/ad-manager/api/reference/v201905/ForecastService.Targeting. ```{r setup-targeting} # recreate the targeting field from scratch this_sample_line$targeting <- NULL this_sample_line$targeting$geoTargeting <- geo_targets # re-use the inventoryTargeting criteria from the original sample this_sample_line$targeting$inventoryTargeting <- sample_line$targeting$inventoryTargeting ``` #### Requesting Targeting Criteria Breakdowns Finally, we'll make the request to determine availability. There are 2 important details in getting back the breakdown of availability for each county in this example. 1. Ensure that `includeTargetingCriteriaBreakdown = 'true'` 2. Ensure that `as_df = FALSE` in `dfp_getAvailabilityForecast()` Targeting criteria breakdowns are omitted by default so we need to tell DFP to return them. Those breakdowns are formatted as a nested list, so if you do not specify, you'll be stuck with a very wide data.frame containing one column for every element in the nested list and it is not easy to work with. ```{r make-county-request} # request the targeting criteria breakdown this time to get that detail forecast_request <- list(lineItem = list(lineItem = this_sample_line), forecastOptions = list(includeTargetingCriteriaBreakdown = 'true', includeContendingLineItems = 'false')) # get the forecasted availability and make sure to specify as_df=FALSE this_result <- dfp_getAvailabilityForecast(forecast_request, as_df=FALSE) breakdowns <- this_result[c(names(this_result) %in% 'targetingCriteriaBreakdowns')] ``` #### Parsing Targeting Criteria Breakdowns Once you've got the result, you'll see an element in the list called `targetingCriteriaBreakdowns`. This element contains a breakdown for each targeting field specified, including the inventory targeting that is for the ad unit. We are not interested in that breakdown and it is not mutually exclusive from the geoTargeting, so we need to exclude. There are many ways to determine the break downs you want. In the example below we use `sapply` to find all breakdowns that have a dimension of `'GEOGRAPHY'`, but we could also check that the targetingCriteriaId is in the list of county ids. This all depends on which breakdowns you're interested in and how you want to pull them out of the forecasted response. ```{r availability-by-county} # only select the breakdowns that are GEOGRAPHY # this omits the Ad Unit and Ad Size breakdowns geo_breakdowns <- breakdowns[sapply(breakdowns, FUN=function(x){ x$targetingDimension == 'GEOGRAPHY' })] avails <- plyr::ldply(geo_breakdowns, .fun=function(x){ return(as.data.frame(x)) }, .id=NULL) avails <- avails %>% mutate(ImpressionsTotal=as.integer(matchedUnits), ImpressionsAvailable=as.integer(availableUnits), ImpressionsBooked=as.integer(ImpressionsTotal-ImpressionsAvailable), PctAvail=ImpressionsAvailable/ImpressionsTotal) %>% select(targetingCriteriaName, ImpressionsTotal, ImpressionsBooked, ImpressionsAvailable, PctAvail) ``` The breakdowns are not necessarily mutually exclusive, so be careful when totaling them up. The counties are exclusive, so it's okay to sum them. ```{r total-texas-county-availability} # total availability across Texas counties sum(avails$ImpressionsAvailable) / sum(avails$ImpressionsTotal) ``` ### Availability for an Existing Line Sometimes you just want to check the availability on an existing line, say, to see if you can renew an existing contract. DFP makes this very easy, by providing the function `dfp_getAvailabilityForecastById()`. You'll need to first determine the Id of the line item you would like to check, then plug it into the function. Note the empty `list()` provided to the `forecastOptions` argument. The API will error out if you do not provide it, but an empty list will quell those error messages and utilize the default forecast options. ```{r availability-by-id} # request for this id with no special forecastOptions forecast_request <- list(lineItemId=single_item$id, forecastOptions = list()) this_result <- dfp_getAvailabilityForecastById(forecast_request) this_result[,c('lineItemId', 'orderId', 'availableUnits', 'deliveredUnits', 'matchedUnits')] ``` ### Check out the Tests The **rdfp** package has quite a bit of unit test coverage to track any changes made between newly released versions of DFP (typically 4 each year). These tests are an excellent source of examples because they cover most all cases of utilizing the package functions. For example, if you're not sure on how to use custom date ranges when requesting a report through the ReportService, just check out the tests at https://github.com/StevenMMortimer/rdfp/blob/master/tests/testthat/test-ReportService.R If you want to know how to create a user, just look at the test for `dfp_createUsers()` ```{r create-users-test, eval=FALSE} request_data <- list(users=list(name="TestUser - 1", email="testuser123456789@gmail.com", roleId=-1) ) dfp_createUsers_result <- dfp_createUsers(request_data) ```