17 Server-side linking with shiny

Section 16.1 covers an approach to linking views client-side with graphical database queries, but not every linked data view can be reasonably framed as a database query. If you need more control, you have at least two more options: add custom JavaScript (covered in section 18) and/or link views server-side via a web application. Some concepts useful for the former approach are covered in 18, but this chapter is all about the latter approach.

There are several different frameworks for creating web applications via R, but we’ll focus our attention on linking plotly graphs with shiny – an R package for creating reactive web applications entirely in R. Shiny’s reactive programming model allows R programmers to build upon their existing R knowledge and create data-driven web applications without any prior web programming experience. Shiny itself is largely agnostic to the engine used to render data views (that is, you can incorporate any sort of R output), but shiny itself also adds some special support for interacting with static R graphics and images (Chang 2017).

When linking graphics in a web application, there are tradeoffs to consider when using static R plots over web-based graphics. As it turns out, those tradeoffs complement nicely with the relative strengths and weaknesses of linking views with plotly, making their combination a powerful toolkit for linking views on the web from R. Shiny itself provides a way to access events with static graphics made with any of the following R packages: graphics, ggplot2, and lattice. These packages are very mature, fully-featured, well-tested, and support a incredibly wide range of graphics, but since they must be regenerated on the server, they are fundamentally limited from an interactive graphics perspective. Comparatively speaking, plotly does not have the same range and history, but it does provide more options and control over interactivity. More specifically, because plotly is inherently web-based, it allows for more control over how the graphics update in response to user input (e.g., change the color of a few points instead of redrawing the entire image). This idea is explored in more depth in section 17.3.1.

This chapter teaches you how to use plotly graphs inside shiny, how to get those graphics communicating with other types of data views, and how to do it all efficiently. Section 17.1 provides an introduction to shiny it’s reactive programming model, Section 17.2 shows how to leverage plotly inputs in shiny to coordinate multiple views, Section 17.3.1 shows how to respond to input changes efficiently, and Section 17.4 demonstrates some advanced applications.

17.1 Embedding plotly in shiny

Before linking views with plotly inside shiny, let’s first talk about how to embed plotly inside a basic shiny app! Through a couple basic examples, you’ll learn the basic components of a shiny and get a feel for shiny’s reactive programming model, as well as pointers to more learning materials.

17.1.1 Your first shiny app

The most common plotly+shiny pattern uses a shiny input to control a plotly output. Figure 17.1 gives a simple example of using shiny’s selectizeInput() function to create a dropdown that controls a plotly graph. This example, as well as every other shiny app, has two main parts:

The user interface, ui, defines how inputs and output widgets are displayed on the page. The fluidPage() function offers a nice and quick way get a grid-based responsive layout²⁸, but it’s also worth noting the UI is completely customizable²⁹, and packages such as shinydashboard make it easy to leverage more sophisticated layout frameworks (Chang and Borges Ribeiro 2018).
The server function, server, defines a mapping from input values to output widgets. More specifically, the shiny server is an R function() between input values on the client and outputs generated on the web server.

Every input widget, including the selectizeInput() in Figure 17.1, is tied to a input value that can be accesssed on the server inside a reactive expression. Shiny’s reactive expressions build a dependency graph between outputs (aka, reactive endpoints) and inputs (aka, reactive sources). The true power of reactive expressions lies in their ability to chain together and cache computations, but let’s first focus on generating outputs. In order to generate an output, you have to choose a suitable function for rendering the result of a reactive expression.

Figure 17.1 uses the renderPlotly() function to render a reactive expression that generates a plotly graph. This expression depends in the input value input$cities (i.e., the input value tied to the input widget with an inputId of "cities") and stores the output as output$p. This instructs shiny to insert the reactive graph into the plotlyOutput(outputId = "p") container defined in the user interface.

Click to show/hide the code

library(shiny)
library(plotly)

ui <- fluidPage(
  selectizeInput(
    inputId = "cities", 
    label = "Select a city", 
    choices = unique(txhousing$city), 
    selected = "Abilene",
    multiple = TRUE
  ),
  plotlyOutput(outputId = "p")
)

server <- function(input, output, ...) {
  output$p <- renderPlotly({
    plot_ly(txhousing, x = ~date, y = ~median) %>%
      filter(city %in% input$cities) %>%
      group_by(city) %>%
      add_lines()
  })
}

shinyApp(ui, server)

FIGURE 17.1: Using a shiny input widget to control which time series are shown on a plotly graph.

If, instead of a plotly graph, a reactive expression generates a static R graphic, simply use renderPlot() (instead of renderPlotly()) to render it and plotOutput() (instead of plotlyOutput()) to position it. Other shiny output widgets also use this naming convention: renderDataTable()/datatableOutput(), renderPrint()/verbatimTextOutput(), renderText()/textOutput(), renderImage()/imageOutput(), etc. Packages that are built on the htmlwidgets standard (e.g. plotly and leaflet) are, in some sense, also shiny output widgets that are encouraged to follow this same naming convention (e.g. renderPlotly()/plotlyOutput() and renderLeaflet()/leafletOutput()).

Shiny also comes pre-packaged with a handful of other useful input widgets. Although many shiny apps use them straight “out-of-the-box”, input widgets can easily be stylized with CSS and/or SASS, and even custom input widgets can be integrated (Mastny 2018, @shiny-custom-inputs).

selectInput()/selectizeInput() for dropdown menus.
numericInput() for a single number.
sliderInput() for a numeric range.
textInput() for a character string.
dateInput() for a single date.
dateRangeInput() for a range of dates.
fileInput() for uploading files.
checkboxInput()/checkboxGroupInput()/radioButtons() for choosing a list of options.

Going forward our focus is to link multiple graphs in shiny through direct manipulation, so we focus less on using these input widgets, and more on using plotly and static R graphics as inputs to other output widgets. Section 17.2 provides an introduction to this idea, but before we learn how to access these input events, you may want to know a bit more about rendering plotly inside shiny.

17.1.2 Hiding and redrawing on resize

The renderPlotly() function renders anything that the plotly_build() function understands, including plot_ly(), ggplotly(), and ggplot2 objects.³⁰ It also renders NULL as an empty HTML div, which is handy for certain cases where it doesn’t make sense to render a graph. Figure 17.2 leverages these features to render an empty div while selectizeInput()’s placeholder is shown, but then render a plotly graph via ggplotly() once cities have been selected. Figure 17.2 also shows how to make the plotly output depend on the size of the container that holds the plotly graph. By default, when a browser is resized, the graph size is changed purely client-side, but this reactive expression will re-execute when the browser window is resized. Due to technical reasons this can improve ggplotly() resizing behavior³¹, but should be used with caution when handling large data and long render times.

Click to show/hide the code

library(shiny)
library(plotly)

cities <- unique(txhousing$city)

ui <- fluidPage(
  selectizeInput(
    inputId = "cities", 
    label = NULL,
    # placeholder prompt is triggered when first choice is an empty string
    choices = c("Please choose a city" = "", cities), 
    multiple = TRUE
  ),
  plotlyOutput(outputId = "p")
)

server <- function(input, output, session, ...) {
  output$p <- renderPlotly({
    req(input$cities)
    if (identical(input$cities, "")) return(NULL)
    p <- ggplot(data = filter(txhousing, city %in% input$cities)) + 
      geom_line(aes(date, median, group = city))
    height <- session$clientData$output_p_height
    width <- session$clientData$output_p_width
    ggplotly(p, height = height, width = width)
  })
}

shinyApp(ui, server)

FIGURE 17.2: Rendering a plotly graph in shiny if and only if the selectizeInput()’s dropdown is non-empty. When the graph is present, and the window is resized, then the reactive expression is re-evaluated.

When a reactive expression inside renderPlotly() is re-executes, it triggers a full redraw of the plotly graph on the client. Generally speaking, this makes your shiny app logic easy to reason about, but it’s not always performant enough. For example, say you have a scatterplot with 10s of thousands of points, and you just want to add a fitted line to those points (in respond to input event)? Instead of redrawing the whole plot from scratch, it can be way more performant to partially update specific components of the visual. Section 17.3.1 covers this idea through a handful of examples.

17.2 Leveraging plotly input events

Section 17.1 covered how to render shiny output widgets (e.g., plotlyOutput()) that depend on a input widget, but what about having an output act like an input to another output? For example, say we’d like to dynamically generate a bar chart (i.e., an output) based on a point clicked on a scatter-plot (i.e., an input event tied to an output widget). In addition to shiny’s static graph and image rendering functions (e.g., plotOutput()/imageOutput()), there are a handful of other R packages that expose user interaction with “output” widget(s) as input value(s). Cheng (2018 c) and Xie (2018) describe the interface for the leaflet and DT packages. This section outlines the interface for plotlyOutput(). This sort of functionality plays a vital role in linking of views through direct manipulation, similar to what we’ve already seen in section 16.1, but having access to plotly events on a shiny server allows for much more flexibility than linking views purely client-side.

The event_data() function is the most straight-forward way to access a plotly input events in shiny. Although event_data() is function, it references and returns a shiny input value, so event_data() needs to be used inside a reactive context. Most of these available events are data-specific traces (e.g., "plotly_hover", "plotly_click", "plotly_selected", etc), but there are also some that are layout-specific (e.g., "plotly_relayout"). Most plotly.js events³² are accessible through this interface – for a complete list see the help(event_data) documentation page.

Numerous Figures in the following sections show how to access common plotly events in shiny and do something with the result. When using these events to inform another view of the data, it’s often necessary to know what portion of data was queried in the event (i.e., the x/y positions alone may not be enough to uniquely identify the information of interest). For this reason, it’s often a good idea to supply a key (and/or customdata) attribute, so that you can map the event data back to the original data. The key attribute is only supported in shiny, but customdata is officially supported by plotly.js, and thus can also be used to attach meta-information to event – see section 18 for more details.

17.2.1 Dragging events

There are currently four different modes for mouse click+drag behavior (i.e., dragmode) in plotly.js: zoom, pan, rectangular selection, and lasso selection. This mode may be changed interactively via the modebar that appears above a plotly graph, but the default mode can also be set from the command-line. The default dragmode in Figure 17.3 is set to 'select', so that dragging draws a rectangular box which highlights markers. When in this mode, or in the lasso selection mode, information about the drag event can be accessed in four different ways: "plotly_selecting", "plotly_selected", "plotly_brushing", and "plotly_brushed". Both the "plotly_selecting" and "plotly_selected" events emit information about trace(s) appearing within the interior of the brush – the only difference is that "plotly_selecting" fires repeatedly during drag events, whereas "plotly_selected" fires after drag events (i.e., after the mouse has been released). The semantics behind "plotly_brushing" and "plotly_brushed" are similar, but these emit the x/y limits of the selection brush. As for the other two dragging modes (zoom and pan), since they modify the range of the x/y axes, information about these events can be accessed through "plotly_relayout". Sections 17.3.1 and 17.4 both have advanced applications of these dragging events.

plotly_example("shiny", "event_data")

FIGURE 17.3: Accessing event data from click and drag events.

17.2.2 3D events

Drag selection events (i.e., "plotly_selecting") are currently only available for 2D charts, but other common events are generally supported for any type of graph, including 3D charts. Figure 17.4 accesses various events in 3D including: "plotly_hover", "plotly_click", "plotly_legendclick", "plotly_legenddoubleclick", and "plotly_relayout". The data emitted via "plotly_hover" and "plotly_click" is structured similarly to data emitted from "plotly_selecting"/"plotly_selected". Figure 17.4 also demonstrates how one can react to particular components of a conflated event like "plotly_relayout". That is, "plotly_relayout" will fire whenever any part of the layout has changed, so if we want to trigger behavior if and only if there are changes to the camera eye, one could first check if the information emitted contains information about the camera eye.

plotly_example("shiny", "event_data_3D")

FIGURE 17.4: Accessing 3D events.

17.2.3 Edit events

A little known fact about plotly is that you can directly manipulate annotations, title, shapes (e.g., circle, lines, rectangles), legends, and more by simply adding config(p, editable = TRUE) to a plot p. Moreover, since these are all layout components, we can access and respond to these ‘edit events’ by listening to the "plotly_relayout" events. Figure 17.5 demonstrates how display access information about changes in annotation positioning and content.

Click to show/hide the code

library(shiny)

ui <- fluidPage(
  plotlyOutput("p"),
  verbatimTextOutput("info")
)

server <- function(input, output, session) {
  
  output$p <- renderPlotly({
    plot_ly() %>%
      layout(
        annotations = list(
          list(
            text = emo::ji("fire"),
            x = 0.5, 
            y = 0.5, 
            xref = "paper",
            yref = "paper",
            showarrow = FALSE
          ),
          list(
            text = "fire",
            x = 0.5, 
            y = 0.5, 
            xref = "paper",
            yref = "paper"
          )
        )) %>%
      config(editable = TRUE)
  })
  
  output$info <- renderPrint({
    event_data("plotly_relayout")
  })
  
}

shinyApp(ui, server)

FIGURE 17.5: Accessing information about direct manipulation of annotations.

Figure 17.6 demonstrates directly manipulating a circle shape and accessing the new positions of the circle. In constrast to Figure 17.5, which made everything (e.g. the plot and axis titles) editable via config(p, editable = TRUE), note how Figure 17.6 makes use of the edits argument to make only the shapes editable.

Click to show/hide the code

library(shiny)
library(plotly)

ui <- fluidPage(
  plotlyOutput("p"),
  verbatimTextOutput("event")
)

server <- function(input, output, session) {
  
  output$p <- renderPlotly({
    plot_ly() %>%
      layout(
        xaxis = list(range = c(-10, 10)),
        yaxis = list(range = c(-10, 10)),
        shapes = list(
          type = "circle", 
          fillcolor = "gray",
          line = list(color = "gray"),
          x0 = -10, x1 = 10,
          y0 = -10, y1 = 10,
          xsizemode = "pixel", 
          ysizemode = "pixel",
          xanchor = 0, yanchor = 0
        )
      ) %>%
      config(edits = list(shapePosition = TRUE))
  })
  
  output$event <- renderPrint({
    event_data("plotly_relayout")
  })
  
}

shinyApp(ui, server)

FIGURE 17.6: Accessing information about direct manipulation of circle shapes.

Figure 17.7 demonstrates a linear model that reacts to edited circle shape positions using the "plotly_relayout" event in shiny. This interactive tool is an effective way to visualize the impact of high leverage points on a linear model fit. The main idea is to have the model fit (as well as it’s summary and predicted values) depend on the current state of x and y values, which here is stored and updated via reactiveValues(). Section 17.2.8 has more examples of using reactive values to maintain state within a shiny application.

plotly_example("shiny", "drag_marker")

FIGURE 17.7: Editing circle shape positions to dynamically alter a linear model summary and fitted line. This is useful mainly as a teaching device to visually demonstrate the effect of high leverage points on a simple linear model.

Figure 17.8 uses an editable vertical line and the plotly_relayout event data to ‘snap’ the line to the closest point in a sequence of x values. It also places a marker on the intersection between the vertical line shape and the line chart of y values. Notice how, by accessing event_data() in this way (i.e., the source and target view of the event is the same), the chart is actually fully redrawn every time the line shape moves. If performance were an issue (i.e., we were dealing with lots of lines), this type of interaction likely won’t be very responsive. In that case, you can use event_data() to trigger side-effects (i.e., partially modify the plot) which is covered in 17.3.1.

plotly_example("shiny", "drag_lines")

FIGURE 17.8: Dragging a vertical line shape and ‘snapping’ the line to match the closest provided x value.

17.2.4 Relayout vs restyle events

Remember every graph has two critical components: data (i.e., traces) and layout. Similar to how "plotly_relayout" reports partial modifications to the layout, the "plotly_restyle" event reports partial modification to traces. Compared to "plotly_relayout", there aren’t very many native direct manipulation events that would trigger a "plotly_restyle" event. For example, zoom/pan events, camera changes, editing annotations/shapes/etc all trigger a "plotly_relayout" event, but not many traces allow you to directly manipulate their properties. One notable exception is the "parcoords" trace type which has native support for brushing lines along an axis dimension(s). As Figure 17.9 demonstrates, these brush events emit a "plotly_restyle" event with the range(s) of the highlighted dimension.

Click to show/hide the code

library(plotly)
library(shiny)

ui <- fluidPage(
  plotlyOutput("parcoords"),
  verbatimTextOutput("info")
)

server <- function(input, output, session) {
  
  d <- dplyr::select_if(iris, is.numeric)
  
  output$parcoords <- renderPlotly({
    
    dims <- Map(function(x, y) {
      list(
        values = x, 
        range = range(x, na.rm = TRUE), 
        label = y
      )
    }, d, names(d), USE.NAMES = FALSE)
    
    plot_ly() %>%
      add_trace(
        type = "parcoords",
        dimensions = dims
      ) %>%
      event_register("plotly_restyle")
  })
  
  output$info <- renderPrint({
    d <- event_data("plotly_restyle")
    if (is.null(d)) "Brush along a dimension" else d
  })
  
}

shinyApp(ui, server)

FIGURE 17.9: Using the "plotly_restyle" event to access brushed dimensions of a parallel coordinates plot.

As Figure 17.10 shows, it’s possible to use this information to infer which data points are highlighted. The logic to do so is fairly sophisticated, and requires accumulation of the event data, as discussed in section 17.2.8.

plotly_example("shiny", "event_data_parcoords")

FIGURE 17.10: Displaying the highlighted observations of a parcoords trace.

17.2.5 Scoping events

This section leverages the interface for accessing plotly input events introduced in section 17.2 to inform other data views about those events. When managing multiple views that communicate with one another, you’ll need to be aware of which views are a source of interaction and which are a target (a view can be both, at once!). The event_data() function provides a source argument to help refine which view(s) serve as the source of an event. The source argument takes a string ID, and when that ID matches the source of a plot_ly()/ggplotly() graph, then the event_data() is “scoped” to that view. To get a better idea of how this works, consider Figure 17.11

Figure 17.11 allows one to click on a cell of correlation heatmap to generate a scatterplot of the two corresponding variables – allowing for a closer look at their relationship. In the case of a heatmap, the event data tied to a plotly_click event contains the relevant x and y categories (e.g., the names of the data variables of interest) and the z value (e.g., the pearson correlation between those variables). In order to obtain click data from the heatmap, and only the heatmap, it’s important that the source argument of the event_data() function matches the source argument of plot_ly(). Otherwise, if the source argument was not specified event_data("plotly_click") would also fire if and when the user clicked on the scatterplot, likely causing an error.

Click to show/hide the code

library(shiny)

# cache computation of the correlation matrix
correlation <- round(cor(mtcars), 3)

ui <- fluidPage(
  plotlyOutput("heat"),
  plotlyOutput("scatterplot")
)

server <- function(input, output, session) {
  
  output$heat <- renderPlotly({
    plot_ly(source = "heat_plot") %>%
      add_heatmap(
        x = names(mtcars), 
        y = names(mtcars), 
        z = correlation
      )
  })
  
  output$scatterplot <- renderPlotly({
    # if there is no click data, render nothing!
    clickData <- event_data("plotly_click", source = "heat_plot")
    if (is.null(clickData)) return(NULL)
    
    # Obtain the clicked x/y variables and fit linear model to those 2 vars
    vars <- c(clickData[["x"]], clickData[["y"]])
    d <- setNames(mtcars[vars], c("x", "y"))
    yhat <- fitted(lm(y ~ x, data = d))
    
    # scatterplot with fitted line
    plot_ly(d, x = ~x) %>%
      add_markers(y = ~y) %>%
      add_lines(y = ~yhat) %>%
      layout(
        xaxis = list(title = clickData[["x"]]), 
        yaxis = list(title = clickData[["y"]]), 
        showlegend = FALSE
      )
  })
  
}

shinyApp(ui, server)

FIGURE 17.11: Linking each cell of a correlation heatmap to their corresponding scatterplots.

17.2.6 Event priority

By default, event_data() only invalidates a reactive expression when the value of it’s corresponding shiny input changes. Sometimes, you might want a particular event, say "plotly_click", to always invalidate a reactive expression. Figure 17.12 shows the difference between this default behavior versus setting priority = 'event'. By default, repeatedly clicking the same marker won’t update the clock, but when setting the priority argument to event, repeatedly clicking the same marker will update the clock (i.e., it will invalidate the reactive expression).

Click to show/hide the code

library(shiny)

ui <- fluidPage(
  plotlyOutput("p"),
  textOutput("time1"),
  textOutput("time2")
)

server <- function(input, output, session) {
  
  output$p <- renderPlotly({
    plot_ly(x = 1:2, y = 1:2, size = I(c(100, 150)))  %>%
      add_markers()
  })
  
  output$time1 <- renderText({
    event_data("plotly_click")
    paste("Input priority: ", Sys.time())
  })
  
  output$time2 <- renderText({
    event_data("plotly_click", priority = "event")
    paste("Event priority: ", Sys.time())
  })
  
}

shinyApp(ui, server)

FIGURE 17.12: A demo of input priority versus event priority. Clicking on the same marker repeatedly, by default, won’t invalidate a reactive expression that depends on ‘plotly_click’, but it will invalidate when given event priority.

There are numerous events accessible through event_data() that don’t contain any information (e.g., "plotly_doublelick", "plotly_deselect", "plotly_afterplot", etc). These events are automatically given an event priority since their corresponding shiny input value never changes. One common use case for events like "plotly_doublelick" (fired when double-clicking in a zoom or pan dragmode) and "plotly_deselect" (fired when double-clicking in a selection mode) is to clear or reset accumulating event data.

17.2.7 Handling discrete axes

For events that are trace-specific (e.g. "plotly_click", "plotly_hover", "plotly_selecting", etc), the positional data (e.g., x/y/z) is always numeric, so if you have a plot with discrete axes, you might want to know how to map that numeric value back to the relevant input data category. In some cases, you can avoid the problem by assigning the discrete variable of interest to the key/customdata attribute, but you might also want to reserve that attribute to encode other information, like a fill aesthetic. Figure 17.13 shows how to map the numerical x value emitted in a click event back to the discrete variable that it corresponds to (mpg$class) and leverages customdata to encode the fill mapping allowing us to display the data records a clicked bar corresponds to. In both ggplotly() and plot_ly(), categories associated with a character vector are always alphabetized, so if you sort() the unique() character values, then the vector indices will match the x event data values. On the other hand, if x were a factor variable, the x event data would match the ordering of the levels() attribute.

Click to show/hide the code

library(shiny)
library(dplyr)

ui <- fluidPage(
  plotlyOutput("bars"),
  verbatimTextOutput("click")
)

classes <- sort(unique(mpg$class))

server <- function(input, output, session) {
  
  output$bars <- renderPlotly({
    ggplot(mpg, aes(class, fill = drv, customdata = drv)) + geom_bar()
  })
  
  output$click <- renderPrint({
    d <- event_data("plotly_click")
    if (is.null(d)) return("Click a bar")
    mpg %>%
      filter(drv %in% d$customdata) %>%
      filter(class %in% classes[d$x])
  })
  
}

shinyApp(ui, server)

FIGURE 17.13: Retrieving the data observations that correspond to a particular bar in a stacked bar chart.

17.2.8 Accumulating and managing event data

Currently all the events accessible through event_data() are transient. This means that, given an event like "plotly_click", the value of event_data() will only reflect the most recent click information. However, in order to implement complex linked graphics with persistent qualities, like Figure 16.3 or 17.25, you’ll need someway to accumulate and manage event data. The general mechanism that shiny provides to achieve this kind of task is reactiveVal() (or, the plural version, reactiveValues()), which essentially provides a way to create and manage input values entirely server-side.

Figure 17.14 demonstrates a shiny app that accumulates hover information and paints the hovered points in red. Every time a hover event is triggered, the corresponding car name is added to the set of selected cars, and everytime the plot is double-clicked that set is cleared. This general pattern of initializing a reactive value (i.e., cars <- reactiveVal()), updating that value upon a suitable observeEvent() event with relevant customdata, and clearing that reactive value (i.e., cars(NULL)) in response to another event is a very useful pattern to can support essentially any sort of linked views paradigm because the logic behind the resolution of selection sequences is under your complete control in R. For example, 17.14 simply adds accumulates the event data from "plotly_hover" (which is like a logical OR operations), but for other applications, you may need different logic, like the AND, XOR, etc.

plotly_example("shiny", "event_data_persist")

FIGURE 17.14: Using reactiveVals() to enable a persistent brush via mouse hover. In this example, the brush can be cleared through a double-click event.

Figure 17.15 demonstrates a shiny gadget for interactively removing/adding points from a linear model via a scatterplot. A shiny gadget is similar to a normal shiny app except that it allows you to return object(s) from the application back to into your R session. In this case, Figure 17.15 returns the fitted model with the outliers removed and the choosen polynomial degree. The logic behind this app does more than simply accumulate event data everytime a point is clicked. Instead, it adds points to the ‘outlier’ set only if it isn’t already an outlier, and removes points that are already in the “outlier” set (so, it’s essentially XOR logic).

plotly_example("shiny", "lmGadget")

FIGURE 17.15: Interactively removing observations from a linear model. Credit to Winston Chang for the initial implementation of this shiny gadget using shiny::plotOutput() instead of plotly::plotlyOutput().

As you can already see, the ability to accumulate and manage event data is a critical skill to have in order to implement shiny applications with complex interactive capabilities. The pattern demonstrates here is known more generally as “maintaining state” of a shiny app based on user interactions and has a variety of applications. So far, we’ve really only see how to maintain state of a single view, but as we’ll see later in section 17.4, the ability to maintain state is required to implement many advanced applications of multiple linked views. Also, it should be noted that Figure 17.14 and 17.15 perform a full redraw when updated – these apps would feel a bit more responsive if they leveraged strategies from section 17.3.1.

17.3 Improving performance

Multiple linked views are known to help facilitate data exploration, but latency in the user interface is also known to reduce exploratory findings (Heer 2014). In addition to the advice and techniques offered in section 17.3.1 for improving plotly’s performance in general, there are also techniques specifically for shiny apps that you can leverage to help improve the user experience.

When trying to speed-up any slow code, the first step is always to identify the main contributor(s) to the poor performance. In some cases, your intuition may serve as a helpful guide, but in order to really see what’s going on, consider using a code profiling tool like profvis (Chang and Luraschi 2018). The profvis package provides a really nice way to visualize and isolate slow running R code in general, but it also works well for profiling shiny apps (RStudio 2014 b).

A lot of different factors can contribute to poor performance in a shiny app, but thankfully, the shiny ecosystem provides an extensive toolbox for diagnosing and improving performance. The profvis package is great for identifying “universal” performance issues, but when deploying shiny apps into production, there may be other potential bottlenecks that surface. This is largely due to R’s single-threaded nature – a single R server has difficulty scaling to many users because, by default, it can only handle one job at a time. The shinyloadtest package helps to identify those bottlenecks and shiny’s support for asynchronous programming with promises is one way to address them without increasing computational infrastructure (e.g. multiple servers) (Dipert, Schloerke, and Borges 2018); (Cheng 2018 b).

To reiterate the section on “Improving performance and scalability” in shiny from Cheng (2018 a), you have a number of tools available to address performance:

The profvis package for profiling code.
Cache computations ahead-of-time.
Cache computations at run time.
Cache computations through chaining reactive expressions.
Leverage multiple R processes and/or servers.
Async programming with promises

We won’t directly cover these topics, but it’s worth noting that all these tools are primarily designed for improving server-side performance of a shiny app. It could be that sluggish plots in your shiny app are due to sluggish server-side code, but it could also be that some of the sluggishness is due to redundant work being done client-side by plotly. Avoiding this redundancy, as covered in section 17.3.1, can be difficult, and it doesn’t always lead to noticable improvements. However, when you need to put lots of graphical elements on a plot, then update just a portion of the plot in response to user event(s), the added complexity can be worth the effort.

17.3.1 Partial plotly updates

By default, when renderPlotly() renders a new plotly graph it’s essentially equivalent to executing a block of R code from your R prompt and generating a new plotly graph from scratch. That means, not only does the R code need to re-execute to generate a new R object, but it also has to re-serialize that object as JSON, and your browser has to re-render the graph from the new JSON object (more on this in section 24). In cases where your plotly graph does not need to serialize a lot data and/or render lots of graphical elements, as in Figure 17.1, you can likely perform a full redraw without noticable glitches, especially if you use canvas-based rendering rather than SVG (i.e., toWebGL()). Generally speaking, you should try very hard to make your app responsive before adopting partial plotly updates in shiny. It makes your app logic easy to reason about because you don’t have to worry about maintaining the state of the graph, but sometimes you have no other choice.

On initial page load, plotly graphs must be drawn from stratch, but when responding to certain user events, often times a partial update to an existing plot is sufficient and more responsive. Take, for instance, the difference between Figure 17.16, which does a full redraw on every update, and Figure 17.17, which does a partial update after initial load. Both of these shiny apps display a scatterplot with 100,000 points and allow a user to overlay a fitted line through a checkbox. The key difference is that in Figure 17.16, the plotly graph is regenerated from scratch everytime the value of input$smooth changes, whereas in Figure 17.17 only the fitted line is added/removed from the plotly. Since the main bottleneck lies in redrawing the points, Figure 17.17 can add/remove the fitted line is a much more responsive fashion.

Click to show/hide the code

library(shiny)
library(plotly)

# Generate 100,000 observations from 2 correlated random variables
d <- MASS::mvrnorm(1e6, mu = c(0, 0), Sigma = matrix(c(1, 0.5, 0.5, 1), 2, 2))
d <- setNames(as.data.frame(d), c("x", "y"))

# fit a simple linear model
m <- lm(y ~ x, data = d)

# generate y predictions over a grid of 10 x values
dpred <- data.frame(
  x = seq(min(d$x), max(d$x), length.out = 10)
)
dpred$yhat <- predict(m, newdata = dpred)

ui <- fluidPage(
  plotlyOutput("scatterplot"),
  checkboxInput("smooth", label = "Overlay fitted line?", value = FALSE)
)

server <- function(input, output, session) {
  
  output$scatterplot <- renderPlotly({
    
    p <- plot_ly(d, x   = ~x, y = ~y) %>%
      add_markers(color = I("black"), alpha = 0.05) %>%
      toWebGL() %>%
      layout(showlegend = FALSE)
    
    if (!input$smooth) return(p)
    
    add_lines(p, data = dpred, x = ~x, y = ~yhat, color = I("red"))
  })
  
}

shinyApp(ui, server)

FIGURE 17.16: Naive implementation of a shiny app that optionally overlays a fitted line to a scatterplot. A full redraw of the plot is performed everytime the checkbox is clicked, leading to an unnecessarily slow plot.

In terms of the implementation behind Figure 17.16 and 17.17, the only difference resides in the server definition. In Figure 17.17, the renderPlotly() statement no longer has a dependency on input values, so that code is only executed once (on page load) to generate the initial view of the scatterplot. The logic behind adding and removing the fitted line is handled through an observe() block – this reactive expression watches the input$smooth input value and modifies the output$scatterplot widget whenever it changes. To trigger a modification of a plotly output widget, you must create a proxy object with plotlyProxy() that references the relevant output ID. Once a proxy object is created, you can invoke any sequence of plotly.js function(s) on it with plotlyProxyInvoke(). Invoking a method with the correct arguments can be tricky and requires knowledge of plotly.js because plotlyProxyInvoke() will send these arguments directly to the plotly.js method and therefore doesn’t support the same ‘high-level’ semantics that plot_ly() does.

Click to show/hide the code

server <- function(input, output, session) {
  
  output$scatterplot <- renderPlotly({
    plot_ly(d, x = ~x, y = ~y) %>%
      add_markers(color = I("black"), alpha = 0.05) %>%
      toWebGL()
  })
  
  observe({
    if (input$smooth) {
      # this is essentially the plotly.js way of doing
      # `p %>% add_lines(x = ~x, y = ~yhat) %>% toWebGL()`
      # without having to redraw the entire plot
      plotlyProxy("scatterplot", session) %>%
        plotlyProxyInvoke(
          "addTraces", 
          list(
            x = dpred$x,
            y = dpred$yhat,
            type = "scattergl",
            mode = "lines",
            line = list(color = "red")
          )
        )
    } else {
      # JavaScript index starts at 0, so the '1' here really means
      # "delete the second traces (i.e., the fitted line)"
      plotlyProxy("scatterplot", session) %>%
        plotlyProxyInvoke("deleteTraces", 1)
    }
  })
}

FIGURE 17.17: A more responsive version of Figure 17.16.

Figure 17.16 demonstrates a common use case where partial updates can be helpful, but there are other not-so-obvious cases. The next section covers a range of examples where you’ll see how to leverage partial updates to implement smooth ‘streaming’ visuals, avoid resetting axis ranges, avoid flickering basemap layers, and more.

17.3.2 Partial update examples

The last section explains why you may want to leverage partial plotly updates in shiny to get more responsive updates through an example. That example leveraged the plotly.js functions Plotly.addTraces() and Plotly.deleteTraces() to add/remove a layer to a plot after it’s initial draw. There are numerous other plotly.js functions that can be handy for a variety of use cases, some of the most widely used ones are: Plotly.restyle() for updating data visuals (section 17.3.2.1), Plotly.relayout() for updating the layout (section 17.3.2.2), and Plotly.extendTraces() for streaming data (section 17.3.2.3).

17.3.2.1 Modifying traces

All plotly figures have two main components: traces (i.e., mapping from data to visuals) and layout. The plotly.js function Plotly.restyle() is for modifying any existing traces. In addition to being a performant way to modify existing data and/or visual properties, it also has the added benefit of not affecting the current layout of the graph. Notice how, in Figure 17.18 for example, when the size of the marker/path changes, it doesn’t change the camera’s view of the 3D plot that the user altered after initial draw. If these input widgets triggered a full redraw of the plot, the camera would be reset to it’s initial state.

plotly_example("shiny", "proxy_restyle_economics")

FIGURE 17.18: Using Plotly.restyle() to change just the width of a path and markers along that path in response to changes to shiny input sliders.

One un-intuitive thing about Plotly.restyle() is that it fully replaces object (i.e., attributes that contain attributes) definitions like marker by default. To modify just a particular attribute of an object, like the size of a marker, you must replace that attribute directly (hence marker.size). As mentioned in the official documentation, by default, modifications are applied to all traces, but specific traces can be targeted through their trace index (which starts at 0, because JavaScript)!

17.3.2.2 Updating the layout

All plotly figures have two main components: traces (i.e., mapping from data to visuals) and layout. The plotly.js function Plotly.relayout() modifies the layout component, so it can control a wide variety of things such titles, axis definitions, annotations, shapes, and many other things. It can even be used to change the basemap layer of a Mapbox-powered layout, as in Figure 17.19. Note how this example uses schema() to grab all the pre-packaged basemap layers and create a dropdown of those options, but you can also provide a URL to a custom basemap style.

plotly_example("shiny", "proxy_relayout")

FIGURE 17.19: Using a shiny::selectInput() to modify the basemap style of plot_mapbox() via Plotly.relayout().

Figure 17.20 demonstrates a clever use of Plotly.relayout() to set the y-axis range in response to changes in the x-axis range.

plotly_example("shiny", "proxy_relayout")

FIGURE 17.20: Using Plotly.relayout() to ‘auto-range’ the y-axis in response to changes in the x-axis range.

17.3.2.3 Streaming data

At this point, we’ve seen how to add/remove traces (e.g., add/remove a fitted line, as in Figure 17.17), and how to edit specific trace properties (e.g., change marker size or path width, as in Figure 17.18), but what about adding more data to existing trace(s)? This is a job for the plotly.js function Plotly.extendTraces() and/or Plotly.prependTraces() which can used to efficiently ‘stream’ data into an existing plot, as done in Figure 17.21.

The implementation behind Figure 17.21, an elementary example of a random walk, makes use of some fairly sophisicated reactive programming tools from shiny. Similar to most examples from this section, the renderPlotly() statement is executed once on initial load to draw the initial line with just two data points. By default, the plot is not streaming, but streaming can be turned on or off through the click of a button, which will require the app to know (at all times) whether or not we are in a streaming state. One way to do this is to leverage shiny’s reactiveValues(), which act like input values, but can be created and modified entirely server-side, making them quite useful for maintaining state of an application. In this case, the reactive value rv$stream is used to store the streaming state, which is turned on/off whenever the actionButton() is clicked (via the observeEvent() logic).

Even if the app is not streaming, there is still constant client/server communication because of the use of invalidateLater() inside the observe(). This effectively tells shiny to re-evaluate the observe() block every 100 milliseconds. If the app isn’t in streaming mode, then it exits early without doing anything. If the app is streaming, then we first use sample() to randomly draw either -1 or 1 (with equal probability) and use the result to update the most recent (x, y) state. This is done by assigning a new value to the reactive values rv$y and rv$n within an isolate()d context – if this assignment happened outside of an isolate()d context it would cause the reactive expression to be invalidated and cause an infinite loop! Once we have the new (x, y) point stored away, Plotly.extendTraces() can be used to add the new point to the plotly graph.

plotly_example("shiny", "stream")

FIGURE 17.21: Using Plotly.extendTraces() to efficiently stream data into a plotly chart. This specific example implements a random walk (using R’s random number generator) which updates every 100 milliseconds.

To see more examples that leverage partial updating, see section 17.4.2.

17.4 Advanced applications

This section combines concepts from prior sections in linking views with shiny and applies them towards some popular use cases.

17.4.1 Drill-down

The so-called ‘drill-down’ chart tends to include a vague grouping of interactive graphics that allow one to generate new views based on interesting subset(s) of the data. In Figure 17.22, the first bar chart shows store sales broked down by their main categories. By clicking on a bar, one can drill-down into that category to see the breakdown of sub-categories within that category. From there, one may pick a sub-category to populate a time series of the corresponding sales; and futhermore, click on a particular date to see the actual sales records for that day.

In a drill-down chart, a change in the top-level category should trigger changes to sub-categories, so to handle this logic correctly, you’ll want to store selections in reactive values and update those values accordingly when relevant events occur. For example, note how in Figure 17.22, a click of a category clears the sub-category and order-date. Moreover, a change in a sub-category will clear the order-date, but doesn’t change the category.

Click to show/hide the code

library(shiny)
library(plotly)
library(dplyr)

# CSV available from https://github.com/cpsievert/plotly_book
sales <- readr::read_csv("data/sales.csv")

ui <- fluidPage(
  plotlyOutput("category", height = 200),
  plotlyOutput("sub_category", height = 200),
  plotlyOutput("sales", height = 300),
  dataTableOutput("datatable")
)

# avoid repeating this code
axis_titles <- . %>%
  layout(
    xaxis = list(title = ""),
    yaxis = list(title = "Sales")
  )

server <- function(input, output, session) {
  
  # for maintaining the state of drill-down variables
  category <- reactiveVal()
  sub_category <- reactiveVal()
  order_date <- reactiveVal()
  
  # when clicking on a category, 
  observeEvent(event_data("plotly_click", source = "category"), {
    category(event_data("plotly_click", source = "category")$x)
    sub_category(NULL)
    order_date(NULL)
  })
  
  observeEvent(event_data("plotly_click", source = "sub_category"), {
    sub_category(event_data("plotly_click", source = "sub_category")$x)
    order_date(NULL)
  })
  
  observeEvent(event_data("plotly_click", source = "order_date"), {
    order_date(event_data("plotly_click", source = "order_date")$x)
  })
  
  output$category <- renderPlotly({
    sales %>%
      count(category, wt = sales) %>%
      plot_ly(x = ~category, y = ~n, source = "category") %>%
      axis_titles() %>% 
      layout(title = "Sales by category")
  })
  
  output$sub_category <- renderPlotly({
    if (is.null(category())) return(NULL)
    
    sales %>%
      filter(category %in% category()) %>%
      count(sub_category, wt = sales) %>%
      plot_ly(x = ~sub_category, y = ~n, source = "sub_category") %>%
      axis_titles() %>%
      layout(title = category())
  })
  
  output$sales <- renderPlotly({
    if (is.null(sub_category())) return(NULL)
    
    sales %>%
      filter(sub_category %in% sub_category()) %>%
      count(order_date, wt = sales) %>%
      plot_ly(x = ~order_date, y = ~n, source = "order_date") %>%
      add_lines() %>%
      axis_titles() %>%
      layout(title = paste(sub_category(), "sales over time"))
  })
  
  output$datatable <- renderDataTable({
    if (is.null(order_date())) return(NULL)
    
    sales %>%
      filter(
        sub_category %in% sub_category(),
        as.Date(order_date) %in% as.Date(order_date())
      )
  })
  
}

shinyApp(ui, server)

FIGURE 17.22: Using a drill-down approach to navigating through sales data by category, sub-category, and order date.

17.4.2 Cross-filter

Somewhat related to the drill-down idea is so-called ‘cross-filter’ chart. The main difference between drill-down and cross-filter is that, with cross-filter, interactions don’t generate new charts – interactions impose a filter on the data shown in a fixed set of multiple views. The typical cross-filter implementation allows for multiple brushes (i.e., filters) and uses the intersection of those filters to the dataset displayed in those views. Implementing a scalable and responsive crossfilter with 3 or more views can get quite complicated – we’ll walk through some simple examples first for learning purposes, then progress to more sophicated and complex applications.

Figure 17.23 demonstrates the simplest way to implement a cross-filter between two histograms. It uses the arrival (arr_time) and departure (dep_time) from the flights dataset via the nycflights13 package (Wickham 2018 a). Notice how, in the implementation, the dep_time view is re-drawn from stratch everytime the arr_time brush changes (and vice-versa). Not only is it completely redrawn (i.e., it relies on renderPlotly() to perform the update), but it also uses add_histogram() which performs binning client-side (in the web browser). That means, every time a brush changes, the shiny server sends all the raw data to the browser and plotly.js redraws the histogram from scratch.

Click to show code

library(shiny)
library(dplyr)
library(nycflights13)

ui <- fluidPage(
  plotlyOutput("arr_time"),
  plotlyOutput("dep_time")
)

server <- function(input, output, session) {
  
  output$arr_time <- renderPlotly({
    p <- plot_ly(flights, x = ~arr_time, source = "arr_time") 
    
    brush <- event_data("plotly_brushing", source = "dep_time")
    if (is.null(brush)) return(p)
    
    p %>%
      filter(between(dep_time, brush$x[1], brush$x[2])) %>%
      add_histogram()
  })
  
  output$dep_time <- renderPlotly({
    p <- plot_ly(flights, x = ~dep_time, source = "dep_time") 
    
    brush <- event_data("plotly_brushing", source = "arr_time")
    if (is.null(brush)) return(p)
    
    p %>%
      filter(between(arr_time, brush$x[1], brush$x[2])) %>%
      add_histogram()
  })
  
}

shinyApp(ui, server)

FIGURE 17.23: A ‘naive’ crossfilter implementation linking arrival time with departure time of about 350,000 flights from the nycflights13 package.

Although the video behind Figure 17.23 demonstrates the app is fairly responsive at 350,000 observations, this implementation won’t scale to much larger data, especially if being viewed a poor internet connection. I call this a ‘naive’ implementation because the reactive logic is easy to reason about, but it illustrates two common issues that we can address to gain speed improvements:

More data than necessary being sent ‘over-the-wire’ (i.e., between the server and the client). This idea is not unique to shiny applications – with any web application framework it’s important to minimize the amount of data you’re requesting from a server if you want a responsive website.
More client-side rendering work than necessary to achieve the request update.

The implementation behind Figure 17.23 could improve in these areas by doing the following:

Perform the binning server-side instead of client-side. This will reduce the amount of data needed to be sent from server to client so that responsiveness is less dependent on a good internet connection. Here we propose using ggstat for the server-side binning since it’s fairly fast and simple if you’re data can fit into memory (Wickham 2016). If your data does not fit into memory you could use something like dbplot to perform the binning in a database (Ruiz 2018).
Perform less rendering work client-side. That is, instead of relying on renderPlotly() to re-render the chart from scratch everytime the charts need an update, we could instead modify just the bar heights (using the techniques from Section 17.3.1).

Click to below to see the responsive implementation

library(shiny)
library(dplyr)
library(nycflights13)
library(ggstat)

arr_time <- flights$arr_time
dep_time <- flights$dep_time
arr_bins <- bin_fixed(arr_time, bins = 250)
dep_bins <- bin_fixed(dep_time, bins = 250)
arr_stats <- compute_stat(arr_bins, arr_time) %>%
  filter(!is.na(xmin_))
dep_stats <- compute_stat(dep_bins, dep_time) %>%
  filter(!is.na(xmin_))

ui <- fluidPage(
  plotlyOutput("arr_time", height = 250),
  plotlyOutput("dep_time", height = 250)
)

server <- function(input, output, session) {
  
  output$arr_time <- renderPlotly({
    plot_ly(arr_stats, source = "arr_time") %>%
      add_bars(x = ~xmin_, y = ~count_) 
  })
  
  output$dep_time <- renderPlotly({
    plot_ly(dep_stats, source = "dep_time") %>%
      add_bars(x = ~xmin_, y = ~count_) 
  })
  
  # arr_time brush updates dep_time view
  observe({
    brush <- event_data("plotly_brushing", source = "arr_time")
    p <- plotlyProxy("dep_time", session)
    
    # if brush is empty, restore default
    if (is.null(brush)) {
      plotlyProxyInvoke(p, "restyle", "y", list(dep_stats$count_), 0)
    } else {
      dep_time_filter <- dep_time[between(dep_time, brush$x[1], brush$x[2])]
      dep_count <- dep_bins %>%
        compute_stat(dep_time_filter) %>%
        filter(!is.na(xmin_)) %>%
        pull(count_)
      
      plotlyProxyInvoke(p, "restyle", "y", list(dep_count), 0)
    }
  })
  
  observe({
    brush <- event_data("plotly_brushing", source = "dep_time")
    p <- plotlyProxy("arr_time", session)
    
    # if brush is empty, restore default
    if (is.null(brush)) {
      plotlyProxyInvoke(p, "restyle", "y", list(arr_stats$count_), 0)
    } else {
      arr_time_filter <- arr_time[between(arr_time, brush$x[1], brush$x[2])]
      arr_count <- arr_bins %>%
        compute_stat(arr_time_filter) %>%
        filter(!is.na(xmin_)) %>%
        pull(count_)
      
      plotlyProxyInvoke(p, "restyle", "y", list(arr_count), 0)
    }
  })
  
}

shinyApp(ui, server)

Before we address the additional complexity that comes with linking 3 or more views, let’s consider targetting a 2D distribution in the cross-filter, as in Figure 17.24. This approach uses kde2d() from the MASS package to summarize the 2D distribution server-side rather than attempting to show all the raw data in a scatterplot.³³

plotly_example("shiny", "crossfilter_kde")

FIGURE 17.24: Using kernel density estimation to responsively crossfilter a 2D distribution. This particular example shows how the relationship between diamond carat and price is dependent upon it’s depth.

When linking 3 or more views in a crossfilter, it’s important to have a mechanism to maintain the state of all the active brushes. This is because, when updating a given view, it needs to know about all of the active brushes. The implementation behind Figure 17.25 maintains the range of every active brush through a reactiveValues() variable named brush_ranges. Everytime a brush changes, the state of brush_ranges is updated, then used to filter the data down to the relevant observations. That filtered data is then binned and used to modify the bar heights of every view (except for the one being brushed).

plotly_example("shiny", "crossfilter")

FIGURE 17.25: Crossfiltering 6 variables in the flights data from the nycflights13 package (Wickham 2018 a). The filtering and binning logic occurs server-side resulting in a very minimal amount of data being sent between server and client (just the brush range and bar heights). Moreover, to perform the UI update, the client only has to tweak existing bar heights, so the overall user experience is quite responsive.

One weakness of a typical crossfilter interface like Figure 17.25 is that it’s difficult to make visual comparisons. That is, when a filter is applied, you lose a visual reference to the overall distribution and/or prior filter, making difficult to make meaningful comparisons. Figure 17.25 modifies the logic of Figure 17.26 to enable filter comparisons by adding the ability to change the color of the brush. Moreover, for sake of demonstration and simplicity, it also allows for only one active filter per color (i.e., brushing within color is transient). One could borrow logic from Figure 17.25 to allow multiple filters for each color, but this would require multiple brush_ranges.

Since brushing within color is transient, in constrast to Figure 17.25, Figure 17.26 doesn’t have to track the state of all the active brushes. It does, however, need to display relative rather than absolute frequencies to facilitate comparison of small filter range(s) to the overall distribution. This particular implementation takes the overall distribution as a “base layer” that remains fixed and overlays a handful of “selection layers” – one for each possible brush color. These selection layers begin with a height of 0, but when the relevant brush fires the heights of the bars for the relevant layer is modified. This approach may seem like a hack, but it leads to a fluid user experience because it’s not much work to adjust the height of a bar that already exists.

plotly_example("shiny", "crossfilter_compare")

FIGURE 17.26: Comparing filters with a dynamic color brush. This particular example compares ‘red eye’ flights (in green) to early morning flights (in orange). This makes it easier to see that delays occur more often for ‘red eye’ flights.

17.4.3 A draggable brush

A draggable linked brush is great for conditioning via a moving window. For example, in a cross-filtering example like Figure 17.25, it would be nice to condition on a certain hour of day, then drag that hour interval along the axis to explore how the hourly distribution changes throughout the day. At the time of writing, plotly.js does not support a draggable brush, but we could implement one with a clever use of a editable rectangle shape. Figure 17.27 demonstrates this idea in a shiny application by drawing a rectangle shape that mimics the plotly.js brush, then uses the "plotly_relayout" event to determine the limits of the brush (instead of the "plotly_brushed" or "plotly_brushing").

plotly_example("shiny", "drag_brush")

FIGURE 17.27: A draggable brush with both a transient and persistent mode. The dragging ability is done by mimicing the native plotly.js brush with an editable rectangle shape and listening to changes in that brush. The implementation of transient vs persistent mode is a matter of forgetting or remembering previous state(s) of the brush.

17.5 Discussion

Compared to the linking framework covered in section 16.1, the ability to link views server-side with shiny opens the door to many new possibilities. This chapter focuses mostly on using just plotly within shiny, but the shiny ecosystem is vast and these techniques can of course be used to inform other views, such as static plots, other htmlwidgets packages (e.g., leaflet, DT, network3D, etc), and other custom shiny bindings. In fact, I have a numerous shiny apps publically available via an R package that use numerous tools to provide exploratory interfaces to a variety of domain-specific problems, including zikar::explore() for exploring Zika virus reports, eechidna::launch() for exploring Australian election and census data, and bcviz::launch() for exploring housing and census information in British Colombia (Sievert 2018 d); (Cook et al. 2017); (Sievert 2018 a). These complex applications also serve as a reference as to how can use the client-side linking framework (i.e., crosstalk) inside a larger shiny application. See this video for an example:

FIGURE 17.28: Example of a shiny app that has crosstalk functionality embedded.

Sometimes shiny gets a bad rap for being too slow or unresponsive, but as we learned in sections 17.3.1 and 17.4, we can still have very advanced functionality as well as a good user experience – it just takes a bit more effort to optimize performance in some cases. In fact, one could argue that a server-client approach to crossfiltering, as done in Figure 17.25 is more scalable than a purely client-side approach since the client wouldn’t need to know about the raw data – just the summary statistics. Nevertheless, sometimes linking views server side simply isn’t an option for you or your organization.

Maybe your IT administrator simply won’t allow you to distribute your work outside of a standalone HTML file. Figure 17.11 is just one example of a linked graphic that could be replicated using the graphical querying framework from section 16.1, but it would require pre-computing every possible view (which becomes un-manageable when there are many possible selections) and posing the update logic as a database query. When users are only allowed to select (e.g. click/hover) a single element at a time, the number of possible selections increases linearly with the number of elements, but when users are allowed to select any subset of elements (e.g., scatterplot brushing), the number of possible selection explodes (increases at a factorial rate). For example, adding a cell to Figure 17.11 only adds one possible selection, but if we added more states to Figure 17.11, the number of possible states goes from 50! to 51!.

Even in the case that you need a standalone HTML file and the R API that plotly provides doesn’t support the type of interactivity that you desire, you can always layer on additional JavaScript to hopefully achieve the functionality you desire. This can be useful for something as simple as opening a hyperlink when clicking on marker of a plotly graph – this topic is covered introduced in Chapter 18.

References

Chang, Winston. 2017. “Interactive Plots (in Shiny).” Blog. http://shiny.rstudio.com/articles/plot-interaction.html.

Chang, Winston, and Barbara Borges Ribeiro. 2018. Shinydashboard: Create Dashboards with ’Shiny’. https://CRAN.R-project.org/package=shinydashboard.

Mastny, Timothy. 2018. Sass: Syntactically Awesome Stylesheets (Sass) Compiler. https://github.com/rstudio/sass.

Cheng, Joe. 2018c. Using Leaflet with Shiny. Blog. https://rstudio.github.io/leaflet/shiny.html.

Xie, Yihui. 2018. Using Leaflet with Shiny. Blog. https://rstudio.github.io/DT/shiny.html.

Heer, Zhicheng Liu AND Jeffrey. 2014. “The Effects of Interactive Latency on Exploratory Visual Analysis.” IEEE Trans. Visualization & Comp. Graphics (Proc. InfoVis). http://idl.cs.washington.edu/papers/latency.

Chang, Winston, and Javier Luraschi. 2018. Profvis: Interactive Visualizations for Profiling R Code. https://CRAN.R-project.org/package=profvis.

RStudio. 2014b. “Profvis — Interactive Visualizations for Profiling R Code.” Blog. https://rstudio.github.io/profvis/examples.html.

Dipert, Alan, Barret Schloerke, and Barbara Borges. 2018. Shinyloadtest: Load Test Shiny Applications.

Cheng, Joe. 2018b. Promises: Abstractions for Promise-Based Asynchronous Programming. https://CRAN.R-project.org/package=promises.

Cheng, Joe. 2018a. “Case Study: Converting a Shiny App to Async.” Blog. https://rstudio.github.io/promises/articles/casestudy.html.

Wickham, Hadley. 2018a. Nycflights13: Flights That Departed Nyc in 2013. https://CRAN.R-project.org/package=nycflights13.

Wickham, Hadley. 2016. Ggstat: Statistical Computations for Visualisation. https://github.com/hadley/ggstat.

Ruiz, Edgar. 2018. Dbplot: Simplifies Plotting Data Inside Databases. https://CRAN.R-project.org/package=dbplot.

Sievert, Carson. 2018d. Zikar: Tools for Exploring Publicly Available Zika Data. https://github.com/cpsievert/zikar.

Cook, Di, Anthony Ebert, Heike Hofmann, Rob Hyndman, Thomas Lumley, Ben Marwick, Carson Sievert, et al. 2017. Eechidna: Exploring Election and Census Highly Informative Data Nationally for Australia. https://CRAN.R-project.org/package=eechidna.

Sievert, Carson. 2018a. Bcviz: A Shiny App for Exploring Bc Housing and Census Data. https://github.com/cpsievert/bcviz.

Read more about shiny’s responsive layout here https://shiny.rstudio.com/articles/layout-guide.html ↩
Read more about using custom HTML templates here https://shiny.rstudio.com/articles/html-ui.html ↩
The plotly_build() function is an S3 generic, so you can list all relevant the methods with methods(plotly_build), and write you’re own method to translate a custom object to plotly.↩
In order to convert grid grobs that are relatively sized, the ggplotly() function uses the size of the current graphics device at print-time, meaning that resizing the browser window without a hook back to R can create wonky sizes.↩
These events are documented at https://plot.ly/javascript/plotlyjs-events/↩
It’s possible to do this responsively with about 50,000 observations, but it won’t scale to anything much larger than that. Run plotly_example("shiny", "crossfilter_scatter") to see it action as well as a corresponding video at https://vimeo.com/318129005 ↩