9  Data Visualisation

9.1 Why Visualisation Matters in Data Science

Effective visualisation is crucial for data science: it helps you communicate findings and enables pattern discovery. A well-designed chart can convey relationships instantly that would take paragraphs to explain in words. The reporting chapter covered the document that wraps your analysis; this chapter covers the charts and diagrams that go inside it.

Data visualisation serves multiple purposes in the data science workflow:

1. Exploratory Data Analysis (EDA): Discovering patterns, outliers, and relationships
2. Communication: Sharing insights with stakeholders
3. Decision Support: Helping decision-makers understand complex data
4. Monitoring: Tracking metrics and performance over time

A chart works because it hands the reader a shape to recognise instead of a column of numbers to compare one pair at a time. A good one earns its place by saving the reader that effort.

This chapter focuses on three practical categories:

• Python visualisation libraries: matplotlib, seaborn, plotly
• R visualisation libraries: ggplot2 and interactive wrappers
• Code-based diagramming with Mermaid for flowcharts, ER diagrams, and architecture sketches

Interactive web dashboards built on top of these libraries (Shiny, Dash, Streamlit) are covered in the Web Development for Data Scientists chapter, since they’re fundamentally running web applications rather than visualisations embedded in a document.

9.2 Python Visualisation Libraries

Python has several established visualisation libraries, each with different strengths and use cases.

9.2.1 Matplotlib: The Foundation

Matplotlib is the original Python visualisation library and serves as the foundation for many others. It provides precise control over every element of a plot.

import matplotlib.pyplot as plt
import numpy as np

# Generate data
x = np.linspace(0, 10, 100)
y = np.sin(x)

# Create a figure and axis
fig, ax = plt.subplots(figsize=(10, 6))

# Plot data
ax.plot(x, y, 'b-', linewidth=2, label='sin(x)')

# Add labels and title
ax.set_xlabel('X-axis', fontsize=14)
ax.set_ylabel('Y-axis', fontsize=14)
ax.set_title('Sine Wave', fontsize=16)

# Add grid and legend
ax.grid(True, linestyle='--', alpha=0.7)
ax.legend(fontsize=12)

# Save and show the figure
plt.savefig('sine_wave.png', dpi=300, bbox_inches='tight')
plt.show()

Matplotlib provides a blank canvas approach where you explicitly define every element. This gives you complete control but requires more code for complex visualisations. It’s the right tool when you need exact control over every line, tick, and label, for example when preparing a figure for a journal submission.

9.2.2 Seaborn: Statistical Visualisation

Seaborn builds on Matplotlib to provide high-level functions for common statistical visualisations.

import seaborn as sns
import pandas as pd
import matplotlib.pyplot as plt

# Set the theme
sns.set_theme(style="whitegrid")

# Load example data
tips = sns.load_dataset("tips")

# Create a visualisation
plt.figure(figsize=(12, 6))
sns.boxplot(x="day", y="total_bill", hue="smoker", data=tips, palette="Set3")
plt.title("Total Bill by Day and Smoker Status", fontsize=16)
plt.xlabel("Day", fontsize=14)
plt.ylabel("Total Bill ($)", fontsize=14)
plt.tight_layout()
plt.show()

Seaborn simplifies the creation of statistical visualisations like box plots, violin plots, and regression plots. It also comes with built-in themes that improve the default appearance of plots. Reach for Seaborn when the chart you want is a standard statistical view of a pandas DataFrame: it will usually be a single function call, where the equivalent matplotlib code would be a dozen lines.

9.2.3 Plotly: Interactive Visualisations

Plotly creates interactive visualisations that can be embedded in web applications or Jupyter notebooks.

import plotly.express as px
import pandas as pd

# Load example data
df = px.data.gapminder().query("year == 2007")

# Create an interactive scatter plot
fig = px.scatter(
    df, x="gdpPercap", y="lifeExp", size="pop", color="continent",
    log_x=True, size_max=60,
    title="GDP per Capita vs Life Expectancy (2007)",
    labels={"gdpPercap": "GDP per Capita", "lifeExp": "Life Expectancy (years)"}
)

# Update layout
fig.update_layout(
    width=900, height=600,
    legend_title="Continent",
    font=dict(family="Arial", size=14)
)

# Show the figure
fig.show()

Plotly’s interactive features include zooming, panning, hovering for details, and the ability to export plots as images. These features make exploration more intuitive and presentations more engaging. Plotly is available in both Python and R, so you can use the same underlying library across languages.

9.2.4 Choosing Between Python Libraries

A pragmatic decision tree for the non-CS reader:

• Need absolute control or a publication-quality static figure? Matplotlib.
• Want a standard statistical plot from a DataFrame with one function call? Seaborn.
• Producing an HTML report or notebook where hover, zoom, and tooltips add value? Plotly.

You’ll usually end up with all three in your toolkit: seaborn for quick EDA, plotly for HTML-bound output, and matplotlib when one of the other two can’t quite render what you want.

9.3 R Visualisation Libraries

R has its own visualisation ecosystem, with ggplot2 the most widely used library by a wide margin.

9.3.1 ggplot2: Grammar of Graphics

ggplot2 is the default choice for data visualisation in R, built on the Grammar of Graphics.

library(ggplot2)
library(dplyr)

# Load dataset
data(diamonds, package = "ggplot2")

# Create a sample of the data
set.seed(42)
diamonds_sample <- diamonds |>
  slice_sample(n = 1000)

# Create basic plot
p <- ggplot(diamonds_sample, aes(x = carat, y = price, color = cut)) +
  geom_point(alpha = 0.7) +
  geom_smooth(method = "lm", se = FALSE) +
  scale_color_brewer(palette = "Set1") +
  labs(
    title = "Diamond Price vs. Carat by Cut Quality",
    subtitle = "Sample of 1,000 diamonds",
    x = "Carat (weight)",
    y = "Price (USD)",
    color = "Cut Quality"
  ) +
  theme_minimal() +
  theme(
    plot.title = element_text(size = 16, face = "bold"),
    plot.subtitle = element_text(size = 12, color = "gray50"),
    axis.title = element_text(size = 12),
    legend.position = "bottom"
  )

# Display the plot
print(p)

# Save the plot
ggsave("diamond_price_carat.png", p, width = 10, height = 6, dpi = 300)

The philosophy behind ggplot2 is that you build a visualisation layer by layer, which corresponds to how we think about visualisations conceptually. First, you define your data and aesthetic mappings (which variables map to which visual properties), then add geometric objects (points, lines, bars), then statistical transformations, scales, coordinate systems, and finally visual themes. This layered approach makes it possible to create complex visualisations by combining simple, understandable components.

9.3.2 Interactive R Visualisations

R also offers interactive visualisation libraries:

library(plotly)
library(dplyr)

# install.packages("gapminder") if you don't have it - it isn't part of
# the tidyverse, and nothing else in this book depends on it
data(gapminder, package = "gapminder")
data_2007 <- gapminder |>
  filter(year == 2007)

# Create interactive plot
p <- plot_ly(
  data = data_2007,
  x = ~gdpPercap,
  y = ~lifeExp,
  size = ~pop,
  color = ~continent,
  type = "scatter",
  mode = "markers",
  sizes = c(5, 70),
  marker = list(opacity = 0.7, sizemode = "diameter"),
  hoverinfo = "text",
  text = ~paste(
    "Country:", country, "<br>",
    "Population:", format(pop, big.mark = ","), "<br>",
    "Life Expectancy:", round(lifeExp, 1), "years<br>",
    "GDP per Capita:", format(round(gdpPercap), big.mark = ","), "USD"
  )
) |>
  layout(
    title = "GDP per Capita vs. Life Expectancy (2007)",
    xaxis = list(
      title = "GDP per Capita (USD)",
      type = "log",
      gridcolor = "#EEEEEE"
    ),
    yaxis = list(
      title = "Life Expectancy (years)",
      gridcolor = "#EEEEEE"
    ),
    legend = list(title = list(text = "Continent"))
  )

# Display the plot
p

The R version of plotly can convert ggplot2 visualisations to interactive versions with a single function call:

# Convert a ggplot to an interactive plotly visualisation
ggplotly(p)

Turning a static ggplot2 chart into an interactive one with a single function call means you can develop in familiar ggplot2 syntax and add interactivity at the end. That matters most when the same analysis has to ship as both a PDF and a web page: build the chart once, render it static for the PDF and interactive for the HTML.

9.4 Code-Based Diagramming with Mermaid

Diagrams are essential for data science documentation, helping to explain workflows, architectures, and relationships. Rather than creating images with external tools, you can write diagrams as code directly in your Quarto documents using Mermaid.

9.4.1 Why Use Mermaid for Data Science?

Code-based diagramming offers several advantages:

1. Reproducibility: Diagrams are defined as code and rendered during document compilation
2. Version control: Diagram definitions can be tracked in git alongside your code
3. Consistency: Apply the same styling across all diagrams in your project
4. Editability: Easily update diagrams without specialised software
5. Integration: Diagrams are rendered directly within your documents

For data scientists, this means your entire workflow (code, analysis, explanations, and diagrams) can all be maintained in the same reproducible environment.

9.4.2 Creating Mermaid Diagrams in Quarto

Quarto has built-in support for Mermaid diagrams. To create a diagram, use a code block with the mermaid engine:

flowchart LR
    A[Raw Data] --> B[Data Cleaning]
    B --> C[Exploratory Analysis]
    C --> D[Feature Engineering]
    D --> E[Model Training]
    E --> F[Evaluation]
    F --> G[Deployment]

flowchart LR
    A[Raw Data] --> B[Data Cleaning]
    B --> C[Exploratory Analysis]
    C --> D[Feature Engineering]
    D --> E[Model Training]
    E --> F[Evaluation]
    F --> G[Deployment]

The syntax starts with the diagram type (flowchart), followed by the direction (LR for left-to-right), and then the definition of nodes and connections.

9.4.3 Diagram Types for Data Science

Mermaid supports several diagram types that are particularly useful for data science:

9.4.3.1 Flowcharts

Flowcharts are perfect for documenting data pipelines and analysis workflows:

flowchart TD
    A[Raw Data] --> B{Missing Values?}
    B -->|Yes| C[Imputation]
    B -->|No| D[Feature Engineering]
    C --> D
    D --> E[Train Test Split]
    E --> F[Model Training]
    F --> G[Evaluation]
    G --> H{Performance<br>Acceptable?}
    H -->|Yes| I[Deploy Model]
    H -->|No| J[Tune Parameters]
    J --> F

flowchart TD
    A[Raw Data] --> B{Missing Values?}
    B -->|Yes| C[Imputation]
    B -->|No| D[Feature Engineering]
    C --> D
    D --> E[Train Test Split]
    E --> F[Model Training]
    F --> G[Evaluation]
    G --> H{Performance<br>Acceptable?}
    H -->|Yes| I[Deploy Model]
    H -->|No| J[Tune Parameters]
    J --> F

This top-down (TD) flowchart illustrates a complete machine learning workflow with decision points. Notice how you can use different node shapes (rectangles, diamonds) and add text to connections.

9.4.3.2 Class Diagrams

Class diagrams help explain data structures and relationships:

classDiagram
    class Dataset {
        +DataFrame data
        +load_from_csv(filename)
        +split_train_test(test_size)
        +normalize()
    }

    class Model {
        +train(X, y)
        +predict(X)
        +evaluate(X, y)
        +save(filename)
    }

    class Pipeline {
        +steps
        +add_step(transformer)
        +fit_transform(data)
    }

    Dataset --> Model: provides data to
    Pipeline --> Dataset: processes
    Pipeline --> Model: feeds into

classDiagram
    class Dataset {
        +DataFrame data
        +load_from_csv(filename)
        +split_train_test(test_size)
        +normalize()
    }

    class Model {
        +train(X, y)
        +predict(X)
        +evaluate(X, y)
        +save(filename)
    }

    class Pipeline {
        +steps
        +add_step(transformer)
        +fit_transform(data)
    }

    Dataset --> Model: provides data to
    Pipeline --> Dataset: processes
    Pipeline --> Model: feeds into

This diagram shows the relationships between key classes in a machine learning system. It’s useful for documenting the architecture of your data science projects.

9.4.3.3 Sequence Diagrams

Sequence diagrams show interactions between components over time:

sequenceDiagram
    participant U as User
    participant API as REST API
    participant ML as ML Model
    participant DB as Database

    U->>API: Request prediction
    API->>DB: Fetch features
    DB-->>API: Return features
    API->>ML: Send features for prediction
    ML-->>API: Return prediction
    API->>DB: Log prediction
    API-->>U: Return results

sequenceDiagram
    participant U as User
    participant API as REST API
    participant ML as ML Model
    participant DB as Database

    U->>API: Request prediction
    API->>DB: Fetch features
    DB-->>API: Return features
    API->>ML: Send features for prediction
    ML-->>API: Return prediction
    API->>DB: Log prediction
    API-->>U: Return results

This diagram illustrates the sequence of interactions in a model deployment scenario, showing how data flows between the user, API, model, and database.

9.4.3.4 Gantt Charts

Gantt charts are useful for project planning and timelines:

gantt
    title Data Science Project Timeline
    dateFormat YYYY-MM-DD

    section Data Preparation
    Collect raw data       :a1, 2026-01-01, 10d
    Clean and validate     :a2, after a1, 5d
    Exploratory analysis   :a3, after a2, 7d
    Feature engineering    :a4, after a3, 8d

    section Modeling
    Split train/test       :b1, after a4, 1d
    Train baseline models  :b2, after b1, 5d
    Hyperparameter tuning  :b3, after b2, 7d
    Model evaluation       :b4, after b3, 4d

    section Deployment
    Create API            :c1, after b4, 6d
    Documentation         :c2, after b4, 8d
    Testing               :c3, after c1, 5d
    Production release    :milestone, after c2 c3, 0d

gantt
    title Data Science Project Timeline
    dateFormat YYYY-MM-DD

    section Data Preparation
    Collect raw data       :a1, 2026-01-01, 10d
    Clean and validate     :a2, after a1, 5d
    Exploratory analysis   :a3, after a2, 7d
    Feature engineering    :a4, after a3, 8d

    section Modeling
    Split train/test       :b1, after a4, 1d
    Train baseline models  :b2, after b1, 5d
    Hyperparameter tuning  :b3, after b2, 7d
    Model evaluation       :b4, after b3, 4d

    section Deployment
    Create API            :c1, after b4, 6d
    Documentation         :c2, after b4, 8d
    Testing               :c3, after c1, 5d
    Production release    :milestone, after c2 c3, 0d

This Gantt chart shows the timeline of a data science project, with tasks grouped into sections and dependencies between them clearly indicated.

9.4.3.5 Entity-Relationship Diagrams

ER diagrams are valuable for database schema design:

erDiagram
    CUSTOMER ||--o{ ORDER : places
    ORDER ||--|{ ORDER_ITEM : contains
    PRODUCT ||--o{ ORDER_ITEM : "ordered in"
    CUSTOMER {
        int customer_id PK
        string name
        string email
        date join_date
    }
    ORDER {
        int order_id PK
        int customer_id FK
        date order_date
        float total_amount
    }
    ORDER_ITEM {
        int order_id PK,FK
        int product_id PK,FK
        int quantity
        float price
    }
    PRODUCT {
        int product_id PK
        string name
        string category
        float unit_price
    }

erDiagram
    CUSTOMER ||--o{ ORDER : places
    ORDER ||--|{ ORDER_ITEM : contains
    PRODUCT ||--o{ ORDER_ITEM : "ordered in"
    CUSTOMER {
        int customer_id PK
        string name
        string email
        date join_date
    }
    ORDER {
        int order_id PK
        int customer_id FK
        date order_date
        float total_amount
    }
    ORDER_ITEM {
        int order_id PK,FK
        int product_id PK,FK
        int quantity
        float price
    }
    PRODUCT {
        int product_id PK
        string name
        string category
        float unit_price
    }

This diagram shows a typical e-commerce database schema with relationships between tables and their attributes.

9.4.4 Styling Mermaid Diagrams

You can customise the appearance of your diagrams:

flowchart LR
    A[Data Collection] --> B[Data Cleaning]
    B --> C[Analysis]

    style A fill:#f9f,stroke:#333,stroke-width:2px
    style B fill:#bbf,stroke:#33f,stroke-width:2px
    style C fill:#bfb,stroke:#3f3,stroke-width:2px

flowchart LR
    A[Data Collection] --> B[Data Cleaning]
    B --> C[Analysis]

    style A fill:#f9f,stroke:#333,stroke-width:2px
    style B fill:#bbf,stroke:#33f,stroke-width:2px
    style C fill:#bfb,stroke:#3f3,stroke-width:2px

This diagram uses custom colours and border styles for each node to highlight different stages of the process.

9.4.5 Generating Diagrams Programmatically

For complex or dynamic diagrams, you can generate Mermaid code programmatically. The trick is output: asis, which tells Quarto to drop the chunk’s output into the document as raw Markdown rather than as printed R output — so a Mermaid fence you build with cat() is processed exactly as if you’d typed it:

# Define the steps in a data pipeline
steps <- c("Import Data", "Clean Data", "Feature Engineering",
           "Split Dataset", "Train Model", "Evaluate", "Deploy")

# Generate Mermaid flowchart code
mermaid_code <- c(
  "```{mermaid}",
  "flowchart LR"
)

# Add connections between steps
for (i in 1:(length(steps)-1)) {
  mermaid_code <- c(
    mermaid_code,
    sprintf("    %s[\"%s\"] --> %s[\"%s\"]",
            LETTERS[i], steps[i],
            LETTERS[i+1], steps[i+1])
  )
}

mermaid_code <- c(mermaid_code, "```")

# Output the Mermaid code
cat(paste(mermaid_code, collapse = "\n"))

flowchart LR
    A["Import Data"] --> B["Clean Data"]
    B["Clean Data"] --> C["Feature Engineering"]
    C["Feature Engineering"] --> D["Split Dataset"]
    D["Split Dataset"] --> E["Train Model"]
    E["Train Model"] --> F["Evaluate"]
    F["Evaluate"] --> G["Deploy"]

flowchart LR
    A["Import Data"] --> B["Clean Data"]
    B["Clean Data"] --> C["Feature Engineering"]
    C["Feature Engineering"] --> D["Split Dataset"]
    D["Split Dataset"] --> E["Train Model"]
    E["Train Model"] --> F["Evaluate"]
    F["Evaluate"] --> G["Deploy"]

The diagram above was generated by that code when this book was rendered — change the steps vector and the flowchart changes with it. That’s the point: a diagram derived from data or configuration can’t drift out of sync with the thing it documents, which is the usual fate of a hand-drawn one.

9.4.6 Best Practices for Diagrams in Data Science

1. Keep it simple: Focus on clarity over complexity
2. Maintain consistency: Use similar styles and conventions across diagrams
3. Align with text: Ensure your diagrams complement your written explanations
4. Consider the audience: Technical diagrams for peers, simplified ones for stakeholders
5. Update diagrams with code: Treat diagrams as living documents that evolve with your project

A diagram should carry weight the prose would otherwise have to. If it needs its own explanation, it isn’t earning its place.

9.5 Conclusion

Visualisation is the layer where your analysis meets its audience. The libraries covered here (matplotlib, seaborn, plotly, ggplot2) are the workhorses of static and lightly-interactive charts you embed in reports. Mermaid handles the architecture and workflow diagrams that explain how your system fits together rather than what the data looks like.

By the end of this chapter you should have:

• A plotting library installed and working in each language you use — matplotlib or plotly for Python, ggplot2 for R
• One chart saved to a file at a resolution suitable for a report, rather than only ever viewed on screen
• One interactive chart embedded in a rendered HTML document, with hover and zoom working
• A Mermaid diagram rendering inside a Quarto document, so you can draw a pipeline without opening a drawing tool

As your datasets and models grow, you’ll eventually hit the limits of what your laptop can render or compute. The next chapter steps away from the document and visualisation layer to look at cloud platforms: where to run notebooks and training jobs when local hardware isn’t enough, and how to do so without accidentally running up a bill. After that, we return to the “how do I put this in front of a stakeholder” question with web development and interactive dashboards.