Appendix A — Utility Tools for Data Scientists

A.1 Utility Tools for Data Scientists

While programming languages, libraries, and frameworks form the core of your data science toolkit, a handful of smaller utilities remove real friction from day-to-day work. This appendix is a catalogue rather than a curriculum: nothing in the main chapters depends on any of it, and you should adopt items from it when you notice a problem they solve, not because they’re listed here.

A.1.1 AI Coding Assistants

Large-language-model-based coding assistants are probably the single biggest productivity change in data science tooling since this book was first drafted. For readers without a CS background they’re especially useful: they can scaffold boilerplate, explain unfamiliar code, translate between Python and R, suggest fixes for cryptic error messages, and draft regex patterns or SQL queries on request. The caveat is universal: they confidently produce wrong answers too, so treat suggestions as a fast draft to be reviewed, not an oracle.

A few options worth knowing. Pricing differs between them, so the cost note matters as much as the description:

GitHub Copilot: In-editor completion and chat inside VS Code, JetBrains IDEs, Neovim, and the web. The most mature and widely used option. Has a genuine free tier with a monthly allowance of completions and chats, and is free outright for verified students, educators, and maintainers of popular open-source projects.
Claude for VS Code and Claude Code: Anthropic’s CLI and IDE integrations. Claude Code runs in a terminal and can read, edit, and run code in a project with your supervision. Particularly good for multi-file refactors. Requires a paid Claude subscription or API credits — there is no free tier.
Cursor: A fork of VS Code with deep LLM integration throughout the editor. Popular with developers who want a single tool rather than an extension. Free hobby tier, paid beyond it.
Aider: An open-source terminal-based pair-programmer that uses git commits as its unit of work. A great fit if you already live in the terminal. The tool itself is free; you supply an API key and pay the model provider for usage.

For genuinely throwaway tasks, the web chat interfaces to Claude and ChatGPT are fine. For anything you’ll actually check into a repository, prefer an IDE integration so the suggestions have full context of your code.

A few habits that make these tools more useful and less dangerous:

Read the diff. Never accept an edit without skimming what changed.
Don’t paste secrets. Anything you put into a prompt may end up in provider logs. Strip API keys, connection strings, and personal data before sharing code.
Ask for explanations alongside code. “Write this function and explain why you chose this approach” is a much better prompt than “write this function”, because you learn something and can spot mistakes faster.
Run the code. LLMs hallucinate function signatures, flag names, and library behaviours. Trust the interpreter over the chatbot.

A.1.2 Text Editors and IDE Enhancements

Editors and Version Control covers Visual Studio Code (VS Code) as the primary recommended editor for data science across Python, R, SQL, and notebooks. It’s free, cross-platform, scriptable, and has the largest extension ecosystem of any editor. For most readers, VS Code plus the extensions listed in that chapter will cover 90% of your needs.

The editors below are lightweight alternatives for quick edits, very large files, or platform-specific situations.

A.1.2.1 Notepad++

Notepad++ is a free, open-source text editor for Windows, and a substantial step up from the built-in Notepad.

Key features for data scientists:

Syntax highlighting: Supports many languages including Python, R, SQL, JSON, and more
Column editing: Edit multiple lines simultaneously (useful for cleaning data)
Regex search and replace: Search by pattern rather than by literal text — regular expressions, usually shortened to regex, are a compact notation for describing text shapes such as “a date” or “any line ending in a comma”
Macro recording: Automate repetitive text edits
Plugins: Extend functionality with viewers for JSON, HTML, Markdown, and other common data-science file formats.

Installation:

Download from notepad-plus-plus.org
Run the installer and follow the prompts

Useful shortcuts:

Ctrl+H: Find and replace
Alt+Shift+Arrow: Column selection mode
Ctrl+D: Duplicate current line
Ctrl+Shift+Up/Down: Move current line up/down

Notepad++ is useful for quickly viewing and editing text files, CSV extracts, or configuration files without launching a full IDE. It is not a large-file tool: it has a hard limit around 2 GB and becomes sluggish well before that. For a genuinely large CSV, read a sample in pandas or query it in place with DuckDB rather than trying to open it.

Notepad++ (as of writing) is not available on Mac, but the following alternative is.

A.1.2.2 Sublime Text

Sublime Text is a cross-platform editor known for opening very large files quickly — useful when a CSV is too big for anything else to load.

Key features for data scientists:

Multiple selections: Edit many places at once
Command palette: Quickly access commands without menus
Distraction-free mode: Focus on your text without UI elements
Splits and grids: View multiple files or parts of files simultaneously
Customisable key bindings: Create shortcuts tailored to your workflow

Installation:

Download from sublimetext.com
Install and activate (free evaluation with occasional purchase reminder)

Sublime Text’s speed suits manipulating text data, writing scripts, or making quick edits to code without launching a heavier IDE.

A.1.3 API Development and Testing Tools

APIs (Application Programming Interfaces) are crucial for accessing web services and databases. These tools help you test, debug, and document APIs.

A.1.3.1 Postman

Postman is the most widely used tool for API development and testing, and the one whose interface most tutorials assume.

Key features for data scientists:

Request building: Create and save HTTP requests
Collections: Organise and share API requests
Environment variables: Manage different settings (dev/prod)
Automated testing: Create test scripts to validate responses
Mock servers: Simulate API responses without a backend

Installation:

Download from postman.com
Create a free account to sync across devices

Example workflow:

Create a new request to a data API:

GET https://api.example.com/data?limit=100

Add authentication (if required):
```
Authorization: Bearer your_token_here
```
Send the request and analyse the JSON response
Save the request to a collection for future use

Postman earns its place when you’re working with data APIs — financial markets, weather services, or an internal company endpoint — and want to poke at the response before writing any code against it.

One caveat worth knowing before you commit: Postman now requires an account and syncs your collections to its cloud by default. For work under a client NDA, or anywhere requests carry credentials you’d rather not upload, check what your organisation’s policy says about that.

A.1.3.2 Bruno and Insomnia

Two alternatives, both a response to the same concern:

Bruno stores collections as plain text files in a folder you choose, which means they live in Git alongside the code that calls the API and diff like any other source file. Offline by default, no account required. If the cloud-sync caveat above matters to you, this is the one to look at.
Insomnia is lighter than Postman with good GraphQL support. Note that since Kong’s 2023 changes, using it without an account means working in Scratch Pad mode, which limits what you can save — check the current terms before assuming the free tier fits your workflow.

For occasional API work, any of the three is fine. The choice matters when collections need to be shared, version-controlled, or kept off someone else’s servers.

A.1.4 Database Management Tools

These tools provide graphical interfaces for working with databases, making it easier to explore and manipulate data.

A.1.4.1 DBeaver

DBeaver is a universal database tool that works with almost any database system.

Key features for data scientists:

Multi-database support: Works with PostgreSQL, MySQL, SQLite, Oracle, and more
Visual query builder: Create SQL queries without writing code
Data export/import: Move data between different formats and databases
ER diagrams: Visualise database structure
SQL editor: Write and execute queries with syntax highlighting

Installation:

Download from dbeaver.io
Run the installer

Example workflow:

Connect to a database with connection parameters
Browse tables and view structure

Use the SQL editor to write a query:

SELECT 
    product_category,
    COUNT(*) as count,
    AVG(price) as avg_price
FROM products
GROUP BY product_category
ORDER BY count DESC;

Export results to CSV for analysis in Python or R

DBeaver streamlines database interactions, allowing you to explore data structures, write queries, and export results without writing code to establish database connections.

A.1.4.2 pgAdmin

pgAdmin is a specialised tool for PostgreSQL databases.

Key features:

PostgreSQL-specific features: Optimised for PostgreSQL
Server monitoring: View database performance
Backup and restore: Manage database backups
User management: Control access to databases
Procedural language debugging: Test stored procedures

Installation:

Download from pgadmin.org
Run the installer

For data scientists working specifically with PostgreSQL databases, pgAdmin provides specialised features that generic tools may lack.

A.1.5 File Comparison and Merging Tools

These tools help identify differences between files and directories, which is useful for comparing datasets or code versions.

A.1.5.1 Beyond Compare

Beyond Compare compares files and whole directory trees side by side, and handles more than plain text.

Key features for data scientists:

Text comparison: View differences between text files line by line
Table comparison: Compare CSV and Excel files with data-aware features
Directory sync: Compare and synchronise folders
3-way merge: Resolve conflicts between different versions
Byte-level comparison: Analyse binary files

Installation:

Download from scootersoftware.com
Install (trial version available)

Example data science use case: Comparing two versions of a dataset to identify changes:

Open two CSV files in Table Compare mode
Automatically align columns by name
Identify added, removed, or modified rows
Export the differences to a new file

Beyond Compare is particularly valuable when dealing with evolving datasets, where you need to understand what changed between versions.

A.1.5.2 WinMerge

WinMerge is a free, open-source alternative for file and folder comparison.

Key features:

Visual text comparison: Side-by-side differences with highlighting
Folder comparison: Compare directory structures
Image comparison: Visual diff for images
Plugins: Extend functionality for additional file types
Integration: Works with source control systems

Installation:

Download from winmerge.org
Run the installer

WinMerge covers basic comparison needs for free, though it lacks some of the features of commercial alternatives.

A.1.6 Terminal Enhancements

A better-configured shell pays for itself quickly once you’re in one every day.

A.1.6.1 Oh My Zsh

Oh My Zsh is a framework for managing your Zsh configuration, providing themes and plugins for the Z shell.

Key features for data scientists:

Tab completion: Intelligent completion for commands and paths
Git integration: Visual indicators of repository status
Syntax highlighting: Colour-coded command syntax
Command history: Improved search through previous commands
Customisable themes: Visual enhancements for the terminal

Installation (macOS or Linux):

# Install Zsh first if needed
# Ubuntu/Debian:
# sudo apt install zsh
# macOS (usually pre-installed)

# Set Zsh as default shell
chsh -s $(which zsh)

# Install Oh My Zsh. It's good practice to review the installer script
# before executing anything you download from the internet:
curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh -o install-ohmyzsh.sh
less install-ohmyzsh.sh   # read it, check for anything suspicious
sh install-ohmyzsh.sh

Many sites still show a one-liner sh -c "$(curl ... install.sh)", which downloads and immediately executes code from a third-party server. The two-step version above takes five extra seconds and lets you actually see what’s about to run on your machine.

Useful plugins for data scientists:

# Edit ~/.zshrc to activate plugins
plugins=(git python pip conda docker)

Only name plugins that exist — Oh My Zsh prints plugin 'x' not found in every new shell otherwise, and there is no jupyter plugin despite several tutorials listing one. The plugins directory is the authoritative list.

Oh My Zsh makes the command line more user-friendly and efficient, which is valuable when working with data processing tools, running scripts, or managing environments.

A.1.6.2 Windows Terminal and PowerShell 7

On Windows, Windows Terminal (pre-installed on Windows 11 and free from the Microsoft Store on Windows 10) is a huge upgrade over the legacy cmd.exe and the old PowerShell window: tabs, split panes, GPU-accelerated rendering, and sensible keyboard shortcuts. Pair it with PowerShell 7 (winget install Microsoft.PowerShell), which is cross-platform and much more pleasant to script than the built-in Windows PowerShell 5.1. If you want a Unix-like experience on Windows, use WSL (covered in The Command Line) from inside Windows Terminal, which handles both worlds side-by-side.

A.1.6.3 Modern Rust-based CLI replacements

A family of newer command-line tools, mostly written in Rust, have become the de-facto replacements for the standard Unix utilities. They’re all faster than the originals and, more importantly for this book’s audience, tend to have friendlier defaults and output. Every one is installable with Homebrew, apt, winget, choco, or cargo:

Instead of	Use	What you get
`grep`	ripgrep (`rg`)	Much faster recursive search; respects `.gitignore` by default
`find`	fd	Saner syntax, colour output, skips `.git` and anything in your `.gitignore`
`cat`	bat	Syntax highlighting and line numbers
`ls`	eza	Git status, icons, tree view
`cd`	zoxide	`z projectname` jumps to any directory you’ve visited
`htop`	btop	Prettier resource monitor
Bash/Zsh prompt	starship	Fast, configurable, works in any shell including PowerShell

For Python and R tooling specifically, three newer tools are now standard enough to mention:

uv: a Python package manager and virtual-environment tool written in Rust, dramatically quicker than pip at resolving and installing. It’s a drop-in replacement for pip, pip-tools, virtualenv, and pyenv. Create an environment with uv venv, install from a requirements.txt with uv pip install -r requirements.txt, and run project scripts with uv run.
ruff: a Python linter and formatter, also from Astral, fast enough to run on every keystroke where the tools it replaces (flake8, isort, black) were not. Modern Python projects almost always use it.
pixi: a project-based package manager built on the conda ecosystem. Think of it as “uv for conda”: fast, lockfile-driven, cross-platform reproducible environments, especially useful for projects that mix Python with R, C++, CUDA, or system libraries.

None of these are required to follow this book. The classic pip/conda/grep/find commands still work fine. But once you’ve tried them it’s hard to go back.

A.1.7 Data Wrangling Tools

These specialised tools help with specific data manipulation tasks that complement programming languages.

A.1.7.1 CSVKit

CSVKit is a suite of command-line tools for working with CSV files.

Key features for data scientists:

csvstat: Generate descriptive statistics on CSV files
csvcut: Extract specific columns
csvgrep: Filter rows based on patterns
csvsort: Sort CSV files
csvjoin: SQL-like join operations between CSV files

Installation:

pip install csvkit

Example commands:

# View basic statistics of a CSV file
csvstat data.csv

# Extract specific columns
csvcut -c 1,3,5 data.csv > extracted.csv

# Filter rows containing a pattern
csvgrep -c 2 -m "Pattern" data.csv > filtered.csv

# Sort by a column
csvsort -c 3 data.csv > sorted.csv

CSVKit earns its place for quick exploration directly from the command line — checking what’s in a file, or pulling two columns out of it, without opening an editor and writing a script.

A.1.7.2 jq

jq is a lightweight command-line JSON processor that helps manipulate JSON data.

Key features:

Filtering: Extract specific data from complex JSON
Transformation: Reshape JSON structures
Combination: Merge multiple JSON sources
Computation: Perform calculations on numeric values
Formatting: Pretty-print and compact JSON

Installation:

# macOS
brew install jq

# Ubuntu/Debian
sudo apt install jq

# Windows (with Chocolatey)
choco install jq

Example commands:

# Pretty-print JSON
cat data.json | jq '.'

# Extract specific fields
cat data.json | jq '.results[] | {name, value}'

# Filter based on a condition
cat data.json | jq '.results[] | select(.value > 100)'

# Calculate statistics
cat data.json | jq '[.results[].value] | {count: length, sum: add, average: add/length}'

jq is invaluable when working with APIs that return JSON data or when preparing JSON data for visualisation or further analysis.

A.1.8 Diagramming and Visualisation Tools

Code-based diagrams version well, but sometimes a drawing tool is the faster route to a one-off architecture sketch.

A.1.8.1 diagrams.net (formerly draw.io)

diagrams.net is a free, open-source online diagramming tool that works with various diagram types.

Key features for data scientists:

Flowcharts: Document data pipelines and workflows
ER diagrams: Model database relationships
Network diagrams: Visualise system architecture
Multiple export formats: PNG, SVG, PDF, etc.
Integration: Works with Google Drive, Dropbox, etc.

Access:

Go to app.diagrams.net in your browser
Choose where to save your diagrams (local, Google Drive, etc.)

Example data science use case: Creating a data flow diagram to document an ETL process — extract, transform, load, the standard name for a pipeline that pulls data from a source, reshapes it, and writes it somewhere queryable:

Select the flowchart template
Add data sources, transformation steps, and outputs
Connect components with arrows showing data flow
Add annotations explaining transformations
Export as PNG for inclusion in documentation

Clear diagrams are essential for communicating complex data processing workflows to stakeholders or documenting them for future reference.

A.1.8.2 Graphviz

Graphviz is a command-line tool for creating structured diagrams from text descriptions.

Key features:

Programmatic diagrams: Generate diagrams from code
Automatic layout: Optimal arrangement of elements
Various diagram types: Directed graphs, hierarchies, networks
Integration: Works with Python, R, and other languages
Scriptable: Automate diagram generation

Installation:

# macOS
brew install graphviz

# Ubuntu/Debian
sudo apt install graphviz

# Windows (with Chocolatey)
choco install graphviz

Example DOT file (graph.dot):

digraph DataPipeline {
  rankdir=LR;
  
  raw_data [label="Raw Data"];
  cleaning [label="Data Cleaning"];
  features [label="Feature Engineering"];
  modeling [label="Model Training"];
  evaluation [label="Evaluation"];
  deployment [label="Deployment"];
  
  raw_data -> cleaning;
  cleaning -> features;
  features -> modeling;
  modeling -> evaluation;
  evaluation -> deployment;
  evaluation -> features [label="Iterate", style="dashed"];
}

Generate the diagram:

dot -Tpng graph.dot -o pipeline.png

Graphviz is particularly useful for generating diagrams programmatically as part of automated documentation processes or for visualising complex relationships that would be tedious to draw manually.

A.1.9 Screenshot and Recording Tools

These tools help create visual documentation and tutorials.

A.1.9.1 Your operating system’s built-in tool

Screenshot utilities were once worth installing separately. They mostly aren’t any more — every current OS ships one that annotates:

Windows: Snipping Tool (Win+Shift+S for a region capture straight to the clipboard). Annotates, and records short screen videos.
macOS: Cmd+Shift+4 for a region, Cmd+Shift+5 for the capture-and-record panel. Markup opens from the thumbnail that appears after a capture.
Linux: varies by desktop — GNOME’s Screenshot and KDE’s Spectacle both cover region capture and annotation.

You’ll still see older guides recommending Greenshot for this on Windows. It works, but its last stable release was in 2017 and the built-in tool now does the same job.

For anything beyond a static image — a bug you can only demonstrate by reproducing it — record a short clip instead. Screenshots of error messages are a common source of confusion in bug reports: paste the text.

A.1.9.2 OBS Studio

OBS (Open Broadcaster Software) Studio records your screen and streams it, free and on every platform.

Key features:

High-quality recording: Capture screen activity with audio
Multiple sources: Record specific windows or regions
Scene composition: Create layouts combining different sources
Flexible output: Record to file or stream online
Cross-platform: Available for Windows, macOS, and Linux

Installation:

Download from obsproject.com
Run the installer

OBS suits tutorial videos, recorded presentations, or documenting an analysis process for training purposes.

A.1.10 Productivity and Note-Taking Tools

These tools help organise your thinking, document your work, and manage your projects.

A.1.10.1 Obsidian

Obsidian is a knowledge base and note-taking application that works on Markdown files.

Key features for data scientists:

Markdown format: Write notes with the same syntax used in Jupyter notebooks
Bidirectional linking: Connect related notes
Graph view: Visualise relationships between notes
Local storage: Files stored on your computer, not in the cloud
Extensible: Plugins for additional functionality

Installation:

Download from obsidian.md
Run the installer

Example data science use case: Creating a personal knowledge base for your data science projects:

Create notes for each project with objectives and findings
Link to related techniques and concepts
Embed code snippets and results
Use tags to categorise by domain or technology
Visualise the connections in your knowledge with the graph view

Obsidian helps capture the thought process behind your data science work, creating a valuable reference for future projects.

A.1.10.2 Notion

Notion is an all-in-one workspace that combines notes, tasks, databases, and more.

Key features:

Rich content: Mix text, code, embeds, and databases
Templates: Pre-built layouts for different use cases
Collaboration: Share and work together with others
Web-based: Access from any device
Integration: Connect with other tools and services

Installation:

Sign up at notion.so
Download desktop and mobile apps if desired

Notion is particularly useful for team-based data science projects, where you need to coordinate tasks, share documentation, and track progress in one place.

A.1.11 Finding Things

Searching across files is common enough in analytical work — where is this variable used, which script produced this column — that it’s worth having a fast tool for it.

ripgrep (rg) is the answer on every platform. It’s already listed in the command-line tools section above; this is just to say that it also replaces the category of dedicated desktop search applications you may have used on Windows. It respects .gitignore by default, so it won’t waste time searching your .venv or your raw data.

# Every occurrence of a column name across a project
rg "customer_lifetime_value"

# Only in Python files, showing three lines of context
rg -t py -C 3 "def build_features"

# Files that mention it, without the matching lines
rg -l "POPIA"

VS Code’s search panel (Ctrl+Shift+F) uses ripgrep underneath, so if you prefer a graphical interface you already have one with the same speed and the same .gitignore behaviour.

For finding files rather than text inside them, fd is the equivalent — also listed above.

A.2 Conclusion

The tools in this appendix are optional by definition. Nothing in the main chapters depends on any of them, and adopting all of them at once would be a poor use of a week.

The productive approach is to add one when you notice friction. If you keep opening a CSV in Excel to look at it, install a better editor. If you’re testing an API by writing throwaway scripts, install Bruno. If you’re taking notes on projects in a text file you can’t search, try Obsidian.

Two of these are worth adopting sooner rather than later regardless of whether you’ve noticed the friction:

An AI coding assistant, because the productivity difference is large and the learning curve is short.
ripgrep, because searching a codebase is something you do many times a day once you have a tool fast enough to make it worth doing.

Everything else can wait until you want it.