irishbuoys

The irishbuoys package provides tools to download, process, and analyze data from the Irish Weather Buoy Network. It includes functions for accessing real-time and historical data via the Marine Institute’s ERDDAP server, storing data in DuckDB for efficient querying, and building predictive models for wave height and weather conditions.

Installation

Standard R

You can install the development version of irishbuoys from GitHub:

# install.packages("remotes")
# remotes::install_github("johngavin/irishbuoys")
install.packages("irishbuoys", repos = c('https://johngavin.r-universe.dev', 'https://cloud.r-project.org'))

Nix (Recommended)

For a reproducible development environment using Nix:

# Clone the repository
git clone https://github.com/johngavin/irishbuoys.git
cd irishbuoys

# Enter Nix shell (first time may take a while, subsequent entries are fast)
./default.sh

The default.sh script automatically: - Generates default.nix from DESCRIPTION if needed (via default.R) - Creates a GC root to prevent garbage collection - Enters pure mode for reproducibility

Pure Mode

This project enforces Nix --pure mode to guarantee:

Reproducibility: Only Nix-provided tools are available
Security: No accidental use of system tools with different versions
Consistency: Same environment locally and in CI

Entering and verifying pure mode

# Recommended: Use default.sh (pure mode enforced automatically)
./default.sh
# Output shows: 🔒 SECURITY: Running in --pure mode

# Or manually with pure flag:
nix-shell --pure default.nix

# Verify pure mode
echo $IN_NIX_SHELL  # Expected: pure
which R             # Expected: /nix/store/...

Passing environment variables:

nix-shell --pure --keep GITHUB_TOKEN --keep MY_API_KEY default.nix

Using with rix

To integrate this package into your own Nix environment:

library(rix)

rix(
  r_ver = "4.5.2",  # Must match project's R version
  r_pkgs = c(
    # Core dependencies (from Imports)
    "arrow", "cli", "dbplyr", "DBI", "dplyr", "duckdb",
    "glue", "httr2", "jsonlite", "lubridate", "pointblank",
    "purrr", "rlang", "tibble",
    # Visualization (from Suggests)
    "plotly", "ggplot2", "dygraphs", "DT"
  ),
  git_pkgs = list(
    list(
      package_name = "irishbuoys",
      repo_url = "https://github.com/johngavin/irishbuoys",
      commit = "main"  # Use specific SHA for reproducibility
    )
  ),
  ide = "other",
  project_path = "."
)

Cachix Binary Cache (Faster Builds)

For Other Machines

Use the pre-built R packages from rstats-on-nix Cachix cache for much faster builds:

# Install cachix (one-time setup)
nix-shell -p cachix --run "cachix use rstats-on-nix"

# Now nix-shell will download pre-built packages instead of compiling
cd irishbuoys
./default.sh  # Much faster with cache!

Two-Tier Caching Strategy

This project uses a two-tier Cachix strategy:

Priority	Cache	Contains
1st	`rstats-on-nix`	All standard R packages (public, pre-built)
2nd	`johngavin`	Project-specific custom packages only

Important: Standard R packages (dplyr, targets, etc.) are ALL available from rstats-on-nix. The johngavin cache is only for custom packages not in rstats-on-nix.

For irishbuoys: - All dependencies come from rstats-on-nix - The irishbuoys package itself is loaded via pkgload::load_all() (development mode) - Nothing needs to be pushed to johngavin cache

GitHub Actions CI

CI workflows automatically use both caches:

# In .github/workflows/r-cmd-check.yaml
- uses: cachix/cachix-action@v15
  with:
    name: rstats-on-nix  # Public cache FIRST

- uses: cachix/cachix-action@v15
  with:
    name: johngavin      # Project cache SECOND
    authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
    skipPush: true       # Don't push during checks

Documentation

Comprehensive vignettes on the package website:

Dashboards

Static Dashboard - Interactive visualization of Irish Weather Buoy Network data.

Real-time station data: Max Wave, Signif Wave, and Wind Speed by station
Time series analysis: Hourly measurements with interactive plotly charts
Extreme event detection: Rogue wave identification (Max Wave > 2× Signif Wave)

Analysis

Wave Analysis - Scientific analysis of wave patterns and extreme value statistics.

Key metrics dashboard: Station statistics, date coverage, and data quality summary
Extreme value modeling: GEV/GPD fits for return period estimation
Joint distribution analysis: Cross-correlations, copula tail dependence

Data Quality

Interactive validation reports via pointblank:

Analysis Data Validation: Column checks, physical bounds (wave 0-30m, wind 0-100 m/s)
Rogue Wave Validation: Ensures Hmax > 2 × Hs criterion

Telemetry

Pipeline Telemetry - Pipeline metrics and performance tracking.

Data coverage metrics: Completeness by station and time period
Pipeline performance: Target execution times and dependencies

Quick Start

Download Recent Data

library(irishbuoys)

# Download last 7 days of data
# Show structure of first 3 rows for efficiency
data <- download_buoy_data(
  start_date = Sys.Date() - 7,
  end_date = Sys.Date()
)
str(head(data, 3))

# Get data for specific station
m3_data <- download_buoy_data(
  stations = "M3",
  start_date = "2024-01-01"
)
dim(m3_data)

# Get earliest available data (buoy network started 2001-02-05)
# Show structure of first 3 rows for efficiency
waves <- download_buoy_data(
  start_date = "2001-02-05",
  end_date = "2001-02-06"
)
str(head(waves, 3))

Initialize and Query Database

# Initialize database with historical data (chunk_days=365 for faster downloads)
initialize_database(
  start_date = "2024-01-01",  # Default: recent data for quick start
  end_date = Sys.Date(),
  chunk_days = 365  # Download in 1-year chunks for efficiency
)

# Check database statistics immediately after initialization
stats <- get_database_stats()

# Connect to database
con <- connect_duckdb()

# Check QC flag distribution by station FIRST
# qc_flag: 0=unknown, 1=good, 9=missing
qc_tally <- tbl(con, "buoy_data") |>
  group_by(station_id, qc_flag) |>
  summarise(n = n(), .groups = "drop") |>
  collect() |>
  tidyr::pivot_wider(names_from = qc_flag, values_from = n, names_prefix = "qc_")
print(qc_tally)

# Query wave data (qc_filter=FALSE returns all data; TRUE filters for qc_flag==1)
wave_data <- query_buoy_data(
  con,
  stations = c("M3", "M4"),
  variables = c("time", "station_id", "wave_height", "wave_period"),
  start_date = "2024-01-01",
  qc_filter = FALSE  # Set TRUE to filter for qc_flag==1 only
)

# Custom SQL query: Find top 10 most extreme rogue waves
# Ordered by hmax (highest individual wave) because "extreme" = largest waves
extreme_waves <- query_buoy_data(
  con,
  sql_query = "
    SELECT station_id, time, wave_height, hmax
    FROM buoy_data
    WHERE hmax > 2 * wave_height
      AND wave_height > 0
      AND qc_flag = 1
    ORDER BY hmax DESC
    LIMIT 10
  "
)

Tidyverse Alternative (duckplyr)

The same query using dplyr verbs with duckplyr backend:

# Tidyverse alternative using duckplyr
# Same query as SQL above, ordered by hmax (highest waves first)
library(dplyr)
library(duckplyr)

extreme_waves_tidy <- tbl(con, "buoy_data") |>
  filter(
    hmax > 2 * wave_height,
    wave_height > 0,
    qc_flag == 1
  ) |>
  select(station_id, time, wave_height, hmax) |>
  arrange(desc(hmax)) |>
  head(10) |>
  collect()

Why duckplyr? - Familiar tidyverse syntax - Lazy evaluation - query runs only on collect() - Automatic SQL translation for performance - Works with any DBI connection

# Don't forget to disconnect
DBI::dbDisconnect(con)

Incremental Updates

# Perform incremental update (for scheduled jobs)
result <- incremental_update()
print(result$summary)

Data Dictionary

# Get complete data dictionary (returns tibble)
dict <- get_data_dictionary()
print(dict)

# Get detailed documentation for specific variable
(wave_docs <- get_variable_docs("WaveHeight"))

# Merge dictionary with database column info
# Useful for creating documentation or understanding data
library(dplyr)
db_cols <- tibble(
  variable = c("wave_height", "hmax", "wind_speed", "gust"),
  db_column = c("wave_height", "hmax", "wind_speed", "gust")
)
dict |>
  filter(tolower(variable) %in% db_cols$variable |
         variable %in% c("WaveHeight", "Hmax", "WindSpeed", "Gust")) |>
  select(variable, units, description)

Data Source

Data is sourced from the Marine Institute’s ERDDAP server, which provides real-time and historical measurements from the Irish Weather Buoy Network.

Available Stations

M2: Southwest of Ireland
M3: Southwest of Ireland
M4: Southeast of Ireland
M5: West of Ireland
M6: Northwest of Ireland
M1: Historical data (decommissioned)
FS1: Historical data

Measured Parameters

Meteorological: Air temperature, pressure, humidity, wind speed/direction
Oceanographic: Wave height/period/direction, sea temperature, salinity
Quality: QC flags for data validation

Project Structure

Click to expand project tree

.
├── DESCRIPTION
├── LICENSE
├── LICENSE.md
├── NAMESPACE
├── R
│   ├── data_dictionary.R
│   ├── database.R
│   ├── database_parquet.R
│   ├── dev
│   │   ├── check_data_gaps.R
│   │   ├── generate_dashboard_data.R
│   │   └── issues
│   ├── email_summary.R
│   ├── erddap_client.R
│   ├── extreme_values.R
│   ├── irishbuoys-package.R
│   ├── joint_analysis.R
│   ├── plot_functions.R
│   ├── plotly_helpers.R
│   ├── rogue_waves.R
│   ├── tar_plans
│   │   ├── plan_dashboard.R
│   │   ├── plan_dashboard_captions.R
│   │   ├── plan_data_acquisition.R
│   │   ├── plan_doc_examples.R
│   │   ├── plan_evidence.R
│   │   ├── plan_joint_analysis.R
│   │   ├── plan_pkgctx.R
│   │   ├── plan_quality_control.R
│   │   ├── plan_telemetry.R
│   │   ├── plan_vignettes.R
│   │   └── plan_wave_analysis.R
│   ├── trend_analysis.R
│   ├── update.R
│   ├── validation.R
│   ├── wave_model.R
│   └── wave_science.R
├── README.md
├── README.qmd
├── README.rmarkdown
├── _extensions
│   └── quarto-ext
│       └── shinylive
├── _output
│   ├── shinylive-sw.js
│   └── vignettes
│       ├── dashboard_shinylive.html
│       ├── dashboard_shinylive_files
│       └── data
├── _pkgdown.yml
├── _quarto.yml
├── _targets.R
├── _targets.yaml
├── data-raw
├── default.R
├── default.nix
├── default.sh
├── inst
│   ├── docs
│   │   └── parquet_storage_guide.md
│   ├── extdata
│   │   ├── analysis_questions.md
│   │   ├── ctx
│   │   ├── dashboard_buoy_data.rds
│   │   ├── dashboard_stats.rds
│   │   ├── dashboard_timeseries.rds
│   │   ├── irish_buoys.duckdb
│   │   ├── return_levels.rds
│   │   ├── rogue_wave_events.rds
│   │   ├── seasonal_analysis.rds
│   │   └── wave_analysis_summary.rds
│   └── scripts
│       ├── example_usage.R
│       └── storage_comparison.R
├── man
│   ├── add_wave_metrics.Rd
│   ├── analyze_gust_factor.Rd
│   ├── analyze_joint_extremes.Rd
│   ├── analyze_parquet_storage.Rd
│   ├── analyze_rogue_statistics.Rd
│   ├── analyze_station_pairs.Rd
│   ├── buoy_tbl.Rd
│   ├── calculate_annual_trends.Rd
│   ├── calculate_hs_from_elevation.Rd
│   ├── calculate_return_levels.Rd
│   ├── calculate_rms_wave_height.Rd
│   ├── calculate_seasonal_means.Rd
│   ├── calculate_wave_steepness.Rd
│   ├── compare_rogue_wave_gust.Rd
│   ├── connect_duckdb.Rd
│   ├── convert_duckdb_to_parquet.Rd
│   ├── create_buoy_schema.Rd
│   ├── create_email_summary.Rd
│   ├── create_plot_annual_trends.Rd
│   ├── create_plot_gust_by_category.Rd
│   ├── create_plot_gusts_vs_waves.Rd
│   ├── create_plot_monthly_wave.Rd
│   ├── create_plot_monthly_wind.Rd
│   ├── create_plot_return_levels.Rd
│   ├── create_plot_rogue_all.Rd
│   ├── create_plot_rogue_by_station.Rd
│   ├── create_plot_rogue_gusts.Rd
│   ├── create_plot_rogue_gusts_all.Rd
│   ├── create_plot_rogue_gusts_by_station.Rd
│   ├── create_plot_stl.Rd
│   ├── create_plot_time_of_day.Rd
│   ├── create_plot_week_of_year.Rd
│   ├── create_plot_wind_beaufort.Rd
│   ├── create_return_level_plot_data.Rd
│   ├── create_validation_summary.Rd
│   ├── cross_correlation_stations.Rd
│   ├── decompose_stl.Rd
│   ├── detect_anomalies.Rd
│   ├── detect_rogue_waves.Rd
│   ├── download_buoy_data.Rd
│   ├── evaluate_wave_model.Rd
│   ├── explain_hourly_averaging.Rd
│   ├── explain_hs_formula.Rd
│   ├── explain_measurement_period.Rd
│   ├── explain_wave_height_measurement.Rd
│   ├── extreme_values.Rd
│   ├── fit_bivariate_copula.Rd
│   ├── fit_gev_annual_maxima.Rd
│   ├── fit_gpd_threshold.Rd
│   ├── generate_and_send_summary.Rd
│   ├── generate_validation_reports.Rd
│   ├── generate_weekly_summary.Rd
│   ├── get_data_dictionary.Rd
│   ├── get_database_stats.Rd
│   ├── get_latest_timestamp.Rd
│   ├── get_station_info.Rd
│   ├── get_stations.Rd
│   ├── get_variable_docs.Rd
│   ├── haversine_distance.Rd
│   ├── hs_from_rms.Rd
│   ├── incremental_update.Rd
│   ├── incremental_update_parquet.Rd
│   ├── init_parquet_storage.Rd
│   ├── initialize_database.Rd
│   ├── irishbuoys-package.Rd
│   ├── irishbuoys_ggplotly.Rd
│   ├── irishbuoys_layout.Rd
│   ├── joint_analysis.Rd
│   ├── joint_analysis_summary.Rd
│   ├── load_to_duckdb.Rd
│   ├── log_update.Rd
│   ├── plot_functions.Rd
│   ├── predict_station_lagged.Rd
│   ├── predict_wave_height.Rd
│   ├── prepare_wave_features.Rd
│   ├── query_buoy_data.Rd
│   ├── query_parquet.Rd
│   ├── rogue_wave_report.Rd
│   ├── save_to_parquet.Rd
│   ├── station_distance_matrix.Rd
│   ├── train_wave_model.Rd
│   ├── trend_analysis.Rd
│   ├── trend_summary_report.Rd
│   ├── update_station_metadata.Rd
│   ├── validate_buoy_data.Rd
│   ├── validate_rogue_events.Rd
│   ├── validation.Rd
│   ├── wave_glossary.Rd
│   ├── wave_model.Rd
│   ├── wave_model_report.Rd
│   ├── wave_science.Rd
│   └── wave_science_documentation.Rd
├── pkgdown
│   └── extra.css
├── plans
│   └── PLAN_telemetry_overhaul.md
├── push_to_cachix.sh
├── tests
│   ├── testthat
│   │   ├── _snaps
│   │   ├── test-data-consistency.R
│   │   ├── test-defensive-programming.R
│   │   └── test-doc-dependencies.R
│   └── testthat.R
└── vignettes
    ├── _targets.yaml
    ├── custom.scss
    ├── dashboard_shinylive.qmd
    ├── dashboard_shinylive_files
    │   └── mediabag
    ├── dashboard_static.qmd
    ├── data
    │   ├── buoy_data.json
    │   ├── buoy_data.parquet
    │   ├── buoy_data_raw.csv
    │   └── stations.json
    ├── debug.qmd
    ├── debug_fixed_files
    ├── shinylive-sw.js
    ├── telemetry.qmd
    └── wave_analysis.qmd

Note: _targets/ (pipeline cache) and docs/ (generated site) excluded for clarity.

Key Features

Efficient Data Storage: Uses DuckDB for fast querying of large datasets
Incremental Updates: Smart updating to only download new data
Quality Control: Built-in filtering for data quality
Rogue Wave Detection: Identify extreme wave events (Hmax > 2 × Hs)
Comprehensive Documentation: Full data dictionary with scientific definitions

Use Cases

Marine safety and operations planning
Climate and oceanographic research
Extreme event analysis
Wave energy resource assessment
Weather forecast validation

Contributing, License & Acknowledgments

Contributing: Contributions are welcome! Please feel free to submit a Pull Request.

License: MIT License. See LICENSE file for details.

Acknowledgments: Data provided by the Marine Institute Ireland in collaboration with Met Éireann and the UK Met Office.

Sources:

*Last updated: 2026-02-15 13:19 UTC *