Using the Copernicus Seasonal Forecast Tools package to create a hazard object#

Introduction#

The copernicus-seasonal-forecast-tools package was developed to manage seasonal forecast data from the Copernicus Climate Data Store (CDS) for the U-CLIMADAPT project. It offers comprehensive tools for downloading, processing, computing climate indices, and generating hazard objects based on seasonal forecast datasets, particularly Seasonal forecast daily and subdaily data on single levels. The package is tailored to integrate seamlessly with the CLIMADA, supporting climate risk assessment and the development of effective adaptation strategies.

Features:

Automated download of the high-dimensional seasonal forecasts data via the Copernicus API
Preprocessing of sub-daily forecast data into daily formats
Calculation of heat-related climate indices (e.g., heatwave days, tropical nights)
Conversion of processed indices into CLIMADA hazard objects ready for impact modelling
Flexible modular architecture to accommodate additional indices or updates to datasets

In this tutorial, you can see a simple example of how to retrieve and process data from Copernicus, calculate a heat-related index, and create a hazard object. For more detailed documentation and advanced examples, please visit the repository or the documentation.

Prerequisites:

CDS account and API key: Register at https://cds.climate.copernicus.eu
CDS API client installation: pip install cdsapi
CDS API configuration: Create a .cdsapirc file in your home directory with your API key and URL. For instructions, visit: https://cds.climate.copernicus.eu/how-to-api#install-the-cds-api-client
Dataset Terms and Conditions: After selecting the dataset to download, make sure to accept the terms and conditions on the corresponding dataset webpage in the CDS portal before running this notebook. Here, https://cds.climate.copernicus.eu/datasets/seasonal-original-single-levels?tab=download.

For more information, visit the comprehensive CDS API setup guide, which walks you through each step of the process. Once configured, you’ll be ready to explore and analyze seasonal forecast data.

Note: Ensure you have the necessary permissions and comply with the CDS data usage policies when using this package. You can view the terms and conditions at https://cds.climate.copernicus.eu/datasets/seasonal-original-single-levels?tab=download. You can find them at the bottom of the download page.

# Import packages

import warnings
import datetime as dt

warnings.filterwarnings("ignore")
from seasonal_forecast_tools import SeasonalForecast, ClimateIndex
from seasonal_forecast_tools.utils.coordinates_utils import bounding_box_from_countries
from seasonal_forecast_tools.utils.time_utils import month_name_to_number

Set up parameters#

To configure the package for working with Copernicus forecast data and converting it into a hazard object for CLIMADA, you will need to define several essential parameters. These settings are crucial as they specify the type of data to be retrieved, the format, the forecast period, and the geographical area of interest. These parameters influence how the forecast data is processed and transformed into a hazard object.

Below, we outline these parameters and use an example for the Tmax – Maximum Temperature index to demonstrate the seasonal forecast functionality.

To learn more about what these parameters entail and their significance, please refer to the documentation on the CDS webpage.

Overview of parameters#

index_metric: Defines the type of index to be calculated. There are currently 12 predefined options available, including temperature-based indices (Tmean – Mean Temperature, Tmin – Minimum Temperature, Tmax – Maximum Temperature), heat stress indicators (HIA – Heat Index Adjusted, HIS – Heat Index Simplified, HUM – Humidex, AT – Apparent Temperature, WBGT – Wet Bulb Globe Temperature (Simple)), and extreme event indices (HW – Heat Wave, TR – Tropical Nights, TX30 – Hot Days).

Heat Waves (“HW”):
If index_metric is set to ‘HW’ for heat wave calculations, additional parameters can be specified to fine-tune the heat wave detection:
- threshold: Temperature threshold above which days are considered part of a heat wave. Default is 27°C.
- min_duration: Minimum number of consecutive days above the threshold required to define a heat wave event. Default is 3 days.
- max_gap: Maximum allowable gap (in days) between two heat wave events to consider them as one single event. Default is 0 days.
Tropical Nights (“TR”):
If index_metric is set to ‘TR’ for tropical nights, an additional parameter can be specified to set the threshold:
- threshold: Nighttime temperature threshold, above which a night is considered “tropical.” Default is 20°C.
⚠️ Flexibility: Users can define and integrate their own indices into the pipeline to extend the analysis according to their specific needs.

format : Specifies the format of the data to be downloaded, “grib” or “netcdf”. Copernicus do NOT recommended netcdf format for operational workflows since conversion to netcdf is considered experimental. More information here.

originating_centre: Identifies the source of the data. A standard choice is “dwd” (German Weather Service), one of eight providers including ECMWF, UK Met Office, Météo France, CMCC, NCEP, JMA, and ECCC.

system: Refers to a specific model or configuration used for forecasts. In this script, the default value is “21,” which corresponds to the GCSF (German Climate Forecast System) version 2.1. More details can be found in the CDS documentation.

year_list: A list of years for which data should be downloaded and processed.

initiation_month: A list of the months in which the forecasts are initiated. Example: [“March”, “April”].

forecast_period: Specifies the months relative to the forecast’s initiation month for which the data is forecasted. Example: [“June”, “July”, “August”] indicates forecasts for these months. The maximum available is 7 months.

⚠️ Important: When an initiation month is in one year and the forecast period in the next, the system recognizes the forecast extends beyond the initial year. Data is retrieved based on the initiation month, with lead times covering the following year. The forecast is stored under the initiation year’s directory, ensuring consistency while spanning both years.

area_selection: This determines the geographical area for which the data should be downloaded. It can be set to

Global coverage:
- Use the predefined function bounding_box_global() to select the entire globe.
Custom geographical bounds (cardinal coordinates):
- Input explicit latitude/longitude limits (in EPSG:4326).
- bounds = bounding_box_from_cardinal_bounds(northern=49, eastern=20, southern=40, western=10)
Country codes (ISO alpha-3):
- Provide a list of ISO 3166-1 alpha-3 country codes (e.g., “DEU” for Germany, “CHE” for Switzerland). The bounding box is constructed as the union of all selected countries.See this wikipedia page for the country codes.
- bounds = bounding_box_from_countries([“CHE”, “DEU”])

overwrite: Boolean flag that, when set to True, forces the system to redownload and reprocess existing files.

# We define above parameters for an example on Tmax
index_metric = ClimateIndex.Tmax.name
data_format = "grib"  # 'grib' or 'netcdf'
originating_centre = "dwd"
system = "21"
forecast_period = [
    "December",
    "February",
]  # from December to February including January
year_list = [2022]
initiation_month = ["November"]
overwrite = False
bounds = bounding_box_from_countries(["URY"])

# Parameters for Heat Waves
hw_threshold = 27
hw_min_duration = 3
hw_max_gap = 0

# Parameters for Tropical Nights
threshold_tr = 20

# Describe the selected climate index and the associated input data
forecast = SeasonalForecast(
    index_metric=index_metric,
    year_list=year_list,
    forecast_period=forecast_period,
    initiation_month=initiation_month,
    bounds=bounds,
    data_format=data_format,
    originating_centre=originating_centre,
    system=system,
)

The variables required for your selected index will be printed below. This allows you to see which data will be accessed and helps estimate the data volume.

forecast.explain_index()

'Explanation for Maximum Temperature: Maximum Temperature: Tracks the highest temperature recorded over a specified period. Required variables: 2m_temperature'

Calculate a Hazard Object#

Then you can call forecast.process_and_save_hazards method to convert processed index from Copernicus forecast data into a hazard object.

Hazard Object Creation: The method processes seasonal forecast data for specified years and months, converting these into hazard objects. These objects encapsulate potential risks associated with specific weather events or conditions, such as Maximum Temperature (‘Tmax’) indicated in the parameters, over the forecast period.
Data Storage: The hazard data for each ensemble member of the forecast is saved as HDF5 files. These files are automatically stored in specific directories corresponding to each month and type of hazard. The file paths are printed below the processing steps. For example, “/SYSTEM_DIR/copernicus_data/seasonal_forecasts/dwd/sys21/2023/init03/valid06_08/hazard/TX30/TX30_boundsW4_S44_E11_N48.hdf5”. HDF5 is a versatile data model that efficiently stores large volumes of complex data.

Each file is specific to a particular month and hazard scenario (‘Tmax’ in this case) and covers all ensemble members for that forecast period, aiding in detailed risk analysis.

Completeness of Data Processing: Messages like ‘Completed processing for 2022-07. Data saved in…’ confirm the successful processing and storage of the hazard data for that period, ensuring that all requested data has been properly handled and stored.

from climada.hazard import Hazard

forecast.save_index_to_hazard()

{'2022_init11_valid12_02': PosixPath('/Users/daraya/climada/data/copernicus_data/seasonal_forecasts/dwd/sys21/2022/init11/valid12_02/hazard/Tmax/Tmax_boundsN-59_S-35_E-52_W-29.hdf5')}

You can always inspect the properties of the Hazard object or visualize its contents. Noted the date attribute uses serial date numbers (ordinal format), which is common in climate data. To convert these to standard datetime format, you can use datetime.datetime.fromordinal.

# Load the hazard and plot intensity for the selected grid
forecast.save_index_to_hazard()
initiation_month_str = f"{month_name_to_number(initiation_month[0]):02d}"
forecast_month_str = f"{forecast.valid_period_str[-2:]}"  # Last month in valid period
forecast_year = year_list[0]

path_to_hazard = forecast.get_pipeline_path(
    forecast_year, initiation_month_str, "hazard"
)
hazard = Hazard.from_hdf5(path_to_hazard)

# Access hazard attributes
print("Hazard attributes:")
print(" - Shape of intensity (time, gridpoint):", hazard.intensity.shape)
print(" - Centroids:", hazard.centroids.shape)
print(" - Units:", hazard.units)
print(" - event_id:", hazard.event_id)
print(" - frequency:", hazard.frequency)
print(" - min, max fraction:", hazard.fraction.min(), hazard.fraction.max())
print(" - Date:", hazard.date)
print("min, max fraction: ", hazard.fraction.min(), hazard.fraction.max())
print(" - event_name:"), hazard.event_name

Hazard attributes:
 - Shape of intensity (time, gridpoint): (150, 56)
 - Centroids: (7, 8)
 - Units: °C
 - event_id: [  1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17  18
  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36
  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54
  55  56  57  58  59  60  61  62  63  64  65  66  67  68  69  70  71  72
  73  74  75  76  77  78  79  80  81  82  83  84  85  86  87  88  89  90
  91  92  93  94  95  96  97  98  99 100 101 102 103 104 105 106 107 108
 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126
 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144
 145 146 147 148 149 150]
 - frequency: [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
 1. 1. 1. 1. 1. 1.]
 - min, max fraction: 0.0 0.0
 - Date: [738490 738521 738552 738490 738521 738552 738490 738521 738552 738490
 738521 738552 738490 738521 738552 738490 738521 738552 738490 738521
 738552 738490 738521 738552 738490 738521 738552 738490 738521 738552
 738490 738521 738552 738490 738521 738552 738490 738521 738552 738490
 738521 738552 738490 738521 738552 738490 738521 738552 738490 738521
 738552 738490 738521 738552 738490 738521 738552 738490 738521 738552
 738490 738521 738552 738490 738521 738552 738490 738521 738552 738490
 738521 738552 738490 738521 738552 738490 738521 738552 738490 738521
 738552 738490 738521 738552 738490 738521 738552 738490 738521 738552
 738490 738521 738552 738490 738521 738552 738490 738521 738552 738490
 738521 738552 738490 738521 738552 738490 738521 738552 738490 738521
 738552 738490 738521 738552 738490 738521 738552 738490 738521 738552
 738490 738521 738552 738490 738521 738552 738490 738521 738552 738490
 738521 738552 738490 738521 738552 738490 738521 738552 738490 738521
 738552 738490 738521 738552 738490 738521 738552 738490 738521 738552]
min, max fraction:  0.0 0.0
 - event_name:

(None,
 ['member0',
  'member0',
  'member0',
  'member1',
  'member1',
  'member1',
  'member2',
  'member2',
  'member2',
  'member3',
  'member3',
  'member3',
  'member4',
  'member4',
  'member4',
  'member5',
  'member5',
  'member5',
  'member6',
  'member6',
  'member6',
  'member7',
  'member7',
  'member7',
  'member8',
  'member8',
  'member8',
  'member9',
  'member9',
  'member9',
  'member10',
  'member10',
  'member10',
  'member11',
  'member11',
  'member11',
  'member12',
  'member12',
  'member12',
  'member13',
  'member13',
  'member13',
  'member14',
  'member14',
  'member14',
  'member15',
  'member15',
  'member15',
  'member16',
  'member16',
  'member16',
  'member17',
  'member17',
  'member17',
  'member18',
  'member18',
  'member18',
  'member19',
  'member19',
  'member19',
  'member20',
  'member20',
  'member20',
  'member21',
  'member21',
  'member21',
  'member22',
  'member22',
  'member22',
  'member23',
  'member23',
  'member23',
  'member24',
  'member24',
  'member24',
  'member25',
  'member25',
  'member25',
  'member26',
  'member26',
  'member26',
  'member27',
  'member27',
  'member27',
  'member28',
  'member28',
  'member28',
  'member29',
  'member29',
  'member29',
  'member30',
  'member30',
  'member30',
  'member31',
  'member31',
  'member31',
  'member32',
  'member32',
  'member32',
  'member33',
  'member33',
  'member33',
  'member34',
  'member34',
  'member34',
  'member35',
  'member35',
  'member35',
  'member36',
  'member36',
  'member36',
  'member37',
  'member37',
  'member37',
  'member38',
  'member38',
  'member38',
  'member39',
  'member39',
  'member39',
  'member40',
  'member40',
  'member40',
  'member41',
  'member41',
  'member41',
  'member42',
  'member42',
  'member42',
  'member43',
  'member43',
  'member43',
  'member44',
  'member44',
  'member44',
  'member45',
  'member45',
  'member45',
  'member46',
  'member46',
  'member46',
  'member47',
  'member47',
  'member47',
  'member48',
  'member48',
  'member48',
  'member49',
  'member49',
  'member49'])

# load an example hazard
initiation_year = year_list[0]
initiation_month_str = f"{month_name_to_number(initiation_month[0]):02d}"

forecast_month = int(forecast.valid_period_str[-2:])
forecast_year = (
    initiation_year + 1
    if int(initiation_month_str) > forecast_month
    else initiation_year
)

path_to_hazard = forecast.get_pipeline_path(
    initiation_year, initiation_month_str, "hazard"
)
haz = Hazard.from_hdf5(path_to_hazard)

if haz:
    available_dates = sorted(set(haz.date))
    readable_dates = [
        dt.datetime.fromordinal(d).strftime("%Y-%m-%d") for d in available_dates
    ]
    print("Available Dates Across Members:", readable_dates)

    target_date = dt.datetime(
        forecast_year, forecast_month, 1
    ).toordinal()  # Look for the first day of the last forecast month
    closest_date = min(available_dates, key=lambda x: abs(x - target_date))
    closest_date_str = dt.datetime.fromordinal(closest_date).strftime("%Y-%m-%d")

    print(f"Selected Date for Plotting: {closest_date_str}")
    haz.select(date=[closest_date, closest_date]).plot_intensity(event=0, smooth=False)

else:
    print("No hazard data found for the selected period.")

Available Dates Across Members: ['2022-12-01', '2023-01-01', '2023-02-01']
Selected Date for Plotting: 2023-02-01

../_images/eba4c6fa00791c990302da5273b1af9095c1d79cee61efd90a87b2469cbea829.png

Now you have a Hazard object that you can use in your specific impact assessment. In addition, you also have access to daily and monthly index estimates, along with ensemble statistics. Of course, the original hourly data of the climate variables related to your index of interest is also available, including their daily statistics.

If you would like to explore more advanced examples, please visit the package repository. There, you will find additional Python notebooks as well as links to plug-and-play Google Colab notebooks demonstrating the full capabilities of the package.

Resources#

Additional resources:

Using the Copernicus Seasonal Forecast Tools package to create a hazard object#

Introduction#

Set up parameters#

Overview of parameters#

Download and Process Data#

Calculate Climate Indices#

Calculate a Hazard Object#

Resources#

This Page