ML Bias Correction

We trained a machine-learning model to predict that error, and the result is a single global field you can apply to the simulations yourself.

The correction field

File ml_bias_correction.nc

Size 897 MiB

Archive Quality assurance vault

You’ll find it in the quality assurance collection Data Catalogue.

The function

import numpy as np


def apply_bias_correction(wtd, average_wtd, bias, protect_depth=0.1, ramp_width=4.9):
    """Correct a simulated water table depth with the ML bias field.

    wtd          the water table depth you want to correct, in metres below the
                 surface. Any time resolution, or none at all.
    average_wtd  the long-term historical average depth for the same simulation,
                 also in metres below the surface.
    bias         the `bias` variable from ml_bias_correction.nc, in metres, on
                 the same grid as `wtd`.

    Comes back the same shape as `wtd`.
    """
    ramp_factor = ((average_wtd - protect_depth) / ramp_width).clip(min=0.0, max=1.0)

    # Cap the adjustment at half the long-term depth in either direction.
    limit = np.abs(average_wtd / 2)
    delta = (bias * ramp_factor).clip(min=-limit, max=limit)

    return wtd + delta

Applying it

You need three:

The simulation you want to correct - any wtd product from the Data Access page.

Its long-term average. This has to be the historical average for the same forcing: wtd_reference_gswp3-w5e5_average_1960_2019.nc

The bias field, on your grid. Reindex it rather than hoping it already lines up:

import xarray as xr
import zarr

# The annual and monthly products are deposited as zipped Zarr stores, the
# averages as NetCDF; both carry a length-one `model` dimension.
sim = xr.open_zarr(
    zarr.storage.ZipStore("wtd_annual_2015_2100_ssp370_gfdl-esm4.zarr.zip", mode="r")
).isel(model=0, drop=True)

average = xr.open_dataset(
    "wtd_average_1960_2014_historical_gfdl-esm4.nc"
).isel(model=0, drop=True)

bias = xr.open_dataset("ml_bias_correction.nc")["bias"].reindex(
    latitude=sim.latitude,
    longitude=sim.longitude,
    layer=sim.layer,
    method="nearest",
)

corrected = apply_bias_correction(sim["wtd"], average["wtd"], bias)