Loading [MathJax]/extensions/Safe.js

1 2 3 4

# larger dataset with 1000 year-time slices import xarray as xr import gcsfs gcs = gcsfs.GCSFileSystem(token=None, access='read_only')

1	snapshots1ka = xr.open_zarr(gcs.get_mapper('gs://ldeo-glaciology/paleo_ensemble/snapshots1ka.zarr'))

1	# snapshots1ka # these chunks are HUGE (we usually recommend chunks around 100MB size. These are around 900!

1 2 3 4

# Proposed solution # snapshots1ka = snapshots1ka.chunk(chunks={'time':1}) # chunks={'par_esia':-1, 'time':1} gives 150 MB chunks...quite large... snapshots1ka

<xarray.Dataset>
Dimensions:     (time: 125, y: 381, x: 381, par_esia: 4, par_ppq: 4,
                 par_prec: 4, par_visc: 4)
Coordinates:
  * par_esia    (par_esia) float64 1.0 2.0 4.0 7.0
  * par_ppq     (par_ppq) float64 0.25 0.5 0.75 1.0
  * par_prec    (par_prec) float64 0.02 0.05 0.07 0.1
  * par_visc    (par_visc) float64 1e+20 5e+20 2.5e+21 1e+22
  * time        (time) float64 -1.24e+05 -1.23e+05 -1.22e+05 ... -1e+03 0.0
  * x           (x) float64 -3.04e+06 -3.024e+06 ... 3.024e+06 3.04e+06
  * y           (y) float64 -3.04e+06 -3.024e+06 ... 3.024e+06 3.04e+06
Data variables:
    bmelt       (time, y, x, par_esia, par_ppq, par_prec, par_visc) float32 dask.array<chunksize=(25, 381, 381, 1, 4, 4, 4), meta=np.ndarray>
    dbdt        (time, y, x, par_esia, par_ppq, par_prec, par_visc) float32 dask.array<chunksize=(25, 381, 381, 1, 4, 4, 4), meta=np.ndarray>
    index       (par_esia, par_ppq, par_prec, par_visc) int64 dask.array<chunksize=(4, 4, 4, 4), meta=np.ndarray>
    mask        (time, y, x, par_esia, par_ppq, par_prec, par_visc) int8 dask.array<chunksize=(25, 381, 381, 1, 4, 4, 4), meta=np.ndarray>
    score       (time, par_esia, par_ppq, par_prec, par_visc) float64 dask.array<chunksize=(125, 4, 4, 4, 4), meta=np.ndarray>
    thk         (time, y, x, par_esia, par_ppq, par_prec, par_visc) float32 dask.array<chunksize=(25, 381, 381, 1, 4, 4, 4), meta=np.ndarray>
    topg        (time, y, x, par_esia, par_ppq, par_prec, par_visc) float32 dask.array<chunksize=(25, 381, 381, 1, 4, 4, 4), meta=np.ndarray>
    usurf       (time, y, x, par_esia, par_ppq, par_prec, par_visc) float32 dask.array<chunksize=(25, 381, 381, 1, 4, 4, 4), meta=np.ndarray>
    velbar_mag  (time, y, x, par_esia, par_ppq, par_prec, par_visc) float32 dask.array<chunksize=(25, 381, 381, 1, 4, 4, 4), meta=np.ndarray>
Attributes:
    Conventions:      CF-1.5
    parameter_space:  {'visc': [1e+20, 5e+20, 2.5e+21, 1e+22], 'sia_e': [1.0,...
    proj4:            +lon_0=0.0 +ellps=WGS84 +datum=WGS84 +lat_ts=-71.0 +pro...
    source:           PISM (stable v1.0-123-gf2e24e88f committed by Torsten A...

xarray.Dataset

• Dimensions:
  • time: 125
  • y: 381
  • x: 381
  • par_esia: 4
  • par_ppq: 4
  • par_prec: 4
  • par_visc: 4
• Coordinates: (7)
  • par_esia
    (par_esia)
    float64
    1.0 2.0 4.0 7.0

    array([1., 2., 4., 7.])

  • par_ppq
    (par_ppq)
    float64
    0.25 0.5 0.75 1.0

    array([0.25, 0.5 , 0.75, 1.  ])

  • par_prec
    (par_prec)
    float64
    0.02 0.05 0.07 0.1

    array([0.02, 0.05, 0.07, 0.1 ])

  • par_visc
    (par_visc)
    float64
    1e+20 5e+20 2.5e+21 1e+22

    array([1.0e+20, 5.0e+20, 2.5e+21, 1.0e+22])

  • time
    (time)
    float64
    -1.24e+05 -1.23e+05 ... -1e+03 0.0

    array([-124000., -123000., -122000., -121000., -120000., -119000., -118000.,
           -117000., -116000., -115000., -114000., -113000., -112000., -111000.,
           -110000., -109000., -108000., -107000., -106000., -105000., -104000.,
           -103000., -102000., -101000., -100000.,  -99000.,  -98000.,  -97000.,
            -96000.,  -95000.,  -94000.,  -93000.,  -92000.,  -91000.,  -90000.,
            -89000.,  -88000.,  -87000.,  -86000.,  -85000.,  -84000.,  -83000.,
            -82000.,  -81000.,  -80000.,  -79000.,  -78000.,  -77000.,  -76000.,
            -75000.,  -74000.,  -73000.,  -72000.,  -71000.,  -70000.,  -69000.,
            -68000.,  -67000.,  -66000.,  -65000.,  -64000.,  -63000.,  -62000.,
            -61000.,  -60000.,  -59000.,  -58000.,  -57000.,  -56000.,  -55000.,
            -54000.,  -53000.,  -52000.,  -51000.,  -50000.,  -49000.,  -48000.,
            -47000.,  -46000.,  -45000.,  -44000.,  -43000.,  -42000.,  -41000.,
            -40000.,  -39000.,  -38000.,  -37000.,  -36000.,  -35000.,  -34000.,
            -33000.,  -32000.,  -31000.,  -30000.,  -29000.,  -28000.,  -27000.,
            -26000.,  -25000.,  -24000.,  -23000.,  -22000.,  -21000.,  -20000.,
            -19000.,  -18000.,  -17000.,  -16000.,  -15000.,  -14000.,  -13000.,
            -12000.,  -11000.,  -10000.,   -9000.,   -8000.,   -7000.,   -6000.,
             -5000.,   -4000.,   -3000.,   -2000.,   -1000.,       0.])

  • x
    (x)
    float64
    -3.04e+06 -3.024e+06 ... 3.04e+06

    axis :

    X

    long_name :

    X-coordinate in Cartesian system

    spacing_meters :

    16000.0

    standard_name :

    projection_x_coordinate

    units :

    m

    array([-3040000., -3024000., -3008000., ...,  3008000.,  3024000.,  3040000.])

  • y
    (y)
    float64
    -3.04e+06 -3.024e+06 ... 3.04e+06

    axis :

    Y

    long_name :

    Y-coordinate in Cartesian system

    spacing_meters :

    16000.0

    standard_name :

    projection_y_coordinate

    units :

    m

    array([-3040000., -3024000., -3008000., ...,  3008000.,  3024000.,  3040000.])

• Data variables: (9)
  • bmelt
    (time, y, x, par_esia, par_ppq, par_prec, par_visc)
    float32
    dask.array<chunksize=(25, 381, 381, 1, 4, 4, 4), meta=np.ndarray>

    comment :

    positive basal melt rate corresponds to ice loss

    coordinates :

    lat lon

    long_name :

    ice basal melt rate from energy conservation and subshelf melt, in ice thickness per time

    pism_intent :

    internal

    standard_name :

    land_ice_basal_melt_rate

    units :

    m year-1

    Array Chunk Bytes 17.30 GiB 885.99 MiB Shape (125, 381, 381, 4, 4, 4, 4) (25, 381, 381, 1, 4, 4, 4) Count 2 Graph Layers 20 Chunks Type float32 numpy.ndarray

  • dbdt
    (time, y, x, par_esia, par_ppq, par_prec, par_visc)
    float32
    dask.array<chunksize=(25, 381, 381, 1, 4, 4, 4), meta=np.ndarray>

    coordinates :

    lat lon

    long_name :

    bedrock uplift rate

    pism_intent :

    model_state

    standard_name :

    tendency_of_bedrock_altitude

    units :

    mm year-1

    Array Chunk Bytes 17.30 GiB 885.99 MiB Shape (125, 381, 381, 4, 4, 4, 4) (25, 381, 381, 1, 4, 4, 4) Count 2 Graph Layers 20 Chunks Type float32 numpy.ndarray

  • index
    (par_esia, par_ppq, par_prec, par_visc)
    int64
    dask.array<chunksize=(4, 4, 4, 4), meta=np.ndarray>

    Array Chunk Bytes 2.00 kiB 2.00 kiB Shape (4, 4, 4, 4) (4, 4, 4, 4) Count 2 Graph Layers 1 Chunks Type int64 numpy.ndarray

  • mask
    (time, y, x, par_esia, par_ppq, par_prec, par_visc)
    int8
    dask.array<chunksize=(25, 381, 381, 1, 4, 4, 4), meta=np.ndarray>

    coordinates :

    lat lon

    flag_meanings :

    ice_free_bedrock grounded_ice floating_ice ice_free_ocean

    flag_values :

    [0, 2, 3, 4]

    long_name :

    ice-type (ice-free/grounded/floating/ocean) integer mask

    pism_intent :

    diagnostic

    units :

    Array Chunk Bytes 4.33 GiB 221.50 MiB Shape (125, 381, 381, 4, 4, 4, 4) (25, 381, 381, 1, 4, 4, 4) Count 2 Graph Layers 20 Chunks Type int8 numpy.ndarray

  • score
    (time, par_esia, par_ppq, par_prec, par_visc)
    float64
    dask.array<chunksize=(125, 4, 4, 4, 4), meta=np.ndarray>

    Array Chunk Bytes 250.00 kiB 250.00 kiB Shape (125, 4, 4, 4, 4) (125, 4, 4, 4, 4) Count 2 Graph Layers 1 Chunks Type float64 numpy.ndarray

  • thk
    (time, y, x, par_esia, par_ppq, par_prec, par_visc)
    float32
    dask.array<chunksize=(25, 381, 381, 1, 4, 4, 4), meta=np.ndarray>

    coordinates :

    lat lon

    long_name :

    land ice thickness

    pism_intent :

    model_state

    standard_name :

    land_ice_thickness

    units :

    m

    valid_min :

    0.0

    Array Chunk Bytes 17.30 GiB 885.99 MiB Shape (125, 381, 381, 4, 4, 4, 4) (25, 381, 381, 1, 4, 4, 4) Count 2 Graph Layers 20 Chunks Type float32 numpy.ndarray

  • topg
    (time, y, x, par_esia, par_ppq, par_prec, par_visc)
    float32
    dask.array<chunksize=(25, 381, 381, 1, 4, 4, 4), meta=np.ndarray>

    coordinates :

    lat lon

    long_name :

    bedrock surface elevation

    pism_intent :

    model_state

    standard_name :

    bedrock_altitude

    units :

    m

    Array Chunk Bytes 17.30 GiB 885.99 MiB Shape (125, 381, 381, 4, 4, 4, 4) (25, 381, 381, 1, 4, 4, 4) Count 2 Graph Layers 20 Chunks Type float32 numpy.ndarray

  • usurf
    (time, y, x, par_esia, par_ppq, par_prec, par_visc)
    float32
    dask.array<chunksize=(25, 381, 381, 1, 4, 4, 4), meta=np.ndarray>

    coordinates :

    lat lon

    long_name :

    ice upper surface elevation

    pism_intent :

    diagnostic

    standard_name :

    surface_altitude

    units :

    m

    Array Chunk Bytes 17.30 GiB 885.99 MiB Shape (125, 381, 381, 4, 4, 4, 4) (25, 381, 381, 1, 4, 4, 4) Count 2 Graph Layers 20 Chunks Type float32 numpy.ndarray

  • velbar_mag
    (time, y, x, par_esia, par_ppq, par_prec, par_visc)
    float32
    dask.array<chunksize=(25, 381, 381, 1, 4, 4, 4), meta=np.ndarray>

    coordinates :

    lat lon

    long_name :

    magnitude of vertically-integrated horizontal velocity of ice

    pism_intent :

    diagnostic

    units :

    m year-1

    valid_min :

    0.0

    Array Chunk Bytes 17.30 GiB 885.99 MiB Shape (125, 381, 381, 4, 4, 4, 4) (25, 381, 381, 1, 4, 4, 4) Count 2 Graph Layers 20 Chunks Type float32 numpy.ndarray

• Attributes: (4)

  Conventions :

  CF-1.5

  parameter_space :

  {'visc': [1e+20, 5e+20, 2.5e+21, 1e+22], 'sia_e': [1.0, 2.0, 4.0, 7.0], 'prec': [0.02, 0.05, 0.07, 0.1], 'ppq': [0.25, 0.5, 0.75, 1.0]}

  proj4 :

  +lon_0=0.0 +ellps=WGS84 +datum=WGS84 +lat_ts=-71.0 +proj=stere +x_0=0.0 +units=m +y_0=0.0 +lat_0=-90.0

  source :

  PISM (stable v1.0-123-gf2e24e88f committed by Torsten Albrecht on 2018-01-23 14:46:59 +0100) CMake 3.2.3. Petsc Release Version 3.7.5, unknown PETSc configure: --known-level1-dcache-size=32768 --known-level1-dcache-linesize=64 --known-level1-dcache-assoc=8 --known-sizeof-char=1 --known-sizeof-void-p=8 --known-sizeof-short=2 --known-sizeof-int=4 --known-sizeof-long=8 --known-sizeof-long-long=8 --known-sizeof-float=4 --known-sizeof-double=8 --known-sizeof-size_t=8 --known-bits-per-byte=8 --known-memcmp-ok=1 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-mpi-long-double=1 --known-mpi-int64_t=1 --known-mpi-c-double-complex=1 --known-sdot-returns-double=0 --known-snrm2-returns-double=0 --known-has-attribute-aligned=1 --with-cc=mpiicc --with-cxx=mpiicpc --with-fc=0 --with-mpi-lib=/p/system/packages/intel/parallel_studio_xe_2017_update1/compilers_and_libraries_2017.1.132/linux/mpi/lib64/libmpi.so --with-mpi-include=/p/system/packages/intel/parallel_studio_xe_2017_update1/compilers_and_libraries_2017.1.132/linux/mpi/include64 --with-blas-lapack-dir=/p/system/packages/intel/parallel_studio_xe_2017_update1/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64 --with-64-bit-indices=1 --known-mpi-shared-libraries=1 --with-debugging=1 --with-valgrind=1 --with-valgrind-dir=/p/system/packages/valgrind/3.10.1/gnu --with-x=0 --with-ssl=0 --with-batch=1 --with-shared-libraries=1 --with-mpiexec=srun CFLAGS="-fp-model precise -no-fast-transcendentals" CXXFLAGS="-fp-model precise -no-fast-transcendentals" Intel(R) MPI Library 2017 Update 1 for Linux* OS NetCDF 4.4.1.1 of Jan 30 2017 12:20:18 $. FFTW fftw-3.3.6-pl2. GSL 2.3. PROJ.4 Rel. 4.9.3, 15 August 2016.

1 2 3 4 5

from dask.distributed import Client import dask_gateway gateway = dask_gateway.Gateway() cluster = gateway.new_cluster()

Solution: Increase worker memory to accomodate large chunks

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

# from dask.distributed import Client # import dask_gateway # gateway = dask_gateway.Gateway() # # close existing clusters # open_clusters = gateway.list_clusters() # print(list(open_clusters)) # if len(open_clusters)>0: # for c in open_clusters: # cluster = gateway.connect(c.name) # cluster.shutdown() # # You can actually adjust the memory (and cores) of your workers! For this example we need more memory, # # since the standard worker (4GB RAM) can barely load in 4 datasets at once!) # options = gateway.cluster_options() # options.worker_memory = 16 # cluster = gateway.new_cluster(cluster_options=options)

[ClusterReport<name=prod.ae0e1f43bb884f9db463f77f30fb460b, status=RUNNING>]

1 2	cluster.scale(20) client = Client(cluster)

Lets show the dashboard link for the cluster to see what is going wrong

1

client

Client

Client-09a689b8-611d-11ed-82c5-a2e042606cc6

Connection method: Cluster object	Cluster type: dask_gateway.GatewayCluster
Dashboard: /services/dask-gateway/clusters/prod.ce27cd039adb423b9995e370373c7467/status

Cluster Info

GatewayCluster

• Name: prod.ce27cd039adb423b9995e370373c7467
• Dashboard: /services/dask-gateway/clusters/prod.ce27cd039adb423b9995e370373c7467/status

When you go shift+right click on the link above, and select copy link you can past that into the dask panel on the right.

1	cellArea = snapshots1ka.x.attrs['spacing_meters'] * snapshots1ka.y.attrs['spacing_meters']

1 2	# Preprocessing data V = (snapshots1ka.thk.where(snapshots1ka.mask == 2).sum(['x','y'])*cellArea)#.persist()

1 2	# Computing the mean volume at each time step across the ensemble Vmean = V.mean({'par_esia','par_ppq','par_prec','par_visc'})

1	# Vmean_test = V.mean({'par_ppq','par_prec','par_visc'}).load()

1 2 3

# Plotting mean volume # Got error Vmean.plot(size=10)

[<matplotlib.lines.Line2D at 0x7fe250666b80>]

1