_images/veros-logo-400px.png

Versatile Ocean Simulation in Pure Python

Veros, the versatile ocean simulator, aims to be the swiss army knife of ocean modeling. It is a full-fledged GCM that supports anything between highly idealized configurations and realistic set-ups, targeting students and seasoned researchers alike. Thanks to its seamless interplay with Bohrium, Veros runs efficiently on your laptop, gaming PC (with experimental GPU support through OpenCL & CUDA), and small cluster. Veros has a clear focus on simplicity, usability, and adaptability - because the Baroque is over.

If you want to learn more about the background and capabilities of Veros, you should check out A short introduction to Veros. If you are already convinced, you can jump right into action, and learn how to get started instead!

A short introduction to Veros

The vision

Veros is an adaptation of pyOM2 (v2.1.0), developed by Carsten Eden (Institut für Meereskunde, Hamburg University). In contrast to pyOM2, however, this implementation does not rely on a Fortran backend for computations - everything runs in pure Python, down to the last parameterization. We believe that using this approach it is possible to create an open source ocean model that is:

  1. Easy to access: Python modules are simple to install, and projects like Anaconda are doing a great job in creating platform-independent environments.
  2. Easy to use: Anyone with some experience can use their favorite Python tools to set up, control, and post-process Veros.
  3. Easy to verify: Python code tends to be concise and easy to read, even for people with little practical programming experience. This enables a wide range of people to spot errors in our code, solidifying it in the process.
  4. Easy to modify: Due to the popularity of Python, its dynamic code structure, and OOP-capabilities, Veros can be extended and modified with minimal effort.

However, using Python over a compiled language like Fortran usually comes at a high computational cost. We try to overcome this gap for large models by providing an interface to Bohrium, a framework that acts as a high-performance replacement for NumPy. Bohrium takes care of all parallelism in the background for us, so we can concentrate on writing a nice, readable ocean model.

In case you are curious about how Veros is currently stacking up against pyOM2 in terms of performance, you should check out our benchmarks.

Features

Note

This section provides a quick overview of the capabilities and limitations of Veros. For a comprehensive description of the physics and numerics behind Veros, please refer to the documentation of pyOM2. You can also obtain a copy of the PDF documentation here.

The model domain

The numerical solution is calculated using finite differences on an Arakawa C-grid, which is staggered in every dimension. Tracers (like temperature and salinity) are calculated at different positions than zonal, meridional, and vertical fluxes (like the velocities u, v, and w). The following figure shows the relative positions of the so-called T, U, V, and ζ grid points (W not shown):

_images/c-grid.svg

The structure of the Arakawa C-grid.

Veros supports both Cartesian and pseudo-spherical (i.e., including additional metric terms) coordinate systems. Islands or holes in the domain are fully supported by the streamfunction solver. Zonal boundaries can either be cyclic or regraded as walls (with free-slip boundary conditions).

Available parameterizations

At its core, Veros currently offers the following solvers, numerical schemes, parameterizations, and closures:

Surface pressure:
  • a high-performance streamfunction solver via an iterative Poisson solver
Equation of state:
  • the full 48-term TEOS equation of state
  • various linear and nonlinear model equations from [Vallis2006]
Friction:
  • harmonic or biharmonic lateral friction
  • linear or quadratic bottom friction
  • interior Rayleigh friction
  • explicit or fully implicit harmonic vertical friction
Advection:
  • a classical second-order central difference scheme
  • a second-order scheme with a superbee flux-limiter
Diffusion:
  • harmonic or biharmonic lateral diffusion
  • explicit or implicit harmonic vertical diffusion
Isoneutral mixing:
  • lateral mixing of tracers along neutral surfaces following [Griffies1998] (optional)
Internal wave breaking:
EKE model (eddy kinetic energy):
  • meso-scale eddy mixing closure after [Gent1995], either with constant coefficients or calculated using the prognostic EKE closure by [EdenGreatbatch2008] (optional)
TKE model (turbulent kinetic energy):
  • prognostic TKE model for vertical mixing as introduced in [Gaspar1990] (optional)

Diagnostics

Diagnostics are responsible for handling all model output, runtime checks of the solution, and restart file handling. They are implemented in a modular fashion, so additional diagnostics can be implemented easily. Already implemented diagnostics handle snapshot output, time-averaging of variables, monitoring of energy fluxes, and calculation of the overturning streamfunction.

For more information, see Diagnostics.

Pre-configured model setups

Veros supports a wide range of model configurations. Several setups are already implemented that highlight some of the capabilities of Veros, and that serve as a basis for users to set up their own configuration: Setup gallery.

Current limitations

Veros is still in early development. There are several open issues that we would like to fix later on:

Physics:
  • Veros does not yet implement any of the more recent pyOM2.2 features such as the ROSSMIX parameterization, IDEMIX v3.0, open boundary conditions, or cyclic meridional boundaries. It neither implements all of pyOM2.1’s features - missing are e.g. the non-hydrostatic solver, IDEMIX v2.0, and the surface pressure solver.
  • Since the grid is required to be rectilinear, there is currently no natural way to handle the singularity at the North Pole. The northern and southern boundaries of the domain are thus always “walls”.
  • There is currently no ice sheet model in Veros. Some realistic setups employ a simple ice mask that cut off atmospheric forcing for water that gets too cold instead.
Technical issues:
  • For the time being, Veros’ dynamical core is still more or less a direct port of PyOM2. This means that numerics and physics are still tightly coupled, which makes for a far from optimal user experience. In a future version of Veros, we would like to introduce additional abstraction to make the core routines a lot more readable than they are now.

References

[EdenGreatbatch2008]Eden, Carsten, and Richard J. Greatbatch. “Towards a mesoscale eddy closure.” Ocean Modelling 20.3 (2008): 223-239.
[OlbersEden2013]Olbers, Dirk, and Carsten Eden. “A global model for the diapycnal diffusivity induced by internal gravity waves.” Journal of Physical Oceanography 43.8 (2013): 1759-1779.
[Gent1995]Gent, Peter R., et al. “Parameterizing eddy-induced tracer transports in ocean circulation models.” Journal of Physical Oceanography 25.4 (1995): 463-474.
[Griffies1998]Griffies, Stephen M. “The Gent–McWilliams skew flux.” Journal of Physical Oceanography 28.5 (1998): 831-841.
[Vallis2006]Vallis, Geoffrey K. “Atmospheric and oceanic fluid dynamics: fundamentals and large-scale circulation.” Cambridge University Press, 2006.
[Gaspar1990]Gaspar, Philippe, Yves Grégoris, and Jean‐Michel Lefevre. “A simple eddy kinetic energy model for simulations of the oceanic vertical mixing: Tests at station Papa and Long‐Term Upper Ocean Study site.” Journal of Geophysical Research: Oceans 95.C9 (1990): 16179-16193.

Getting started

Installation

Using pip (multi-platform)

Note

You should only install Veros via pip if you want to get going as quickly as possible, and do not plan to access or modify the Veros source code. The prefered way to install Veros is through Anaconda (see below).

If you already have Python installed, the quickest way to get a working Veros installation is to run:

$ pip install veros --user

and optionally:

$ pip install bohrium --user

to use Veros with Bohrium (Linux and OSX only).

On bare metal (Ubuntu / Debian)

  1. Install some dependencies:

    $ sudo apt-get install git python3-dev python3-pip libhdf5-dev

  2. Clone our repository:

    $ git clone https://github.com/dionhaefner/veros.git

  3. Install Veros (preferably in a virtual environment) via:

    $ pip3 install -e ./veros --user

  4. Optionally, install Bohrium via:

    $ pip3 install bohrium --user

Setting up a model

To run Veros, you need to set up a model - i.e., specify which settings and model domain you want to use. This is done by subclassing the Veros setup base class in a setup script that is written in Python. You should have a look at the pre-implemented model setups in the repository’s setup folder, or use the veros copy-setup command to copy one into your current folder. A good place to start is the ACC model:

$ veros copy-setup acc

By working through the existing models, you should quickly be able to figure out how to write your own simulation. Just keep in mind this general advice:

  • You can (and should) use any (external) Python tools you want in your model setup. Before implementing a certain functionality, you should check whether it is already provided by a common library. Especially the SciPy module family provides countless implementations of common scientific functions (and SciPy is installed along with Veros).

  • If you decorate your methods with @veros_method, the variable np inside that function will point to the currently used backend (i.e., NumPy or Bohrium). Thus, if you want your setup to be able to dynamically switch between backends, you should write your methods like this:

    from veros import Veros, veros_method

class MyVerosSetup(Veros):
    ...
    @veros_method
    def my_function(self):
        arr = np.array([1,2,3,4]) # "np" uses either NumPy or Bohrium

  • If you are curious about the general procedure in which a model is set up and ran, you should read the source code of veros.VerosSetup (especially the setup() and run() methods). This is also the best way to find out about the order in which methods and routines are called.

  • Out of all functions that need to be implemented by your subclass of veros.VerosSetup, the only one that is called in every time step is set_forcing() (at the beginning of each iteration). This implies that, to achieve optimal performance, you should consider moving calculations that are constant in time to other functions.

If you want to learn more about setting up advanced configurations, you should check out our tutorial that walks you through the creation of a realistic configuration with an idealized Atlantic.

Running Veros

After adapting your setup script, you are ready to run your first simulation. It is advisable to include something like:

@veros.tools.cli
def run(*args, **kwargs):
    simulation = MyVerosSetup()
    simulation.setup()
    simulation.run()

if __name__ == "__main__":
    run()

in your setup file, so you can run it as a script:

$ python my_setup.py

However, you are not required to do so, and you are welcome to write include your simulation class into other Python files and call it dynamically or interactively (e.g. in an IPython session).

All Veros setups decorated with veros.tools.cli() accept additional options via the command line when called as a script or as arguments to their __init__() function when called from another Python module. You can check the available commands through

$ python my_setup.py --help

Reading Veros output

All output is handled by the available diagnostics. The most basic diagnostic, snapshot, writes some model variables to netCDF files in regular intervals (and puts them into your current working directory).

NetCDF is a binary format that is widely adopted in the geophysical modeling community. There are various packages for reading, visualizing and processing netCDF files (such as ncview and ferret), and bindings for many programming languages (such as C, Fortran, MATLAB, and Python).

In fact, after installing Veros, you will already have installed the netCDF bindings for Python, so reading data from an output file and plotting it is as easy as:

import matplotlib.pyplot as plt
import h5netcdf

with h5netcdf.File("veros.snapshot.nc", "r") as datafile:
    # read variable "u" and save it to a NumPy array
    u = datafile.variables["u"][...]

# plot surface velocity at the last time step included in the file
plt.imshow(u[-1, -1, ...])
plt.show()

For further reference refer to the netcdf4-python documentation.

Using Bohrium

Warning

While Bohrium yields significant speed-ups for large to very large setups, the overhead introduced by Bohrium often leads to (sometimes considerably) slower execution for problems below a certain threshold size (see also Which backend should I choose to run my model (NumPy / Bohrium)?). You are thus advised to test carefully whether Bohrium is beneficial in your particular use case.

For large simulations, it is often beneficial to use the Bohrium backend for computations. When using Bohrium, all number crunching will make full use of your available architecture, i.e., computations are executed in parallel on all of your CPU cores, or even GPU when using BH_STACK=opencl or BH_STACK=cuda. You may switch between NumPy and Bohrium with a simple command line switch:

$ python my_setup.py -b bohrium

or, when running inside another Python module: (must be done before initializing you setup):

from veros import runtime_settings as rs

rs.backend = "bohrium"

Re-starting from a previous run

Restart data (in HDF5 format) is written at the end of each simulation or after a regular time interval if the setting restart_frequency is set to a finite value. To use this restart file as initial conditions for another simulation, you will have to point restart_input_filename of the new simulation to the corresponding restart file. This can (as all settings) also be given via command line:

$ python my_setup.py -s restart_input_filename /path/to/restart_file.h5

Running Veros on multiple processes via MPI

Note

This assumes that you are familiar with running applications through MPI, and is most useful on large architectures like a compute cluster. For smaller architectures, it is usually easier to stick to Bohrium.

Running Veros through MPI requires some addititonal dependencies:

  • A recent MPI implementation, such as OpenMPI or MPICH
  • mpi4py that is linked to the correct MPI library
  • A parallel-enabled version of the HDF5 library
  • h5py built against this parallel version of HDF5
  • For optimal performance, PETSc and petsc4py, linked to the rest of the stack

After you have installed everything, you can start Veros on multiple processes like so::

$ mpirun -n 4 python my_setup.py -n 2 2

In this case, Veros would run on 4 processes, each process computing one-quarter of the domain. The arguments of the -n flag specify the number of chunks in x and y-direction, respectively.

You can combine MPI and Bohrium like so::

$ OMP_NUM_THREADS=2 mpirun -n 2 python my_setup.py -n 2 1 -b bohrium

This starts 2 independent processes, each being parallelized by Bohrium using 2 threads (hybrid run).

See also

For more information, see Running Veros on a cluster.

Enhancing Veros

Veros was written with extensibility in mind. If you already know some Python and have worked with NumPy, you are pretty much ready to write your own extension. The model code is located in the veros subfolder, while all of the numerical routines are located in veros/core.

We believe that the best way to learn how Veros works is to read its source code. Starting from the Veros base class, you should be able to work your way through the flow of the program, and figure out where to add your modifications. If you installed Veros through pip -e or setup.py develop, all changes you make will immediately be reflected when running the code.

In case you want to add additional output capabilities or compute additional quantities without changing the main solution of the simulation, you should consider adding a custom diagnostic.

A convenient way to implement your modifications is to create your own fork of Veros on GitHub, and submit a pull request if you think your modifications could be useful for the Veros community.

See also

More information is available in our developer guide.

Creating an advanced model setup

Note

This guide is still work in progress.

This is a step-by-step guide that illustrates how even complicated setups can be created with relative ease (thanks to the tools provided by the scientific Python community). As an example, we will re-create the wave propagation setup, which is a global ocean model with an idealized Atlantic.

_images/wave-propagation.png

The resulting stream function after about 1 year of integration.

The vision

The purpose of this model is to examine wave propagation along the eastern boundary of the North Atlantic. Since it is difficult to track propagating waves along ragged geometry or through uneven forcing fields, we will idealize the representation of the North Atlantic; and as the presence of the Pacific in the model is crucial to achieve a realistic ocean circulation, we want to use a global model.

This leaves us with the following requirements for the final wave propagation model:

  1. A global model with a resolution of around 2 degrees and meridional stretching.
  2. Convert the eastern boundary of the Atlantic to a straight line, so analytically derived wave properties hold.
  3. A refined grid resolution at the eastern boundary of the Atlantic.
  4. Zonally averaged forcings in the Atlantic.
  5. A somehow interpolated initial state for cells that have been converted from land to ocean in the North Atlantic.
  6. Options for shelf and continental slope.
  7. A multiplier setting for the Southern Ocean wind stress.

Model skeleton

Instead of starting from scratch, we can use the global one degree model as a template, which looks like this:

#!/usr/bin/env python

import os
import h5netcdf

from veros import VerosSetup, tools, veros_method, time
from veros.variables import Variable, allocate

BASE_PATH = os.path.dirname(os.path.realpath(__file__))
DATA_FILES = tools.get_assets('global_1deg', os.path.join(BASE_PATH, 'assets.yml'))


class GlobalOneDegreeSetup(VerosSetup):
    """Global 1 degree model with 115 vertical levels.

    `Adapted from pyOM2 <https://wiki.zmaw.de/ifm/TO/pyOM2/1x1%20global%20model>`_.
    """

    @veros_method
    def set_parameter(self, vs):
        """
        set main parameters
        """
        vs.nx = 360
        vs.ny = 160
        vs.nz = 115
        vs.dt_mom = 1800.0
        vs.dt_tracer = 1800.0
        vs.runlen = 0.

        vs.coord_degree = True
        vs.enable_cyclic_x = True

        vs.congr_epsilon = 1e-10
        vs.congr_max_iterations = 10000

        vs.enable_hor_friction = True
        vs.A_h = 5e4
        vs.enable_hor_friction_cos_scaling = True
        vs.hor_friction_cosPower = 1
        vs.enable_tempsalt_sources = True
        vs.enable_implicit_vert_friction = True

        vs.eq_of_state_type = 5

        # isoneutral
        vs.enable_neutral_diffusion = True
        vs.K_iso_0 = 1000.0
        vs.K_iso_steep = 50.0
        vs.iso_dslope = 0.005
        vs.iso_slopec = 0.005
        vs.enable_skew_diffusion = True

        # tke
        vs.enable_tke = True
        vs.c_k = 0.1
        vs.c_eps = 0.7
        vs.alpha_tke = 30.0
        vs.mxl_min = 1e-8
        vs.tke_mxl_choice = 2
        vs.enable_tke_superbee_advection = True

        # eke
        vs.enable_eke = True
        vs.eke_k_max = 1e4
        vs.eke_c_k = 0.4
        vs.eke_c_eps = 0.5
        vs.eke_cross = 2.
        vs.eke_crhin = 1.0
        vs.eke_lmin = 100.0
        vs.enable_eke_superbee_advection = True
        vs.enable_eke_isopycnal_diffusion = True

        # idemix
        vs.enable_idemix = False
        vs.enable_eke_diss_surfbot = True
        vs.eke_diss_surfbot_frac = 0.2
        vs.enable_idemix_superbee_advection = True
        vs.enable_idemix_hor_diffusion = True

        # custom variables
        vs.nmonths = 12
        vs.variables.update(
            t_star=Variable('t_star', ('xt', 'yt', 'nmonths'), '', '', time_dependent=False),
            s_star=Variable('s_star', ('xt', 'yt', 'nmonths'), '', '', time_dependent=False),
            qnec=Variable('qnec', ('xt', 'yt', 'nmonths'), '', '', time_dependent=False),
            qnet=Variable('qnet', ('xt', 'yt', 'nmonths'), '', '', time_dependent=False),
            qsol=Variable('qsol', ('xt', 'yt', 'nmonths'), '', '', time_dependent=False),
            divpen_shortwave=Variable('divpen_shortwave', ('zt',), '', '', time_dependent=False),
            taux=Variable('taux', ('xt', 'yt', 'nmonths'), '', '', time_dependent=False),
            tauy=Variable('tauy', ('xt', 'yt', 'nmonths'), '', '', time_dependent=False),
        )

    @veros_method
    def _read_forcing(self, vs, var):
        with h5netcdf.File(DATA_FILES['forcing'], 'r') as infile:
            var = infile.variables[var]
            return np.array(var, dtype=str(var.dtype)).T

    @veros_method
    def set_grid(self, vs):
        dz_data = self._read_forcing(vs, 'dz')
        vs.dzt[...] = dz_data[::-1]
        vs.dxt[...] = 1.0
        vs.dyt[...] = 1.0
        vs.y_origin = -79.
        vs.x_origin = 91.

    @veros_method
    def set_coriolis(self, vs):
        vs.coriolis_t[...] = 2 * vs.omega * np.sin(vs.yt[np.newaxis, :] / 180. * vs.pi)

    @veros_method(dist_safe=False, local_variables=['kbot'])
    def set_topography(self, vs):
        bathymetry_data = self._read_forcing(vs, 'bathymetry')
        salt_data = self._read_forcing(vs, 'salinity')[:, :, ::-1]

        mask_salt = salt_data == 0.
        vs.kbot[2:-2, 2:-2] = 1 + np.sum(mask_salt.astype(np.int), axis=2)

        mask_bathy = bathymetry_data == 0
        vs.kbot[2:-2, 2:-2][mask_bathy] = 0

        vs.kbot[vs.kbot >= vs.nz] = 0

        # close some channels
        i, j = np.indices((vs.nx, vs.ny))

        mask_channel = (i >= 207) & (i < 214) & (j < 5)  # i = 208,214; j = 1,5
        vs.kbot[2:-2, 2:-2][mask_channel] = 0

        # Aleutian islands
        mask_channel = (i == 104) & (j == 134)  # i = 105; j = 135
        vs.kbot[2:-2, 2:-2][mask_channel] = 0

        # Engl channel
        mask_channel = (i >= 269) & (i < 271) & (j == 130)  # i = 270,271; j = 131
        vs.kbot[2:-2, 2:-2][mask_channel] = 0

    @veros_method(dist_safe=False, local_variables=[
        't_star', 's_star', 'qnec', 'qnet', 'qsol', 'divpen_shortwave', 'taux', 'tauy',
        'temp', 'salt', 'forc_iw_bottom', 'forc_iw_surface', 'kbot', 'maskT', 'maskW',
        'zw', 'dzt'
    ])
    def set_initial_conditions(self, vs):
        rpart_shortwave = 0.58
        efold1_shortwave = 0.35
        efold2_shortwave = 23.0

        # initial conditions
        temp_data = self._read_forcing(vs, 'temperature')
        vs.temp[2:-2, 2:-2, :, 0] = temp_data[..., ::-1] * vs.maskT[2:-2, 2:-2, :]
        vs.temp[2:-2, 2:-2, :, 1] = temp_data[..., ::-1] * vs.maskT[2:-2, 2:-2, :]

        salt_data = self._read_forcing(vs, 'salinity')
        vs.salt[2:-2, 2:-2, :, 0] = salt_data[..., ::-1] * vs.maskT[2:-2, 2:-2, :]
        vs.salt[2:-2, 2:-2, :, 1] = salt_data[..., ::-1] * vs.maskT[2:-2, 2:-2, :]

        # wind stress on MIT grid
        vs.taux[2:-2, 2:-2, :] = self._read_forcing(vs, 'tau_x')
        vs.tauy[2:-2, 2:-2, :] = self._read_forcing(vs, 'tau_y')

        qnec_data = self._read_forcing(vs, 'dqdt')
        vs.qnec[2:-2, 2:-2, :] = qnec_data * vs.maskT[2:-2, 2:-2, -1, np.newaxis]

        qsol_data = self._read_forcing(vs, 'swf')
        vs.qsol[2:-2, 2:-2, :] = -qsol_data * vs.maskT[2:-2, 2:-2, -1, np.newaxis]

        # SST and SSS
        sst_data = self._read_forcing(vs, 'sst')
        vs.t_star[2:-2, 2:-2, :] = sst_data * vs.maskT[2:-2, 2:-2, -1, np.newaxis]

        sss_data = self._read_forcing(vs, 'sss')
        vs.s_star[2:-2, 2:-2, :] = sss_data * vs.maskT[2:-2, 2:-2, -1, np.newaxis]

        if vs.enable_idemix:
            tidal_energy_data = self._read_forcing(vs, 'tidal_energy')
            mask = np.maximum(0, vs.kbot[2:-2, 2:-2] - 1)[:, :, np.newaxis] == np.arange(vs.nz)[np.newaxis, np.newaxis, :]
            tidal_energy_data[:, :] *= vs.maskW[2:-2, 2:-2, :][mask].reshape(vs.nx, vs.ny) / vs.rho_0
            vs.forc_iw_bottom[2:-2, 2:-2] = tidal_energy_data

            wind_energy_data = self._read_forcing(vs, 'wind_energy')
            wind_energy_data[:, :] *= vs.maskW[2:-2, 2:-2, -1] / vs.rho_0 * 0.2
            vs.forc_iw_surface[2:-2, 2:-2] = wind_energy_data

        """
        Initialize penetration profile for solar radiation and store divergence in divpen
        note that pen is set to 0.0 at the surface instead of 1.0 to compensate for the
        shortwave part of the total surface flux
        """
        swarg1 = vs.zw / efold1_shortwave
        swarg2 = vs.zw / efold2_shortwave
        pen = rpart_shortwave * np.exp(swarg1) + (1.0 - rpart_shortwave) * np.exp(swarg2)
        pen[-1] = 0.
        vs.divpen_shortwave = allocate(vs, ('zt',))
        vs.divpen_shortwave[1:] = (pen[1:] - pen[:-1]) / vs.dzt[1:]
        vs.divpen_shortwave[0] = pen[0] / vs.dzt[0]

    @veros_method
    def set_forcing(self, vs):
        t_rest = 30. * 86400.
        cp_0 = 3991.86795711963  # J/kg /K

        year_in_seconds = time.convert_time(1., 'years', 'seconds')
        (n1, f1), (n2, f2) = tools.get_periodic_interval(vs.time, year_in_seconds,
                                                         year_in_seconds / 12., 12)

        # linearly interpolate wind stress and shift from MITgcm U/V grid to this grid
        vs.surface_taux[:-1, :] = f1 * vs.taux[1:, :, n1] + f2 * vs.taux[1:, :, n2]
        vs.surface_tauy[:, :-1] = f1 * vs.tauy[:, 1:, n1] + f2 * vs.tauy[:, 1:, n2]

        if vs.enable_tke:
            vs.forc_tke_surface[1:-1, 1:-1] = np.sqrt((0.5 * (vs.surface_taux[1:-1, 1:-1] \
                                                                + vs.surface_taux[:-2, 1:-1]) / vs.rho_0) ** 2
                                                      + (0.5 * (vs.surface_tauy[1:-1, 1:-1] \
                                                                + vs.surface_tauy[1:-1, :-2]) / vs.rho_0) ** 2) ** (3. / 2.)

        # W/m^2 K kg/J m^3/kg = K m/s
        t_star_cur = f1 * vs.t_star[..., n1] + f2 * vs.t_star[..., n2]
        vs.qqnec = f1 * vs.qnec[..., n1] + f2 * vs.qnec[..., n2]
        vs.qqnet = f1 * vs.qnet[..., n1] + f2 * vs.qnet[..., n2]
        vs.forc_temp_surface[...] = (vs.qqnet + vs.qqnec * (t_star_cur - vs.temp[..., -1, vs.tau])) \
            * vs.maskT[..., -1] / cp_0 / vs.rho_0
        s_star_cur = f1 * vs.s_star[..., n1] + f2 * vs.s_star[..., n2]
        vs.forc_salt_surface[...] = 1. / t_rest * \
            (s_star_cur - vs.salt[..., -1, vs.tau]) * vs.maskT[..., -1] * vs.dzt[-1]

        # apply simple ice mask
        mask1 = vs.temp[:, :, -1, vs.tau] * vs.maskT[:, :, -1] <= -1.8
        mask2 = vs.forc_temp_surface <= 0
        ice = ~(mask1 & mask2)
        vs.forc_temp_surface *= ice
        vs.forc_salt_surface *= ice

        # solar radiation
        if vs.enable_tempsalt_sources:
            vs.temp_source[..., :] = (f1 * vs.qsol[..., n1, None] + f2 * vs.qsol[..., n2, None]) \
                * vs.divpen_shortwave[None, None, :] * ice[..., None] \
                * vs.maskT[..., :] / cp_0 / vs.rho_0

    @veros_method
    def set_diagnostics(self, vs):
        average_vars = ['surface_taux', 'surface_tauy', 'forc_temp_surface', 'forc_salt_surface',
                        'psi', 'temp', 'salt', 'u', 'v', 'w', 'Nsqr', 'Hd', 'rho',
                        'K_diss_v', 'P_diss_v', 'P_diss_nonlin', 'P_diss_iso', 'kappaH']
        if vs.enable_skew_diffusion:
            average_vars += ['B1_gm', 'B2_gm']
        if vs.enable_TEM_friction:
            average_vars += ['kappa_gm', 'K_diss_gm']
        if vs.enable_tke:
            average_vars += ['tke', 'Prandtlnumber', 'mxl', 'tke_diss',
                             'forc_tke_surface', 'tke_surf_corr']
        if vs.enable_idemix:
            average_vars += ['E_iw', 'forc_iw_surface', 'forc_iw_bottom', 'iw_diss',
                             'c0', 'v0']
        if vs.enable_eke:
            average_vars += ['eke', 'K_gm', 'L_rossby', 'L_rhines']

        vs.diagnostics['averages'].output_variables = average_vars
        vs.diagnostics['cfl_monitor'].output_frequency = 86400.0
        vs.diagnostics['snapshot'].output_frequency = 365 * 86400 / 24.
        vs.diagnostics['overturning'].output_frequency = 365 * 86400
        vs.diagnostics['overturning'].sampling_frequency = 365 * 86400 / 24.
        vs.diagnostics['energy'].output_frequency = 365 * 86400
        vs.diagnostics['energy'].sampling_frequency = 365 * 86400 / 24.
        vs.diagnostics['averages'].output_frequency = 365 * 86400
        vs.diagnostics['averages'].sampling_frequency = 365 * 86400 / 24.

    @veros_method
    def after_timestep(self, vs):
        pass


@tools.cli
def run(*args, **kwargs):
    simulation = GlobalOneDegreeSetup(*args, **kwargs)
    simulation.setup()
    simulation.run()


if __name__ == '__main__':
    run()

The biggest changes in the new wave propagation setup will be located in the set_grid() set_topography() and set_initial_conditions() methods to accommodate for the new geometry and the interpolation of initial conditions to the modified grid, so we can concentrate on implementing those first.

Step 1: Setup grid

Warning

When using a non-uniform grid,

Step 2: Create idealized topography

Usually, to create an idealized topography, one would simply hand-craft some input and forcing files that reflect the desired changes. However, since we want our setup to have flexible resolution, we will have to write an algorithm that creates these input files for any given number of grid cells. One convenient way to achieve this is by creating some high-resolution masks representing the target topography by hand, and then interpolate these masks to the desired resolution.

Create a mask image

Before we can start, we need to download a high-resolution topography dataset. There are many freely available topographical data sets on the internet; one of them is ETOPO5 (with a resolution of 5 arc-minutes), which we will be using throughout this tutorial. To create a mask image from the topography file, you can use the command line tool veros create-mask, e.g. like

$ veros create-mask ETOPO5_Ice_g_gmt4.nc

This creates a one-to-one representation of the topography file as a PNG image. However, in the case of the 5 arc-minute topography, the resulting image includes a lot of small islands and complicated coastlines that might cause problems when being interpolated to a numerical grid with a much lower resolution. To address this, the create-mask script accepts a scale argument. When given, a Gaussian filter with standard deviation scale (in grid cells) is applied to the resulting image, smoothing out small features. The command

$ veros create-mask ETOPO5_Ice_g_gmt4 --scale 3 3

results in the following mask:

_images/mask-smooth.png

Smoothed topography mask

which looks good enough to serve as a basis for horizontal resolutions of around one degree.

Modify the mask

We can now proceed to mold this realistic version of the global topography into the desired idealized shape. You can use any image editor you have available; one possibility is the free software GIMP. Inside the editor, we can use the pencil tools to create a modified version of the topography mask:

_images/topography_idealized.png

Idealized topography mask

In this modified version, I have

  1. replaced the eastern boundary of the North Atlantic by a meridional line;
  2. removed all lakes and inland seas;
  3. thickened Central America (to prevent North and South America to become disconnected due to interpolation artifacts); and
  4. removed the Arctic Ocean and Hudson Bay.

Now that our topography mask is finished, we can go ahead and implement it in the Veros setup!

Import to Veros

To read the mask in PNG format, we are going to use the Python Imaging Library (PIL).

Step 3: Interpolate forcings & initial conditions

_images/na_mask.png

Mask to identify grid cells in the North Atlantic

Step 4: Set up diagnostics & final touches

Making changes in Veros

Code conventions

When contributing to Veros, please adhere to the following general guidelines:

  • Your first guide should be the surrounding Veros code. Look around, and be consistent with your modifications.
  • Unless you have a very good reason not to do so, please stick to the PEP8 style guide throughout your code. One exception we make in Veros is in regard to the maximum line length - since numerical operations can take up quite a lot of horizontal space, you may use longer lines if it increases readability.
  • Please follow the PEP8 naming conventions, and use meaningful, telling names for your variables, functions, and classes. The variable name stretching_factor is infinitely more meaningful than k. This is especially important for settings and generic helper functions.
  • “Private” helper functions that are not meant to be called from outside the current source file should be prefixed with an underscore (_).
  • Use double quotes (") for all strings longer than a single character.
  • Document your functions using Google-style docstrings. This is especially important if you are implementing a user-facing API (such as a diagnostic, a setup, or tools that are meant to be called from setups).

Distributed memory support

By default, all core routines should support distributed execution via MPI. In this case, every processor only operates on a chunk of the total data. By using veros.variables.allocate(), you can make sure that allocated data always has the right shape.

Since none of the processes have access to the global data, you need to take special care during reductions (e.g. sum) and accumulations (e.g. cumsum) along horizontal dimensions. Use functions from veros.distributed (e.g. veros.distributed.global_max()) where appropriate.

The dist_safe keyword

If you are not comfortable writing code that is safe for distributed execution, you can use the dist_safe keyword to veros.decorators.veros_method()::

@veros_method(dist_safe=False, local_variables=["temp"])
def my_function(vs):
    # this function is now guaranteed to be executed on the main process

    # since temp is declared as a local variable, we have access to all of the data
    vs.temp[2:-2, 2:-2] = np.max(vs.temp)

    # this would throw an error, since salt is not in local_variables
    # vs.salt[...] = 0

    # after execution, the updated contents of vs.temp are scattered to all processes,
    # and distributed execution continues

When encountering a veros_method that is marked as not safe for distributed execution (dist_safe=False), Veros gathers all relevant data from the worker processes, copies it to the main process, and executes the function there. This ensures that you can write your code exactly as in the non-distributed case (but it comes with a performance penalty, of course).

Running tests and benchmarks

If you want to make sure that your changes did not break anything, you can run our test suite that compares the results of each subroutine to pyOM2. To do that, you will need to compile the Python interface of pyOM2 on your machine, and then point the testing suite to the library location, e.g. through:

$ pytest -v . --pyom2-lib /path/to/pyOM2/py_src/pyOM_code.so

from the main folder of the Veros repository.

If you deliberately introduced breaking changes, you can disable them during testing by prefixing them with:

if not vs.pyom_compatibility_mode:
    # your changes

Automated benchmarks are provided in a similar fashion. The benchmarks run some dummy problems with varying problem sizes and all available computational backends: numpy, bohrium-openmp, bohrium-opencl, bohrium-cuda, fortran (pyOM2), and fortran-mpi (parallel pyOM2). For options and further information run:

$ python run_benchmarks.py --help

from the test folder. Timings are written in YAML format.

Performance tweaks

If your changes to Veros turn out to have a negative effect on the runtime of the model, there several ways to investigate and solve performance problems:

  • Run your model with the -v debug option to get additional debugging output (such as timings for each time step, and a timing summary after the run has finished).

  • Run your model with the -p option to profile Veros with pyinstrument. You may have to run pip install pyinstrument before being able to do so. After completion of the run, a file profile.html will be written that can be opened with a web browser and contains timings for the entire call stack.

  • You should try and avoid explicit loops over arrays at all cost (even more so when using Bohrium). You should always try to work on the whole array at once.

  • When using Bohrium, it is sometimes beneficial to copy an array to NumPy before passing it to an external module or performing an operation that cannot be vectorized efficiently. Just don’t forget to copy it back to Bohrium after you are finished, e.g. like so:

    from veros import runtime_settings as rs

if rs.backend == "bohrium":
    u_np = vs.u.copy2numpy()
else:
    u_np = vs.u

vs.u[...] = np.asarray(external_function(u_np))

  • If you are still having trouble, don’t hesitate to ask for help (e.g. on GitHub).

Running Veros on a cluster

This tutorial walks you through some of the most common challenges that are specific to large, shared architectures like clusters and supercomputers. In case you are still having trouble setting up or running Veros on a large architecture after reading it, you should first contact the administrator of your cluster. Otherwise, you should of course feel free to open an issue.

Installation

Probably the easiest way to install Veros on a cluster is to, once again, use Anaconda. Since it is mostly platform independent and does not require elevated permissions, Anaconda is the perfect way to try out Veros without too much hassle.

If you are an administrator and want to make Veros accessible to multiple users on your cluster, we recommend that you do not install Veros system-wide, since it severely limits the possibilities of the users: First of all, they won’t be able to install additional Python modules they might want to use for post-processing or development. And second of all, the source code (and playing with it) is supposed to be a critical part of the Veros experience. Instead, you could e.g. use virtualenv to create a lightweight Python environment for every user that they can freely manage.

Usage

If you want to run Veros on a shared computing architecture, there are several issues that require special handling:

  1. Preventing timeouts: In cloud computing, it is common that scheduling constraints limit the maximum execution time of a given process. Processes that exceed this time are killed. To prevent that long-running processes have to be restarted manually after each timeout, one usually makes use of a resubmit mechanism: The long-running process is split into chunks that each finish before a timeout is triggered, with subsequent runs starting from the restart files that the previous process has written.
  2. Allocation of resources: Most applications use MPI to distribute work across processors; however, this is not supported by Bohrium. We therefore need to make sure that just one single process on a single node is started for our simulation (Bohrium will then divide the workload among different threads using OpenMP).

To solve these issues, the scheduling manager needs to be told exactly how it should run our model, which is usually being done by writing a batch script that prepares the environment and states which resources to request. The exact set-up of such a script will vary depending on the scheduling manager running on your cluster, and how exactly you chose to install Veros and Bohrium. One possible way to write such a batch script for the scheduling manager SLURM is presented here:

#!/bin/bash -l
#
#SBATCH -p mycluster
#SBATCH -A myaccount
#SBATCH --job-name=veros_mysetup
#SBATCH --nodes=2
#SBATCH --ntasks=16
#SBATCH --cpus-per-task=4
#SBATCH --exclusive
#SBATCH --mail-type=ALL
#SBATCH --mail-user=your@email.xyz

# load module dependencies
module load bohrium

# only needed if not found automatically
export BH_CONFIG=/path/to/bohrium/config.ini

# if needed, you can modify the internal Bohrium compiler flags
export BH_OPENMP_COMPILER_FLG="-x c -fPIC -shared -std=gnu99 -O3 -Werror -fopenmp"

# set number of threads to cpus-per-task
export OMP_NUM_THREADS=4

# adapt srun command to your available scheduler / MPI implementation
veros resubmit -i my_run -n 8 -l 7776000 \
    -c "srun --mpi=pmi2 -- python my_setup.py -b bohrium -v debug -n 4 4" \
    --callback "sbatch veros_batch.sh"

which is saved as veros_batch.sh in the model setup folder and called using sbatch.

This script makes use of the veros resubmit command and its --callback option to create a script that automatically re-runs itself in a new process after each successful run (see also Command line tools). Upon execution, a job is created on one node, using 16 processors in one process, that runs the Veros setup located in my_setup.py a total of eight times for 90 days (7776000 seconds) each, with identifier my_run. Note that the --callback "sbatch veros_batch.sh" part of the command is needed to actually create a new job after every run, to prevent the script from being killed after a timeout.

Available model settings

The following list of available settings is automatically created from the file settings.py in the Veros main folder. They are available as attributes of all instances of the Veros state class, e.g.:

>>> simulation = MyVerosSetup()
>>> vs = simulation.state
>>> print(vs.eq_of_state_type)
1
identifier = UNNAMED

Identifier of the current simulation

nx = 0

Grid points in zonal (x) direction

ny = 0

Grid points in meridional (y,j) direction

nz = 0

Grid points in vertical (z,k) direction

dt_mom = 0.0

Time step in seconds for momentum

dt_tracer = 0.0

Time step for tracers, can be larger than dt_mom

runlen = 0.0

Length of simulation in seconds

AB_eps = 0.1

Deviation from Adam-Bashforth weighting

coord_degree = False

either spherical (True) or cartesian (False) coordinates

enable_cyclic_x = False

enable cyclic boundary conditions

eq_of_state_type = 1

equation of state: 1: linear, 3: nonlinear with comp., 5: TEOS

enable_implicit_vert_friction = False

enable implicit vertical friction

enable_explicit_vert_friction = False

enable explicit vertical friction

enable_hor_friction = False

enable horizontal friction

enable_hor_diffusion = False

enable horizontal diffusion

enable_biharmonic_friction = False

enable biharmonic horizontal friction

enable_biharmonic_mixing = False

enable biharmonic horizontal mixing

enable_hor_friction_cos_scaling = False

scaling of hor. viscosity with cos(latitude)**cosPower

enable_ray_friction = False

enable Rayleigh damping

enable_bottom_friction = False

enable bottom friction

enable_bottom_friction_var = False

enable bottom friction with lateral variations

enable_quadratic_bottom_friction = False

enable quadratic bottom friction

enable_tempsalt_sources = False

enable restoring zones, etc

enable_momentum_sources = False

enable restoring zones, etc

enable_superbee_advection = False

enable advection scheme with implicit mixing

enable_conserve_energy = True

exchange energy consistently

enable_store_bottom_friction_tke = False

transfer dissipated energy by bottom/rayleig fric. to TKE, else transfer to internal waves

enable_store_cabbeling_heat = False

transfer non-linear mixing terms to potential enthalpy, else transfer to TKE and EKE

enable_noslip_lateral = False

enable lateral no-slip boundary conditions in harmonic- and biharmonic friction.

congr_epsilon = 1e-12

convergence criteria for Poisson solver

congr_max_iterations = 1000

maximum number of Poisson solver iterations

A_h = 0.0

lateral viscosity in m^2/s

K_h = 0.0

lateral diffusivity in m^2/s

r_ray = 0.0

Rayleigh damping coefficient in 1/s

r_bot = 0.0

bottom friction coefficient in 1/s

r_quad_bot = 0.0

qudratic bottom friction coefficient

hor_friction_cosPower = 3
A_hbi = 0.0

lateral biharmonic viscosity in m^4/s

K_hbi = 0.0

lateral biharmonic diffusivity in m^4/s

kappaH_0 = 0.0
kappaM_0 = 0.0

fixed values for vertical viscosity/diffusivity which are set for no TKE model

enable_neutral_diffusion = False

enable isopycnal mixing

enable_skew_diffusion = False

enable skew diffusion approach for eddy-driven velocities

enable_TEM_friction = False

TEM approach for eddy-driven velocities

K_iso_0 = 0.0

constant for isopycnal diffusivity in m^2/s

K_iso_steep = 0.0

lateral diffusivity for steep slopes in m^2/s

K_gm_0 = 0.0

fixed value for K_gm which is set for no EKE model

iso_dslope = 0.0008

parameters controlling max allowed isopycnal slopes

iso_slopec = 0.001

parameters controlling max allowed isopycnal slopes

enable_idemix = False
tau_v = 172800.0

time scale for vertical symmetrisation

tau_h = 1296000.0

time scale for horizontal symmetrisation

gamma = 1.57
jstar = 5.0

spectral bandwidth in modes

mu0 = 0.3333333333333333

dissipation parameter

enable_idemix_hor_diffusion = False
enable_eke_diss_bottom = False
enable_eke_diss_surfbot = False
eke_diss_surfbot_frac = 1.0

fraction which goes into bottom

enable_idemix_superbee_advection = False
enable_idemix_upwind_advection = False
enable_tke = False
c_k = 0.1
c_eps = 0.7
alpha_tke = 1.0
mxl_min = 1e-12
kappaM_min = 0.0
kappaM_max = 100.0
tke_mxl_choice = 1
enable_tke_superbee_advection = False
enable_tke_upwind_advection = False
enable_tke_hor_diffusion = False
K_h_tke = 2000.0

lateral diffusivity for tke

enable_eke = False
eke_lmin = 100.0

minimal length scale in m

eke_c_k = 1.0
eke_cross = 1.0

Parameter for EKE model

eke_crhin = 1.0

Parameter for EKE model

eke_c_eps = 1.0

Parameter for EKE model

eke_k_max = 10000.0

maximum of K_gm

alpha_eke = 1.0

factor vertical friction

enable_eke_superbee_advection = False
enable_eke_upwind_advection = False
enable_eke_isopycnal_diffusion = False

use K_gm also for isopycnal diffusivity

enable_eke_leewave_dissipation = False
c_lee0 = 1.0
eke_Ri0 = 200.0
eke_Ri1 = 50.0
eke_int_diss0 = 5.787037037037037e-07
kappa_EKE0 = 0.1
eke_r_bot = 0.0

bottom friction coefficient

eke_hrms_k0_min = 0.0

min value for bottom roughness parameter

kappaH_min = 0.0

minimum value for vertical diffusivity

enable_Prandtl_tke = True

Compute Prandtl number from stratification levels in TKE routine

Prandtl_tke0 = 10.0

Constant Prandtl number when stratification is neglected for kappaH computation in TKE routine

use_io_threads = False

Start extra threads for disk writes

io_timeout = 20

Timeout in seconds while waiting for IO locks to be released

enable_netcdf_zlib_compression = True

Use netCDF4’s native zlib interface, which leads to smaller output files (but carries some computational overhead).

enable_hdf5_gzip_compression = True

Use h5py’s native gzip interface, which leads to smaller restart files (but carries some computational overhead).

restart_input_filename =

File name of restart input. If not given, no restart data will be read.

restart_output_filename = {identifier}_{itt:0>4d}.restart.h5

File name of restart output. May contain Python format syntax that is substituted with Veros attributes.

restart_frequency = 0

Frequency (in seconds) to write restart data

force_overwrite = False

Overwrite existing output files

pyom_compatibility_mode = False

Force compatibility to pyOM2 (even reproducing bugs and other quirks). For testing purposes only.

diskless_mode = False

Suppress all output to disk. Mainly used for testing purposes.

default_float_type = float64

Default type to use for floating point arrays (e.g. float32 or float64).

Model variables

The variable meta-data (i.e., all instances of veros.variables.Variable) are available in a dictionary as the attribute Veros.variables. The actual data arrays are added directly as attributes to Veros. The following code snippet (as commonly used in the Diagnostics) illustrates this behavior:

var_meta = {key: val for key, val in vs.variables.items() if val.time_dependent and val.output}
var_data = {key: getattr(veros, key) for key in var_meta.keys()}

In this case, var_meta is a dictionary containing all metadata for variables that are time dependent and should be added to the output, while var_data is a dictionary with the same keys containing the corresponding data arrays.

Variable class

class veros.variables.Variable(name, dims, units, long_description, dtype=None, output=False, time_dependent=True, scale=1.0, write_to_restart=False, extra_attributes=None)[source]

Bases: object

Available variables

There are two kinds of variables in Veros. Main variables are always present in a simulation, while conditional variables are only available if their respective condition is True at the time of variable allocation.

Attributes:
: Time-dependent
: Included in snapshot output by default
: Written to restart files by default

Main variables

Veros.dxt
Units:m
Dimensions:xt
Type:float
Attributes:

Zonal (x) spacing of T-grid point

Veros.dxu
Units:m
Dimensions:xu
Type:float
Attributes:

Zonal (x) spacing of U-grid point

Veros.dyt
Units:m
Dimensions:yt
Type:float
Attributes:

Meridional (y) spacing of T-grid point

Veros.dyu
Units:m
Dimensions:yu
Type:float
Attributes:

Meridional (y) spacing of U-grid point

Veros.zt
Units:m
Dimensions:zt
Type:float
Attributes:

Vertical coordinate

Veros.zw
Units:m
Dimensions:zw
Type:float
Attributes:

Vertical coordinate

Veros.dzt
Units:m
Dimensions:zt
Type:float
Attributes:

Vertical spacing

Veros.dzw
Units:m
Dimensions:zw
Type:float
Attributes:

Vertical spacing

Veros.cost
Units:1
Dimensions:yt
Type:float
Attributes:

Metric factor for spherical coordinates

Veros.cosu
Units:1
Dimensions:yu
Type:float
Attributes:

Metric factor for spherical coordinates

Veros.tantr
Units:1
Dimensions:yt
Type:float
Attributes:

Metric factor for spherical coordinates

Veros.coriolis_t
Units:1/s
Dimensions:xt, yt
Type:float
Attributes:

Coriolis frequency at T grid point

Veros.coriolis_h
Units:1/s
Dimensions:xt, yt
Type:float
Attributes:

Horizontal Coriolis frequency at T grid point

Veros.kbot
Units:
Dimensions:xt, yt
Type:int
Attributes:

Index of the deepest grid cell (counting from 1, 0 means all land)

Veros.ht
Units:m
Dimensions:xt, yt
Type:float
Attributes:

Total depth of the water column

Veros.hu
Units:m
Dimensions:xu, yt
Type:float
Attributes:

Total depth of the water column

Veros.hv
Units:m
Dimensions:xt, yu
Type:float
Attributes:

Total depth of the water column

Veros.hur
Units:m
Dimensions:xu, yt
Type:float
Attributes:

Total depth of the water column (masked)

Veros.hvr
Units:m
Dimensions:xt, yu
Type:float
Attributes:

Total depth of the water column (masked)

Veros.beta
Units:1/(ms)
Dimensions:xt, yt
Type:float
Attributes:

Change of Coriolis frequency with latitude

Veros.area_t
Units:m^2
Dimensions:xt, yt
Type:float
Attributes:

Area of T-box

Veros.area_u
Units:m^2
Dimensions:xu, yt
Type:float
Attributes:

Area of U-box

Veros.area_v
Units:m^2
Dimensions:xt, yu
Type:float
Attributes:

Area of V-box

Veros.maskT
Units:
Dimensions:xt, yt, zt
Type:int8
Attributes:

Mask in physical space for tracer points

Veros.maskU
Units:
Dimensions:xu, yt, zt
Type:int8
Attributes:

Mask in physical space for U points

Veros.maskV
Units:
Dimensions:xt, yu, zt
Type:int8
Attributes:

Mask in physical space for V points

Veros.maskW
Units:
Dimensions:xt, yt, zw
Type:int8
Attributes:

Mask in physical space for W points

Veros.maskZ
Units:
Dimensions:xu, yu, zt
Type:int8
Attributes:

Mask in physical space for Zeta points

Veros.rho
Units:kg/m^3
Dimensions:xt, yt, zt, timesteps
Type:float
Attributes:

In-situ density anomaly, relative to the surface mean value of 1024 kg/m^3

Veros.prho
Units:kg/m^3
Dimensions:xt, yt, zt
Type:float
Attributes:

Potential density anomaly, relative to the surface mean value of 1024 kg/m^3 (equal to in-situ density anomaly for equation of state 1 to 4)

Veros.int_drhodT
Units:?
Dimensions:xt, yt, zt, timesteps
Type:float
Attributes:

Partial derivative of dynamic enthalpy by temperature

Veros.int_drhodS
Units:?
Dimensions:xt, yt, zt, timesteps
Type:float
Attributes:

Partial derivative of dynamic enthalpy by salinity

Veros.Nsqr
Units:1/s^2
Dimensions:xt, yt, zw, timesteps
Type:float
Attributes:

Square of stability frequency

Veros.Hd
Units:m^2/s^2
Dimensions:xt, yt, zt, timesteps
Type:float
Attributes:

Dynamic enthalpy

Veros.dHd
Units:m^2/s^3
Dimensions:xt, yt, zt, timesteps
Type:float
Attributes:

Change of dynamic enthalpy due to advection

Veros.temp
Units:deg C
Dimensions:xt, yt, zt, timesteps
Type:float
Attributes:

Conservative temperature

Veros.dtemp
Units:deg C/s
Dimensions:xt, yt, zt, timesteps
Type:float
Attributes:

Conservative temperature tendency

Veros.salt
Units:g/kg
Dimensions:xt, yt, zt, timesteps
Type:float
Attributes:

Salinity

Veros.dsalt
Units:g/(kg s)
Dimensions:xt, yt, zt, timesteps
Type:float
Attributes:

Salinity tendency

Veros.dtemp_vmix
Units:deg C/s
Dimensions:xt, yt, zt
Type:float
Attributes:

Change of temperature due to vertical mixing

Veros.dtemp_hmix
Units:deg C/s
Dimensions:xt, yt, zt
Type:float
Attributes:

Change of temperature due to horizontal mixing

Veros.dsalt_vmix
Units:deg C/s
Dimensions:xt, yt, zt
Type:float
Attributes:

Change of salinity due to vertical mixing

Veros.dsalt_hmix
Units:deg C/s
Dimensions:xt, yt, zt
Type:float
Attributes:

Change of salinity due to horizontal mixing

Veros.dtemp_iso
Units:deg C/s
Dimensions:xt, yt, zt
Type:float
Attributes:

Change of temperature due to isopycnal mixing plus skew mixing

Veros.dsalt_iso
Units:deg C/s
Dimensions:xt, yt, zt
Type:float
Attributes:

Change of salinity due to isopycnal mixing plus skew mixing

Veros.forc_temp_surface
Units:m K/s
Dimensions:xt, yt
Type:float
Attributes:

Surface temperature flux

Veros.forc_salt_surface
Units:m g/s kg
Dimensions:xt, yt
Type:float
Attributes:

Surface salinity flux

Veros.flux_east
Units:?
Dimensions:xu, yt, zt
Type:float
Attributes:

Multi-purpose flux

Veros.flux_north
Units:?
Dimensions:xt, yu, zt
Type:float
Attributes:

Multi-purpose flux

Veros.flux_top
Units:?
Dimensions:xt, yt, zw
Type:float
Attributes:

Multi-purpose flux

Veros.u
Units:m/s
Dimensions:xu, yt, zt, timesteps
Type:float
Attributes:

Zonal velocity

Veros.v
Units:m/s
Dimensions:xt, yu, zt, timesteps
Type:float
Attributes:

Meridional velocity

Veros.w
Units:m/s
Dimensions:xt, yt, zw, timesteps
Type:float
Attributes:

Vertical velocity

Veros.du
Units:m/s
Dimensions:xu, yt, zt, timesteps
Type:float
Attributes:

Zonal velocity tendency

Veros.dv
Units:m/s
Dimensions:xt, yu, zt, timesteps
Type:float
Attributes:

Meridional velocity tendency

Veros.du_cor
Units:m/s^2
Dimensions:xu, yt, zt
Type:float
Attributes:

Change of u due to Coriolis force

Veros.dv_cor
Units:m/s^2
Dimensions:xt, yu, zt
Type:float
Attributes:

Change of v due to Coriolis force

Veros.du_mix
Units:m/s^2
Dimensions:xu, yt, zt
Type:float
Attributes:

Change of u due to implicit vertical mixing

Veros.dv_mix
Units:m/s^2
Dimensions:xt, yu, zt
Type:float
Attributes:

Change of v due to implicit vertical mixing

Veros.du_adv
Units:m/s^2
Dimensions:xu, yt, zt
Type:float
Attributes:

Change of u due to advection

Veros.dv_adv
Units:m/s^2
Dimensions:xt, yu, zt
Type:float
Attributes:

Change of v due to advection

Veros.p_hydro
Units:m^2/s^2
Dimensions:xt, yt, zt
Type:float
Attributes:

Hydrostatic pressure

Veros.kappaM
Units:m^2/s
Dimensions:xt, yt, zt
Type:float
Attributes:

Vertical viscosity

Veros.kappaH
Units:m^2/s
Dimensions:xt, yt, zw
Type:float
Attributes:

Vertical diffusivity

Veros.surface_taux
Units:N/s^2
Dimensions:xu, yt
Type:float
Attributes:

Zonal surface wind stress

Veros.surface_tauy
Units:N/s^2
Dimensions:xt, yu
Type:float
Attributes:

Meridional surface wind stress

Veros.forc_rho_surface
Units:?
Dimensions:xt, yt
Type:float
Attributes:

Surface potential density flux

Veros.psi
Units:m^3/s
Dimensions:xu, yu, timesteps
Type:float
Attributes:

Barotropic streamfunction

Veros.dpsi
Units:m^3/s^2
Dimensions:xu, yu, timesteps
Type:float
Attributes:

Streamfunction tendency

Veros.land_map
Units:
Dimensions:xt, yt
Type:float
Attributes:

Land map

Veros.isle
Units:
Dimensions:isle
Type:float
Attributes:

Island number

Veros.psin
Units:m^3/s
Dimensions:xu, yu, isle
Type:float
Attributes:

Boundary streamfunction

Veros.dpsin
Units:?
Dimensions:isle, timesteps
Type:float
Attributes:

Boundary streamfunction factor

Veros.line_psin
Units:?
Dimensions:isle, isle
Type:float
Attributes:

Boundary line integrals

Veros.boundary_mask
Units:
Dimensions:xt, yt, isle
Type:float
Attributes:

Boundary mask

Veros.line_dir_south_mask
Units:
Dimensions:xt, yt, isle
Type:float
Attributes:

Line integral mask

Veros.line_dir_north_mask
Units:
Dimensions:xt, yt, isle
Type:float
Attributes:

Line integral mask

Veros.line_dir_east_mask
Units:
Dimensions:xt, yt, isle
Type:float
Attributes:

Line integral mask

Veros.line_dir_west_mask
Units:
Dimensions:xt, yt, isle
Type:float
Attributes:

Line integral mask

Veros.K_gm
Units:m^2/s
Dimensions:xt, yt, zw
Type:float
Attributes:

GM diffusivity, either constant or from EKE model

Veros.K_iso
Units:m^2/s
Dimensions:xt, yt, zw
Type:float
Attributes:

Along-isopycnal diffusivity

Veros.K_diss_v
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Kinetic energy dissipation by vertical, rayleigh and bottom friction

Veros.K_diss_bot
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Mean energy dissipation by bottom and rayleigh friction

Veros.K_diss_h
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Kinetic energy dissipation by horizontal friction

Veros.K_diss_gm
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Mean energy dissipation by GM (TRM formalism only)

Veros.P_diss_v
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Potential energy dissipation by vertical diffusion

Veros.P_diss_nonlin
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Potential energy dissipation by nonlinear equation of state

Veros.P_diss_iso
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Potential energy dissipation by isopycnal mixing

Veros.P_diss_skew
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Potential energy dissipation by GM (w/o TRM)

Veros.P_diss_hmix
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Potential energy dissipation by horizontal mixing

Veros.P_diss_adv
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Potential energy dissipation by advection

Veros.P_diss_comp
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Potential energy dissipation by compression

Veros.P_diss_sources
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Potential energy dissipation by external sources (e.g. restoring zones)

Veros.u_wgrid
Units:m/s
Dimensions:xt, yt, zw
Type:float
Attributes:

Zonal velocity interpolated to W grid points

Veros.v_wgrid
Units:m/s
Dimensions:xt, yt, zw
Type:float
Attributes:

Meridional velocity interpolated to W grid points

Veros.w_wgrid
Units:m/s
Dimensions:xt, yt, zw
Type:float
Attributes:

Vertical velocity interpolated to W grid points

Conditional variables

coord_degree
Veros.xt
Units:degrees_east
Dimensions:xt
Type:float
Attributes:

Zonal (x) coordinate of T-grid point

Veros.xu
Units:degrees_east
Dimensions:xu
Type:float
Attributes:

Zonal (x) coordinate of U-grid point

Veros.yt
Units:degrees_north
Dimensions:yt
Type:float
Attributes:

Meridional (y) coordinate of T-grid point

Veros.yu
Units:degrees_north
Dimensions:yu
Type:float
Attributes:

Meridional (y) coordinate of U-grid point

not coord_degree
Veros.xt
Units:km
Dimensions:xt
Type:float
Attributes:

Zonal (x) coordinate of T-grid point

Veros.xu
Units:km
Dimensions:xu
Type:float
Attributes:

Zonal (x) coordinate of U-grid point

Veros.yt
Units:km
Dimensions:yt
Type:float
Attributes:

Meridional (y) coordinate of T-grid point

Veros.yu
Units:km
Dimensions:yu
Type:float
Attributes:

Meridional (y) coordinate of U-grid point

enable_tempsalt_sources
Veros.temp_source
Units:K/s
Dimensions:xt, yt, zt
Type:float
Attributes:

Non-conservative source of temperature

Veros.salt_source
Units:g/(kg s)
Dimensions:xt, yt, zt
Type:float
Attributes:

Non-conservative source of salt

enable_momentum_sources
Veros.u_source
Units:m/s^2 (?)
Dimensions:xu, yt, zt
Type:float
Attributes:

Non-conservative source of zonal velocity

Veros.v_source
Units:m/s^2 (?)
Dimensions:xt, yu, zt
Type:float
Attributes:

Non-conservative source of meridional velocity

enable_neutral_diffusion
Veros.K_11
Units:?
Dimensions:xt, yt, zt
Type:float
Attributes:

Isopycnal mixing tensor component

Veros.K_13
Units:?
Dimensions:xt, yt, zt
Type:float
Attributes:

Isopycnal mixing tensor component

Veros.K_22
Units:?
Dimensions:xt, yt, zt
Type:float
Attributes:

Isopycnal mixing tensor component

Veros.K_23
Units:?
Dimensions:xt, yt, zt
Type:float
Attributes:

Isopycnal mixing tensor component

Veros.K_31
Units:?
Dimensions:xt, yt, zt
Type:float
Attributes:

Isopycnal mixing tensor component

Veros.K_32
Units:?
Dimensions:xt, yt, zt
Type:float
Attributes:

Isopycnal mixing tensor component

Veros.K_33
Units:?
Dimensions:xt, yt, zt
Type:float
Attributes:

Isopycnal mixing tensor component

Veros.Ai_ez
Units:?
Dimensions:xt, yt, zt, tensor1, tensor2
Type:float
Attributes:

?

Veros.Ai_nz
Units:?
Dimensions:xt, yt, zt, tensor1, tensor2
Type:float
Attributes:

?

Veros.Ai_bx
Units:?
Dimensions:xt, yt, zt, tensor1, tensor2
Type:float
Attributes:

?

Veros.Ai_by
Units:?
Dimensions:xt, yt, zt, tensor1, tensor2
Type:float
Attributes:

?

enable_skew_diffusion
Veros.B1_gm
Units:m^2/s
Dimensions:xt, yu, zt
Type:float
Attributes:

Zonal component of GM streamfunction

Veros.B2_gm
Units:m^2/s
Dimensions:xu, yt, zt
Type:float
Attributes:

Meridional component of GM streamfunction

enable_bottom_friction_var
Veros.r_bot_var_u
Units:?
Dimensions:xu, yt
Type:float
Attributes:

Zonal bottom friction coefficient

Veros.r_bot_var_v
Units:?
Dimensions:xt, yu
Type:float
Attributes:

Meridional bottom friction coefficient

enable_TEM_friction
Veros.kappa_gm
Units:m^2/s
Dimensions:xt, yt, zw
Type:float
Attributes:

Vertical diffusivity

enable_tke
Veros.tke
Units:m^2/s^2
Dimensions:xt, yt, zw, timesteps
Type:float
Attributes:

Turbulent kinetic energy

Veros.sqrttke
Units:m/s
Dimensions:xt, yt, zw
Type:float
Attributes:

Square-root of TKE

Veros.dtke
Units:m^2/s^3
Dimensions:xt, yt, zw, timesteps
Type:float
Attributes:

Turbulent kinetic energy tendency

Veros.Prandtlnumber
Units:
Dimensions:xt, yt, zw
Type:float
Attributes:

Prandtl number

Veros.mxl
Units:m
Dimensions:xt, yt, zw
Type:float
Attributes:

Mixing length

Veros.forc_tke_surface
Units:m^3/s^3
Dimensions:xt, yt
Type:float
Attributes:

TKE surface flux

Veros.tke_diss
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

TKE dissipation

Veros.tke_surf_corr
Units:m^3/s^3
Dimensions:xt, yt
Type:float
Attributes:

Correction of TKE surface flux

enable_eke
Veros.eke
Units:m^2/s^2
Dimensions:xt, yt, zw, timesteps
Type:float
Attributes:

meso-scale energy

Veros.deke
Units:m^2/s^3
Dimensions:xt, yt, zw, timesteps
Type:float
Attributes:

meso-scale energy tendency

Veros.sqrteke
Units:m/s
Dimensions:xt, yt, zw
Type:float
Attributes:

square-root of eke

Veros.L_rossby
Units:m
Dimensions:xt, yt
Type:float
Attributes:

Rossby radius

Veros.L_rhines
Units:m
Dimensions:xt, yt, zw
Type:float
Attributes:

Rhines scale

Veros.eke_len
Units:m
Dimensions:xt, yt, zt
Type:float
Attributes:

Eddy length scale

Veros.eke_diss_iw
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Dissipation of EKE to internal waves

Veros.eke_diss_tke
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Dissipation of EKE to TKE

Veros.eke_bot_flux
Units:m^3/s^3
Dimensions:xt, yt
Type:float
Attributes:

Flux by bottom friction

enable_eke_leewave_dissipation
Veros.eke_topo_hrms
Units:?
Dimensions:xt, yt
Type:float
Attributes:

?

Veros.eke_topo_lam
Units:?
Dimensions:xt, yt
Type:float
Attributes:

?

Veros.hrms_k0
Units:?
Dimensions:xt, yt
Type:float
Attributes:

?

Veros.c_lee
Units:1/s
Dimensions:xt, yt
Type:float
Attributes:

Lee wave dissipation coefficient

Veros.eke_lee_flux
Units:m^3/s^3
Dimensions:xt, yt
Type:float
Attributes:

Lee wave flux

Veros.c_Ri_diss
Units:1/s
Dimensions:xt, yt, zw
Type:float
Attributes:

Interior dissipation coefficient

enable_idemix
Veros.E_iw
Units:m^2/s^2
Dimensions:xt, yt, zw, timesteps
Type:float
Attributes:

Internal wave energy

Veros.dE_iw
Units:m^2/s^2
Dimensions:xt, yt, zw, timesteps
Type:float
Attributes:

Internal wave energy tendency

Veros.c0
Units:m/s
Dimensions:xt, yt, zw
Type:float
Attributes:

Vertical internal wave group velocity

Veros.v0
Units:m/s
Dimensions:xt, yt, zw
Type:float
Attributes:

Horizontal internal wave group velocity

Veros.alpha_c
Units:?
Dimensions:xt, yt, zw
Type:float
Attributes:

?

Veros.iw_diss
Units:m^2/s^3
Dimensions:xt, yt, zw
Type:float
Attributes:

Internal wave dissipation

Veros.forc_iw_surface
Units:m^3/s^3
Dimensions:xt, yt
Type:float
Attributes:

Internal wave surface forcing

Veros.forc_iw_bottom
Units:m^3/s^3
Dimensions:xt, yt
Type:float
Attributes:

Internal wave bottom forcing

Diagnostics

Diagnostics are separate objects (instances of subclasses of VerosDiagnostic) responsible for handling I/O, restart mechanics, and monitoring of the numerical solution. All available diagnostics are instantiated and added to a dictionary attribute VerosState.diagnostics (with a key determined by their name attribute). Options for diagnostics may be set during the VerosSetup.set_diagnostics() method:

class MyModelSetup(VerosSetup):
    ...
    def set_diagnostics(self, vs):
        vs.diagnostics['averages'].output_variables = ['psi','u','v']
        vs.diagnostics['averages'].sampling_frequency = 3600.
        vs.diagnostics['snapshot'].output_variables += ['du']

Base class

This class implements some common logic for all diagnostics. This makes it easy to write your own diagnostics: Just derive from this class, and implement the virtual functions.

class veros.diagnostics.diagnostic.VerosDiagnostic(vs)[source]

Bases: object

Base class for diagnostics. Provides an interface and wrappers for common I/O.

Any diagnostic needs to implement the five interface methods and set some attributes.

name = None

Name that identifies the current diagnostic

initialize(vs)

Called at the end of setup. Use this to process user settings and handle setup.

diagnose(vs)

Called with frequency sampling_frequency.

output(vs)

Called with frequency output_frequency.

write_restart(vs)

Responsible for writing restart files.

read_restart(vs)

Responsible for reading restart files.

Available diagnostics

Currently, the following diagnostics are implemented and added to VerosState.diagnostics:

Snapshot

class veros.diagnostics.snapshot.Snapshot(vs)[source]

Bases: veros.diagnostics.diagnostic.VerosDiagnostic

Writes snapshots of the current solution. Also reads and writes the main restart data required for restarting a Veros simulation.

output_path = '{identifier}.snapshot.nc'

File to write to. May contain format strings that are replaced with Veros attributes.

name = 'snapshot'
output_frequency = None

Frequency (in seconds) in which output is written.

output_variables = None

Variables to be written to output. Defaults to all Veros variables that have the attribute output.

restart_variables = None

Variables to be written to restart. Defaults to all Veros variables that have the attribute write_to_restart.

Averages

class veros.diagnostics.averages.Averages(vs)[source]

Bases: veros.diagnostics.diagnostic.VerosDiagnostic

Time average output diagnostic.

All registered variables are summed up when diagnose() is called, and averaged and output upon calling output().

name = 'averages'
output_path = '{identifier}.averages.nc'

File to write to. May contain format strings that are replaced with Veros attributes.

output_variables = None

Iterable containing all variables to be averaged. Changes have no effect after initialize has been called.

output_frequency = None

Frequency (in seconds) in which output is written.

sampling_frequency = None

Frequency (in seconds) in which variables are accumulated.

CFL monitor

class veros.diagnostics.cfl_monitor.CFLMonitor(vs)[source]

Bases: veros.diagnostics.diagnostic.VerosDiagnostic

Diagnostic monitoring the maximum CFL number of the solution to detect instabilities.

Writes output to stdout (no binary output).

name = 'cfl_monitor'

Tracer monitor

class veros.diagnostics.tracer_monitor.TracerMonitor(vs)[source]

Bases: veros.diagnostics.diagnostic.VerosDiagnostic

Diagnostic monitoring global tracer contents / fluxes.

Writes output to stdout (no binary output).

name = 'tracer_monitor'
output_frequency = None

Frequency (in seconds) in which output is written.

Energy

class veros.diagnostics.energy.Energy(vs)[source]

Bases: veros.diagnostics.diagnostic.VerosDiagnostic

Diagnose globally averaged energy cycle. Also averages energy in time.

name = 'energy'
output_path = '{identifier}.energy.nc'

File to write to. May contain format strings that are replaced with Veros attributes.

output_frequency = None

Frequency (in seconds) in which output is written.

sampling_frequency = None

Frequency (in seconds) in which variables are accumulated.

Overturning

class veros.diagnostics.overturning.Overturning(vs)[source]

Bases: veros.diagnostics.diagnostic.VerosDiagnostic

Isopycnal overturning diagnostic. Computes and writes vertical streamfunctions (zonally averaged).

name = 'overturning'
output_path = '{identifier}.overturning.nc'

File to write to. May contain format strings that are replaced with Veros attributes.

output_frequency = None

Frequency (in seconds) in which output is written.

sampling_frequency = None

Frequency (in seconds) in which variables are accumulated.

p_ref = 2000.0

Reference pressure for isopycnals

Command line tools

After installing Veros, you can call these scripts from the command line from any location on your system.

veros

This is a wrapper script that provides easy access to all Veros command line tools.

Usage: veros [OPTIONS] COMMAND [ARGS]...

  Veros command-line tools

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  copy-setup   Copy a standard setup to another directory
  create-mask  Creates a mask image from a given netCDF file
  resubmit     Re-run a Veros setup several times

veros-create-mask

Usage: veros create-mask [OPTIONS] INFILE

  Creates a mask image from a given netCDF file

Options:
  -v, --variable TEXT     Variable holding topography data (default: z)
  -o, --outfile TEXT      Output filename (default: topography.png)
  -s, --scale INTEGER...  Standard deviation in grid cells for Gaussian
                          smoother (default: disable smoother)
  --help                  Show this message and exit.

veros-copy-setup

Usage: veros copy-setup [OPTIONS] [acc|acc_sector|global_1deg|global_4deg|glob
                        al_flexible|north_atlantic|wave_propagation]

  Copy a standard setup to another directory

Options:
  --to PATH  Target directory (default: copy to current working directory)
  --help     Show this message and exit.

veros-resubmit

Usage: veros resubmit [OPTIONS]

  Performs several runs of Veros back to back, using the previous run as
  restart input.

  Intended to be used with scheduling systems (e.g. SLURM or PBS).

Options:
  -i, --identifier TEXT       Base identifier of the simulation  [required]
  -n, --num-runs INTEGER      Total number of runs to execute  [required]
  -l, --length-per-run FLOAT  Length (in seconds) of each run  [required]
  -c, --veros-cmd COMMAND     The command that is used to call veros (quoted)
                              [required]
  --callback CMD              Command to call after each run has finished
                              (quoted, default: call self)
  --help                      Show this message and exit.

Python API

Veros setup class

class veros.VerosSetup(state=None, override=None)[source]

Bases: object

Main class for Veros, used for building a model and running it.

Note

This class is meant to be subclassed. Subclasses need to implement the methods set_parameter(), set_topography(), set_grid(), set_coriolis(), set_initial_conditions(), set_forcing(), and set_diagnostics().

Parameters:
  • backend (bool, optional) – Backend to use for array operations. Possible values are numpy and bohrium. Defaults to None, which tries to read the backend from the command line (set via a flag -b/--backend), and uses numpy if no command line argument is given.
  • loglevel (one of {debug, info, warning, error, critical}, optional) – Verbosity of the model. Tries to read value from command line if not given (-v/--loglevel). Defaults to info.

Example

>>> import matplotlib.pyplot as plt
>>> from veros import VerosSetup
>>>
>>> class MyModel(VerosSetup):
>>>     ...
>>>
>>> simulation = MyModel(backend='bohrium')
>>> simulation.run()
>>> plt.imshow(simulation.state.psi[..., 0])
>>> plt.show()
set_parameter(vs)[source]

To be implemented by subclass.

First function to be called during setup. Use this to modify the model settings.

Example

>>> def set_parameter(self, vs):
>>>     vs.nx, vs.ny, vs.nz = (360, 120, 50)
>>>     vs.coord_degree = True
>>>     vs.enable_cyclic = True
set_initial_conditions(vs)[source]

To be implemented by subclass.

May be used to set initial conditions.

Example

>>> @veros_method
>>> def set_initial_conditions(self, vs):
>>>     vs.u[:, :, :, vs.tau] = np.random.rand(vs.u.shape[:-1])
set_grid(vs)[source]

To be implemented by subclass.

Has to set the grid spacings dxt, dyt, and dzt, along with the coordinates of the grid origin, x_origin and y_origin.

Example

>>> @veros_method
>>> def set_grid(self, vs):
>>>     vs.x_origin, vs.y_origin = 0, 0
>>>     vs.dxt[...] = [0.1, 0.05, 0.025, 0.025, 0.05, 0.1]
>>>     vs.dyt[...] = 1.
>>>     vs.dzt[...] = [10, 10, 20, 50, 100, 200]
set_coriolis(vs)[source]

To be implemented by subclass.

Has to set the Coriolis parameter coriolis_t at T grid cells.

Example

>>> @veros_method
>>> def set_coriolis(self, vs):
>>>     vs.coriolis_t[:, :] = 2 * vs.omega * np.sin(vs.yt[np.newaxis, :] / 180. * vs.pi)
set_topography(vs)[source]

To be implemented by subclass.

Must specify the model topography by setting kbot.

Example

>>> @veros_method
>>> def set_topography(self, vs):
>>>     vs.kbot[:, :] = 10
>>>     # add a rectangular island somewhere inside the domain
>>>     vs.kbot[10:20, 10:20] = 0
set_forcing(vs)[source]

To be implemented by subclass.

Called before every time step to update the external forcing, e.g. through forc_temp_surface, forc_salt_surface, surface_taux, surface_tauy, forc_tke_surface, temp_source, or salt_source. Use this method to implement time-dependent forcing.

Example

>>> @veros_method
>>> def set_forcing(self, vs):
>>>     current_month = (vs.time / (31 * 24 * 60 * 60)) % 12
>>>     vs.surface_taux[:, :] = vs._windstress_data[:, :, current_month]
set_diagnostics(vs)[source]

To be implemented by subclass.

Called before setting up the diagnostics. Use this method e.g. to mark additional variables for output.

Example

>>> @veros_method
>>> def set_diagnostics(self, vs):
>>>     vs.diagnostics['snapshot'].output_vars += ['drho', 'dsalt', 'dtemp']
after_timestep(vs)[source]

Called at the end of each time step. Can be used to define custom, setup-specific events.

run(show_progress_bar=None)[source]

Main routine of the simulation.

Note

Make sure to call setup() prior to this function.

Parameters:show_progress_bar (bool, optional) – Whether to show fancy progress bar via tqdm. By default, only show if stdout is a terminal and Veros is running on a single process.

Veros state class

class veros.state.VerosState[source]

Bases: object

Holds all settings and model state for a given Veros run.

veros_method decorator

veros.veros_method(function=None, **kwargs)[source]

Decorator that injects the current backend as variable np into the wrapped function.

Note

This decorator should be applied to all functions that make use of the computational backend (even when subclassing veros.Veros). The first argument to the decorated function must be a Veros instance.

Example

>>> from veros import Veros, veros_method
>>>
>>> class MyModel(Veros):
>>>     @veros_method
>>>     def set_topography(self):
>>>         self.kbot[...] = np.random.randint(0, self.nz, size=self.kbot.shape)

Runtime settings & state

Runtime settings

class veros.runtime.RuntimeSettings[source]

Bases: object

Runtime state

class veros.runtime.RuntimeState[source]

Bases: object

Unifies attributes from various modules in a simple read-only object

proc_rank
proc_num
proc_idx
backend_module
vector_engine

Tools & utilities

Assets

veros.tools.assets.get_assets(asset_id, asset_file)[source]

Handles automatic download and verification of external assets (such as forcing files).

By default, assets are stored in $HOME/.veros/assets (can be overwritten by setting VEROS_ASSET_DIR environment variable to the desired location).

Parameters:
  • asset_id (str) – Identifier of the collection of assets. Should be unique for each setup.
  • asset_file (str) – YAML file containing URLs and (optionally) MD5 hashsums of each asset.
Returns:

A dict mapping identifier of each asset to file name on disk.

Example

>>> get_assets('mysetup', 'assets.yml')
{
    "forcing": "/home/user/.veros/assets/mysetup/mysetup_forcing.h5",
    "initial_conditions": "/home/user/.veros/assets/mysetup/initial.h5"
}

In this case, assets.yml contains:

forcing:
    url: https://mywebsite.com/veros_assets/mysetup_forcing.h5
    md5: ef3be0a58782771c8ee5a6d0206b87f6

initial_conditions:
    url: https://mywebsite.com/veros_assets/initial.h5
    md5: d1b4e0e199d7a5883cf7c88d3d6bcb28

CLI

veros.tools.cli.cli(run)[source]

Decorator that wraps the decorated function with the Veros setup command line interface.

Example

>>> @veros.tools.cli.cli()
>>> def run_setup(override):
...     sim = MyVerosSetup(override=override)
...     sim.run()
...
>>> if __name__ == '__main__':
...     run_setup()

This script then automatically supports settings to be specified from the command line:

$ python my_setup.py --help
Usage: my_setup.py [OPTIONS]

Options:
-b, --backend [numpy|bohrium]   Backend to use for computations (default:
                                numpy)
-v, --loglevel [trace|debug|info|warning|error|critical]
                                Log level used for output (default: info)
-s, --override SETTING VALUE    Override default setting, may be specified
                                multiple times
-p, --profile-mode              Write a performance profile for debugging
                                (default: false)
-n, --num-proc INTEGER...       Number of processes in x and y dimension
                                (requires execution via mpirun)
--help                          Show this message and exit.

Setup tools

veros.tools.setup.interpolate(coords, var, interp_coords, missing_value=None, fill=True, kind='linear')[source]

Interpolate globally defined data to a different (regular) grid.

Parameters:
  • coords – Tuple of coordinate arrays for each dimension.
  • var (ndarray of dim (nx1, …, nxd)) – Variable data to interpolate.
  • interp_coords – Tuple of coordinate arrays to interpolate to.
  • missing_value (optional) – Value denoting cells of missing data in var. Is replaced by NaN before interpolating. Defaults to None, which means no replacement is taking place.
  • fill (bool, optional) – Whether NaN values should be replaced by the nearest finite value after interpolating. Defaults to True.
  • kind (str, optional) – Order of interpolation. Supported are nearest and linear (default).
Returns:

ndarray containing the interpolated values on the grid spanned by interp_coords.
veros.tools.setup.fill_holes(data)[source]

A simple inpainting function that replaces NaN values in data with the nearest finite value.

veros.tools.setup.get_periodic_interval(current_time, cycle_length, rec_spacing, n_rec)[source]

Used for linear interpolation between periodic time intervals.

One common application is the interpolation of external forcings that are defined at discrete times (e.g. one value per month of a standard year) to the current time step.

Parameters:
  • current_time (float) – Time to interpolate to.
  • cycle_length (float) – Total length of one periodic cycle.
  • rec_spacing (float) – Time spacing between each data record.
  • n_rec (int) – Total number of records available.
Returns:

Indices and weights for the interpolated record array.
Return type:

tuple containing (n1, f1), (n2, f2)

Example

The following interpolates a record array data containing 12 monthly values to the current time step:

>>> year_in_seconds = 60. * 60. * 24. * 365.
>>> current_time = 60. * 60. * 24. * 45. # mid-february
>>> print(data.shape)
(360, 180, 12)
>>> (n1, f1), (n2, f2) = get_periodic_interval(current_time, year_in_seconds, year_in_seconds / 12, 12)
>>> data_at_current_time = f1 * data[..., n1] + f2 * data[..., n2]
veros.tools.setup.make_cyclic(longitude, array=None, wrap=360.0)[source]

Create a cyclic version of a longitude array and (optionally) another array.

Parameters:
  • longitude (ndarray) – Longitude array of shape (nlon, …).
  • array (ndarray) – Another array that is to be made cyclic of shape (nlon, …).
  • wrap (float) – Wrapping value, defaults to 360 (degrees).
Returns:

Tuple containing (cyclic_longitudes, cyclic_array) if array is given, otherwise just the ndarray cyclic_longitudes of shape (2 * nlon, …).
veros.tools.setup.get_coastline_distance(coords, coast_mask, spherical=False, radius=None, num_candidates=None, n_jobs=-1)[source]

Calculate the (approximate) distance of each water cell from the nearest coastline.

Parameters:
  • coords (tuple of ndarrays) – Tuple containing x and y (longitude and latitude) coordinate arrays of shape (nx, ny).
  • coast_mask (ndarray) – Boolean mask indicating whether a cell is a land cell (must be same shape as coordinate arrays).
  • spherical (bool) – Use spherical instead of Cartesian coordinates. When this is True, cyclical boundary conditions are used, and the resulting distances are only approximate. Cells are pre-sorted by Euclidean lon-lat distance, and great circle distances are calculated for the first num_candidates elements. Defaults to False.
  • radius (float) – Radius of spherical coordinate system. Must be given when spherical is True.
  • num_candidates (int) – Number of candidates to calculate great circle distances for for each water cell. The higher this value, the more accurate the returned distances become when spherical is True. Defaults to the square root of the number of coastal cells.
  • n_jobs (int) – Number of parallel jobs to determine nearest neighbors (defaults to -1, which uses all available threads).
Returns:

ndarray of shape (nx, ny) indicating the distance to the nearest land cell (0 if cell is land).

Example

The following returns coastal distances of all T cells for a spherical Veros setup.

>>> coords = np.meshgrid(self.xt[2:-2], self.yt[2:-2], indexing='ij')
>>> dist = tools.get_coastline_distance(coords, self.kbot > 0, spherical=True, radius=self.radius)
veros.tools.setup.get_uniform_grid_steps(total_length, stepsize)[source]

Get uniform grid step sizes in an interval.

Parameters:
  • total_length (float) – total length of the resulting grid
  • stepsize (float) – grid step size
Returns:

ndarray of grid steps

Example

>>> uniform_steps = uniform_grid_setup(6., 0.25)
>>> uniform_steps
[ 0.25,  0.25,  0.25,  0.25,  0.25,  0.25,  0.25,  0.25,  0.25,
  0.25,  0.25,  0.25,  0.25,  0.25,  0.25,  0.25,  0.25,  0.25,
  0.25,  0.25,  0.25,  0.25,  0.25,  0.25 ]
veros.tools.setup.get_stretched_grid_steps(n_cells, total_length, minimum_stepsize, stretching_factor=2.5, two_sided_grid=False, refine_towards='upper')[source]

Computes stretched grid steps for regional and global domains with either one or two-sided stretching using a hyperbolic tangent stretching function.

Parameters:
  • n_cells (int) – Number of grid points.
  • total_length (float) – Length of the grid interval to be covered (sum of the resulting grid steps).
  • minimum_stepsize (float) – Grid step size on the lower end of the interval.
  • stretching_factor (float, optional) – Coefficient of the tanh stretching function. The higher this value, the more abrupt the step sizes change.
  • two_sided_grid (bool, optional) – If set to True, the resulting grid will be symmetrical around the center. Defaults to False.
  • refine_towards ('upper' or 'lower', optional) – The side of the interval that is to be refined. Defaults to ‘upper’.
Returns:

ndarray of shape (n_cells) containing grid steps.

Examples

>>> dyt = get_stretched_grid_steps(14, 180, 5)
>>> dyt
[  5.10517337   5.22522948   5.47813251   5.99673813   7.00386752
   8.76808565  11.36450896  14.34977676  16.94620006  18.71041819
  19.71754758  20.2361532   20.48905624  20.60911234]
>>> dyt.sum()
180.0

>>> dyt = get_stretched_grid_steps(14, 180, 5, stretching_factor=4.)
>>> dyt
[  5.00526979   5.01802837   5.06155549   5.20877528   5.69251688
   7.14225176  10.51307232  15.20121339  18.57203395  20.02176884
  20.50551044  20.65273022  20.69625734  20.70901593]
>>> dyt.sum()
180.0
veros.tools.setup.get_vinokur_grid_steps(n_cells, total_length, lower_stepsize, upper_stepsize=None, two_sided_grid=False, refine_towards='upper')[source]

Computes stretched grid steps for regional and global domains with either one or two-sided stretching using Vinokur stretching.

This stretching function minimizes discretization errors on finite difference grids.

Parameters:
  • n_cells (int) – Number of grid points.
  • total_length (float) – Length of the grid interval to be covered (sum of the resulting grid steps).
  • lower_stepsize (float) – Grid step size on the lower end of the interval.
  • upper_stepsize (float or None, optional) – Grid step size on the upper end of the interval. If not given, the one-sided version of the algorithm is used (that enforces zero curvature on the upper end).
  • two_sided_grid (bool, optional) – If set to True, the resulting grid will be symmetrical around the center. Defaults to False.
  • refine_towards ('upper' or 'lower', optional) – The side of the interval that is to be refined. Defaults to ‘upper’.
Returns:

ndarray of shape (n_cells) containing grid steps.
Reference:
Vinokur, Marcel, On One-Dimensional Stretching Functions for Finite-Difference Calculations, Journal of Computational Physics. 50, 215, 1983.

Examples

>>> dyt = get_vinokur_grid_steps(14, 180, 5, two_sided_grid=True)
>>> dyt
[ 18.2451554   17.23915939  15.43744632  13.17358802  10.78720589
   8.53852027   6.57892471   6.57892471   8.53852027  10.78720589
  13.17358802  15.43744632  17.23915939  18.2451554 ]
>>> dyt.sum()
180.

>>> dyt = get_vinokur_grid_steps(14, 180, 5, upper_stepsize=10)
>>> dyt
[  5.9818365    7.3645667    8.92544833  10.61326984  12.33841985
  13.97292695  15.36197306  16.3485688   16.80714121  16.67536919
  15.97141714  14.78881918  13.27136448  11.57887877 ]
>>> dyt.sum()
180.

Numerical operators

veros.core.utilities.enforce_boundaries(vs, arr, local=False)[source]
veros.core.utilities.where(vs, cond, arr1, arr2)[source]
veros.core.utilities.pad_z_edges(vs, array)[source]

Pads the z-axis of an array by repeating its edge values

veros.core.utilities.solve_implicit(vs, ks, a, b, c, d, b_edge=None, d_edge=None)[source]

Distributed communication primitives

veros.distributed.ascontiguousarray(arr)[source]
veros.distributed.get_array_buffer(vs, arr)[source]
veros.distributed.validate_decomposition(vs)[source]
veros.distributed.get_chunk_size(vs)[source]
veros.distributed.get_global_size(vs, arr_shp, dim_grid, include_overlap=False)[source]
veros.distributed.get_local_size(vs, arr_shp, dim_grid, include_overlap=False)[source]
veros.distributed.proc_rank_to_index(rank)[source]
veros.distributed.proc_index_to_rank(ix, iy)[source]
veros.distributed.get_chunk_slices(vs, dim_grid, proc_idx=None, include_overlap=False)[source]
veros.distributed.get_process_neighbors(vs)[source]
veros.distributed.exchange_overlap(vs, arr, var_grid)[source]
veros.distributed.exchange_cyclic_boundaries(vs, arr)[source]
veros.distributed.global_and(vs, arr)[source]
veros.distributed.global_or(vs, arr)[source]
veros.distributed.global_max(vs, arr)[source]
veros.distributed.global_min(vs, arr)[source]
veros.distributed.global_sum(vs, arr)[source]
veros.distributed.gather(vs, arr, var_grid)[source]
veros.distributed.broadcast(vs, obj)[source]
veros.distributed.scatter(vs, arr, var_grid)[source]
veros.distributed.barrier()[source]
veros.distributed.abort()[source]

Frequently asked questions

Which backend should I choose to run my model (NumPy / Bohrium)?

Because in its current state Bohrium carries some computational overhead, this mostly depends on your problem size and the architecture you want to use. As a rule of thumb, switching from NumPy to Bohrium is beneficial if your set-up contains at least 1,000,000 elements (total number of elements in a 3-dimensional array, i.e., \(n_x n_y n_z\)). You can also use our benchmarks for general orientation.

Benchmarks

Note

The following benchmarks are for general orientation only. Benchmark results are highly platform dependent; your mileage may vary.

Veros compared to PyOM2

The following figurew present some benchmarks that compare the performance of Veros and pyOM 2.1 depending on the problem size:

Without MPI

_images/acc.svg

Benchmarks on a Desktop PC with 4 CPU cores (I) and a cluster node with 24 CPU cores and an NVidia Tesla P100 GPU (II). Line fits suggest a linear scaling with constant overhead for all components.

With MPI

_images/acc-4node.png

Benchmarks on 4 cluster nodes with 32 CPUs each (128 processes in total).

Veros runtime estimates on the DC3 cluster

The following figure presents estimates of Veros runtime required for 1 year prediction on CPU and GPU nodes of the DC3 cluster. The estimates were done for ACC, Wave propagation and Global cases with different spatial and temporal resolution.

For more details on the cases configuration see the setup gallery.

_images/veros_dc3perf.svg

Runtime estimates on a cluster node with 32 CPU cores (Bh CPU) and on a cluster node with 24 CPU cores and an NVidia Tesla P100 GPU (Bh GPU).

Publications

Papers

  • Häfner, D., Jacobsen, R. L., Eden, C., Kristensen, M. R. B., Jochum, M., Nuterman, R., and Vinter, B.: Veros v0.1 – a fast and versatile ocean simulator in pure Python, Geosci. Model Dev., 11, 3299-3312, https://doi.org/10.5194/gmd-11-3299-2018, 2018.

Talks

  • Veros — A High-Performance Ocean Simulator Written in Pure Python. A talk held at the 98th Annual Meeting of the American Meteorological Society (AMS). Abstract and slides available.
  • A Fast Versatile Ocean Simulator (Veros) in Pure Python. A talk held at the European Geosciences Union General Assembly 2018, Vienna, Austria. Abstract

Contact

If you want to report a bug in Veros, have a technical inquiry, or want to ask for a missing feature, please use our issue tracker on GitHub.

In case you have general questions about Veros, please contact the maintainer of the Veros repository.