# What is OpenAgua?

OpenAgua is a web-based application for modeling water systems for water resources planning and management. It aims to help develop insights to complex water issues in a collaborative online space. To achieve this, it offers a powerful set of basic tools for most use cases, as well as an increasing number of advanced tools for more complex modeling and computing cases. While OpenAgua is oriented toward water systems, it can also be easily adapted for other kinds of systems, such as energy systems.

OpenAgua is built by [a dedicated team](/learn-more/the-team) of developers, water experts, and funders.

{% hint style="info" %}
This documentation is under development. Please contact the developers for specific help, or to highlight areas for priority documentation.
{% endhint %}

## Key Innovations

OpenAgua includes a range of key innovations over existing desktop-based software applications:

* **Integrated collaboration and sharing** - The modern web facilitates technical collaboration and sharing for transparency and data discovery, central motivations of OpenAgua; collaboration and sharing features are deeply integrated into OpenAgua, though much additional work remains. [Read more](/how-to-use/user-guide/collaboration).
* **Cloud-based computing** - Although desktop computers are continually increasing speed, on-demand access to even more powerful and numerous cloud computers significantly increases our ability to discover water management solutions in an increasingly deeply uncertain world. OpenAgua allows users to use any internet-connected computer to take on tough computing tasks, from commercial cloud computers to personal laptops. [Read more](/how-to-use/user-guide/configuration/model-engines).
* **Open source (mostly)** - As much of OpenAgua as is possible and reasonable is open source. In particular, the default water allocation model is completely open source ([see it here](https://github.com/openagua/waterlp)). The core OpenAgua API is not yet open source, but will be. [Read more](/learn-more/contributing/open-source).

## Availability

For now, OpenAgua is only available online at [www.openagua.org](https://www.openagua.org), though we are working to create a self-hosting option.

## Main Features

### Water system modelling

Water system modeling needs is built into the DNA of OpenAgua, even though OpenAgua is also flexible enough to accommodate any arbitrary model. OpenAgua by default includes a demand-driven, priority-based simulation model, similar in concept to WEAP. [Read more](https://openagua.github.io/waterlp).

### **Standardized data organization**

All data of OpenAgua is organized using the [Hydra Platform](http://www.hydraplatform.org) software, which ensures consistent and quality-controlled data organization. The use of Hydra Platform also ensures cross-compatibility between OpenAgua and other Hydra-compliant software systems, such as the Hydra-native web application <https://hydra.org.uk>

### **Advanced scenario builder**

In order to adequately asses several decision frameworks, aside from the baseline scenario, OpenAgua has the feature of constructing scenarios. The user will be able to incorporate a mix of potential uncertainties and make projections in a specified time frame. By implementing scenarios, the user’s project will developed a more robust and deep analysis. Evaluation, identification and establishment of the best strategies into potential plans will ensure the user to select the best future projection for water planning.

Learn more in the [user guide](/how-to-use/user-guide/setup-model/view-edit-scenarios).

### **Collaborative modeling and scenario analysis**

OpenAgua is designed from the ground up for collaboration and sharing. Practical examples of collaboration include:

* Technical collaboration between modelers.
* A modeling consultant can share a model with a client, with either view-only access, or full edit access.
* Sharing a model with the public for transparency in decision-making.

Despite these options, OpenAgua requires neither collaboration nor transparency; access to data is controlled, and private data is secure.

### **Flexible computing**

Flexible computing is another core feature of OpenAgua. As noted above, OpenAgua ships with a default model that can be run without any configuration on the part of the user. However, models can also be configured to use a range of different computing modes, with the only requirement being that the computer is connected to the internet. Read more about [model engine configuration](/how-to-use/user-guide/configuration/model-engines).

### **Advanced analytical tools**

OpenAgua integrates modern visualization tools, based on the popular [Plotly library](https://plot.ly/javascript/). Since no single platform can meet all user needs, users can also either download data for offline analysis or connect directly to their data from other data analysis platforms, such as R or Jupyter Notebook.

### **Security**

Standard website data security best practices ensure that your data is protected and sharing is controlled. OpenAgua has employed a tiered ownership system. The user controls who has access to their projects, networks and other resources, as well as the level of permission they have. Inheriting the Hydra Platform permissions model, sharing chains are limited: trust levels for further sharing extend only one level beyond the project or network owner: If John allows Jane to share a project, Jane cannot allow others to further share the project.

### **Open source (mostly)**

OpenAgua fully appreciates the open source paradigm, while also recognizing the need to protect and control some source code. All of the technologies employed directly by OpenAgua are open source, and the default water system planning model is 100% open source. Currently, the OpenAgua API is not open source, but will be in the future. The website user interface code will likely not be open source for the foreseeable future. The driving principle here is that back-end \*processes\* are open source, while front-end (what you see and interact with in the browser) are not currently. The OpenAgua team is exploring the development of an \[open core]\(<https://en.wikipedia.org/wiki/Open-core_model>) approach to development, whereby a core, basic version of OpenAgua is 100% open source, keeping some advanced features non-open source.

This hybrid (and evolving!) open source model recognizes that at the very least the core generalized modeling logic should be open to all, and free for all to modify and improve, but that some of the value-added tools developed by the OpenAgua team represent unique intellectual property that should be protected. However, this only represents the current state of practice, and not necessarily the long-term outcome.  We would very much welcome your opinion about this.


# User Guide

The purpose of this user guide is to help get started modeling right away; describe key concepts that underpin the data organization and modeling principles used by OpenAgua, and describe how to use specific application features.

For now, this documentation is organized around the main user areas of the website, focusing on technical aspects (technologies involved, methods used, etc.), but also use in some cases. General help for the registered user is found on the site itself, under "Help".

This guide is primarily targeted toward the application user who does not want to customize the underlying code. For users who would like to better understand the more technical aspects of OpenAgua itself, see the [Development section](https://github.com/openagua/openagua-documentation/tree/8ca81ea384353fa9c727246381b2eede12d0662f/docs/how-to-use/user-guide/learn-more/development/README.md)


# Key concepts

There are several concepts that are important to understand when using OpenAgua. Specifically the user should be familiar with general *data organization*, the concept of *templates*, how *model engines* work (optional), and OpenAgua's *data visualization* capabilities.

## Data organization

OpenAgua builds on the data organizing scheme defined by [Hydra Platform](https://www.hydraplatform.org) (Hydra). Data is broadly organized as follows, yet the actual data organization is much more complex (refer to Hydra documentation):

* Water system model components are grouped into *networks*, which are further grouped into *projects*.
* Data is grouped into *scenarios*

OpenAgua builds on this and other Hydra data organization concepts with a wide range of other data and metadata that either are directly tied to Hydra data structures (e.g., a river's shape) or are stored independently (e.g., results charts and tables).

### Project & networks

In Hydra Platform, a *network* is a user-created collection of nodes (e.g., reservoirs and demand sites) connected to each other by links (e.g., rivers and conveyances). A *project* is a collection of one or more networks; a network may belong to only one project.

All the projects and networks are stored in the library called **Home**. In here, the user will be able to create new networks and arrange them by the user's preferences. Networks can be shared, edited, exported, deleted and cleaned.

### Scenarios

Under construction.

## Templates

In OpenAgua, a "template" is a definition of the overall structure of a network. Specifically, a template defines the kinds (*types*) of resources (nodes, links, and global resources) and attributes (i.e., constants and variables) that a network may have. For example, a template would indicate that there is a node type called "reservoir" and link type called "river". The template would then further indicate that, for example, a reservoir has a "capacity" and an "inactive zone". Thus, although OpenAgua is inspired by solving water system problems, templates can be created for any network type.

The user does not need to deal with templates to get started with OpenAgua. However, modelers will often find the need to modify the template associated with their networks. For this reason, OpenAgua includes the ability to edit templates. Two template viewers/editors exist for this purpose:

1. **General templates** - Templates that are not used by a specific network are included listed as "general templates". Templates in this area are editable only by their owner. If the owner makes a template public, this template may be viewed and copied by others for use in their own projects.
2. **Project templates** - Templates that are in use by specific networks are listed in the project viewer, where they may be edited by anybody who can edit the project.

## Model engines (optional)

Under construction.

## Data visualization

Under construction.


# Sharing & collaboration

Under construction


# Configuration


# Model engines & cloud computing

To successfully run a model using OpenAgua, there must be a working connection between OpenAgua and a *model engine*. A *model engine* is a generalized executable modeling application that is designed to 1) connect with and read data from the OpenAgua database, 2) perform some action using the data (most likely a simulation or optimization of the system), and 3) save results back to OpenAgua.

There is no hard constraint on what action is performed within the model engine, so this setup actually affords a significant amount of flexibility in terms of customization of the model, from the trivial (e.g., do nothing) to the complex (e.g., simulate a coupled human-water system using multiple sub-models in various physical and management domains).

## Conceptual overview

When [running a model](https://github.com/openagua/openagua-documentation/tree/e5fde739d6daf4b5cb8dc0d39118d38f234eab0d/docs/user-guide/configuration/user-guide/running-models.md), a user initiates a set of actions that results in the model engine starting. The model engine may be located on one or more computers connected to the internet. Generally, there are three modes that OpenAgua can use to communicate with a model engine.

1. **Directly** on the host computer. In this case, the model engine is located on the same computer as the OpenAgua server. OpenAgua sends the command to run the model engine directly (using Python's [Popen](https://docs.python.org/2/library/subprocess.html#popen-constructor) constructor).&#x20;
2. **Directly via** [**SSH**](https://en.wikipedia.org/wiki/Secure_Shell) on a remote computer: In this case, the model engine is located on a computer different than the main OpenAgua server. OpenAgua sends the command to run the model using SSH (secure shell). SSH is a way of logging directly into a computer remotely to execute commands on that computer.
3. **Indirectly via a task \[scheduler]\(**<https://en.wikipedia.org/wiki/Scheduling_(computing>**))**: In this case, the model engine may be on one or more computers, and may include the OpenAgua server itself. Running the model involves two steps behind-the-scenes. First, OpenAgua sends the command to a *scheduler* (specifically, a [RabbitMQ](https://www.rabbitmq.com/) task queue). Second, one of the model engines, which are setup as *workers* to continually wait for new tasks, takes the task out of the queue and performs the model computations.

The practical implications of these are that model engine setup requires configuration in two places: in OpenAgua and in the model engine itself. Both of these are described here.

### Cloud computing

\*Cloud computing\* is a widely used, and at times ambiguous term. Here, \*cloud computing\* refers to the ability to use any computer connected to the internet for computationally challenging tasks, including modeling. This is achieved using the third method above, i.e. via a task scheduler. This allows any cloud-connected computer to be used for the computational part of modeling. Examples include:

* Commercial Infrastructure-as-a-Service (IaaS) providers, such as \[Amazon Web Services EC2]\(<https://aws.amazon.com/ec2/>) or \[Google Compute Engine]\(<https://cloud.google.com/compute/>).
* An always-on server, such as a university-owned server.
* A personal laptop, such as might be used during model development.

The configuration of a model engine for cloud computing using the task scheduler approach is described below.

## Configuration of OpenAgua for model engine use

**NOTE**: For the time being, only administrative users can set up and configure models within OpenAgua for different computing modes, but this will change in the future as OpenAgua is improved. However, non-administrative users can still set up model engines to perform the modeling computations, as explained below.

Under construction.

## Configuration of the model engine

Under construction (see WaterLP as an example)


# Network templates

For each resource type (node, link, network), a template defines the name, icon to be used (nodes) or line style (links), as well as attributes. In the case of the "network" (i.e., the network as a whole, not specific nodes and links), "resources" may be added to represent arbitrary groupings of global attributes for the network.

Templates are matched with models: a template should describe (at least) the input and output expected by a core model. A network defined by a particular template may be used by one ore more models. A model might also use more than one template, but this might be more restrictive, as a model's input requirements are typically fixed; any variation on a template would need to consider this.

## Types

Under construction.

## Attributes

Under construction.

For each attribute (variable), a template further defines the following:

* Data type (*scalar*, *timeseries*, etc.)
* Dimension (*volume*, *flow rate*, etc.)
* Units (*cubic meters*, *cubic meters per second*, etc.)
* Scope (whether or not the attribute is an input or output)


# Setting up a model


# First steps

This describes how to set up a network so that you may start building the network, adding data, and so forth.

## Create project a project

All the projects and networks are stored in the library called Home. In here, the user will be able to create new networks and arrange them by the user's preferences. Networks can be shared, edited, exported, deleted and cleaned.

## Create a network

Once logged-in, click "add network" in order to start building your project. A window will pop up so you can name and describe your project. Each network can have different versions where you will be able to switch within without loosing any data, we recommend selecting the latest version and looking at the "version history" link before modifying the template. This feature will function in later versions of OpenAgua.

Once you are logged-in, click "add network" in order to start building your project. A window will pop up so you can name and describe your project. Each network can have different versions where you will be able to switch within without loosing any data, we recommend selecting the latest version and looking at the "version history" link before modifying the template. This feature will function in later versions of OpenAgua.

Once your network is set you can start building it by clicking "overview" and then "Network Editor", also a drop-down menu is shown at the upper right where sharing and editing functions are available.


# Viewing/editing a network

## Edit a network

The spatial representation of a water system developed in the **Network Editor**. The Network Editor section is destined to construct your model in a schematic view. In here, the graphical interface will help the user to visualize and comprenhend its project in form of the physical relationship between demand and supply nodes as well as their connection among them.

To begin, start by selecting the geographic area of your project. There are a set of different map types (light, terrain, satellite, etc.) located on the right side of your map, pick the one that best suit your project. On top of the list, there is a "go to my location" button, by clicking it, the map will display the current location of the user. Zoom in and out are features identified by the plus (+) and minus (-) icons, these may help you to select a very specific location of the network. However if the user prefers a simplistic schematic model there is a -No based Map- element on the list of maps.

Basemaps of OpenAgua are high-resolution imagery maps which can be added to the network. A list of useful maps is located at the right side of the Network Editor; topographic, satellite, street maps, terrain etc. are some of the most useful maps.

Once you set the area, start adding nodes and links. To set a point or a node: Click on the marker and place it on the map, a feature window will appear and here you have to select the type of feature, name and description. To set a link/river/conveyance: Click on the line icon located at the top left of your map, then click anywhere in the map to start building your polyline, do a double-click on the last point to finish your polyline or simply press finish on the right side of the line icon.

You are also able to edit the schematic location of points and polylines just click the edit icon at the top left of the map and move your nodes and links freely then save any changes. On the other side if you want to edit the nature of any node or link, do a right click on it and you will be able to: see the coordinates of the feature and center the data there. Moreover you will be able to delete, edit the name/description and data of the feature.

The Network Editor section is destined to construct your model in a schematic view. First zoom in to select the geographic area of your project. There are a set of different map types (light, terrain, satellite, etc.) on the right side of your map, pick the one that best suit your project.

Once you set the area, start adding nodes and links. To set a point or a node: Click on the marker and place it on the map, a feature window will appear and here you have to select the type of feature, name and description. To set a link/river/conveyance: Click on the line icon located at the top left of your map, then click anywhere in the map to start building your polyline, do a double-click on the last point to finish your polyline or simply press finish on the right side of the line icon.

You are also able to edit the schematic location of points and polylines just click the edit icon at the top left of the map and move your nodes and links freely then save any changes. It is important to know that every polyline is managed as an independent link (we have to discuss this part). On the other side if you want to edit the nature of any node or link, do a right click on it and you will be able to: see the coordinates of the feature and center the data there. Moreover you will be able to delete, edit the name/description and data of the feature.

## Customization

Under construction.


# Viewing/editing scenarios

\[Under construction]


# View/edit data

Two tools are provided for entering data. First, the **Basic Data Input** allows input of data on a per-scenario, per-resource, per-variable basis. In contrast, the **Input Table Creator** allows input for multiple scenarios/resources/variables at once, using a pivot table-style input mechanism. Both of these are described.

## Basic Data Input

The basic data editor consists of several areas:

1. Scenario selector
2. Resource tree
3. Variable selector
4. Data input
5. Notes & metadata input
6. Misc. supporting functions

### Scenario selector

### Resource tree

### Variable selector

### Data input

### Data preview

## Multi Data editor

The advanced data editor is a powerful tool to input data, it is based on using a pivot table to arrange all data and information. By using pivot tables, this editor gives the user flexibility and analytical skills resulting on well-organized source data.

To start using the advanced data editor first identify the filters bar and select the feature, variable and scenario. In here the user is able to select more than one data to edit. Click Load to boot your pivot table. Start dragging features to the row and column bank. Notice there is a drop down menu of aggregators, by using this, data is set to be original, average or a sum. Arrange your table as you prefer and start filling it either manually or by copy-paste from another data sheet. There is an extra button called Block, this allows to add columns to every feature of the table which might be useful for adding more data to specific variables. The user is able to make as many pivot tables as it like, just save your set up and open it at any time.

The key limitation of this pivot table is it may become unstable or unusable after 100,000 rows of data. If you reach the limit you will be warned to use the filter at the top to help reduce the amount of data retrieved from the server. This filter enables you to select more specific data based on an arrange of filter criteria including feature type and variable.


# Running a model

## Model Dashboard

The **Model Dashboard** enables you to run models for one or more scenarios. Once the model is run, progress will be reported in one or more progress bars. Additionally, more detailed information about both, the model run as a whole and about each specific scenario run is shown. The main log information is always shown providing feedback including the parameters used in the current model run (e.g. time span, level foresight, scenario ids, solver etc.) and the general status of the model run.

The Scenario log is shown by clicking the view log button next to the progress bar. It provides text updates about the model run. Additionally, it provides limited information from Pyomo about the optimization problem (e.g. number of objectives, constraints, variables etc.) Importantly this information also includes the status of the problem after the model run including specifically the termination condition. Termination conditions could include optimal, infeasible and many others (see <http://www.pyomo.org/blog/2015/1/8/accessing-solver>).

During the run, the modeling can be stopped by clicking Stop model.


# Viewing & saving results


# Visualize results

## Results Explorer / Chart maker

The results explorer consists of:

* A suite of pivot tables and charts
* Options to change the plot renderer (Plotly and Google Charts)
* Options to filter the data from the server.
* The ability to save charts for later viewing and editing.

First, select in the filters bar the feature or variables and click load. Then select which kind of plot you prefer from the suit of charts and tables at the drop down menu. Select the type of aggregator and then start building your results pivot table.

To add dynamism, there are two render options in which the user is able to see and edit their results: Plotly and Google charts. Both are online analytics and data visualization tools, find this option on the drop down menu located at the top left corner then select the most adequate for your project. If you want to change from one render to another remember to click Load after switching this option.

**Plotly:** By hovering the pointer on the chart, an edit menu will appear at the right corner of your plot. Here you will be able download your result, zoom in and out, edit etc. If you choose to edit your plot, it will automatically take you to the plotly site where you will be able to style and analyze it with several statistical tests.

Note: To create a scatter chart the filter should be used to select more specific data.

## My Charts or Chart Collection

The chart collection contains all the saved graphs and plots from the results explorer window. By clicking View on the image, a slide show will display the selected image. If there are some changes to be performed, simply click edit and this command will take the saved result into the Results Explorer window in order to perform any modification.


# Results dashboard


# Exporting results

\[Under construction]


# Advanced


# OpenAgua data specifications

Data in OpenAgua builds on the data schema of Hydra Platform. Understanding these OpenAgua-specific data specifications is helpful if uploading data via Hydra API.

Hydra enables extensions through the `layout` field of many data objects (or, in the case of template types, the `properties` field). OpenAgua thus uses the `layout` field (and `properties` field) extensively.

## Scenarios

OpenAgua uses the following fields within the `scenario.layout` field:

* `class`: Options include `baseline`, `option`, `portfolio`, `scenario` and `results`.
  * The `baseline` scenario represents the main scenario from which all other scenarios are derived.
  * Scenarios with the `option`, `portfolio` and `scenario` class are shown in their respective parts of the application.
  * Scenarios with the `results` class are available for viewing in the results explorer.

{% hint style="warning" %}
All scenarios in OpenAgua should be classified with layout.class. Unclassified scenarios will not be deleted, but may not be visible.
{% endhint %}


# Advanced data tools

\[Under construction]


# WaterLP

WaterLP is the default generalized water system model integrated into OpenAgua. This integration, though optional, allows you to start modeling water system right away. As with that for OpenAgua generally, this documentation is currently under construction.

See also the [code repository for WaterLP](https://github.com/openagua/waterlp-pywr).

Here is a high level flow chart of the modeling steps:

![General flowchart of the modeling process (click to zoom).](/files/-LXaWgpXDVyAmERrZsPR)


# Getting started

## Data input

Data may be input either directly in the form of fundamental data types of *time series*, *array*, *scalar*, or *descriptor* (text), or indirectly via Python-based functions.

## Functions

Data input as a code-based script are evaluated as functions either once at the beginning of a model run, or on a per-time step basis, depending on whether the fundamental variable data type is fixed (array, scalar, or descriptor) or temporally variable (time series), respectively.

The Python-based scheme enables the user to import data, enter custom functions directly into the code, etc. The most important principles to understand when entering functions are described below, though a full description of the Python programming language is beyond the scope of this documentation.

### Returning a value

Let's start with two very simple functions:

```python
5
```

and

```python
return 5
```

Both of these result in a value of `5` every time step (assuming this function is for a time series). The code evaluator needs to know which value the code should return. In general, this is achieved with the `return` statement. However, the evaluator will automatically add `return` if it is missing, so that if the value to be returned is on the last line, then no `return` statement is needed.

In many cases, including `return` is simply a matter of personal preference. But this is particularly useful if a return is nested in the last part of a conditional statement. To demonstrate, the following three versions of code input yield the exact same result when evaluated:

```python
if date.month in [6,7,8]:
    x = 0.5
else:
    x = 1
x
```

```python
if date.month in [6,7,8]:
    x = 0.5
else:
    x = 1
return x
```

```python
if date.month in [6,7,8]:
    return 0.5
else:
    return 1
```

In the first case, because `x` is on the last line, `return` is not needed; `x` will be the value returned. In the second two cases, values are returned explicitly. As a side note, Python can be quite succinct at times, allowing the above to be written, for example, as:

```python
0.5 if date.month in [6,7,8] else 1
```

The last few examples above should raise a question: where does `date` come from? WaterLP contains several convenient built-in variables, as described in the next section.


# Built-in variables

OpenAgua includes a growing list of built-in variables available for use in functions. As of writing, these include the following:

### `timestep`

An ordinal integer representing the current timestep (i.e., 1, 2, 3, ..., N).

### `date`

A the current timestep's date as a [Pendulum](https://pendulum.eustace.io) date object, which has numerous useful [attributes and properties](https://pendulum.eustace.io/docs/#attributes-and-properties) (e.g., `date.month`, etc.).

### `periodic_timestep`

An ordinal integer representing the timestep in the current water year. For example, a monthly time step model starting in October would have `periodic_timestep` of 1, 2, 3, ..., 12, with 1 in October and 12 the following September. A daily time step model would similarly range from 1 to 365 (or 366 if leap years are considered).

### `water_year`

The current water year, which is defined by the calendar year of the *last month* in the water year. If it helps, here is the specific equation for `water_year`:

```python
water_year = date.year + (0 if date.month < start_date.month else 1)
```

where `start_date` is the start date of the model. Speaking of start date...

### `start_date`

This is a [Pendulum](https://pendulum.eustace.io) date object representing (you guessed it!) the start date of the model.

{% hint style="info" %}
This list is preliminary, and we would happily accommodate additional variable needs.
{% endhint %}


# Functions reference

The following functions are available for use within WaterLP.

### `get`

Get data from another variable, or another time step, possibly aggregated.

#### Arguments

| Argument          | Data type                               | Description                                                                                                                                                                                                                                                                                                                                                                                                             |
| ----------------- | --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `path` (required) | string                                  | The path of the variable (aka attribute) of interest. See note below for more detail on path construction.                                                                                                                                                                                                                                                                                                              |
| `flatten`         | boolean                                 | Whether or not the value returned should be summed across multiple columns if the variable obtained is a multi-column dataset. Currently, most datasets are single columns, so this normally will not be used. Defaults to `True`.                                                                                                                                                                                      |
| `offset`          | integer                                 | Get a value from a specific time step offset. Values can be negative or positive. E.g., `offset=-1` returns the value from the last time step, while `offset=1` returns the value from the last time step. Offset values beyond the range of the data will result in `None`. Default is `0`.                                                                                                                            |
| `start`           | <p>date string <br>or Pendulum date</p> | This indicates that the value should be aggregated from more than one time step, with start indicating the start date of the aggregation period. The value should be either a [Pendulum](https://pendulum.eustace.io/) date object or a string (e.g., `1999-10-01`) that Pendulum can properly parse. Future `start` values are valid, though should be accompanied by a future `end` value as well. Default is `None`. |
| `end`             | <p>date string<br>or Pendulum date</p>  | This indicates that the value should be aggregated from more than one time step, with `end` indicating the end date of the aggregation period. The value should be either a [Pendulum](https://pendulum.eustace.io/) date object or a string (e.g., `1999-10-01`) that Pendulum can properly parse. Future `end` values are valid. Default is `None`.                                                                   |
| `agg`             | string                                  | If aggregating a value (i.e., with `start` and/or `end)`, this indicates how the value should be aggregated. Options include `mean` and `sum`. Default is `mean`.                                                                                                                                                                                                                                                       |

#### Returns

A scalar representing the value of the target variable during the time step.

#### Notes

Currently, path names are constructed in one of two ways, depending on the resource type. For *nodes* and *links*:

**Example 3**: Get the mean water year runoff-to-date.

```python
# Assume water year starts Oct. 1.
start_date = '{wy}-10-01'.format(wy=water_year-1)
get("Monterrey/node/Catchment Rio San Juan/Runoff", start=start_date, agg="mean")
```

`"[network_name]/[resource_type]/[resource_name]/[attribute_name]"`

where network\_name is the name of the network (of course!), resource\_type is either node or link, resource\_name is the name of the node or link (e.g., El Cuchillo Reservoir), and attribute\_name is the name of the variable (e.g., Storage Capacity).

For network attributes (global variables ascribed to the system as a whole, as opposed to a specific resource or facility):

`"[network_name]/[attribute_name]"`

#### Examples

**Example 1**: Get the runoff from a catchment in the current time step.

```python
get("Monterrey/node/Catchment Rio San Juan/Runoff")
```

**Example 2**: Get the reservoir storage from the previous time step.

```python
get("Monterrey/node/El Cuchillo/Storage", offset=-1)
```

**Example 3**: Get the mean runoff-to-date for the current water year.

```python
start_date = "{year}-10-01".format(year=water_year-1)
get("Monterrey/node/Catchment Rio San Juan/Runoff", start=start_date, agg="mean")
```

### `read_csv`

Read a CSV file from a specified path. For now, this is limited to reading from the current network's file storage (using AWS S3), as viewable in the OpenAgua app.

Returns a [Pandas DataFrame](https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.DataFrame.html) object.

{% hint style="info" %}
Since this function relies on Pandas read\_csv, you can prepare and test your function outside of OpenAgua with your Python programming environment of choice, using Pandas instead of OpenAgua.
{% endhint %}

#### Arguments

This function uses the [Pandas read\_csv](https://pandas.pydata.org/pandas-docs/version/0.23.3/generated/pandas.read_csv.html) function (version 0.23.4), and generally accepts the same arguments, which will be passed through directly. The `filepath_or_buffer` argument of the native Pandas read\_csv function should be replaced by the `path` argument. Some of the Pandas default argument values are over-ridden to accommodate the most typical CSV format, as shown in Example 1 below; these can still be changed in the function call. Arguments listed below include both over-ridden arguments and custom additional arguments.

| Argument          | Data type       | Description                                                                                                                                                                             |
| ----------------- | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `path` (required) | string          | The path of the data of interest.                                                                                                                                                       |
| index\_col        | integer or list | Pandas `index_col` argument. Default is `0`.                                                                                                                                            |
| parse\_dates      | boolean         | Pandas `parse_dates` argument. Default is `True`.                                                                                                                                       |
| flavor            | string          | The type of Python object to return. Options include `native` (a Python dictionary), `pandas`, and `json`. Note that this is not a Pandas `read_csv` argument. The default is `native`. |

#### Returns

A Python data object that depends on the `flavor` argument. The default is a native Python dictionary (`flavor='native'`).

#### Notes

For the time being, this must be called prepended with `self.` and with the last arguments as `**kwargs`, as in the examples below.

{% hint style="info" %}
CSV files loaded using `read_csv` are cached, so loading the same CSV file from different functions will *not* significantly impact performance.
{% endhint %}

#### Examples

**Example 1**: Load inflow hydrology (all at once).

For this first example, let's assume we have a CSV file as follows:

```
Date,Hetch Hetchy
2007-10-10,52.116
2007-10-20,96.703
2007-10-31,51.93
2007-11-10,87.213
2007-11-20,72.233
```

This can be loaded using the default arguments, as:

```python
self.read_csv("data/runoff.csv", **kwargs)
```

Since this function returns an entire time series (as a Python dictionary object by default), it will not be called again, as the data will already be readily available to the model. I.e., all data is loaded in one go. Note again that `return` is optional, so is omitted here.

**Example 2**: Load inflow hydrology (per time step).

```python
data = self.read_csv("data/runoff.csv", usecols=[0,1], **kwargs)
return data.iloc[timestep-1][1]
```

This example loads the dataset within the function, but since it returns only a single value, it will be called again every time step. This can be useful if some time-dependent manipulation of the data is needed, but is generally less efficient than the approach in Example 1. Not that this uses the built-in variable `timestep`. Since Python objects are indexed starting at zero, but `timestep` starts at 1, when using the [DataFrame iloc index method](https://pandas.pydata.org/pandas-docs/version/0.23.4/indexing.html), `timestep-1` must be used.


# Where to get help

Since this documentation is currently under development, the best way to get help is to reach out to the lead developer, David Rheinheimer, at <drheinheimer@umass.edu>.


# Frequently Asked Questions

Under construction.


# Web API

The mostly RESTful web application programming interface (API).

OpenAgua provides a mostly RESTful API for most objects in the database. This API is documented (as a work-in-progress) at: [www.openagua.org/api/v1](https://docs.openagua.org/how-to-use/www.openagua.org/api/v1).

Here some of the more useful API elements are described in more detail.

## Organization

\[to-be-described: API organization]

## Authentication


# networks

API for requests related to networks.

## networks

<mark style="color:blue;">`GET`</mark> `https://www.openagua.org/api/v1/networks/:network_id`

Get a network, with various options for downloading in different formats and including related data.

#### Path Parameters

| Name        | Type   | Description               |
| ----------- | ------ | ------------------------- |
| network\_id | number | ID of the network to get. |

#### Query Parameters

| Name    | Type   | Description                                                                                                                                                                                       |
| ------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| purpose | string | <p>This is a way of specifying the purpose of the request. If omitted, the normal, JSON-formatted version of the network will be returned.<br>For now, the only option is "download".</p>         |
| format  | string | <p>If the purpose is "download", the "format" parameter is used to specify the file format of the download.<br>Options include "json", "csv", "xlsx", "adjacency", "geojson" and "shapefile".</p> |

{% tabs %}
{% tab title="200 " %}

```
```

{% endtab %}
{% endtabs %}


# Publications

## Peer-reviewed journal articles

Under construction

## Conference presentations

Under construction


# The Team

OpenAgua is developed and maintained by a core group of university researchers and funding partners, with development and technical support from many others.

## Institutional Partners

Current:

[**Hydrosystems Research Group**](http://blogs.umass.edu/hydrosystems/about/)**, University of Massachusetts Amherst.**

Previous:

[**Center for Water for Latin America and the Caribbean**](http://www.centrodelagua.org/)**, Tecnológico de Monterrey.**

[**Center for Watershed Sciences**](https://watershed.ucdavis.edu/)**, University of California, Davis.**

## Development Team

### Current team

**David Rheinheimer, Ph.D., Post-doctoral Researcher, Hydrosystems Group, University of Massachusetts Amherst**. Dr. Rheinheimer conceptualized OpenAgua and is involved in all aspects of its development, including vision, strategic planning, project management and technical leadership; to-date, he has been the primary developer of OpenAgua. His primary interests are in developing and leveraging decision support technologies to improve strategic management of water and energy resources.

[**Casey Brown, PhD, Hydrosystems Research Group, University of Massachusetts Amherst**](https://cee.umass.edu/faculty/casey-brown). Dr. Brown helps guide the vision of OpenAgua and helps ensure its continued short-term development in the context of multiple high profile water systems.

**Don Park**. Don, who currently a graduate student in the UMass Hydrosystems Research Group, has previous experience in the startup world and cloud software management, and provides valuable insights into the application development process. Don also maintains a list of useful articles related to programming conventions, [Programming and You!](https://projects.cloudwaterlab.com/donpark/ProgrammingKnowledge).

### Past contributors

**Laura Elisa-Garza, M.S., Center for Water for Latin America and the Caribbean, Tecnológico de Monterrey**. Laura led the study to demonstrate OpenAgua for the city of Monterrey, Mexico, and contributed to documentation and visual design.

[**Josué Medellín-Azuara, Ph.D.**](https://watershed.ucdavis.edu/people/joshmd)**, Senior Researcher, Center for Watershed Sciences, University of California, Davis**. Dr. Medellín-Azuara was the U.S. co-Principal Investigator for the \[CITRIS]\(<https://citris-uc.org/>)-funded seed project to develop OpenAgua, helping to oversee project organization and coordinate with other related projects.

[**Aldo Ramírez-Orozco, Ph.D.**](http://www.centrodelagua.org/draldo.aspx)**, Professor, Center for Water for Latin America and the Caribbean, and General Director, Núcleo Estrategico de Decisiones (NED), at Tecnológico de Monterrey**. Dr. Ramírez was the Mexico co-Principal Investigator for the CITRIS-funded seed project to develop OpenAgua, helping to guide project organization and liaise with funding partners in Mexico.

[**Michael Zarozinski**](https://www.cics.umass.edu/people/zarozinski-michael)**, Senior Software Engineer, College of Information and Computer Sciences, University of Massachusetts Amherst**. Michael was a lead designer and implementer of the "heart and soul" of OpenAgua, the facility to run and monitor models. Michael is interested in all aspects of artificial intelligence and the art and science of software engineering.

### Collaborators

In addition, core development is in close collaboration with others. In particular:

**Stephen Knox, Ph.D., Lead Developer, Water Resources Group, Manchester University**. Dr. Knox is the lead developer of [Hydra Platform](http://hydraplatform.org/), the core data organization framework used by OpenAgua. Steve provides critical support for Hydra Platform and advice on strategic development.

**Adel Abdallah, Ph.D. Candidate, Utah State University, Logan Utah and Senior Hydroinformatics Specialist at the Western States Water Council**. Adel designed the Water Management Data Model [(WaMDaM)](https://github.com/WamdamProject) to organize water systems data across multiple datasets and models. He has contributed to OpenAgua with ideas and bug reports while developing interoperability between WaMDaM and Hydra Platform/OpenAgua. Adel is interested in improving information methods and tools like OpenAgua to better organize, analyze, and communicate water management data and models to ultimately support effective and transparent decisions.

\[Adel Abdullah]\(<https://www.hydroshare.org/user/300/>), PhD Candidate, Utah State University. An expert in hydroinformatics and water resources management, Adel has provided critical feedback on OpenAgua, pushing it further while integrating it into his own data workflow, including \[WaMDaM]\(<http://wamdam.org/>) and \[HydroShare]\(<https://www.hydroshare.org/>).

## Funding Partners

The OpenAgua team appreciates working with our initial funding partners, for both their financial support and strategic vision.

[**CITRIS and the Banatao Institute**](http://citris-uc.org/)**, University of California**. CITRIS provided the seed funding for OpenAgua in 2016 under the 2016 joint CITRIS-ITESM seed funding program. Under this program, "Bi-national research teams in Mexico and California develop promising, early-stage IT projects that can attract larger funding and support resources." See the [anouncement of the award](http://citris-uc.org/three-california-mexico-research-teams-win-seed-funds-from-citris-itesm/).

[**FEMSA Foundation**](http://www.femsa.com/en/femsa-foundation). The FEMSA Foundation supports the Núcleo Estrategico de Decisiones (Nucleus for Strategic Decisions, or NED) at the Water Center for Latin America at Tecnológico de Monterrey, under which OpenAgua was initiated.

[**Inter-American Development Bank (IADB)**](http://www.iadb.org/). IADB also supports NED.

**Tecnológico de Monterrey**. Tecnológico de Monterrey also supports NED, including with physical space.

## Translation Team

Once we get translations going again, we'll list the translators here.

## The Open Source Community

OpenAgua uses primarily Open Source software, much of which is highly undervalued by society, yet nonetheless otherwise extremely appreciated. We consider the many thuosands of developers who have contributed to the various technologies used by OpenAgua to be implicitly part of the team. See the list of [core technologies](/learn-more/contributing/open-source).


# Development

Under construction.


# Open source software


# Contributing