Reference - Sub-Packages

You can get an overview of the purpose of the sub-packages from the previous section. The (detailed) description of all user-relevant functions in rivus are listed below. In addition to the documentation of the individual functions, a brief description is provided per sub-package or module to enlighten the used packages or design logic.

Note

Prerequisites:
In the code examples, we assume that you have a installed rivus (git clone) and the required packages for your work. (conda install see in installation)

rivus.main

rivus

The formulation of the MILP model is implemented with the help of Pyomo’s ConcreteModel. That means, the needed constrains are formulated with the help of python functions. These have descriptive names following the *_rule or *_balance naming convention. In general, you do not have to touch them.

As for now, the object returned by create_model() (often named as prob) serves as the base type of input for the other function. This may change in future versions. But as for now, it is important you get familiar with its format. Following attributes are directly accessible and esp. interesting in the first round.

  • peak : (Pandas.DataFrame) - Maximum demands per edge.

  • r_in, r_out : (Pandas.Series) - Process-Commodity ratios split by direction.

  • params : (dict)

    • commodity - processed Excel sheet
    • process - processed Excel sheet
    • hub - filtered version of process
    • process_commodity - processed Excel sheet
    • time - processed Excel sheet
    • area_demand - processed Excel sheet
    • vertex - processed spatial input
    • edge - processed spatial input

Warning

Following functions will be migrated to the io sub-package:

  • read_excel
  • plot - result_figures
  • save - load
  • save_log

Following function will be migrated to rivus.main.getters:

  • get_entity - get_entities - list_entities
  • get_onset_names
  • get_constants
  • get_timeseries

Modelling

create_model(data, vertex, edge, peak_multiplier=None, hub_only_in_edge=True)[source]

Return a rivus model instance from input file and spatial input.

Parameters:
  • data (dict) – Processed Excel spreadsheet by read_excel
  • vertex (GeoDataFrame) – DataFrame with vertex IDs as column ‘Vertex’ and other columns named like source commodities (e.g. ‘Gas’, ‘Elec’, ‘Pellets’), containing source vertex capacities (in kW)
  • edge (GeoDataFrame) – DataFrame with vertex IDs in columns ‘Vertex1’ and ‘Vertex2’ and other columns named like area types (in spreadsheet/Area-Demand), containing total areas (square metres) to be supplied
  • peak_multiplier (callable, optional) – If not None and bool-casts to True, then m.peak will be calculated by calling m.peak = peak_multiplier(m)
  • hub_only_in_edge (bool, optional) – Temporary switch between original and fixed process handling.
Returns:

Pyomo ConcreteModel object

Note

This function will change its input objects! (Reindex, insertion of values.)

line_length(line)[source]

Calculate length of a line in meters, given in geographic coordinates.

Parameters:line – a shapely LineString object with WGS 84 coordinates
Returns:Length of line in meters

IO

Following functions will be migrated to the io sub-package

read_excel(filepath)[source]

Read Excel input file and prepare rivus input data dict.

Reads an Excel spreadsheet that adheres to the structure shown in the example dataset data/mnl/mnl.xlsx. Must contain

Parameters:filepath (str) – absolute or relative filepath to an Excel spreadsheet.
Returns:dict – 5 DataFrames, one for each sheet

Example

data = read_excel('./data/mnl/data.xlsx')
plot(prob, commodity, plot_demand=False, mapscale=False, tick_labels=True, annotations=True, buildings=None, shapefiles=None, decoration=True, boundary=False)[source]

Plot a map of supply, conversion, transport and consumption.

For given commodity, plot a map of all locations where the commodity is introduced (Rho), transported (Pin/Pot/Pmax), converted (Epsilon_*) and consumed (Sigma, peak).

Parameters:
  • prob – rivus ConcreteModel
  • commodity – str like Elec, Heat etc.
  • plot_demand – If True, plot demand, else plot capacities
  • mapscale (Boolean) – If True, add mapscale to plot (default: False)
  • tick_labels (Boolean) – If True, add lon/lat tick labels (default: True)
  • annotations (Boolean) – If True, add numeric labels to graph (default: True)
  • buildings (tuple) – tuple of (filename to shapefile, boolean) if true, color buildings according to attribute column “type” and colors in constant rivus.COLORS; else use default COLOR[‘building’] for all
  • shapefiles (list) – list of dicts of shapefiles that shall be drawn by basemap function readshapefile. is passed as **kwargs
  • decoration (Boolean) – Switch for map decoration (meridian, parallels)
  • boundary (Boolean) – Draw map border or not.
Returns:

fig – the map figure object
result_figures(prob, file_basename, buildings=None, shapefiles=None)[source]

Call rivus.plot with hard-coded combinations of plot_type and commodity.

This is a convenience wrapper to shorten scripts. TODO: Generalise so that no hard-coding of commodity names is needed.

Parameters:
  • prob – a rivus model instance
  • file_basename – filename prefix for figures
  • buildings – optional filename to buildings shapefile
  • shapefiles – list of dicts of shapefiles that shall be drawn by basemap function readshapefile. is passed as **kwargs
Returns:

Nothing
report(prob, filename)[source]

Write result summary to a spreadsheet file.

Create a concise result spreadsheet with values of all key variables, inclduing costs, pipe capacities, process and hub capacities, source flows, and process input/output/throughput per time step.

Parameters:
  • prob – a rivus model instance
  • filename – Excel spreadsheet filename, will be overwritten if exists
Returns:

Nothing
save_log(result, filename)[source]

Save rivus result and solver information to a log file.

Parameters:
  • result – as returned by the solve method of a solver object
  • filename – log file to be written
Returns:

Nothing
save(prob, filepath)[source]

Save rivus model instance to a gzip’ed pickle file

Pickle is the standard Python way of serializing and de-serializing Python objects. By using it, saving any object, in case of this function a Pyomo ConcreteModel, becomes a twoliner. GZip is a standard Python compression library that is used to transparently compress the pickle file further. It is used over the possibly more compact bzip2 compression due to the lower runtime.

Parameters:
  • prob – a rivus model instance
  • filepath – pickle file to be written

Notes

Pickle GZIP <>bzip2

load(filepath)[source]

Load a rivus model instance from a gzip’ed pickle file

Parameters:filepath – absolute or relative path to gzip’d pickle file
Returns:prob – the unpickled rivus model instance

Getters

Following function will be migrated to rivus.main.getters

get_entity(instance, name)[source]

Return a DataFrame for an entity in model instance.

Parameters:
  • instance – a Pyomo ConcreteModel instance
  • name – name of a Set, Param, Var, Constraint or Objective
Returns:

a single-columned Pandas DataFrame with domain as index
get_entities(instance, names)[source]

Return one DataFrame with entities in columns and a common index.

Works only on entities that share a common domain (set or set_tuple), which is used as index of the returned DataFrame.

Parameters:
  • instance – a Pyomo ConcreteModel instance
  • names – list of entity names (as returned by list_entities)
Returns:

a Pandas DataFrame with entities as columns and domains as index
list_entities(instance, entity_type)[source]

Return list of sets, params, variables, constraints or objectives.

Parameters:
  • instance – a Pyomo ConcreteModel object
  • entity_type – “set”, “par”, “var”, “con” or “obj”
Returns:

DataFrame of entities

Example:

>>> data = read_excel('data-example.xlsx')
>>> model = create_model(data, vertex, edge)
>>> list_entities(model, 'obj')
Name
obj   minimize(cost = sum of all cost types)     []
[1 rows x 2 columns]
get_onset_names(entity)[source]

Get onset names of model entity Example:

vertex, edge = create_square_grid()
data = read_excel('data-example.xlsx')
model = create_model(data, vertex, edge)
get_onset_names(model.hub)
get_constants(prob)[source]

Retrieve time-independent variables/quantities.

Parameters:prob – a rivus model instance
Returns:(costs, pmax, kappa_hub, kappa_process) tuple

Example

costs, pmax, kappa_hub, kappa_process = get_constants(prob)

get_timeseries(prob)[source]

Retrieve time-dependent variables/quantities.

Example

source, flows, hubs, proc_io, proc_tau = get_timeseries(prob)

Parameters:prob – a rivus model instance
Returns:(source, flows, hubs, proc_io, proc_tau) tuple

rivus.graph

Comparison of the common graph analysis tools:

  • NetworkX (preferred):

    • + /- Pure python implementation.
    • + Widely used and tested.
    • + Docs are quite good.
    • + Easy (platform independent) installation
    • - Slower than igraph (and graph-tools)

  • python-igraph (fall-back):

    • + C based with python wrappers.
    • + Mature library package.
    • + Included for speed and so for scalability.
    • × Docs are OK.
    • - Windows install can be somewhat tedious (with unofficial wheel files). But it works.

  • graph-tools: (eventually added in the future, if there is reeeaaly big need for efficiency..)

    • + Self proclaimed: fastest in graph analyses
    • - Not really windows user friendly (docker install should be tested)

These all facilitate pretty advanced graph theoretical analysis. Moreover, the file export functions of the used libraries were bridged through the to_graph module. Preferred file format is .gml which is supported by all common graph analysis tools. As such, specialized, stand-alone tools for graph visualisation and analysis can be used. E.g. Gephi with its exceptional geolayout plug-in can be of tremendous help for deeper graph visualization and analysis.

to_graph

Functions to convert tabular data to popular python graph structures

to_igraph(vdf, edf, pmax, comms=None, peak=None, save_dir=None, ext='gml')[source]

Convert Data from (Geo)DataFrames to python-igraph’s Graph class Each commodity gets its own graph. Weights are derived from built capacity.

Parameters:
  • vdf ([Geo]DataFrame) – Holding Vertex Data id=Vertex and Commodity Sources as columns
  • edf ([Geo]DataFrame) – Holding (V1,V2) Multi-indexed Edge data To be sure, that all edges are created.
  • pmax (DataFrame) – Commodities as columns with max capacity per edge returned by rivus.get_constants()
  • comms (iterable, optional) – Names of the commodities from which we build the graphs. (Each as separate graph.) If omitted, the columns of pmax will be used.
  • peak (DataFrame, optional) – Commodities as columns with demands in t_peak time-step. Calculated in main.rivus
  • save_dir (path string, optional) – Path to a dir to save graphs as ext Path preferably constructed using the os.path module. If dir does not exit yet, it will be created.
  • ext (str, optional) – Description
  • (string) file extension, supported by igraph.save() (ext) – If not one of the following, the default ‘gml’ will be applied. ‘adjacency’, ‘dimacs’, ‘dot’, ‘graphviz’, ‘edgelist’, ‘edges’, ‘edge’, ‘gml’, ‘graphml’, ‘graphmlz’, ‘gw’, ‘leda’, ‘lgl’, ‘lgr’, ‘ncol’, ‘net’, ‘pajek’, ‘pickle’, ‘picklez’, ‘svg’
Returns:

list – List of igraph.Graph objects in order of comms. Graphs are undirected and weighted.

Example

_, pmax, _, _ = get_constants(prob)
graphs = to_igraph(vertex, edge, pmax, ['Gas', 'Heat'])
to_nx(vdf, edf, pmax, comms=None, save_dir=None)[source]

Convert to networkx graph representation

Parameters:
  • vdf ([Geo]DataFrame) – Holding Vertex Data id=Vertex and Commodity Sources as columns
  • edf ([Geo]DataFrame) – Holding (V1,V2) Multi-indexed Edge data To be sure, that all edges are created.
  • pmax (DataFrame) – Commodities as columns with max capacity per edge returned by rivus.get_constants()
  • comms (iterable, optional) – Names of the commodities from which we build the graphs. (Each as separate graph.) If omitted, the columns of pmax will be used.
  • save_dir (path string, optional) – Path to a dir to save graphs as GML. Path preferably constructed using the os.path module If dir does not exit yet, it will be created.
Returns:

list – nx_graph objects in accordance with input comms or all commodities found in pmax.columns

Example

_, pmax, _, _ = get_constants(prob)
graphs = to_nx(vertex, edge, pmax, ['Gas', 'Heat'])

Note

nx.from_pandas_dataframe() was also investigated for conversion, but it is a bit slower and does not improve code quality in my opinion.

analysis

Functions to hold dedicated analysis runs on the graph objects. networkx is the de-facto graph package, but igraph compatibility is also eligible.

minimal_graph_anal(graphs, calc_spanning=True, graph_package='NX')[source]

Showcase interoperable connectivity analysis.

Parameters:
  • graphs (list) – networkx or igraph Graph objects. Awaited is the resulting ITERABLE of rivus.graph.to_graph functions.
  • calc_spanning (bool, optional) – Default: True, sets whether to investigate: Is the graph of the built commodity also a minimal spanning tree of the street network?
  • graph_package (str, optional) – Default: ‘NX’, Accepted: ‘NX’ or ‘IGRAPH’. To ease the decision of which package is used.
Returns:

list of dicts per graph –

  • commodity
  • is_connected
  • connected_components (number of them)
  • is_minimal (if calc_spanning is True)

rivus.gridder

The birth of this module originates to the intention to regard the linear optimization model as a natural phenomenon. As usual approach in natural sciences, we abstract the specific question in focus, removing the “noise” of real-world data and homogenize the otherwise heterogeneous input data.

After gaining understanding of the highly symmetric model, the noise and asymmetries can be added back in an iterative process. Thus testing the validity of our findings.

As the first approach, a symmetric grid generator is implemented, which can be highly parametrized and can create input for create_model() only in few lines of code.

_images/sq_grids1.png

Adding noise to the grid. (x, and y edge length can be also set separately.)

Moreover, the possibility to have an input generator integrated into rivus itself has proven itself useful in many ways. Testing the implemented paradigms can be more intuitive in a symmetric environment, and the ability to rapidly showcase little optimization projects can serve as a demonstration tool accompanying relevant lectures.

Note

A highly symmetric square grid may seem far-fetched from real-life street structures. However, remember the grid street plan which has been around since the Greeks and is to be found all over the world. So, it is maybe not such a bad starting point as considered it may seem at the first glance.

Short example set of cities Google Maps Set.

create_grid

create_square_grid(origo_latlon=(48.26739, 11.66842), num_edge_x=1, num_edge_y=None, dx=100, dy=None, noise_prop=0.0, epsg=None, match=0)[source]

Create chessboard grid with edges and vertices on WGS84 suface with vincenty distance calculation lat ~ x, lon ~ y

Parameters:
  • origo_latlon (tuple, optional) – WGS84 latlon coordinates of the bottom left grid point defaults to some the TUM-ENS dep. ;]
  • num_edge_x (int, optional) – how many edges horizontally
  • num_edge_y (None, optional) – How many edgey vertically
  • dx (int, optional) – length of the horizontal edges (in meters)
  • dy (None, optional) – length of the vertical edges (in meters)
  • noise_prop (float, optional) – 0.0 to MAX_NOISE (< 1.0) missplacement radius relative to dx and dy.
  • epsg (int, optional) – If a valid epsg code which is supported py pyproy, the coordinates are calculated in the carthesian UTM CRS and then transformed into epsg4326 (latlon). If None or omitted, then the coordinates are calculated directly in epsg4326 with vincenty’s formula for distance and the grid lines up with the North and East directions
  • match (enumerated values, optional) –
    • 0 - vertices and edges are matched by the logic of generation
      (faster as less calculation is needed.)
    • 1 - matching is done geographicaly
      with pandashp helper (slower, but flexible)
Returns:

list of GeoDataFrames –

  • vertices : with [geometry, Vertex] columns
  • edges : with [geometry, Edge, Vertex1, Vertex2] columns

Note

Sequence of IDs: From buttom left to upper right. From row to row. From left to right.

 bearing 0

       (6)══04══(7)══05══(8)
        ║        ║        ║
        7        9        11
        ║        ║        ║
       (3)══02══(4)══03══(4)
 ^      ║        ║        ║
(y)     6        8        10
 L      ║        ║        ║
 A     (0)══00══(1)══01══(2)
 T
   LON (x) -> bearing 90
Raises:ValueError – Not supported epsg number
get_source_candidates(vdf, dim_x, dim_y, logic='sym')[source]

Calculate the set of indexes of the vertices, which are worth testing as source vertex in a single commodity case. A square grid is assumed. “Worth” means: The minimal set of vertices which cover the main symmetrical positions.

Parameters:
  • vdf (pandas DataFrame) – The vertex frame. (Created by create_square_grid())
  • dim_x (int) – Number of vertices along the x axis.
  • dim_y (int) – Number of vertices along the y axis.
  • logic (str, optional default='sym') –

    what kind or source candidates are looked for.

    • sym - Minimal(ish) set of vertices based on symmetry.
      E.g. here the indices marked with * are selected.
      18, 19, 20, 21, 22, 23
12, 13, 14, 15, 16, 17
*6, *7, *8,  9, 10, 11
*0, *1, *2,  3,  4,  5
    • extrema - Pairs of vertices possibly further away from each other.
      Say: combination of the corners. 0: diagonal (0-23) 1: x-edge (0-5) 2: y-edge (0-18) if x-y have different lengths
    • center - One corner and one center-ish ID
Returns:

List of different dimensions –

  • smy : 1D list [1,2,6,7,8]
  • extrema, center : 2D list - list of lists [[0,23],[0,5],[0,18]]
Raises:

ValueError – Unsupported source vertex calculation logic

extend_grid

extend_edge_data(edge_df, sorts=None, inits=None, strat='equal', strat_param=None)[source]

Add demand data to the edges in a (Geo)DataFrame

Parameters:
  • edge_df ((Geo)DataFrame) – edge dataframe to be extended
  • sorts (list of str, optional) – The names of new columns (extensions) Defaults to [‘residential’]
  • inits (list of int/float , optional) – The parameter values, matching to sorts argument. Defaults to [1000] for each sort.
  • strat (str, optional) – TODO now only ‘equal’ has an effect How the data values will be created + ‘equal’ - all edge demand is the same + ‘linear’ - linearly decreasing + ‘exp’ - exponentially decreasing + ‘manual’ - provide mapper in strat_param
  • strat_param (optional) – TODO not implemented yet Parameter for linear | exp | manual strategies. + ‘equal’ - None - no effect + ‘linear’ - minimum (lowest demand) + ‘exp’ - minimum (lowest demand) + ‘manual’ - function/dict to fetch value per edge

Example

sorts = ('residential', 'other')
inits = (1000, 800)
extend_edge_data(edge, sorts=sorts, inits=inits)
Raises:ValueError – If inputs differ from awaited
vert_init_commodities(vertex_df, commodities, sources=None, inplace=True)[source]

Add commodity columns to the vertex DataFrame with zeros to vertices without commodity source and source capacity at the vertices provided by sources.

Parameters:
  • vertex_df ((Geo)DataFrame) – vertex dataframe input
  • commodities (list of str) – Like (‘Elec’, ‘Gas’, ‘Heat’)
  • sources (list of tuples, optional) – Init the source nodes. Tuple form:(Commodity, Index, Value)
  • inplace (Boolean, default: True) – If False, vertex_df is not changed and the result is returned.
Returns:

None or DataFrame – DataFrame if inplace == False
Raises:

ValueError – If parameters differ from awaited

Example

comms = ('Elec', 'Gas')
sources = [('Elec', 0, 1000), ('Gas', 1, 500)]
vert_init_commodities(vert, comms, sources)

rivus.io

rivus.db

To advocate the possibilities provided by a good database connection, a throughout description of the set-up process is documented in rivus-db. There you can find help from the entry level (install, create database) to more advanced topics (queries, data archive).

In this module presents a convenient way to interact with your PostgreSQL database.

Store example

from sqlalchemy import create_engine
from rivus.io import db as rdb
engine = create_engine('postgresql://postgres:pass@localhost/rivus')
# ...
# Modelling, Solving, Analysing
# ...
this_run = dict(comment='testing graph table and features with networx',
            profiler=profile_log)
rdb.store(engine, rivus_model, run_data=run_dict)

Import example

from sqlalchemy import create_engine
from rivus.io import db as rdb
from rivus.main.rivus import create_model
engine = create_engine('postgresql://postgres:pass@localhost/rivus')
run_id = 4242
data_dfs = ['process', 'commodity', 'process_commodity', 'time', 'area_demand']
data = {df_name: rdb.df_from_table(engine, df_name, run_id)
        for df_name in data_dfs}
vertex = rdb.df_from_table(engine, 'vertex', run_id)
edge = rdb.df_from_table(engine, 'edge', run_id)

prob = create_model(data, vertex, edge, hub_only_in_edge=False)

Example queries with results and short descriptions are part of the separate documentation.

df_from_table(engine, fname, run_id)[source]

Extract data form the database into a dataframe in a form, that is common during the rivus work-flow.

Implemented DataFrames:

  • rivus_model.params[] dataframes:

    • process
    • commodity
    • process_commodity
    • edge
    • vertex
    • time
    • area_demand

  • get_timeseries dataframes:

    • source

  • get_constants dataframes:

    • cost
    • pmax
    • kappa_hub
    • kappa_process
Parameters:
  • engine (sqlalchemy engine whit psycopg2 driver) – For managing connection to the DB.
  • fname (str) – One of the implemented dataframes. (See summary.)
  • run_id (int) – run_id of an initialized run row in the DB. You could query the run table for e.g. start date, or join it vertex table and execute a geographical query and get the run_id(s) you want to work with
Returns:

DataFrame or Series – depending on the data’s dimensions. Only cost returns a Series to be consequent with get_constants.
init_run(engine, runner='Havasi', start_ts=None, status='prepared', outcome='not_run', comment=None, plot_dict=None, profiler=None)[source]

Initialize the run table with basic info.

Parameters:
  • engine (sqlalchemy engine whit psycopg2 driver) – For managing connection to the DB.
  • runner (str, optional) – Person’s name/identifier who created(executed) the data(process).
  • start_ts (datetime.datetime, optional) – Timezone-less datetime object. If omitted, .now() will be used.
  • status (str, optional) – One of the following strings: | ‘prepared’ (default) | ‘run’ | ‘error’
  • outcome (str, optional) – One of the following strings: | ‘not_run’ (default) | ‘optimum’ | ‘optimum_not_found’ | ‘error’
  • comment (str, optional) – Any text based comment. (No length limit.)
  • plot_dict (dict, optional) – Dictionary returned by the rivus.io.plot.fig3d function.
  • profiler (pandas.Series, optional) – Series containing profiled process name and execution time pairs. Execution time is measured in seconds
Returns:

int – run_id of the initialized run row in the DB.
purge_run(engine, run_id)[source]

Delete all rows related to run_id across all tables. This is not a throughout reliable function, and can do some harm. Use it with caution, and at your own risk!

Parameters:
  • engine (sqlalchemy engine whit psycopg2 driver) – For managing connection to the DB.
  • run_id (int) – run_id of the initialized run row in the DB. Used to identify related data to be removed: directly (table has run_in as FK) and indirectly (table has FK of an Entity with run_id FK)
Returns:

None
store(engine, prob, run_id=None, graph_results=None, run_data=None, time_series=None, constants=None)[source]

Store I/O plus extras of a rivus model into a postgres DB.

Parameters:
  • engine (sqlalchemy engine whit psycopg2 driver) – For managing connection to the DB.
  • prob (pyomo ConcreteModel) – Created by rivus.create_model()
  • run_id (int, optional) – run_id of an initialized run row in the DB. If omitted: init_run() will be called with run_data.
  • graph_results (iterable, optional) – Results of the graph analysis. Each graph should have its own dict. For implemented result keys see _handle_graph. E.g. [{‘is_connected’:True, ‘is_minimal’:True}, {‘is_connected’:True}]
  • run_data (dict, optional) – Keyword arguments to be passed to init_run(). runner, start_ts, status, outcome, comment, plot_dict, profiler
  • time_series (None, optional) – TODO If already present at function call, this could save time.
  • constants (None, optional) – TODO If already present at function call, this could save time.
Returns:

None
Raises:

Exception caught during data export.

rivus.plot

Interactive 3D data visualization! See live example here (May take a while to load.)

_images/3dplot.gif

Demo of a 3D interactive plot.

Plotly has a great documentation and an extensive examples library.

It is available also available as a JavaScript library, which can help when a web-app GUI is added to rivus. (Client-side visualization.)

To ensure interoperability, it thinks in dictionaries (JSONs). This gives a greater flexibility then e.g. the API of matplotlib.

Adding extra hove-over information (line length, capacity, you-name-it) is also highly flexible.

The structure can be explained easily:

data = cap_layers + hub_layer + markers
fig = dict(data=data, layout=layout)

where cap_layers, hub_layer and markers simply lists of dictionaries are, which dictionaries have the key-value pairs of the Scatter3d structure (class).

We use raw dicts instead of Plotly’s “classes” because they are mainly the same, but much faster. Still, Loading the plot can take a while if there are a lot of elements to render. (For each edge we have a scatter3d dict in the data list, which are grouped together by the same legendgroup key value.)

The plot consists of layers, stacked upon each other. Each of these represent a commodity. All of the edges are shown in each layer, but if in one no carrier was built, it is displayed dashed. If the edge is stroked through, some amount of capacity was built out there. The width of the edges are in proportion with that amount.

Diamond shapes represent the sources in the vertices.

Vertical lines show processes (commodity conversions).

All of the elements can be toggled with the help of the menu in the upper right corner.

Note

As the legends (with which you can turn off/on the layers) are generated from the first elements per legend-groups in the data list, there are now dummies as first ones to ensure a nicer look.

fig3d(prob, comms=None, linescale=1.0, use_hubs=False, hub_opac=0.55, dz=5, layout=None, verbose=False)[source]

Generate 3D representation of the rivus results using plotly

Parameters:
  • prob (rivus_archive) – A rivus model (later extract of it)
  • comms (None, optional) – list/ndarray of commodity names to plot, Order: [‘C1’, ‘C2’, ‘C3’] -> Bottom: C1, Top: C3
  • linescale (float, optional) – A multiplier to get proportionally thicker lines.
  • use_hubs (bool, optional) – Switch to depict hub processes.
  • hub_opac (float, optional) – 0-1 opacity param.
  • dz (number, optional) – Distance between layers along ‘z’ axis .
  • layout (None, optional) – A plotly layout dict to overwrite default.
  • verbose (bool, optional) – To print out progress and the time it took.

Example

import plotly.offline as po
fig = fig3d(prob, ['Gas', 'Heat', 'Elec'], hub_opac=0.55, linescale=7)
# for static image
# po.plot(fig, filename='plotly-game.html', image='png')
po.plot(fig, filename='plotly-game.html')
Returns:plotly compatible figure *dict (in plotly everything is kinda a dict.)*

Note

Greatly inspired by Example1 and Example2.

rivus.utils

prerun

Collection of small rivus related helper functions

In use to avoid multiple solutions of the same task, like:

  • Setting up the solver.
  • Todo: Create needed directories
setup_solver(optim, logfile='solver.log', guro_time_lim=12000, guro_mip_focus=2, guro_mip_gap=0.001, guro_threads=None, log_to_console=True)[source]

Change solver options to custom values.

Parameters:
  • optim (SolverFactory) – pyomo Solver object from pyomo.opt.base
  • logfile (str, optional) – default=’solver.log’ Name (Path) to the logfile
  • guro_time_lim (int, optional) – unit is seconds | default=12000
  • guro_mip_focus (int, optional) – default=2 1=feasible, 2=optimal, 3=bound
  • guro_mip_gap (float, optional) – our default=.001 (gurobi’s default: 1e-4)
  • guro_threads (None, optional) – parallel solver tasks | default=None If None, no Threads parameter is set (gurobi takes <=CPU_count threads automatically) If greater than CPU_count then it is threshold to CPU_count If less than CPU_count then Thread is set with the parameter
  • log_to_console (bool, optional) – Description
  • (Boolean, optional) If False, the output of the solver (log_to_console) – is not piped to the stdout.

Example

optim = SolverFactory('glpk')
optim = setup_solver(optim, logfile=log_filename)
Returns:SolverFactory – With applied modifications

notify

email_me(message, sender, send_pass, recipient, smtp_addr, smtp_port, subject='[rivus][notification]')[source]

Send notification message through email server.

Parameters:
  • message (str) – Body of the e-mail
  • sender (str) – The e-mail account through which the email will be sent. E.g. tum.robot@gmail.com
  • send_pass (str) – Password of sender. Hopefully read from a file, which is not added to Git…
  • recipient (str) – The e-mail account of you, where you want to get the notification.
  • smtp_addr (str) – SMTP Address. Like “smtp.gmail.com”
  • smtp_port (int) – SMTP Port. Like 587
  • subject (str, optional) – The subject of the mail…
Returns:

integer – 0 - if run through without exception -1 - if encountered with a problem (mainly for unittest)

Example

email_setup = {
    'sender': config['email']['s_user'],
    'send_pass': config['email']['s_pass'],
    'recipient': config['email']['r_user'],
    'smtp_addr': config['email']['smtp_addr'],
    'smtp_port': config['email']['smtp_port']}
...
except Exception as solve_error:
    sub = run_summary + '[rivus][solve-error]'
    email_me(solve_error, subject=sub, **email_setup)

# or with traceback:

except Exception as plot_error:
    err_tb = tb.format_exception(
                None, plot_error, plot_error.__traceback__)
    sub = run_summary + '[rivus][plot-error]'
    email_me(err_tb, subject=sub, **email_setup)

runmany

Functions are collected here, which can be useful in case of a massive runs involving analysis of a broader parameter-space.

parameter_range(data_df, index, column, lim_lo=None, lim_up=None, step=None, zero_root=None)[source]

Yield values of the parameter in a given range

Parameters:
  • data_df (DataFrame) – Original data frame, where the target parameter can be found.
  • index (valid pandas DataFrame row label) – DataFrame .loc parameter to locate the parameter value. E.g.: [‘Gas power plant’, ‘CO2’, ‘Out’] or ‘Gas’
  • column (str) – Label of the column, where the parameter is. E.g.: ‘ratio’ or ‘cap-max’
  • lim_lo (None, optional) – Proportional parameter. If omitted, 90% of the original.
  • lim_up (None, optional) – Proportional parameter. If omitted 110% of the original.
  • step (None, optional) – Proportional parameter. The difference between two following yielded values.
  • zero_root (None, optional) – If the selected parameter is 0, then the default method using proportions will fail. Use this value to set the root for the parameter range.
Yields:

DataFrame – A modified version of xls[df_name]

Example

data = read_excel(data_spreadsheet)
interesting_parameters = [
    {'df_name': 'commodity',
     'args': {'index': 'Heat',
              'column': 'cost-inv-fix',
              'lim_lo': 0.5, 'lim_up': 1.6, 'step': 0.5}},
    {'df_name': 'commodity',
     'args': {'index': 'Heat',
              'column': 'cost-fix',
              'lim_lo': 0.5, 'lim_up': 1.6, 'step': 0.5}}]
for param in interesting_parameters :
    sheet = data[param['df_name']]
    param_path = param['args']
    for variant in parameter_range(sheet, **param_path):
        ...

pandashp

pandashp: read/write shapefiles to/from special DataFrames

Offers two functions read_shp and write_shp that convert ESRI shapefiles to pandas DataFrames that can be manipulated at will and then written back to shapefiles. Opens up data manipulation capabilities beyond a simple GIS field calculator.

Usage:
import pandashp as pdshp # calculate population density from shapefile of cities (stupid, I know) cities = pdshp.read_shp(‘cities_germany_projected’) cities[‘popdens’] = cities[‘population’] / cities[‘area’] pdshp.write_shp(cities, ‘cities_germany_projected_popdens’)
bounds(df)[source]

Return a DataFrame of minx, miny, maxx, maxy of each geometry.

find_closest_edge(polygons, edges, to_attr='index', column='nearest')[source]

Find closest edge for centroid of polygons.

Parameters:
  • polygons – a pandas DataFrame with geometry column of Polygons
  • edges – a pandas DataFrame with geometry column of LineStrings
  • to_attr – a column name in DataFrame edges (default: index)
  • column – a column name to be added/overwrite in DataFrame polygons with the value of column to_attr from the nearest edge in edges
Returns:

a list of LineStrings connecting polygons’ centroids with the nearest point in in edges. Side effect: polygons recieves new column with the attribute value of nearest edge. Warning: if column exists, it is overwritten.
match_vertices_and_edges(vertices, edges, vertex_cols=('Vertex1', 'Vertex2'))[source]

Adds unique IDs to vertices and corresponding edges.

Identifies, which nodes coincide with the endpoints of edges and creates matching IDs for matching points, thus creating a node-edge graph whose edges are encoded purely by node ID pairs. The optional argument vertex_cols specifies which DataFrame columns of edges are added, default is ‘Vertex1’ and ‘Vertex2’.

Parameters:
  • vertices – pandas DataFrame with geometry column of type Point
  • edges – pandas DataFrame with geometry column of type LineString
  • vertex_cols – tuple of 2 strings for the IDs numbers
Returns:

Nothing, the mathing IDs are added to the columns vertex_cols in argument edges
read_shp(filename)[source]

Read shapefile to dataframe w/ geometry.

Parameters:filename – ESRI shapefile name to be read (without .shp extension)
Returns:pandas DataFrame with column geometry, containing individual shapely Geometry objects (i.e. Point, LineString, Polygon) depending on the shapefiles original shape type
total_bounds(df)[source]

Return bounding box (minx, miny, maxx, maxy) of all geometries.

write_shp(filename, dataframe, write_index=True)[source]

Write dataframe w/ geometry to shapefile.

Parameters:
  • filename – ESRI shapefile name to be written (without .shp extension)
  • dataframe – a pandas DataFrame with column geometry and homogenous shape types (Point, LineString, or Polygon)
  • write_index – add index as column to attribute tabel (default: true)
Returns:

Nothing.

rivus.test

Write tests to every function where there are clearly definable input(s) and output(s).

Enlighten the main logic of the test here if it is not trivial.

test_db

class RivusDBTest(methodName='runTest')[source]
test_df_insert_query()[source]

Are the stored dataframes and the retrieved ones identical?

  • Comparison form of frames is after create_model. (index is set)
  • Comparison form expects that input dataframes only have meaningful columns. (See pull request #23)
  • Only implemented dataframes are tested.

Note

Requires a config.json file in the root of rivus-repo with the database credentials. For Example:

{
    "db" : {
        "user" : "postgres",
        "pass" : "postgres",
        "host" : "localhost",
        "base" : "rivus"
    }
}

test_gridder

class RivusGridTest(methodName='runTest')[source]

test_main

class RivusMainTest(methodName='runTest')[source]

test_utils

class RivusUtilsTest(methodName='runTest')[source]
test_email_notification()[source]

It only can test, whether the notification function run trhrough successfully.

Can be useful for quick testing the email parameters in the config file

Note

Requires a config.json file in the root of rivus-repo with the database credentials. For Example:

{
    "email" : {
        "s_user" : "robot.mail@gmail.com",
        "s_pass" : "TheAnswerIs42!",
        "r_user" : "my.mail@gmail.com",
        "smpt_addr" : "smtp.gmail.com",
        "smpt_port" : "587"
    }
}
test_parameter_range()[source]

Load minimal example data and test with one of the parameters

  • Tests for side-effects on the original.
  • Tests for the awaited numer of parameters.
  • Tests for range of the parameters