Linking consumers and suppliers in space

Preparation

Successful linking of datasets requires that the steps be followed in the right order, as they build upon each other.

First, we add the field reference product to each dataset.

ocelot.transformations.utils.label_reference_product(data)

Add field reference product to all datasets.

The field reference product has the name of the reference product exchange.

Raises InvalidMultioutputDataset if multiple reference products are present, or ValueError if no reference products are present.

We also need to clean up the data. There are some specific datasets which we can safely modify or delete.

ocelot.transformations.locations.markets.delete_allowed_zero_pv_market_datsets(data)

Remove some (but not all) global markets with zero production volume.

Uses a white list of markets which are not in the ecoinvent 3.2 release.

ocelot.transformations.locations.markets.assign_fake_pv_to_confidential_datasets(data)

Confidential datasets have production volumes of zero, because. Just because.

In order to allocate these datasets to markets. In the absence of data, we assume all confidential datasets contribute equally.

Next, we change global datasets to rest-of-world datasets whenever this is necessary, i.e. whenever there is also a region-specific dataset for this reference product. We need to do this before calculating the attribute-specific uniquely identifying hash, as this hash depends on the location field.

ocelot.transformations.locations.rest_of_world.relabel_global_to_row(data)

Change GLO locations to RoW if there are region-specific datasets in the activity group.

We then calculate the unique codes from each dataset, which are derived from a number of dataset attributes that together should uniquely identify a dataset.

ocelot.transformations.identifying.add_unique_codes(data)

Add the field code based on unique attributes of datasets.

Raises IdenticalDatasets if two datasets with the same attributes were found.

Uses activity_hash, which hashes the dataset name, reference product, unit, location, start and end date.

ocelot.transformations.utils.activity_hash(dataset)

Return the hash string that uniquely identifies an activity.

Uses the following fields, in order:
  • name
  • reference product
  • unit
  • location
  • start date
  • end date

An empty string is used if a field isn’t present. All fields are cast to lower case.

Finding suppliers

In this section, we will fill up market datasets with their inputs. These inputs are not defined in the dataset, but should be filled automatically from other input datasets by matching the name of the market reference product.

Using our new identifying codes, we can turn activity links (hard links that are not resolved by our linking algorithm, but rather specified already in the undefined datasets) from an UUID to an actual link with a specific dataset. This can only happen after allocation because the activity link reference could have been to a multi-output dataset which is now split into multiple datasets (which would make the UUID non-unique). Actual linking means that we add the code attribute to the exchange, and this code refers to the dataset which produces the product that is being consumed.

Add code field to activity links.

In the cutoff model, we have special recycled content datasets. These need to be found in a separate function.

ocelot.transformations.locations.markets.add_recycled_content_suppliers_to_markets(data)

Link markets to recycled content producing activities.

At this point, the markets have not been modified in any way, but the recycled content cut-off processes have been created by create_recycled_content_datasets. So, we have the following:

  • A market activity has a name market for foo, and a reference product of foo.
  • A new transforming activity has the name foo, Recycled Content cut-off, and a reference product of foo, Recycled Content cut-off.

We need to correctly add the recycled content suppliers to these markets. The general purpose add_suppliers_to_markets doesn’t work because the reference products are different.

We can then apply the general purpose algorithm for finding suppliers, from transforming activities to market activities.

ocelot.transformations.locations.markets.add_suppliers_to_markets(data, from_type='transforming activity', to_type='market activity', topo_func=None)

Add references to supplying exchanges to markets in field suppliers.

By default works with inputs to markets, but can be curried to work with market groups.

Should only be run after ensuring that each dataset has one labeled reference product. Need to add actual exchange data because we need production volumes.

Does not change the exchanges list or do allocation between various suppliers.

We do the same for market groups, which are broad regional collections of markets.

Markets don’t start with production volumes - instead, their production volume is defined by the production volumes of their inputs. We need to add these production volume amounts to markets.

ocelot.transformations.locations.markets.update_market_production_volumes(data, kind='market activity')

Update market production volumes to sum to the production volumes of all applicable inputs, minus any hard (activity) links to the market and to the market suppliers.

By default works only on markets with type market activity, but can be curried to work on market group types as well.

Activity link amounts are added by add_hard_linked_production_volumes and are currently given in rp_exchange['production volume']['subtracted activity link volume'].

Production volume is set to zero is the net production volume is negative.

Our add_suppliers_to_markets function has not allocated between the possible suppliers - it has only created the list suppliers which has these possible suppliers. We then choose how much of each possible supplier will contribute to each market, based on the supplier’s production volumes.

ocelot.transformations.locations.markets.allocate_all_market_suppliers(data)

Allocate all market activity suppliers.

Uses the function allocate_suppliers, which modifies data in place.

ocelot.transformations.locations.markets.allocate_suppliers(dataset)

Allocate suppliers to a market dataset and create input exchanges.

The sum of the suppliers inputs should add up to the production amount of the market (reference product exchange amount), minus any constrained market links. Constrained market exchanges should already be in the list of dataset exchanges, with the attribute constrained.

Finding consumers

In this section, we link undefined technosphere inputs to the activities that provide these flows. We do this in an iterative fashion - first, find any inputs of cutoff-specific dataset created by the system model. Next, if no provider was found, a suitable regional market is sought. Finally, if no region-specific provider is found, a global supplier will be selected.

ocelot.transformations.locations.linking.link_consumers_to_recycled_content_activities(data)

Link technosphere exchange inputs to activities that produce Recycled Content products.

Only for inputs of recyclable byproducts in cutoff system model. In these dataset, linking doesn’t go through a market, but goes directly to the transforming activity.

Add the field code to each exchange with the code of the linked market activity.

Link technosphere exchange inputs to markets.

Should only be run after add_suppliers_to_markets. Skips hard (activity) links, and exchanges which have already been linked.

Add the field code to each exchange with the code of the linked market activity.

Link technosphere exchange inputs to GLO or RoW markets.

Add the field code to each exchange with the code of the linked market activity.

ocelot.transformations.locations.linking.log_and_delete_unlinked_exchanges(data)

Log and delete exchanges which haven’t been linked.