DyPy API

dypy Package

DyPy Core

Dynamic Program class - implements only backward dynamic programming. Ideal usage is to have an objective function defined by the user, however they’d like.

The user defines as many StateVariable instances as they have state variables in their DP and they define a DecisionVariable. The objective function should be prepared to take arguments with keyword values for each of these, where the keyword is determined by the name attribute on each instance. It then returns a value.

For situations with multiple state variables, we have reducers, which, prior to minimization or maximization will reduce the number of state variables to one so that we can get a single F* for each input scenario. This is a class Reducer with a defined interface, and can then be extended. For our levee problem, we have a ProbabilisticReducer which keeps track of the probability of each set of state variables and can collapse by a master variable. Probabilities must be provided by the user.

# TODO: Make sure to check how we can make the decision variable properly interact with the state variable - thinking # of making sure that the decision variable adds to the correct state variable items # TODO: Missing plan for how we assign probabilities here - needs to be incorporated somewhere # TODO: Missing choice constraints (eg, minimim possible selection at stage ___ is ___)

At that point, usage for the levee problem should be something like:

import support  # user's extra code to support the objective function
import dypy as dp

objective_function = support.objective_function
height_var = dp.StateVariable("height")
flow_var = dp.StateVariable("peak_flow")
variance_var = dp.StateVariable("variance")
decision_var = dp.DecisionVariable()
decision_var.related_state = height_var  # tell the decision variable which state it impacts

dynamic_program = dp.DynamicProgram(objective_function=objective_function,
                                                                        state_variables=(height_var, flow_var, variance_var),
                                                                        decison_variable=decision_var)
dynamic_program.optimize()  # runs the backward recursion
dynamic_program.get_optimal_values()  # runs the forward method to obtain choices

class dypy.DiscountedSimplePrior(stage, data=None, matrix=None)

Bases: dypy.SimplePrior

Same as SimplePrior, but discounts values from future stages before applying them to the current stage. This discounting is cumulative (as in stage n+2 will be discounted and applied to stage n+1 and then that summed value will then be discounted and applied to stage n.

Discount rate is taken from the DynamicProgram object and will be transformed (simply) to cover the timestep size on the DynamicProgram - see the DynamicProgram parameter documentation for more.

setUp()

class dypy.DynamicProgram(objective_function=<function default_objective>, timestep_size=None, time_horizon=None, discount_rate=0, state_variables=None, max_selections=None, selection_constraints=None, decision_variable=None, calculation_function=None, prior=None)

Bases: object

This object actually runs the DP - doesn’t currently support multiple decision variables or probabilities.

Currently designed to only handle backward DPs.

Attributes:

DynamicProgram.stages list of Stage objects

Parameters:

• objective_function – What function provides values to populate each cell in each stage? Not required if you build your own stages with their own values
• selection_constraints – Is there a minimum value that must be achieved in the selection? If so, this should be a list with the required quantity at each time step
• decision_variable – a DecisionVariable instance object
• discount_rate – give the discount rate in timestep_size units. Though timesteps don’t need to be in years, think of the discount rate as applying per smallest possible timestep size, so if your timestep_size is 40, then your discount rate will be for one timestep in 40 and will compound across the 40 timesteps in the single stage. Defaults to 0, which results in no discounting, and discounting is only enabled automatically if you use the DiscountedSimplePrior, or another Prior object that handles discounting.
• calculation_function – What function are we using to evaluate? Basically, is this a maximization (benefit) or minimization (costs) setup. Provide the function object for max or min. Provide the actual min or `max functions (don’t run it, just the name) or if convenient, use the shortcuts dp.MINIMIZE or dp.MAXIMIZE
• max_selections – Up to what value of the state variable can we use to make our decision? If not provided, all values of the state variable will be allowed
• prior – Which class should be used to handle priors (best values from future stages) - SimplePrior is best in many cases, but may not apply everywhere and will be slow for large tables. Just provide the class object, not an instance.

add_stage(name)

Adds an empty stage to this DynamicProgram, tied to the currently set DecisionVariable and Prior classes associated with this DynamicProgram. Gives the stage the name provided as a parameter, usually generated automatically by the DP.

This method is called automatically by default, but is exposed as a public method primarily so you can control its usage if you desire, or so you can override its behavior in a subclass (such as if you wish to manually handle Stage creation in order to control the process for a specific dynamic programming problem).

Parameters:name – The name associated with the stage. Generated by the name prefix and the stage ID when called via build_stages Returns:None

add_state_variable(variable, setup_state=True)

Adds a state variable object to this dynamic program - automatically sets up the dynamic program to run with the new state variable when setup_state is True. Otherwise waits for you to call it manually. If you provide your state variables at DynamicProgram creation, then this method does not need to be called.

Parameters:

• variable – A StateVariable object - afterward, will be available in .state_variables
• setup_state – If True, runs _setup_state_variables after finishing. Default is True, but when adding a bunch of state variables, it’s faster just to manually run it once at the end. It runs it by default so that for simple DPs, you don’t need to think about it, but advanced, larger DPs, may wnat to set it to False and run _setup_state_variables once all are added

Returns:

build_stages(name_prefix='Step')

Make a stage for every timestep

Parameters:name_prefix – The string that will go before the stage number when printing information Returns:

run()

class dypy.Prior(stage, data=None, matrix=None)

Bases: object

The Prior classes provide different ways of applying future stage values to earlier stages. SimplePrior includes an implementation that will work for most single variable problems, but which may not work for multi-variable problems. This class can be subclassed and have the apply method overridden to provide a different implementation. The apply method should return the new matrix

Parameters:

• stage – The Stage object to attach this prior to - allows for different stages to have different prior application methods, if needed
• data – The values from the future stage to apply back to the previous stage
• matrix – The matrix (2D numpy array) for consideration in the current stage
• transformation – A function - can be used by subclasses to transform values prior to applying them

apply()

exists()

setUp()

class dypy.SimplePrior(stage, data=None, matrix=None)

Bases: dypy.Prior

A simple implementation of the prior class - given a single set of optimal values from the future stage (as a 1D numpy array), it applies these values to the previous stage by flipping the axis of the decision and shifting the values down by 1 as we move across the decisions.

This prior application method won’t work for multiple state variables or for state variables with different discretization than decision variables. For more advanced situations with multiple states, we’ll need to use the linked_state attribute on the decision variables to understand how to apply priors. This functionality may need to be re-engineered or expanded.

apply()

class dypy.Stage(name, decision_variable, parent_dp, previous=None, next_stage=None, prior_handler=None)

Bases: object

build()

Builds the stage table - evaluates the objective function for each location, passing the various values to it so it can provide a result. Does not handle prior data though - that is handled when we actually run the DP. We separate this out so that people who want to intervene between generating the stage table and handling the prior can do so.

Returns:

create_prior_handler()

get_optimal_values(prior=0)

After running the backward DP, moves forward and finds the best choices at each stage.

Parameters:prior – The value of the choice at each stage Returns:

optimize(prior=None)

This handles the actual backward DP calculation - assumes that the stage has a matrix that is built already with appropriate values. Calls the Prior object to apply future values back to the current stage, when necessary.

• Need to think about where reducers fit in
• Need to figure out how we want to handle both selection constraints, but also decision/state interactions

Right now, we can say which state a decision feeds back on - but what’s going through my head about that is:

1. Do we need that if we expect an objective function? I think we do because that determines how we take values from the future and propogate them backward. This is where maybe we might not actually be able to use numpy broadcasting? Or maybe we have to shift the array we’re broadcasting to align based on the best decision - not sure - need to think of how we’re going to handle that a bit more

2. What about where there’s some limit imposed by decision var / state var interaction. For example, with the course studying test problem - in the last stage, the state variable can’t be less than the decision variable. Maybe that’s always the case though, and is only a last stage problem?

Parameters:prior – Returns:

dypy.default_objective(*args, **kwargs)

A bare objective function that makes all cells 0 - provided as a default so we can build stages where someone might do a manual override, but the DP should check to make sure the objective isn’t the default with all cells at 0 before solving.

Parameters:

• args –
• kwargs –

Returns:

DyPy Variables

class dypy.variables.AbstractVariable(name, variable_id=None, minimum=None, maximum=None, step_size=None, values=None, *args, **kwargs)

Bases: object

maximum

minimum

step_size

values

class dypy.variables.DecisionVariable(*args, **kwargs)

Bases: dypy.variables.AbstractVariable

We’ll use this to manage the decision variable - we’ll need columns for each potential value here

Parameters:

• name –
• related_state – the StateVariable object that this DecisionVariable directly feeds back on
• variable_id – will be used as the kwarg name when passing the value of the state into the objective function. If not provided, is generated from name by removing nonalphanumeric or underscore characters, lowercasing, and removing numbers from the beginning. If it is provided, it is still validated into a Python kwarg by removing leading numbers and removing non-alphanumeric/underscore characters, while leaving any capitalization intact

add_constraint(stage, value)

Will be used to add constraints on how much or little of the decision variable is chosen in each stage. Not yet implemented

Want to figure out a way here to store also whether this constraint is a minimum or a maximum value constraint. Need to think how we’d handle that behavior

Parameters:

• stage –
• value –

Returns:

class dypy.variables.StateVariable(*args, **kwargs)

Bases: dypy.variables.AbstractVariable

Parameters:

• name –
• values –
• initial_state – the initial value of this state variable at stage 0 - used when getting the ultimate solution - if not provided or None, then any state can be usd in the first stage, which is often not desired.
• availability_function –

  A numpy function indicating which states in the next stage are valid selections given the value in the current stage plus decisions. Should be: - numpy.equal (default) - only values that match the current state of this variable are available selections - numpy.not_equal - only values not matching the current state are valid - numpy.greater - only state values greater than the current state are available selections - numpy.greater_equal - same as above, but greater than or equal - numpy.less - only state values less than the current state are available selections - numpy.less_equal - same as above, but less than or equal

  Any function that takes a 2D numpy array as parameter one and the state value as parameter 2 and returns a new 2D array is valid.

• variable_id – will be used as the kwarg name when passing the value of the state into the objective function. If not provided, is generated from name by removing nonalphanumeric or underscore characters, lowercasing, and removing numbers from the beginning. If it is provided, it is still validated into a Python kwarg by removing leading numbers and removing non-alphanumeric/underscore characters, while leaving any capitalization intact

dypy.variables.check_variable_id(name, variable_id)

Given a full variable name and a current variable_id returns the keyword argument name for the variable. Designed for private within-package usage, but public for inspection and overriding.

Parameters:

• name – Full name of a variable
• variable_id – the current variable ID

Returns:

sanitized new variable_id, suitable for usage in a Python keyword argument

DyPy State Variable Reducers

class dypy.reducers.ProbabilisticReducer(variable, stage)

Bases: dypy.reducers.Reducer

Given a StateVariable to process (S), and a set of StateVariables to hold constant (Cs), reduces S for each combination of Cs by multiplying the objective values in the records for S by their probabilities and summing them.

We should be able to actually just make this have a single column for probabilities so that we can do the same thing we planned to do for the variable reducer and just (ignoring the first paragraph of this docstring) select a master variable, get all rows for it, multiply those rows by the probability field, and sum them up.

class dypy.reducers.Reducer(variable, stage)

Bases: object

Reduces multiple state variables to a single state variable so we can just minimize

class dypy.reducers.VariableReducer(variable, stage)

Bases: dypy.reducers.Reducer

Given a StateVariable, reduces the table size by collapsing all other variables - can do this by min/max/mean/sum of all options.

Saving implementation here until after we have a better sense for how the rest of this will be implemented’

Parameters:

• variable –
• stage –