BMIv3: Defining sets of variables exported by a model

[PhilMiller]

This is a generalization of the 'input' and 'output' sets of export items defined in BMI v2. This will enable models to present a richer range of variables to frameworks that are calling and driving them.

Example uses:

• Parameters that can be varied for calibration
• Diagnostic data (e.g. estimates of change in conserved values, internal model state)
• Data to be captured for serialization

[PhilMiller]

I posed this question to Nels:

Should there be an API to get the set of set names that a model defines?

I'm leaning toward 'no', because

1. (introspection aside) callers should only really be asking for sets whose semantics they actually know, and
2. set names are potentially a place where models could maybe usefully/meaningfully define arbitrary sets by parsing the name requested, or an unbounded number of sets

Additionally, "what sets can I expect from a given model?" seems like a very natural thing to tie into "what extensions does the model support?", and a given set name might even have different semantics around its contents depending on which of multiple conflicting extensions promise its presence

[PhilMiller]

Initially asking for review from Scott and Nels for help in drafting this well, before really looking for feedback from the broader BMI council.

[PhilMiller]

Discussed this with Nels, and made some small revisions accordingly.

[PhilMiller]

@RolfHut Eric mentioned you may not have seen this suggestion at a previous meeting. It had pretty general support. It replaces input_vars = get_input_var_names with input_vars = get_varset_members("input"), so minimal impact on teachability/readability.

[BSchilperoort]

I still worry that some information is lost from the transition from input/output variables to varsets; the implied "settability" of variables.

Especially for frameworks wrapping BMI it can be very valuable to know if a variable can be set with set_value. I believe the behavior of set_value is not well defined with regards to settability; will the BMI implementation return an exception/error code, will it fail silently, or will it still modify the values of the array even if the variable was not intended as an "input" variable?

For example, in my BMI controller front-end it is nice to only present users with an input field and a set button for variables which you can actually set. Having to read through a model's BMI documentation (if it even exists) is not machine-readable and thus makes it more difficult to implement frameworks on top of BMI.

[hellkite500]

I still worry that some information is lost from the transition from input/output variables to varsets; the implied "settability" of variables.

I think the idea here is to provide/document known sets, e.g. input_vars and output_vars similar to the BMI 2.0 expectation of these variables. A variable queried under the input_vars set carries the same implied settability as a variable name returned from get_input_vars in the current implementation.

Especially for frameworks wrapping BMI it can be very valuable to know if a variable can be set with set_value. I believe the behavior of set_value is not well defined with regards to settability; will the BMI implementation return an exception/error code, will it fail silently, or will it still modify the values of the array even if the variable was not intended as an "input" variable?

Under 2.0 semantics, set_value should return BMI_FAILURE for anything other than "set the value of the variable provided as expected". I don't think the behavior is under-defined in that sense, but many implementations may be incomplete.

For example, in my BMI controller front-end it is nice to only present users with an input field and a set button for variables which you can actually set. Having to read through a model's BMI documentation (if it even exists) is not machine-readable and thus makes it more difficult to implement frameworks on top of BMI.

These exact same semantics would work with the var set, but the controller interface would take a slightly different path. On existing BMI 2.0 the controller (likely) does something like this (python flavored)

model = initialize( config )
inputs = model.get_input_var_names( )
# show user available inputs
# get input from user to pass to model
result = model.set_value( var, value )

The exact same semantics under this proposal would look like

model = initialize( config )
inputs = get_varset_members( "input_vars" )
# show user available inputs
# get input from user to pass to model
result = model.set_value( var, value )

What we ask models to do differently here is to implement a named varset, input_vars in this example, vs a named function (get_input_vars). But the discoverability works the same.

On the topic of "settability", in a more general sense, I have found the following BMI 2.0 functions to be the minimum required to ensure a variable can be set properly -- the variable must be "known" to each of these functions and each must yield a BMI_SUCCESS for "settability" to hold.

• get_var_n_bytes
• get_var_type
• get_var_itemsize
• set_value

But the minimum test is still a call to set_value and if you get BMI_FAILURE, assume it cannot be set, if you get BMI_SUCCESS assume it works.

I would definitely be interested in perhaps a slightly separate, yet definitely related, discussion on expanding/enumerating BMI_FAILURE into some more robust error values/states.

[RolfHut]

my two cents:

• often (in our case) 'input' variables are read from disc, ie. not provided to the model via a BMI set_value call. Many of the models on our platform have a wildly different subset of variables for which 'set_value' has been implemented, than are considered input variables. I agree with @BSchilperoort that it is more graceful to ask a model: "which variables can I set through BMI" than have to check them and see if you catch an error.
• I think the definition of a variable that you can set with a set_value call should be well documented. Something along the lines of: 'setting a value will mean that the model will use this value in its next calculations done when update() is called. We've had instances where a model would overwrite the value of the variable with something read from disc at the start of update(). It needs to be clear from the documentation for anyone implementing BMI on (top of) a model, how to deal with this.

[BSchilperoort]

A variable queried under the input_vars set carries the same implied settability as a variable name returned from get_input_vars in the current implementation.

Yes, but this should then be explicitly stated in the BMI specification. For BMI 2.0 this was largely just implied.

Under 2.0 semantics, set_value should return BMI_FAILURE for anything other than "set the value of the variable provided as expected". I don't think the behavior is under-defined in that sense, but many implementations may be incomplete.

This is only specified for the C/Fortran specs. For C++ and Java it's a void function. It is not specified if e.g. Python should raise an exception here in case the value cannot be set. The main thing to rely on here was just "if it's in the input variables it is probably settable, otherwise it's not".

On existing BMI 2.0 the controller (likely) does something like this (python flavored)

For Python, C++, Java, R, JavaScript, and Julia the set_value function returns nothing.

I would definitely be interested in perhaps a slightly separate, yet definitely related, discussion on expanding/enumerating BMI_FAILURE into some more robust error values/states.

I think that expanding on the expected behavior for when function calls fail would improve BMI. The spec could use with more formalized error handling or at a minimum more guidance on this. It could also be a solution to this problem.

[PhilMiller]

Can I ask that the discussion of "what is settable" and what that means be considered separately from this change? It's as much an issue with BMI 2 get_input_vars right now, and that doesn't change if the means of enumerating "input" variables changes. If anything, I'd suggest that this change facilitates better resolutions to that issue

[PhilMiller]

Regarding the BMI_FAILURE return values, the languages that support exceptions are all documented to throw/raise an exception on failure, equivalent to returning an error code.