Consumption Group Settings
Introduction
A consumption group in OneStream's SensibleAI Forecast is the interface through which project-related data can be exported to a relational table in a database. This includes model predictions and feature insight information. The data export parameters are highly configurable to get the output in a desired format.
This article was originally written against the SensibleAI Forecast v3.0.0 release. Some of the datatypes and settings may have changed in the v4.0.0 and may continue to evolve in future versions.
Here are the possible export options within the Consumption Group page in SensibleAI Forecast:
- Feature Effect: This type contains data that shows how a given feature and its values compare to a target's actual and predicted values. It provides insights into correlations between feature values and predictions or actuals.
- Feature Impact: This type contains data that indicates how much a given feature influences the model for a target. A large Feature Impact Value suggests that the feature is a significant driver for the model predictions.
- Backtest Model Forecast: This type includes backtest results (if an automatic backtest could be generated during SensibleAI Forecast’s model build) made by models for each target.
- Deployed Model Forecast: This type contains predictions created in the Utilization phase for all deployed models for each target.
- Deployed Prediction Explanation: This type contains the values found in the “Tug of War” feature explainability chart, detailing the quantitative influence (positive or negative) that each feature had on the predicted value.
- Backtest Prediction Explanation: The prediction explanations backtest consumption type contains the amount that the feature influenced (positive or negative) the prediction of a given model for a given target for a given date during the backtest of model build.
When creating consumption groups, you will have the options below, or a subset of them, to customize based on the following selection options.
Export Action: The type of action this consumption group should be auto-attached to. If auto-attached, it automatically runs and exports as a part of that type of action. Examples include prediction and pipeline jobs.
Actuals Types: The type of actuals (engine cleaned, engine uncleaned, source) to include in the consumption group.
Target Frequency: The frequency to export data in.
Merge Method: The type of merge to perform on data if there are multiple forecasts across a date.
Models to Return: The number of models to return in the consumption group.
Prediction Intervals: Indicates if prediction intervals should be included in the export.
Extraction Type: Select Batch to export only the latest prediction run. Select Time to use Start Date Type and End Date Type fields for the earliest prediction to the latest prediction or custom time frames.
Start Date Type: If the earliest start date or a custom beginning date should be used.
Start Date Time, Start Date Hour: The beginning date and hour of the day of the consumption group (inclusive). Available if the If Start Date Type is set to Custom.
End Date Type: If the latest end date or a custom end date should be used.
End Date Time, End Date Hour: The end date and end hour of the day of the consumption group (inclusive). Available if the If End Date Type is set to Custom.
Group Name: The consumption group name within SensibleAI Forecast.
Output Table Name: The name of the outputted table; this is how it will be displayed within the database. Note that the consumption group will automatically prepend a “SML_” to the table name (only on pre-4.0 versions). This is in the same database as the target data set.
When creating a consumption group, the option to choose Consumption Type, Group Name, and Table Name will always be present. Specific details regarding inputs and outputs surrounding consumption groups can be found in the subsequent sections.
Deployed Model Forecast Settings
Input Parameters
Export Actions | |
|---|---|
Value | Result |
Prediction Cluster Orchestrator | Selecting Prediction Cluster Orchestrator will automatically export the results of every prediction into the output table after each prediction job. The export action is tied to each prediction job. Forecast Number and Forecast Name within the export will clarify which prediction the table resulted from. If a table already exists within the group, the results are appended onto the existing table following completion. |
None | Selecting None will not automatically export the Consumption Group with the most recent predictions of the deployed model. A manual run of the consumption job must occur before table export. |
Deployed Model Forecast Table Output
Actuals Types | |
|---|---|
Value | Result |
Source Actuals | Source Actuals is the only option for Deployed Model Forecast tables. Actuals are specified in the output table by “Actuals” in the ModelCategory column. |
Backtest Model Forecast
Input Parameters
Target Frequency | |
|---|---|
Value | Result |
D | Value will be exported at the daily level. |
W | Value will be exported at the weekly level. |
W-SUN | Value will be exported at the weekly level, week starting Sunday. |
W-MON | Value will be exported at the weekly level, week starting Monday. |
W-TUE | Value will be exported at the weekly level, week starting Tuesday. |
W-WED | Value will be exported at the weekly level, week starting Wednesday. |
W-THU | Value will be exported at the weekly level, week starting Thursday. |
W-FRI | Value will be exported at the weekly level, week starting Friday. |
W-SAT | Value will be exported at the weekly level, week starting Saturday. |
M | Value will be exported at the monthly level, with the Date column representing the last day of the month. |
MS | Value will be exported at the monthly level, with the Date column representing the first day of the month. |
Q | Value will be exported at the quarterly level, with the Date column representing the last day of the quarter. |
QS | Value will be exported at the quarterly level, with the Date column representing the first day of the quarter. |
A | Value will be exported at the annual level, with the Date column representing the last day of the year. |
AS | Value will be exported at the annual level, with the Date column representing the first day of the year. |
Project Frequency | Value will be exported at the frequency of the actuals within the project. |
Backtest Model Forecast Table Output
Merge Method | |
|---|---|
Value | Result |
No Merge | No Merge is the only option in this instance, as all forecasts are kept distinct and included within the export table. If there are multiple predictions on the same day, the group will capture all of them in the export. |
Feature Impact
Input Parameters
Models to Return | |
|---|---|
Value | Result |
Best | Selecting Best will only export prediction results for the best model for each target in the project. ModelRank in this instance will be “1”, and Model will be the name of the associated model. |
Best Three | Selecting Best Three will export prediction results for the three best models for each target in the project. ModelRank in this instance will be “1”, “2”, or “3”. |
All | Selecting All will export predictions from all deployed models of the project. |
Feature Impact Table Output
Prediction Intervals | |
|---|---|
Value | Result |
Included | In order to export prediction interval data, models must be built with prediction intervals when configuring. By selecting Included, the consumption group export will have populated values for Lower PI and Upper PI , displaying the confidence bands surrounding the prediction. |
Excluded | Choosing Excluded will populate null values in Lower PI and Upper PI regardless of how the model was configured. |
Feature Effect
Input Parameters
Extraction Type | |
|---|---|
Value | Result |
Batch | Selecting Batch will only capture results corresponding to the most recent prediction job. When selecting Batch with Prediction Cluster Orchestrator, this will automatically export the prediction after completion. |
Time | Selecting Time will prompt Start Date Type and End Date Type fields for the earliest prediction to the latest prediction. If a different range is desired, a custom option is also available. |
Feature Effect Table Output
Start Date Type (Extraction Type: Time) | |
|---|---|
Value | Result |
Earliest Start | Earliest Start is a default that chooses a date going back to the start of the Actuals data, allowing all predictions to be captured. |
Custom | Selecting Custom will prompt Start Date and Start Date Hour to manually choose a beginning date for data export (inclusive). |
Deployed Prediction Explanations
Input Parameters
End Date Type (Extraction Type: Time) | |
|---|---|
Value | Result |
Latest End | Latest End is a default that chooses a date corresponding to the end of the most recent prediction. |
Custom | Selecting Custom will prompt End Date and End Date Hour to manually choose the last day (inclusive) to include in the export. |
Deployed Prediction Explanations Table Output
Column Name | Data Type | Desc. |
|---|---|---|
Model | String | Description of model used in forecast, or “Actuals” |
ModelCategory | String | Type of model used. “Actuals”, “baseline”, “statistical”, or “ml” |
TargetName | String | Concatenated target name |
Value | Decimal | Prediction value, or Actuals value |
LowerPI | nullable Decimal | Lower prediction bound - can be null |
UpperPI | nullable Decimal | Upper prediction bound - can be null |
Date | DateTime | Prediction date |
ModelRank | int | Model ranking Actuals is “-1” |
PredictionCallID | String | Unique ID for each prediction job |
PredictionScheduledTime | DateTime | Date and time the prediction job started |
ConsumptionID | String | Specific to the consumption group that was used for export |
XperimentKernelID | String | Specific to a single model and single target Changes upon a rebuild |
XperimentBuildID | String | Specific to a single target or group of targets Changes upon a rebuild |
XperimentSetID | String | Specific to a single target |
BuildInfoID | String | Specific to a pipeline job |
ProjectID | String | Specific to the project |
ConsumptionRunID | String | Specific to the time the consumption group was run |
ConsumptionRunTime | DateTime | Date and time the consumption group started |
ForecastStartDate | DateTime | Date of starting prediction |
ForecastNumber | int | Forecast count for that ForecastStartDate |
ForecastName | String | Scenario Name as entered in prediction run |
[Target Dim 1] | String | Column name of the first target dimension Dimension columns are ordered alphabetically |
… | String | … |
[Target Dim N] | String | Column name of the n-th target dimension Dimension columns are ordered alphabetically |
Backtest Prediction Explanations
Input Parameters
Export Actions | |
|---|---|
Value | Result |
Pipeline Orchestrator | Selecting Pipeline Orchestrator will automatically export the backtest model forecast after the Pipeline job is complete. Any Pipeline job, including from rebuild prompt the automatic export. |
None | Selecting None will not automatically export the Consumption Group after a pipeline. The consumption group will need to be run manually. |
Backtest Prediction Explanations Table Output
Actuals Types | |
|---|---|
Value | Result |
Source Actuals | Selecting Source Actuals includes the target actuals data into the table. In the ModelCategory column of the output table, there will be rows specified for the “Actuals” that are specified in this field. These actuals will cover the range of the backtest period. |
Engine Cleaned | Selecting Engine Cleaned uses the missing method logic in the consumption group for dates with missing actuals. This is only relevant if an option other than “Zero” is chosen in the Clean Missing Method model settings. In this instance, when ModelCategory is “Actuals” and the value is missing, the cleaned value will be represented in the consumption group. |
Engine Uncleaned | Selecting Engine Uncleaned will produce the same results as Source Actuals. |