TimeSeriesDataGenerator

Versions

• 1.0.0

v1.0.0

Basic Information

Class Name: TimeSeriesDataGenerator

Title: Time Series DataForge

Version: 1.0.0

Author: Drew Shea

Organization: OneStream

Creation Date: 2025-11-01

Default Routine Memory Capacity: 2.0 GB

Tags

Data Transformation, Time Series, Synthetic Data

Description

Short Description

DataForge allows you to create synthetic timeseries datasets and generate reusable dataset configurations.

Long Description

Generate timeseries datasets by using a representation of the data called a manifest. A manifest is a lightweight, JSON representation of the features and patterns present inside the data.

There are 3 ways to generate a manifest: natural language, random generation, and inferring from a dataset. Natural language uses state-of-the-art LLMs to generate manifests from plain English, random generation can quickly create a manifest at any scale based on a few user configurations, and dataset inference will reverse-engineer a real dataset, extrapolating its patterns and features into a synthetic copy.

Subsequently, there are 2 ways to generate a dataset: manifest to dataset generation, or random dataset generation. The manifest to dataset generation applies the features and patterns captured within the manifest onto a baseline distribution, and creates the date, value, and dimension columns based on JSON fields. The random dataset generation method will generate a random manifest and create a dataset from it.

Usage: only Initialize DataForge is intended to be invoked directly. Generate Timeseries Dataset and Save Dataset are internal entry points driven by the DataForge web app launched from Initialize DataForge, and will fail with a clear error if run on their own.

Use Cases

1. Timeseries Data Generation

Generate timeseries data in under 15 minutes using a lightweight JSON configuration. The data will be representative of its manifest configuration, capturing high or low degrees of detail. Choose between configuring complex features such as trends, seasonalities, events and other patterns, or a simple representation that focuses on the number of targets, dimension, and date range. Once generated, choose what to do with your data, such as saving it, and use it for scale/stress testing, demoing to potential clients, and any other dataset-related use case you may have. Using a JSON representation allows manifests to be easily shared and programmatically transformed into time series data.

2. Timeseries Data Lightweight Representation

Create and store JSON representations of timeseries datasets as manifests for exportability and shareability. Choose between 3 different ways to generate manifests, ranging from extremely simple to highly-complex representations of your data. Use natural language if you're looking for an intuitive, non-technical way to generate a manifest. Try out random manifest generation to quickly develop data representations that capture basic constraints such as the number of targets, the number of dimensions, and data range configurations such as the time period in months and the data frequency. For the most realistic representation, use dataset inference to replicate the structure of a real-world dataset.

Routine Methods

1. Init (Constructor)

• Method: __init__

  • Type: Constructor

  • Allow In-Memory Execution: Yes

  • Read Only: Yes

  • Method Limits: N/A

  • Outputs Dynamic Artifacts: No

  • Short Description:

    • Constructor method for the DataForge Routine.

  • Detailed Description:

    • This method is responsible for creating all DataForge instances.

  • Inputs:

    • No input parameters

  • Artifacts: No artifacts are returned by this method

2. Generate Timeseries Dataset (Method)

• Method: generate_timeseries_dataset

  • Type: Method

  • Allow In-Memory Execution: No

  • Read Only: Yes

  • Method Limits: N/A

  • Outputs Dynamic Artifacts: No

  • Short Description:

    • [INTERNAL] Invoked by the DataForge web app. Do not run directly; use Initialize DataForge.

  • Detailed Description:

    • DO NOT RUN THIS METHOD DIRECTLY. This method is the dataset-generation entry point that the DataForge web app calls under the hood after you configure a manifest in the UI. It is not intended to be configured or invoked on its own and will fail with a clear error if it is. To generate a timeseries dataset, run the Initialize DataForge method on this routine and use the resulting DataForge web app, which will invoke this method for you with the correct manifest as you click "Generate" in the UI.

  • Inputs:

    • Required Input
      • Manifest Input: JSON-serialized manifest for generation.
        • Name: manifest_json
        • Long Description: The full time series dataset manifest serialized as a JSON string.
        • Tooltip:
          • Validation Constraints:
            • This input may be subject to other validation constraints at runtime.
        • Type: str

  • Artifacts:

    • Generated Dataset: Synthetic timeseries dataset as a Polars DataFrame.

      • Qualified Key Annotation: dataset
      • Aggregate Artifact: False
      • In-Memory Json Accessible: False
      • File Annotations:
        • artifacts_/@dataset/data_/data_<int>.parquet
          • A partitioned set of parquet files where each file will have no more than 1000000 rows.

    • Generation Metadata: JSON string with row_count, col_count, elapsed_seconds, and column info.

      • Qualified Key Annotation: metadata
      • Aggregate Artifact: False
      • In-Memory Json Accessible: True
      • File Annotations:
        • artifacts_/@metadata/data_/string.txt
          • A text file of string data.

3. Initialize Dataforge (Method)

• Method: initialize_dataforge

  • Type: Method

  • Allow In-Memory Execution: No

  • Read Only: Yes

  • Method Limits: N/A

  • Outputs Dynamic Artifacts: No

  • Short Description:

    • Entry point for DataForge.

  • Detailed Description:

    • This method is responsible for initializing the method routine entry point for DataForge in the form of an Artifact. This method also seeds the DataForge ChatAgent if it doesn't exist, responsible for generating the manifest from natural language. If GenAI is disabled in the environment, do not seed the chat agent.

  • Inputs:

    • No input parameters

  • Artifacts:

    • DataForge Web App: An interactive dashboard to generate TimeSeries data and lightweight representations.
      • Qualified Key Annotation: web_app
      • Aggregate Artifact: False
      • In-Memory Json Accessible: False
      • File Annotations:
        • artifacts_/@web_app/data_/data.appref
          • json file of data relating to web app

4. Save Dataset (Method)

• Method: save_dataset

  • Type: Method

  • Allow In-Memory Execution: No

  • Read Only: Yes

  • Method Limits: N/A

  • Outputs Dynamic Artifacts: No

  • Short Description:

    • [INTERNAL] Invoked by the DataForge web app. Do not run directly; use Initialize DataForge.

  • Detailed Description:

    • DO NOT RUN THIS METHOD DIRECTLY. This method is the dataset-save entry point that the DataForge web app calls under the hood after you generate a dataset and choose a destination in the Save panel. It is not intended to be configured or invoked on its own and will fail with a clear error if it is. To save a generated dataset, run the Initialize DataForge method on this routine and use the Save button in the resulting DataForge web app on a generated dataset, which will invoke this method for you with the correct destination configuration.

  • Inputs:

    • No input parameters

  • Artifacts: No artifacts are returned by this method

Interface Definitions

No interface definitions found for this routine

Developer Docs

Routine Typename: TimeSeriesDataGenerator

Method Name	Artifact Keys
__init__	N/A
generate_timeseries_dataset	dataset, metadata
initialize_dataforge	web_app
save_dataset	N/A