Skip to main content

Interface IMetaFileSystemClient

Provides a unified interface for file and directory operations across the XperiFlow MetaFileSystem storage infrastructure.

Namespace: Workspace.XBR.Xperiflow.MetaFileSystem

Assembly: Xperiflow.dll

Declaration
public interface IMetaFileSystemClient

Examples​

Example 1: Working with Framework Location

// Create a Framework-scoped client
var frameworkClient = MetaFileSystemClientFactory.CreateClient(
    sessionInfo, 
    MetaFileSystemLocation.Framework
);

// Read configuration file (implicit path - no protocol needed)
var configJson = frameworkClient.GetFileContentStringAsync("config/app.json").Result;

// Can also use explicit protocol (both resolve to the same file)
var configJson2 = frameworkClient.GetFileContentStringAsync("framework://config/app.json").Result;

// Check if a directory exists
bool configDirExists = frameworkClient.DirectoryExistsAsync("config").Result;

// List directory contents
var configDir = frameworkClient.GetDirectoryAsync("config", maxDepth: 1).Result;
foreach (var file in configDir.Files)
{
    BRApi.ErrorLog.LogMessage(si, $"File: {file.Name}, Size: {file.Size} bytes");
}

Example 2: Working with Routine Location

// Create a Routine-scoped client
var routineClient = MetaFileSystemClientFactory.CreateClient(
    sessionInfo, 
    MetaFileSystemLocation.Routine
);

// Read Python script (implicit path)
var scriptContent = routineClient.GetFileContentStringAsync("scripts/data_processor.py").Result;

// Write execution results
var results = new { status = "completed", recordsProcessed = 1500 };
var resultsJson = Newtonsoft.Json.JsonConvert.SerializeObject(results);
routineClient.WriteFileAsync("output/execution_results.json", resultsJson, overwriteOk: true).Wait();

// Read partial content (first 1000 bytes) for large files
var largeFilePreview = routineClient.GetFileContentStringAsync("logs/execution.log", contentSize: 1000).Result;

// Stream large file efficiently
using (var stream = routineClient.GetFileContentStreamAsync("data/large_dataset.csv").Result)
{
    // Process stream without loading entire file into memory
    using (var reader = new StreamReader(stream))
    {
        string line;
        while ((line = reader.ReadLine()) != null)
        {
            // Process each line
        }
    }
}

Example 3: Working with Shared Location

// Create a Shared-scoped client
var sharedClient = MetaFileSystemClientFactory.CreateClient(
    sessionInfo, 
    MetaFileSystemLocation.Shared
);

// Access shared templates
var templateBytes = sharedClient.GetFileContentByteArrayAsync("templates/report_template.xlsx").Result;

// Write shared reference data
var lookupData = GetLookupTableData();
sharedClient.WriteFileAsync("shared://reference/product_lookup.json", lookupData).Wait();

// Create directory for team collaboration
sharedClient.MakeDirectoriesAsync("team_workspace/project_alpha", existsOk: true).Wait();

// Check if shared resource exists before accessing
if (sharedClient.FileExistsAsync("datasets/master_data.csv").Result)
{
    var masterData = sharedClient.GetFileContentStreamAsync("datasets/master_data.csv").Result;
    // Process master data
}

Example 4: Working with Meta Location (Cross-Location Access)

// Create a Meta client for unified access
var metaClient = MetaFileSystemClientFactory.CreateClient(
    sessionInfo, 
    MetaFileSystemLocation.Meta
);

// IMPORTANT: Meta location requires explicit protocols for all paths

// Read configuration from Framework
var frameworkConfig = metaClient.GetFileContentStringAsync("framework://config/app.json").Result;

// Read script from Routine
var routineScript = metaClient.GetFileContentStringAsync("routine://scripts/process.py").Result;

// Read template from Shared
var sharedTemplate = metaClient.GetFileContentByteArrayAsync("shared://templates/report.xlsx").Result;

// Cross-location file copy operation
var sourceData = metaClient.GetFileContentByteArrayAsync("framework://seed_data/initial_config.json").Result;
metaClient.WriteFileAsync("routine://config/imported_config.json", sourceData, overwriteOk: true).Wait();

// INVALID - implicit paths are NOT supported with Meta location
// var file = metaClient.GetFileContentStringAsync("config/app.json").Result; // This will FAIL!

Example 5: Advanced Operations - Directory Management and Metadata

var client = MetaFileSystemClientFactory.CreateClient(
    sessionInfo, 
    MetaFileSystemLocation.Routine
);

// Create nested directory structure
client.MakeDirectoriesAsync("projects/2024/Q4/reports", existsOk: true).Wait();

// Get file metadata without downloading content
var fileMetadata = client.GetFileMetadataAsync("scripts/processor.py").Result;
BRApi.ErrorLog.LogMessage(si, $"File Size: {fileMetadata.Size} bytes");
BRApi.ErrorLog.LogMessage(si, $"Last Modified: {fileMetadata.LastModified}");

// Get directory metadata
var dirMetadata = client.GetDirectoryMetadataAsync("scripts").Result;

// Recursively list all files in directory tree
var allFiles = client.GetDirectoryAsync("projects", maxDepth: -1, includeDirectories: false).Result;
foreach (var file in allFiles.Files)
{
    BRApi.ErrorLog.LogMessage(si, $"Found: {file.Path}");
}

// Delete file if it exists
client.RemoveFileAsync("temp/old_file.txt", errorWhenDoesNotExist: false).Wait();

// Remove empty directory
client.RemoveDirectoryAsync("temp", recursive: false, errorWhenDoesNotExist: false).Wait();

// Remove directory and all contents
client.RemoveDirectoryAsync("old_project", recursive: true).Wait();

Remarks​

The MetaFileSystem is a distributed storage abstraction layer built on Azure Blob Storage that provides organized, protocol-based access to different logical storage locations within the XperiFlow platform. Each storage location serves a specific purpose and is accessed through a consistent API, enabling seamless file operations across framework resources, routine artifacts, and shared data.

Storage Locations:

1. Framework Location - System-level configuration and framework resources

The Framework location stores core system resources, configuration files, and framework-level assets that are shared across the entire XperiFlow platform. This includes application settings, system templates, and infrastructure configuration.

  • Protocol: framework://

  • Path Resolution: Supports both explicit protocol paths ("framework://config/app.json") and implicit paths ("config/app.json"). When no protocol is specified, the path is automatically resolved with the "framework://" protocol prefix.

2. Routine Location - Executable scripts, workflows, and routine-specific artifacts

The Routine location stores executable code, Python scripts, workflow definitions, and artifacts generated by routine executions. This is the primary storage location for user-defined business logic, data processing scripts, and automation workflows within XperiFlow.

  • Protocol: routine://

  • Path Resolution: Supports both explicit protocol paths ("routine://scripts/process.py") and implicit paths ("scripts/process.py"). When no protocol is specified, the path is automatically resolved with the "routine://" protocol prefix.

3. Shared Location - Cross-application data and collaborative resources

The Shared location stores data and resources that are shared across multiple applications, users, or organizational units within the XperiFlow ecosystem. This includes shared datasets, common templates, reference data, and collaborative workspaces.

  • Protocol: shared://

  • Path Resolution: Supports both explicit protocol paths ("shared://templates/report.xlsx") and implicit paths ("templates/report.xlsx"). When no protocol is specified, the path is automatically resolved with the "shared://" protocol prefix.

4. Meta Location - Unified cross-location access to all storage locations

The Meta location is a special unified storage location that enables access to files across all underlying storage locations (Framework, Routine, and Shared) through a single MetaFileSystem client. This is particularly useful for operations that need to work with resources from multiple locations or when the storage location is determined dynamically at runtime.

  • Protocol: framework://, routine://, or shared:// (must be specified)

  • Path Resolution: REQUIRES explicit protocol specification for all file paths. You must use fully-qualified paths with protocols. Implicit paths without protocols are not supported and will result in errors.

  • Use Cases: Dynamic routing, cross-location operations, unified administrative views, multi-location transactions

Key Features:

  • Protocol-Based Routing: Automatic routing to the correct Azure Blob Storage container based on path protocol

  • Efficient Streaming: Support for partial content retrieval using Azure Blob Storage Range requests

  • Metadata Preservation: Automatic preservation of blob metadata during file updates and overwrites

  • Async Operations: Full async/await support with cancellation tokens for scalable, non-blocking I/O

  • Flexible Content Types: Read and write operations support Stream, string, and byte[] payloads

Methods​

DirectoryExistsAsync(string, CancellationToken)​

Checks if a directory exists in the Meta FileSystem at the provided path.

Declaration
Task<bool> DirectoryExistsAsync(string path, CancellationToken cancellationToken = default)
Returns​

Task<System.Boolean>

Parameters​
TypeNameDescription
System.StringpathThe path to the directory. If the directory path.
System.Threading.CancellationTokencancellationTokenThe cancellation token.

GetDirectoryMetadataAsync(string, bool, CancellationToken)​

Retrieves only the metadata of a directory (no subdirectories or subfiles) from the Meta FileSystem.

Declaration
Task<IDirectoryMetadata?> GetDirectoryMetadataAsync(string path, bool errorWhenDoesNotExist = true, CancellationToken cancellationToken = default)
Returns​

Task< Workspace.XBR.Xperiflow.MetaFileSystem.IDirectoryMetadata >

DirectoryMetadata

Parameters​
TypeNameDescription
System.StringpathThe path to the folder. If the folder path does not exist, an error will be thrown.
System.BooleanerrorWhenDoesNotExistWhether or not to throw an error when the directory does not exist. Defaults to true.
System.Threading.CancellationTokencancellationTokenThe cancellation token.

GetDirectoryAsync(string, int?, bool, bool, CancellationToken)​

Retrieves a directory and its subcontents (other file and folders metdata) from the Meta FileSystem.

Declaration
Task<IDirectoryContent?> GetDirectoryAsync(string path, int? maxDepth = 1, bool includeDirectories = true, bool errorWhenDoesNotExist = true, CancellationToken cancellationToken = default)
Returns​

Task< Workspace.XBR.Xperiflow.MetaFileSystem.IDirectoryContent >

DirectoryContent: This holds accessors to files and directory subcontents.

Parameters​
TypeNameDescription
System.StringpathThe path to the folder. If the folder path does not exist, an error will be thrown.
System.Nullable<System.Int32>maxDepthThe max depth of subcontents to grab. Defaults to 1 level deep of subcontents. Set to -1 or null to grab all subcontents.
System.BooleanincludeDirectoriesWhether or not to include subfolders in the subcontents. Defaults to true.
System.BooleanerrorWhenDoesNotExistWhether or not to throw an error when the directory does not exist. Defaults to true.
System.Threading.CancellationTokencancellationTokenThe cancellation token.

MakeDirectoriesAsync(string, bool, CancellationToken)​

Makes one or more directories in the Meta FileSystem at the provided path.

Note that this method is subject to runtime checks that may not allow the creation of directories in certain locations.

Declaration
Task MakeDirectoriesAsync(string path, bool existsOk = false, CancellationToken cancellationToken = default)
Returns​

System.Threading.Tasks.Task

Parameters​
TypeNameDescription
System.StringpathThe directory path to create.
System.BooleanexistsOkControls whether or not to throw an error if the directory already exists. Defaults to false.
System.Threading.CancellationTokencancellationToken

RemoveDirectoryAsync(string, bool, bool, CancellationToken)​

Removes a directory from the MetaFileSystem. This method is not recursive in that it will only remove the last directory in the path. It also expects that the directory is empty.

Note that this method is subject to runtime checks that may not allow removing a directory in certain locations.

Declaration
Task RemoveDirectoryAsync(string path, bool recursive = false, bool errorWhenDoesNotExist = true, CancellationToken cancellationToken = default)
Returns​

System.Threading.Tasks.Task

Parameters​
TypeNameDescription
System.StringpathThe directory path to remove.
System.BooleanrecursiveWhether or not to attempt to recursively remove the directory and its all of its subcontents. Defaults to false.
System.BooleanerrorWhenDoesNotExistWhether or not to throw an error when the directory in the path does not exist. Defaults to true.
System.Threading.CancellationTokencancellationToken

FileExistsAsync(string, CancellationToken)​

Checks if a file exists in the Meta FileSystem at the provided path.

Declaration
Task<bool> FileExistsAsync(string path, CancellationToken cancellationToken = default)
Returns​

Task<System.Boolean>

bool: true means the file does exist.

Parameters​
TypeNameDescription
System.StringpathThe file path. Ex: folderA/folderB/filename.txt
System.Threading.CancellationTokencancellationTokenThe cancellation token.

GetFileMetadataAsync(string, bool, CancellationToken)​

Retrieves the metadata of a file (not the raw content of the file) from the Meta FileSystem.

Declaration
Task<IFileMetadata?> GetFileMetadataAsync(string path, bool errorWhenDoesNotExist = true, CancellationToken cancellationToken = default)
Returns​

Task< Workspace.XBR.Xperiflow.MetaFileSystem.IFileMetadata >

FileMetadata

Parameters​
TypeNameDescription
System.StringpathThe file path. ex: folderA/folderB/filename.txt
System.BooleanerrorWhenDoesNotExistWhether or not to throw an error when the file does not exist. Defaults to true.
System.Threading.CancellationTokencancellationTokenThe cancellation token.

GetFileContentStreamAsync(string, int?, CancellationToken)​

Delegates the retrieval of file contents to a Stream object.

Declaration
Task<Stream> GetFileContentStreamAsync(string path, int? contentSize = -1, CancellationToken cancellationToken = default)
Returns​

Task<System.IO.Stream>

Stream

Parameters​
TypeNameDescription
System.StringpathThe file path. ex: folderA/folderB/filename.txt
System.Nullable<System.Int32>contentSizeThe amount of content in bytes to pull back from the stream. Default to -1 which means that all the content will be retrieved.
System.Threading.CancellationTokencancellationTokenThe cancellation token.

GetFileContentStringAsync(string, int?, CancellationToken)​

Retrieve the file contents to a string object.

Declaration
Task<string> GetFileContentStringAsync(string path, int? contentSize = -1, CancellationToken cancellationToken = default)
Returns​

Task<System.String>

string

Parameters​
TypeNameDescription
System.StringpathThe file path. ex: folderA/folderB/filename.txt
System.Nullable<System.Int32>contentSizeThe amount of content in bytes to pull back from the stream. Default to -1 which means that all the content will be retrieved.
System.Threading.CancellationTokencancellationToken

GetFileContentByteArrayAsync(string, int?, CancellationToken)​

Retrieve the file contents to a byte array.

Declaration
Task<byte[]> GetFileContentByteArrayAsync(string path, int? contentSize = -1, CancellationToken cancellationToken = default)
Returns​

Task<System.Byte[]>

byte[]

Parameters​
TypeNameDescription
System.StringpathThe file path. ex: folderA/folderB/filename.txt
System.Nullable<System.Int32>contentSizeThe amount of content in bytes to pull back from the stream. Default to -1 which means that all the content will be retrieved.
System.Threading.CancellationTokencancellationToken

WriteFileAsync(string, Stream, bool, CancellationToken)​

Writes a file to the Meta FileSystem using a stream object.

Note that this method is subject to runtime checks that may not allow writing a file in certain locations.

Declaration
Task WriteFileAsync(string path, Stream payload, bool overwriteOk = false, CancellationToken cancellationToken = default)
Returns​

System.Threading.Tasks.Task

Parameters​
TypeNameDescription
System.StringpathThe file path to write to. Note that if the file path does not have directories that exist, an error will be thrown.
System.IO.StreampayloadThe file payload to write.
System.BooleanoverwriteOkControls whether or not to overwrite the file if it already exists. Defaults to false. If false, an error will be thrown if the file already exists.
System.Threading.CancellationTokencancellationToken

WriteFileAsync(string, byte[], bool, CancellationToken)​

Writes a file to the Meta FileSystem using a byte array.

Declaration
Task WriteFileAsync(string path, byte[] payload, bool overwriteOk = false, CancellationToken cancellationToken = default)
Remarks​

Note that this method is subject to runtime checks that may not allow writing a file in certain locations.

Returns​

System.Threading.Tasks.Task

Parameters​
TypeNameDescription
System.StringpathThe file path to write to. Note that if the file path does not have directories that exist, an error will be thrown.
System.Byte[]payloadThe file payload to write.
System.BooleanoverwriteOkControls whether or not to overwrite the file if it already exists. Defaults to false. If false, an error will be thrown if the file already exists.
System.Threading.CancellationTokencancellationToken

WriteFileAsync(string, string, bool, CancellationToken)​

Writes a file to the Meta FileSystem using a string payload.

Note that this method is subject to runtime checks that may not allow writing a file in certain locations.

Declaration
Task WriteFileAsync(string path, string payload, bool overwriteOk = false, CancellationToken cancellationToken = default)
Returns​

System.Threading.Tasks.Task

Parameters​
TypeNameDescription
System.StringpathThe file path to write to. Note that if the file path does not have directories that exist, an error will be thrown.
System.StringpayloadThe file payload to write.
System.BooleanoverwriteOkControls whether or not to overwrite the file if it already exists. Defaults to false. If false, an error will be thrown if the file already exists.
System.Threading.CancellationTokencancellationToken

RemoveFileAsync(string, bool, CancellationToken)​

Removes a file from the MetaFileSystem.

Note that this method is subject to runtime checks that may not allow removing a file in certain locations.

Declaration
Task RemoveFileAsync(string path, bool errorWhenDoesNotExist = false, CancellationToken cancellationToken = default)
Returns​

System.Threading.Tasks.Task

Parameters​
TypeName
System.Stringpath
System.BooleanerrorWhenDoesNotExist
System.Threading.CancellationTokencancellationToken
Last updated on by Jeremy Meiss

Was this page helpful?