Skip to main content

Class MetaFileSystemClient

Provides file system operations for the Xperiflow MetaFileSystem service.

Namespace: Workspace.XBR.Xperiflow.MetaFileSystem

Assembly: Xperiflow.dll

Declaration
public class MetaFileSystemClient : IMetaFileSystemClient

Examples​

Basic file operations:

// Initialize the client
var client = new MetaFileSystemClient(sessionInfo, httpClientManager, storageClient, connectionKey);

// Create a directory
await client.MakeDirectoriesAsync("data/processed", existsOk: true);

// Write a file
await client.WriteFileAsync("data/processed/results.json", jsonData, overwriteOk: true);

// Read file content
var content = await client.GetFileContentStringAsync("data/processed/results.json");

// Get file metadata
var metadata = await client.GetFileMetadataAsync("data/processed/results.json");
Console.WriteLine($"File size: {metadata.Size} bytes, created: {metadata.CreatedTime}");

// List directory contents
var directory = await client.GetDirectoryAsync("data/processed", maxDepth: 2);
foreach (var file in directory.Files)
`{
    Console.WriteLine($"Found file: {file.Name}`");
}

Streaming operations for large files:

// Stream large file content
using (var stream = await client.GetFileContentStreamAsync("data/large-dataset.csv"))
{
    // Process stream without loading entire file into memory
    using (var reader = new StreamReader(stream))
    {
        string line;
        while ((line = await reader.ReadLineAsync()) != null)
        {
            // Process each line
        }
    }
}

Remarks​

The Workspace.XBR.Xperiflow.MetaFileSystem.MetaFileSystemClient is the standard concrete implementation of Workspace.XBR.Xperiflow.MetaFileSystem.IMetaFileSystemClient that provides connectivity to the Xperiflow MetaFileSystem service. It serves as the primary interface for file and directory operations within the Xperiflow ecosystem, offering both synchronous and asynchronous operations for data management.

Key Features:

  • Directory Management Create, delete, and query directories with support for recursive operations

  • File Operations Read, write, and delete files with support for various data formats (streams, strings, byte arrays)

  • Metadata Access Retrieve detailed metadata for files and directories including creation times, sizes, and permissions

  • Azure Blob Integration Seamless integration with Azure Blob Storage using SAS tokens for secure access

  • Error Handling Comprehensive error handling with OneStream logging and exception management

Storage Architecture:

The MetaFileSystem operates as an abstraction layer over underlying storage systems, providing a unified interface for file operations regardless of the actual storage backend (local file system, Azure Blob Storage, etc.). This design enables seamless migration between storage providers and consistent behavior across environments.

Thread Safety:

This class is designed to be thread-safe and supports concurrent operations through proper use of async/await patterns and cancellation tokens. Multiple operations can be performed simultaneously without additional synchronization.

Performance Considerations:

The client optimizes performance through streaming operations for large files and efficient metadata caching. Use streaming methods for large data transfers and consider specifying content size hints when known to improve performance.

Implements: Workspace.XBR.Xperiflow.MetaFileSystem.IMetaFileSystemClient

Methods​

DirectoryExistsAsync(string, CancellationToken)​

Checks if a directory exists in the Meta FileSystem at the provided path.

Declaration
public Task<bool> DirectoryExistsAsync(string path, CancellationToken cancellationToken = default)
Returns​

Task<System.Boolean>

Parameters​
TypeNameDescription
System.StringpathThe path to the directory. If the directory path.
System.Threading.CancellationTokencancellationTokenThe cancellation token.

GetDirectoryMetadataAsync(string, bool, CancellationToken)​

Retrieves only the metadata of a directory (no subdirectories or subfiles) from the Meta FileSystem.

Declaration
public Task<IDirectoryMetadata?> GetDirectoryMetadataAsync(string path, bool errorWhenDoesNotExist = true, CancellationToken cancellationToken = default)
Returns​

Task< Workspace.XBR.Xperiflow.MetaFileSystem.IDirectoryMetadata >

DirectoryMetadata

Parameters​
TypeNameDescription
System.StringpathThe path to the folder. If the folder path does not exist, an error will be thrown.
System.BooleanerrorWhenDoesNotExistWhether or not to throw an error when the directory does not exist. Defaults to true.
System.Threading.CancellationTokencancellationTokenThe cancellation token.

GetDirectoryAsync(string, int?, bool, bool, CancellationToken)​

Retrieves a directory and its subcontents (other file and folders metdata) from the Meta FileSystem.

Declaration
public Task<IDirectoryContent?> GetDirectoryAsync(string path, int? maxDepth = 1, bool includeDirectories = true, bool errorWhenDoesNotExist = true, CancellationToken cancellationToken = default)
Returns​

Task< Workspace.XBR.Xperiflow.MetaFileSystem.IDirectoryContent >

DirectoryContent: This holds accessors to files and directory subcontents.

Parameters​
TypeNameDescription
System.StringpathThe path to the folder. If the folder path does not exist, an error will be thrown.
System.Nullable<System.Int32>maxDepthThe max depth of subcontents to grab. Defaults to 1 level deep of subcontents. Set to -1 or null to grab all subcontents.
System.BooleanincludeDirectoriesWhether or not to include subfolders in the subcontents. Defaults to true.
System.BooleanerrorWhenDoesNotExistWhether or not to throw an error when the directory does not exist. Defaults to true.
System.Threading.CancellationTokencancellationTokenThe cancellation token.

MakeDirectoriesAsync(string, bool, CancellationToken)​

Makes one or more directories in the Meta FileSystem at the provided path.

Note that this method is subject to runtime checks that may not allow the creation of directories in certain locations.

Declaration
public Task MakeDirectoriesAsync(string path, bool existsOk = false, CancellationToken cancellationToken = default)
Returns​

System.Threading.Tasks.Task

Parameters​
TypeNameDescription
System.StringpathThe directory path to create.
System.BooleanexistsOkControls whether or not to throw an error if the directory already exists. Defaults to false.
System.Threading.CancellationTokencancellationToken

RemoveDirectoryAsync(string, bool, bool, CancellationToken)​

Removes a directory from the MetaFileSystem. This method is not recursive in that it will only remove the last directory in the path. It also expects that the directory is empty.

Note that this method is subject to runtime checks that may not allow removing a directory in certain locations.

Declaration
public Task RemoveDirectoryAsync(string path, bool recursive = false, bool errorWhenDoesNotExist = true, CancellationToken cancellationToken = default)
Returns​

System.Threading.Tasks.Task

Parameters​
TypeNameDescription
System.StringpathThe directory path to remove.
System.BooleanrecursiveWhether or not to attempt to recursively remove the directory and its all of its subcontents. Defaults to false.
System.BooleanerrorWhenDoesNotExistWhether or not to throw an error when the directory in the path does not exist. Defaults to true.
System.Threading.CancellationTokencancellationToken

FileExistsAsync(string, CancellationToken)​

Checks if a file exists in the Meta FileSystem at the provided path.

Declaration
public Task<bool> FileExistsAsync(string path, CancellationToken cancellationToken = default)
Returns​

Task<System.Boolean>

bool: true means the file does exist.

Parameters​
TypeNameDescription
System.StringpathThe file path. Ex: folderA/folderB/filename.txt
System.Threading.CancellationTokencancellationTokenThe cancellation token.

GetFileMetadataAsync(string, bool, CancellationToken)​

Retrieves the metadata of a file (not the raw content of the file) from the Meta FileSystem.

Declaration
public Task<IFileMetadata?> GetFileMetadataAsync(string path, bool errorWhenDoesNotExist = true, CancellationToken cancellationToken = default)
Returns​

Task< Workspace.XBR.Xperiflow.MetaFileSystem.IFileMetadata >

FileMetadata

Parameters​
TypeNameDescription
System.StringpathThe file path. ex: folderA/folderB/filename.txt
System.BooleanerrorWhenDoesNotExistWhether or not to throw an error when the file does not exist. Defaults to true.
System.Threading.CancellationTokencancellationTokenThe cancellation token.

GetFileContentStreamAsync(string, int?, CancellationToken)​

Delegates the retrieval of file contents to a Stream object.

Declaration
public Task<Stream> GetFileContentStreamAsync(string path, int? contentSize = -1, CancellationToken cancellationToken = default)
Returns​

Task<System.IO.Stream>

Stream

Parameters​
TypeNameDescription
System.StringpathThe file path. ex: folderA/folderB/filename.txt
System.Nullable<System.Int32>contentSizeThe amount of content in bytes to pull back from the stream. Default to -1 which means that all the content will be retrieved.
System.Threading.CancellationTokencancellationTokenThe cancellation token.

GetFileContentStringAsync(string, int?, CancellationToken)​

Retrieve the file contents to a string object.

Declaration
public Task<string> GetFileContentStringAsync(string path, int? contentSize = -1, CancellationToken cancellationToken = default)
Returns​

Task<System.String>

string

Parameters​
TypeNameDescription
System.StringpathThe file path. ex: folderA/folderB/filename.txt
System.Nullable<System.Int32>contentSizeThe amount of content in bytes to pull back from the stream. Default to -1 which means that all the content will be retrieved.
System.Threading.CancellationTokencancellationToken

GetFileContentByteArrayAsync(string, int?, CancellationToken)​

Retrieve the file contents to a byte array.

Declaration
public Task<byte[]> GetFileContentByteArrayAsync(string path, int? contentSize = -1, CancellationToken cancellationToken = default)
Returns​

Task<System.Byte[]>

byte[]

Parameters​
TypeNameDescription
System.StringpathThe file path. ex: folderA/folderB/filename.txt
System.Nullable<System.Int32>contentSizeThe amount of content in bytes to pull back from the stream. Default to -1 which means that all the content will be retrieved.
System.Threading.CancellationTokencancellationToken

WriteFileAsync(string, Stream, bool, CancellationToken)​

Writes a file to the Meta FileSystem using a stream object.

Note that this method is subject to runtime checks that may not allow writing a file in certain locations.

Declaration
public Task WriteFileAsync(string path, Stream payload, bool overwriteOk = false, CancellationToken cancellationToken = default)
Returns​

System.Threading.Tasks.Task

Parameters​
TypeNameDescription
System.StringpathThe file path to write to. Note that if the file path does not have directories that exist, an error will be thrown.
System.IO.StreampayloadThe file payload to write.
System.BooleanoverwriteOkControls whether or not to overwrite the file if it already exists. Defaults to false. If false, an error will be thrown if the file already exists.
System.Threading.CancellationTokencancellationToken

WriteFileAsync(string, byte[], bool, CancellationToken)​

Writes a file to the Meta FileSystem using a byte array.

Declaration
public Task WriteFileAsync(string path, byte[] payload, bool overwriteOk = false, CancellationToken cancellationToken = default)
Remarks​

Note that this method is subject to runtime checks that may not allow writing a file in certain locations.

Returns​

System.Threading.Tasks.Task

Parameters​
TypeNameDescription
System.StringpathThe file path to write to. Note that if the file path does not have directories that exist, an error will be thrown.
System.Byte[]payloadThe file payload to write.
System.BooleanoverwriteOkControls whether or not to overwrite the file if it already exists. Defaults to false. If false, an error will be thrown if the file already exists.
System.Threading.CancellationTokencancellationToken

WriteFileAsync(string, string, bool, CancellationToken)​

Writes a file to the Meta FileSystem using a string payload.

Note that this method is subject to runtime checks that may not allow writing a file in certain locations.

Declaration
public Task WriteFileAsync(string path, string payload, bool overwriteOk = false, CancellationToken cancellationToken = default)
Returns​

System.Threading.Tasks.Task

Parameters​
TypeNameDescription
System.StringpathThe file path to write to. Note that if the file path does not have directories that exist, an error will be thrown.
System.StringpayloadThe file payload to write.
System.BooleanoverwriteOkControls whether or not to overwrite the file if it already exists. Defaults to false. If false, an error will be thrown if the file already exists.
System.Threading.CancellationTokencancellationToken

RemoveFileAsync(string, bool, CancellationToken)​

Removes a file from the MetaFileSystem.

Note that this method is subject to runtime checks that may not allow removing a file in certain locations.

Declaration
public Task RemoveFileAsync(string path, bool errorWhenDoesNotExist = true, CancellationToken cancellationToken = default)
Returns​

System.Threading.Tasks.Task

Parameters​
TypeName
System.Stringpath
System.BooleanerrorWhenDoesNotExist
System.Threading.CancellationTokencancellationToken

Implements​

Inherited Members​

  • System.Object.Equals(System.Object)
  • System.Object.Equals(System.Object,System.Object)
  • System.Object.GetHashCode
  • System.Object.GetType
  • System.Object.MemberwiseClone
  • System.Object.ReferenceEquals(System.Object,System.Object)
  • System.Object.ToString
Last updated on by Jeremy Meiss

Was this page helpful?