Class ParquetUtil

Provides utility methods for reading and writing Parquet data to and from DataFrame and DataTable objects.

Namespace: Workspace.XBR.Xperiflow.Etl.Tabular.Utilities

Assembly: Xperiflow.dll

Declaration

public static class ParquetUtil

Methods

WriteDataFrameToParquetAsync(DataFrame, Stream, CancellationToken)

Writes the provided DataFrame to the provided Stream in Parquet format.

Declaration

public static Task WriteDataFrameToParquetAsync(DataFrame df, Stream outputStream, CancellationToken cancellationToken = default)

Returns

System.Threading.Tasks.Task

A Task representing the asynchronous operation

Parameters

Type	Name	Description
Microsoft.Data.Analysis.DataFrame	df	The DataFrame to write to the output stream
System.IO.Stream	outputStream	The Stream to write the Parquet data to
System.Threading.CancellationToken	cancellationToken	The System.Threading.CancellationToken to cancel the operation

Exceptions

System.InvalidOperationException

ReadParquetToDataFrameAsync(Stream, CancellationToken)

Reads the provided Stream containing data in a Parquet format to a DataFrame asynchronously.

Declaration

public static Task<DataFrame> ReadParquetToDataFrameAsync(Stream inputStream, CancellationToken cancellationToken = default)

Returns

Task<Microsoft.Data.Analysis.DataFrame>

A Task representing the asynchronous operation. Access the resulting DataFrame with the Task.Result property

Parameters

Type	Name	Description
System.IO.Stream	inputStream	The Stream to read the Parquet data from
System.Threading.CancellationToken	cancellationToken	The System.Threading.CancellationToken to cancel the operation

WriteDataTableToParquetAsync(DataTable, Stream, bool, CancellationToken)

Writes the data from the provided DataTable to the Stream in a Parquet format.

Declaration

public static Task WriteDataTableToParquetAsync(DataTable dt, Stream fs, bool useCustomMetadata = true, CancellationToken cancellationToken = default)

Returns

System.Threading.Tasks.Task

A Task representing the asynchronous operation

Parameters

Type	Name	Description
System.Data.DataTable	dt	The DataTable to write to the Stream
System.IO.Stream	fs	The Stream to write the Parquet data to
System.Boolean	useCustomMetadata	A bool value to determine if custom metadata should be used
System.Threading.CancellationToken	cancellationToken	A System.Threading.CancellationToken to cancel the asynchronous operation

Exceptions

System.ArgumentNullException

System.ArgumentException

ReadParquetToDataTableAsync(Stream, bool)

Reads the data from the provided Stream in a Parquet format to a DataTable asynchronously.

Declaration

public static Task<DataTable> ReadParquetToDataTableAsync(Stream fs, bool useCustomMetadata = true)

Returns

Task<System.Data.DataTable>

A Task representing the asynchronous operation. Access the DataTable from the Task.Result property

Parameters

Type	Name	Description
System.IO.Stream	fs	The Stream to read the Parquet data from
System.Boolean	useCustomMetadata	A bool value to determine if custom metadata should be used

Inherited Members

• System.Object.Equals(System.Object)
• System.Object.Equals(System.Object,System.Object)
• System.Object.GetHashCode
• System.Object.GetType
• System.Object.MemberwiseClone
• System.Object.ReferenceEquals(System.Object,System.Object)
• System.Object.ToString