Lg{'SSKrSSKrSSKJrJrJrJrJrJ r J r J r J r J r JrJrJrJrJrJr SSKJr "SS\R*5rSr"SS\R05r"SS \R45r"S S \R85rS rS rSSS.Sjr SRC\5\ l"SSS.Sjr#SSS.Sjr$SRC\5\$l"SSSS.Sjjr%SSS.Sjr&SS.Sjr'g)N)IpcReadOptionsIpcWriteOptions ReadStats WriteStatsMessage MessageReaderRecordBatchReader_ReadPandasMixinMetadataVersion read_messageread_record_batch read_schema read_tensor write_tensorget_record_batch_sizeget_tensor_sizec(\rSrSrSrSSS.SjrSrg)RecordBatchStreamReader"a Reader for the Arrow streaming binary format. Parameters ---------- source : bytes/buffer-like, pyarrow.NativeFile, or file-like Python object Either an in-memory buffer, or a readable file object. If you want to use memory map use MemoryMappedFile as source. options : pyarrow.ipc.IpcReadOptions Options for IPC deserialization. If None, default values will be used. memory_pool : MemoryPool, default None If None, default memory pool is used. Noptions memory_poolc:[U5nURXUS9 g)Nr _ensure_default_ipc_read_options_open)selfsourcerrs +lib/python3.12/site-packages/pyarrow/ipc.py__init__ RecordBatchStreamReader.__init__2s27; 6 D__name__ __module__ __qualname____firstlineno____doc__r __static_attributes__r#r"rrr"s +/DEEr"raNParameters ---------- sink : str, pyarrow.NativeFile, or file-like Python object Either a file path, or a writable file object. schema : pyarrow.Schema The Arrow schema for data to be written to the file. use_legacy_format : bool, default None Deprecated in favor of setting options. Cannot be provided with options. If None, False will be used unless this default is overridden by setting the environment variable ARROW_PRE_0_15_IPC_FORMAT=1 options : pyarrow.ipc.IpcWriteOptions Options for IPC serialization. If None, default values will be used: the legacy format will not be used unless overridden by setting the environment variable ARROW_PRE_0_15_IPC_FORMAT=1, and the V5 metadata version will be used unless overridden by setting the environment variable ARROW_PRE_1_0_METADATA_VERSION=1.cF\rSrSrSR \5rSSS.SjrSrg)RecordBatchStreamWriterNz0Writer for the Arrow streaming binary format {}Nuse_legacy_formatrc:[X45nURXUS9 gNr_get_legacy_format_defaultrrsinkschemar/rs rr RecordBatchStreamWriter.__init__S,->H 4 1r"r# r%r&r'r(format_ipc_writer_class_docr)r r*r#r"rr,r,Ns( f "# ;?22r"r,c,\rSrSrSrSSSS.SjjrSrg)RecordBatchFileReaderXa Class for reading Arrow record batch data from the Arrow binary file format Parameters ---------- source : bytes/buffer-like, pyarrow.NativeFile, or file-like Python object Either an in-memory buffer, or a readable file object. If you want to use memory map use MemoryMappedFile as source. footer_offset : int, default None If the file is embedded in some larger file, this is the byte offset to the very end of the file data options : pyarrow.ipc.IpcReadOptions Options for IPC serialization. If None, default values will be used. memory_pool : MemoryPool, default None If None, default memory pool is used. Nrc:[U5nURXX4S9 g)N footer_offsetrrr)rrrBrrs rr RecordBatchFileReader.__init__ks"27; 6"  =r"r#Nr$r#r"rr>r>Xs$=d!==r"r>cF\rSrSrSR \5rSSS.SjrSrg)RecordBatchFileWriterrz1Writer to create the Arrow binary file format {}Nr.c:[X45nURXUS9 gr1r3r5s rr RecordBatchFileWriter.__init__xr9r"r#r:r#r"rrFrFrs( f "# ;?22r"rFcUbUb [S5eU(a:[U[5(d#[SR [ U555eU$[ RnUc2[[[RRSS555n[[[RRSS555(a[ Rn[UUS9$)Nz8Can provide at most one of options and use_legacy_formatz expected IpcWriteOptions, got {}ARROW_PRE_0_15_IPC_FORMAT0ARROW_PRE_1_0_METADATA_VERSION)r/metadata_version) ValueError isinstancer TypeErrorr;typer V5boolintosenvirongetV4)r/rrNs rr4r4}s$)< FH H '?33>#VDM24 4&)) RZZ^^$?EF G  C ?E FGG*-- ->,< >>r"cU(a8[U[5(d#[SR[ U555eU=(d [5$)Nzexpected IpcReadOptions, got {})rPrrQr;rRr2s rrrs@z'>:: - 4 4T'] C    &n&&r"r.c[XUUS9$Nr.)r,r6r7r/rs r new_streamr^s "45F+2 44r"zCreate an Arrow columnar IPC stream writer instance {} Returns ------- writer : RecordBatchStreamWriter A writer for the given sink rc[XUS9$)a Create reader for Arrow streaming format. Parameters ---------- source : bytes/buffer-like, pyarrow.NativeFile, or file-like Python object Either an in-memory buffer, or a readable file object. options : pyarrow.ipc.IpcReadOptions Options for IPC serialization. If None, default values will be used. memory_pool : MemoryPool, default None If None, default memory pool is used. Returns ------- reader : RecordBatchStreamReader A reader for the given source r)r)rrrs r open_streamr`s& #6/: <)rrBrrs r open_filerds, ! 22r"nthreadspreserve_indexc<[RRXUS9n[R"5n[R"XCR 5nUR U5 SSS5 UR5$!,(df  UR5$=f)a` Serialize a pandas DataFrame into a buffer protocol compatible object. Parameters ---------- df : pandas.DataFrame nthreads : int, default None Number of threads to use for conversion to Arrow, default all CPUs. preserve_index : bool, default None The default of None will store the index as a column, except for RangeIndex which is stored as metadata only. If True, always preserve the pandas index data as a column. If False, no index information is saved and the result will have a default RangeIndex. Returns ------- buf : buffer An object compatible with the buffer protocol. reN)pa RecordBatch from_pandasBufferOutputStreamr,r7 write_batchgetvalue)dfrfrgbatchr6writers rserialize_pandasrrsy( NN & &r6D ' FE "D # #D,, 765! 8 ==? 8 7 ==?s A>> BT use_threadsc[R"U5n[R"U5nUR5nSSS5 WR US9$!,(df  N=f)aZDeserialize a buffer protocol compatible object into a pandas DataFrame. Parameters ---------- buf : buffer An object compatible with the buffer protocol. use_threads : bool, default True Whether to parallelize the conversion using multiple threads. Returns ------- df : pandas.DataFrame The buffer deserialized as pandas DataFrame Nrs)ri BufferReaderrread_all to_pandas)bufrt buffer_readerreadertables rdeserialize_pandasr} sMOOC(M # #M 2f! 3 ??{? 33 3 2s A A#rD)(rVpyarrowri pyarrow.librrrrrrr r r r r rrrrrlib_RecordBatchStreamReaderrr<_RecordBatchStreamWriterr,_RecordBatchFileReaderr>_RecordBatchFileWriterrFr4rr^r;r)r`rbrdrrr}r#r"rrs( AAAAAEc::E*).2c::2=C66=42C662>('374  F ! $(T<.15d2  F ! 2Tt26&*$8,04r"