lcS.dZddlZGddeZdS)zEImplements ByteBuffer class for amortizing network transfer overhead.NcPeZdZdZejfdZdZd dZd dZ dZ d dZ d Z d S) ByteBufferaImplements a byte buffer that allows callers to read data with minimal copying, and has a fast __len__ method. The buffer is parametrized by its chunk_size, which is the number of bytes that it will read in from the supplied reader or iterable when the buffer is being filled. As primary use case for this buffer is to amortize the overhead costs of transferring data over the network (rather than capping memory consumption), it leads to more predictable performance to always read the same amount of bytes each time the buffer is filled, hence the chunk_size parameter instead of some fixed capacity. The bytes are stored in a bytestring, and previously-read bytes are freed when the buffer is next filled (by slicing the bytestring into a smaller copy). Example ------- Note that while this example works in both Python 2 and 3, the doctest only passes in Python 3 due to the bytestring literals in the expected values. >>> buf = ByteBuffer(chunk_size = 8) >>> message_bytes = iter([b'Hello, W', b'orld!']) >>> buf.fill(message_bytes) 8 >>> len(buf) # only chunk_size bytes are filled 8 >>> buf.peek() b'Hello, W' >>> len(buf) # peek() does not change read position 8 >>> buf.read(6) b'Hello,' >>> len(buf) # read() does change read position 2 >>> buf.fill(message_bytes) 5 >>> buf.read() b' World!' >>> len(buf) 0 c<||_|dS)aJCreate a ByteBuffer instance that reads chunk_size bytes when filled. Note that the buffer has no maximum size. Parameters ----------- chunk_size: int, optional The the number of bytes that will be read from the supplied reader or iterable when filling the buffer. N) _chunk_sizeempty)self chunk_sizes 5lib/python3.11/site-packages/smart_open/bytebuffer.py__init__zByteBuffer.__init__8s& c:t|j|jz S)z9Return the number of unread bytes in the buffer as an intlen_bytes_posrs r __len__zByteBuffer.__len__Es4;$)++r cj||}|xjt|z c_|S)aZRead bytes from the buffer and advance the read position. Returns the bytes in a bytestring. Parameters ---------- size: int, optional Maximum number of bytes to read. If negative or not supplied, read all unread bytes in the buffer. Returns ------- bytes )peekrrrsizeparts r readzByteBuffer.readIs.yy SYY  r c|dks|t|krt|}|j|j|j|z}|S)acGet bytes from the buffer without advancing the read position. Returns the bytes in a bytestring. Parameters ---------- size: int, optional Maximum number of bytes to return. If negative or not supplied, return all unread bytes in the buffer. Returns ------- bytes rrrs r rzByteBuffer.peek[sK !8 tc$ii' t99D{49TYt^34 r c"d|_d|_dS)z Remove all bytes from the bufferr rN)rrrs r rzByteBuffer.emptyos  r cl|dkr|n|j}t||j}|jdkr |j|jd|_d|_t |dr||}n!d}|D]}||z }t ||krn|xj|z c_t |S)a=Fill the buffer with bytes from source until one of these conditions is met: * size bytes have been read from source (if size >= 0); * chunk_size bytes have been read from source; * no more bytes can be read from source; Returns the number of new bytes added to the buffer. Note: all previously-read bytes in the buffer are removed. Parameters ---------- source: a file-like object, or iterable/list that contains bytes The source of bytes to fill the buffer with. If this argument has the `read` attribute, it's assumed to be a file-like object and `read` is called to get the bytes; otherwise it's assumed to be an iterable or list that contains bytes, and a for loop is used to get the bytes. size: int, optional The number of bytes to try to read from source. If not supplied, negative, or larger than the buffer's chunk_size, then chunk_size bytes are read. Note that if source is an iterable or list, then it's possible that more than size bytes will be read if iterating over source produces more than one byte at a time. Returns ------- int, the number of new bytes added to the buffer. rNrr )rminrrhasattrrr)rsourcer new_bytes more_bytess r fillzByteBuffer.fillts8qy6ttd&64)** 9> +dijj1DKDI 66 " "  D))III$   Z' y>>T)E y 9~~r c|j||j}|dkrt|}n ||jz dz}||S)a#Read a line from this buffer efficiently. A line is a contiguous sequence of bytes that ends with either: 1. The ``terminator`` character 2. The end of the buffer itself :param byte terminator: The line terminator character. :rtype: bytes r)rfindrrr)r terminatorindexrs r readlinezByteBuffer.readlinesW   TY77 B; )t99DD49$q(Dyyr N)r) __name__ __module__ __qualname____doc__ioDEFAULT_BUFFER_SIZEr rrrrr#r)r r rr s((T#%"8    ,,,$( ----^r r)r-r.objectrr0r r r2sULK ggggggggggr