c&~dZddlmZddlZddlZddlZdZddgZeddfdZ Gd de Z e d k(r d Z e yy) z``jsonutils`` aims to provide various helpers for working with JSON. Currently it focuses on providing a reliable and intuitive means of working with `JSON Lines`_-formatted files. .. _JSON Lines: http://jsonlines.org/ )print_functionN JSONLIteratorreverse_iter_linesTc#K |xs |j}d}|} |j}d\}}}|r |j dt j|}|j} d| krt|| } | | z} |j | t j|j| } | |z}|j} t| dks| d|k(rs|dd|k(r|r|n|| dddD]} |r| j|n| | d}d| kr|r|r|j|n|yy#t$rd}Y(wxYw#ttjf$rY5wxYww)a;Returns an iterator over the lines from a file object, in reverse order, i.e., last line first, first line last. Uses the :meth:`file.seek` method of file objects, and is tested compatible with :class:`file` objects, as well as :class:`StringIO.StringIO`. Args: file_obj (file): An open file object. Note that ``reverse_iter_lines`` mutably reads from the file and other functions should not mutably interact with the file object after being passed. Files can be opened in bytes or text mode. blocksize (int): The block size to pass to :meth:`file.read()`. Warning: keep this a fairly large multiple of 2, defaults to 4096. preseek (bool): Tells the function whether or not to automatically seek to the end of the file. Defaults to ``True``. ``preseek=False`` is useful in cases when the file cursor is already in position, either at the end of the file or in the middle for relative reverse line generation. zutf-8N) r)encodingAttributeErrordetachioUnsupportedOperationseekosSEEK_ENDtellminSEEK_SETread splitlineslendecode)file_obj blocksizepreseekr orig_obj empty_bytes newline_bytes empty_textbuffcur_pos read_sizecurlineslines 1lib/python3.12/site-packages/boltons/jsonutils.pyrr6s00x00 H ??$.=*K  a% DmmoG g+ 7+ 9 gr{{+mmI&Tz! u:>U1X4  9 % (*k 9%1R%L >D+3$++h' = >Qx g+ '/dkk(#T9 E  B33 4   sPE"D0E"EC+E"E"0 D?;E">D??E"EE"EE"cFeZdZdZ d dZedZdZdZdZ dZ e Z y) raThe ``JSONLIterator`` is used to iterate over JSON-encoded objects stored in the `JSON Lines format`_ (one object per line). Most notably it has the ability to efficiently read from the bottom of files, making it very effective for reading in simple append-only JSONL use cases. It also has the ability to start from anywhere in the file and ignore corrupted lines. Args: file_obj (file): An open file object. ignore_errors (bool): Whether to skip over lines that raise an error on deserialization (:func:`json.loads`). reverse (bool): Controls the direction of the iteration. Defaults to ``False``. If set to ``True`` and *rel_seek* is unset, seeks to the end of the file before iteration begins. rel_seek (float): Used to preseek the start position of iteration. Set to 0.0 for the start of the file, 1.0 for the end, and anything in between. .. _JSON Lines format: http://jsonlines.org/ Nctt||_||_||_||r(d}n%d|cxkrdksnt d|z|dkrd|z }||_d|_||j|jr(t|j|j d|_ yt|j|_ y)N?gz8'rel_seek' expected a float between -1.0 and 1.0, not %rrrF)rr) bool_reverse _file_obj ignore_errors ValueError _rel_seek _blocksize_init_rel_seekr _line_iteriter)selfrr0reverserel_seeks r)__init__zJSONLIterator.__init__sW  !*  &3&57?@A A \X~H!      ! ==0;???9>@DO#4>>2DOrc6|jjS)zBA property representing where in the file the iterator is reading.)r/rr7s r) cur_byte_poszJSONLIterator.cur_byte_poss~~""$$rc|j|j}}d\}}|j}d|vr|j|}||z }d|vr |j d|z|z }|j ||zy#t $rwxYw)z6Aligns the file object's position to the next newline.)r r N)r/r3rrindexr1r)r7fobsizer& total_readr$newline_offsets r)_align_to_newlinezJSONLIterator._align_to_newlinesNNDOOEZ'')#o''%.C % J#o  YYt_z9EAN .()   s A:: Bc|j|j}}|dk(r!|jdtjy|jdtj |j }|dk(r||_yt||z}|j|tj|j|j |_y)zCSets the file object's position to the relative location set above.grr,N) r2r/rrrrr_cur_posintrE)r7rsrAsizetargets r)r4zJSONLIterator._init_rel_seeksB 9 GGAr{{ # GGAr{{ #779DSy $ TBY ,&&( "  rc|S)Nr<s r)__iter__zJSONLIterator.__iter__s rc t|jj}|s' tj|}|S#t $r|j sYWwxYw)zYields one :class:`dict` loaded with :func:`json.loads`, advancing the file object by one line. Raises :exc:`StopIteration` upon reaching the end of the file (or beginning, if ``reverse`` was set to ``True``. )nextr5lstripjsonloads Exceptionr0)r7r(objs r)rPzJSONLIterator.nexts^ (//1D jj& J  )) sAAA)FFN) __name__ __module__ __qualname____doc__r:propertyr=rEr4rNrP__next__rMrr)rr~sA.?C32%% ** "Hr__main__c ddl}d|jvsd|jvr tdyd}d|jvsd|jvrd}d \}}|jd d}|D]}|d vr|d z }t|d 5}t |}d} t ||d z }|d z }|rH|rF|dzdk(r>|jjd|dzr|jjd|za|rtd|ztd|zy#t $r-td |d zd|jd|Ydddyt$rYnwxYw ddd#1swYxYw)Nr-h--helpz7loads one or more JSON Line files for basic validation.F-v --verboseT)rr)r^r_r`rarbzerror reading object #z around byte z in d.i'z%s zfiles checked: %szobjects loaded: %s) sysargvprintopenrrPr1r= StopIterationstdoutwrite) rfverbose file_count obj_count filenamesfilenameriterator cur_obj_counts r)_mainrts 388 x3883 K L  388 {chh6G $ IHHQRL ! AH>> !OJh% A(2 ! XNI!Q&M9SA1E ((-$u,JJ,,Vi-?@ A,  % 2 3 &2 3&!.!2H4I4I8UV A A) A A As=<E D AE *E 6EE  E E  EE" ) rY __future__rrrrRDEFAULT_BLOCKSIZE__all__robjectrrVrtrMrr)rysoB&  0 1,=dUY=:BgFgT z#J GMr