&h~`dZddlZddlZddlZddlZddlmZddlmZ ddl Z ejd dZ dZ GddeZGd d ej jZGd d eZy#e $rdZ YUwxYw)zCode for more fancy file handles. Bio.File defines private classes used in Bio.SeqIO and Bio.SearchIO for indexing files. These are not intended for direct use. N)ABC)abstractmethodc+K t||fi|5}|dddy#1swYyxYw#t$r|YywxYww)a?Context manager to ensure we are using a handle. Context manager for arguments that can be passed to SeqIO and AlignIO read, write, and parse methods: either file objects or path-like objects (strings, pathlib.Path instances, or more generally, anything that can be handled by the builtin 'open' function). When given a path-like object, returns an open file handle to that path, with provided mode, which will be closed when the manager exits. All other inputs are returned, and are *not* closed. Arguments: - handleish - Either a file handle or path-like object (anything which can be passed to the builtin 'open' function, such as str, bytes, pathlib.Path, and os.DirEntry objects) - mode - Mode to open handleish (used only if handleish is a string) - kwargs - Further arguments to pass to open(...) Examples -------- >>> from Bio import File >>> import os >>> with File.as_handle('seqs.fasta', 'w') as fp: ... fp.write('>test\nACGT') ... 10 >>> fp.closed True >>> handle = open('seqs.fasta', 'w') >>> with File.as_handle(handle) as fp: ... fp.write('>test\nACGT') ... 10 >>> fp.closed False >>> fp.close() >>> os.remove("seqs.fasta") # tidy up N)open TypeError) handleishmodekwargsfps W/mounts/lovelace/software/anaconda3/envs/py312/lib/python3.12/site-packages/Bio/File.py as_handler sIV )T ,V , H    s4> ++>(+>+ ;>;>ct|d}|jd}|jd|dk(rddlm} |j d|S|S#t $r0}dt|vsJ|jt d d d }~wwxYw) zOpen a file in binary mode, spot if it is BGZF format etc (PRIVATE). This functionality is used by the Bio.SeqIO and Bio.SearchIO index and index_db functions. If the file is gzipped but not BGZF, a specific ValueError is raised. rbrs)bgzf)r fileobjBGZFz[Gzipped files are not suitable for indexing, please use BGZF (blocked gzip format) instead.N) rreadseekr BgzfReader ValueErrorstrclose)filenamehandlemagicres r _open_for_random_accessr Ns(D !F KKNE KKN  ??f?= = M SV# # LLNA   sA B +BB c6eZdZdZedZedZdZy)_IndexedSeqFileProxya<Abstract base class for file format specific random access (PRIVATE). This is subclasses in both Bio.SeqIO for indexing as SeqRecord objects, and in Bio.SearchIO for indexing QueryResult objects. Subclasses for each file format should define '__iter__', 'get' and optionally 'get_raw' methods. ct)zReturn (identifier, offset, length in bytes) tuples. The length can be zero where it is not implemented or not possible for a particular file format. NotImplementedErrorselfs r __iter__z_IndexedSeqFileProxy.__iter__{s "!ct)z$Return parsed object for this entry.r$r'offsets r getz_IndexedSeqFileProxy.gets "!r)ctd)zReturn the raw record from the file as a bytes string (if implemented). If the key is not found, a KeyError exception is raised. This may not have been implemented for all file formats. z#Not available for this file format.r$r+s r get_rawz_IndexedSeqFileProxy.get_raws""GHHr)N)__name__ __module__ __qualname____doc__rr(r-r/r)r r"r"qs5"""" Ir)r"c@eZdZdZdZdZdZdZdZdZ dZ d Z y ) _IndexedSeqFileDicta5Read only dictionary interface to a sequential record file. This code is used in both Bio.SeqIO for indexing as SeqRecord objects, and in Bio.SearchIO for indexing QueryResult objects. Keeps the keys and associated file offsets in memory, reads the file to access entries as objects parsing them on demand. This approach is memory limited, but will work even with millions of records. Note duplicate keys are not allowed. If this happens, a ValueError exception is raised. As used in Bio.SeqIO, by default the SeqRecord's id string is used as the dictionary key. In Bio.SearchIO, the query's id string is used. This can be changed by supplying an optional key_function, a callback function which will be given the record id and must return the desired key. For example, this allows you to parse NCBI style FASTA identifiers, and extract the GI number to use as the dictionary key. Note that this dictionary is essentially read only. You cannot add or change values, pop values, nor clear the dictionary. c ||_|_||_||_d|_r fd|D}n|}i}|D]B\}}} ||vr3|jj j td|d|||<D||_y)Initialize the class.)NNc3<K|]\}}}|||fywNr4).0keyr,length key_functions r z/_IndexedSeqFileDict.__init__..s+)S&&c"FF3szDuplicate key ''N) _proxy _key_function_repr _obj_repr_cached_prev_record_handlerr_offsets) r'random_access_proxyr>reprobj_repr offset_iteroffsetsr<r,r=s ` r __init__z_IndexedSeqFileDict.__init__s* ) !#/ -@K .K#. & Cg~ ##))+ ?3%q!9::%  &  r)c|jS)z2Return a string representation of the File object.rCr&s r __repr__z_IndexedSeqFileDict.__repr__s zzr)cb|r-dt|jdd|jdSy)z2Create a string representation of the File object.{rz : z (...), ...}z{})listkeysrDr&s r __str__z_IndexedSeqFileDict.__str__s5 TYY[)!,/s4>>2B,O Or)c,t|jS)zReturn the number of records.)lenrGr&s r __len__z_IndexedSeqFileDict.__len__s4==!!r)c,t|jS)Iterate over the keys.)iterrGr&s r r(z_IndexedSeqFileDict.__iter__sDMM""r)c@||jdk(r|jdS|jj|j|}|jr|j |j }n |j }||k7rt d|d|d||f|_|S)aReturn record for the specified key. As an optimization when repeatedly asked to look up the same record, the key and record are cached so that if the *same* record is requested next time, it can be returned without going to disk. rrKey did not match ( vs ))rErAr-rGrBidr)r'r<recordkey2s r __getitem__z_IndexedSeqFileDict.__getitem__s $**1- -++A. .s!34   %%fii0D99D $;23%tD6CD D$'=  r)cR|jj|j|S)Return the raw record from the file as a bytes string. If the key is not found, a KeyError exception is raised. )rAr/rGr'r<s r r/z_IndexedSeqFileDict.get_raws" {{""4==#566r)cL|jjjy)a?Close the file handle being used to read the data. Once called, further use of the index won't work. The sole purpose of this method is to allow explicit handle closure - for example if you wish to delete the file, on Windows you must first close all open handles to that file. N)rArFrr&s r rz_IndexedSeqFileDict.closes !!#r)N) r0r1r2r3rMrPrUrXr(rcr/rr4r)r r6r6s00 B"#(7$r)r6cPeZdZdZ d dZdZdZdZdZdZ dZ d Z d Z d Z y )_SQLiteManySeqFilesDictaKRead only dictionary interface to many sequential record files. This code is used in both Bio.SeqIO for indexing as SeqRecord objects, and in Bio.SearchIO for indexing QueryResult objects. Keeps the keys, file-numbers and offsets in an SQLite database. To access a record by key, reads from the offset in the appropriate file and then parses the record into an object. There are OS limits on the number of files that can be open at once, so a pool are kept. If a record is required from a closed file, then one of the open handles is closed first. ctddlm}|d| t|}||_||_||_||_||_||_ ||_ i|_ tjjtjj||_tjj#|r|j%y|j'y)r8Nr)MissingPythonDependencyErrorz.Python was compiled without the sqlite3 module)sqlite3BiorkrS_index_filename _filenames_formatrB_proxy_factoryrC _max_open_proxiesospathabspathdirname_relative_pathisfile _load_index _build_index) r'index_filename filenames proxy_factoryfmtr>rImax_openrks r rMz _SQLiteManySeqFilesDict.__init__s ? 8.@   YI .# )+ ! !ggoobggoon.MN 77>>. )        r)c  |j}|j}|j}|j}|j}t j j|d}||_ |jddj\}t||_ |jdk(r|jtdd|jdj\}|jt|k7r4|jtd t||jfzd|jdd j\|_|r;||jk7r,|jtd |jd |d |jdd j\}|jdk(}|jdj#D cgc]} | d c} |_|rt$j&j)t$j&j+|}g} |jD]} t$j&j-| r| j/| 4| j/t$j&j1|| j3dt$j&j4| |_~ |r^t7|t7|jk7r=|jtdt7|jt7|fzd|r||jk7rt9|j|D]\} } t$j&j)| t$j&j)| k7sF|j|rLtdt$j&j)| dt$j&j)| dtdt$j&j)| dt$j&j)| d||js)|jtd|jdy#t $rd}YwxYwcc} w#t j:$r$}|jtd|dd}~wwxYw)z8Call from __init__ to reuse an existing index (PRIVATE).F)check_same_threadz(SELECT value FROM meta_data WHERE key=?;)countzUnfinished/partial databaseNz%SELECT MAX(_ROWID_) FROM offset_data;z#Corrupt database? %i entries not %i)formatzIndex file says format z, not )filenames_relative_to_indexTRUEz0SELECT name FROM file_data ORDER BY file_number;r/z Index file says %i files, not %iz)Index file has different filenames, e.g. z != zIndex file has different filenames [This is an old index where any relative paths were relative to the original working directory]. e.g. z Not a Biopython index database? Unsupported format 'r@)rnrxrorprqrldbapi2connect_conexecutefetchoneint_lengthrrupperrfetchallrtrurvrwisabsappendjoinreplaceseprWzipOperationalError)r'r| relative_pathr}rr~conrrrowtmpfoldnewerrs r rzz#_SQLiteManySeqFilesDict._load_indexGs--++ OO ll++ nn$$^u$M U Q{{:Jhj Uu:DL||r!  !>?TI {{#JKTTVHU||s5z)  9SZ42(*/, 05576A,;;F(* ADO + "0O P Aww}}Q' 1  GGLL #rww{{8ST #&S^s4??/CC  64??+S^<=Y$//9 #DOOY ?(HCwws+rwws/CC 6",#%77??3#79M!O#$(( #- $&77??3#79M !O# $( ((,T\\* IIK3DLL>CD D+k 4.3+ 4\'' Q IIK?uEFD P QsQ&D.R-5R !R-+ R(7F7R-/B+R- R%!R-$R%%R--S$SS$cn|j}|j}|j}|j}|j|j }|j }|j}|r|std|||std|dtjj|}||_ |jd|jd|jd|jdd|jdd |f|jdd |jd |jd d } t|D]G\} tj j#| } tj j%| shtj j%|sItj j'| |j)tj j*d} ntj j-tj j#| tj j*zj/|tj j*zr`tj j'| |j)tj j*d} | j/drJ| |jd| f||| } r fd| D} n fd| D} t1t3j4| d}|sn1|j7d||j9| t;|z } St;||kr| |<.| j<j?J| |_ |jd|jd|jd| df|j9y#tjB$r;}||_|j?|j?td|dd}~wwxYw)z3Call from __init__ to create a new index (PRIVATE).z0Filenames to index and format required to build rr@zPRAGMA synchronous=OFFzPRAGMA locking_mode=EXCLUSIVEz.CREATE TABLE meta_data (key TEXT, value TEXT);z0INSERT INTO meta_data (key, value) VALUES (?,?);)rrr)rTruez8CREATE TABLE file_data (file_number INTEGER, name TEXT);zYCREATE TABLE offset_data (key TEXT, file_number INTEGER, offset INTEGER, length INTEGER);rrz../z7INSERT INTO file_data (file_number, name) VALUES (?,?);c3>K|]\}}}|||fywr:r4)r;r<r,r= file_indexr>s r r?z7_SQLiteManySeqFilesDict._build_index..s--ff"#& FFCsc32K|]\}}}|||fywr:r4)r;r<r,r=rs r r?z7_SQLiteManySeqFilesDict._build_index..s(-ff*ff5sdzIINSERT INTO offset_data (key,file_number,offset,length) VALUES (?,?,?,?);z@CREATE UNIQUE INDEX IF NOT EXISTS key_index ON offset_data(key);zDuplicate key? NzPRAGMA locking_mode=NORMALz-UPDATE meta_data SET value = ? WHERE key = ?;r)"rnrxrorprBrqrrrsrrlrrrr enumeratertrurvrrelpathrrrw startswithrS itertoolsislice executemanycommitrWrFrrIntegrityError)r'r|rr}rr~rrandom_access_proxiesrrrrrHrKbatchrrr>s @@r r{z$_SQLiteManySeqFilesDict._build_indexs--++ OO ll)) ++ >> $ )B>BTU S!3C5:; ;nn$$^4  ,- 34 DE F V FSVX > 3 NO  D $-i$80 4 J)A77==*277==3P GGOOHm<DDRWW[[RUV''//"''//("; 1D 1D Y--k3?@_ U#()H44G%j1#++113a0 4b  @ KKR  01 CeWEUV %% @1DM JJL IIKse454 ?  @sO&&P496P//P4c|jSr:rOr&s r rPz _SQLiteManySeqFilesDict.__repr__s zzr)cjt|jjd|fjS)Nz(SELECT key FROM offset_data WHERE key=?;)boolrrrrfs r __contains__z$_SQLiteManySeqFilesDict.__contains__s/ II  :SF hj  r)c|jS)z%Return the number of records indexed.)rr&s r rXz_SQLiteManySeqFilesDict.__len__!s ||r)c#lK|jjdD]}t|dyw)rZz9SELECT key FROM offset_data ORDER BY file_number, offset;rN)rrr)r'rs r r(z _SQLiteManySeqFilesDict.__iter__&s799$$ G  Cc!f+  s24cZ|jjd|fj}|st|\}}|j}||vr||j |}nt ||jk\r+|jdjj|j|j|j|}|j |}|||<|jr|j|j}n |j}||k7rt!d|d|d|S)z$Return record for the specified key.z8SELECT file_number, offset FROM offset_data WHERE key=?;rr]r^r_)rrrKeyErrorrsr-rWrrpopitemrFrrqrprorBr`r) r'r<r file_numberr,proxiesraproxyrbs r rcz#_SQLiteManySeqFilesDict.__getitem__-sii F (* N! V-- ' ![)--f5F7|t~~-!!$,,224'' dook6RSEYYv&F#(GK   %%fii0D99D $;23%tD6CD D r)c|jjd|fj}|st|\}}}|j}||vrG|r1||j }|j ||j|S||j|St||jk\r+|jdj j|j|j|j|}|||<|r.|j }|j ||j|S|j|S)rez@SELECT file_number, offset, length FROM offset_data WHERE key=?;r)rrrrrsrFrrr/rWrrrrrqrpro) r'r<rrr,r=rhrs r r/z_SQLiteManySeqFilesDict.get_rawIs! ii NQTPV (* N&)# VV-- ' !K(00vvvf~%{+33F;;7|t~~-!!$,,224'' dook6RSE#(GK MMvvvf~%}}V,,r)c~|j}|r/|jdjj|r.yy)zClose any open file handles.rN)rsrrFr)r'rs r rz_SQLiteManySeqFilesDict.closens1-- OO a ( ( . . 0r)N) )r0r1r2r3rMrzr{rPrrXr(rcr/rr4r)r riri sE ,+ ZdELgT  8#-J1r)ri)r)r3collections.abc collections contextlibrrtabcrrrl ImportErrorcontextmanagerr r r"Mappingr6rir4r)r rs    ..bF"I3"IJr$+//11r$jg11g1iGsA00A:9A: