+dg%ddlmZddlmZddlmZddlmZmZGddeZed eZ d Z d Z d Z Gd deZ Gdde ZGddeZGddeZdS)) lru_cache) open_catalog) import_name) DataSourceSchemacTeZdZdZdZdZdZd fd ZdZdZ d Z d Z d Z d Z xZS) AliasSourceacRefer to another named source, unmodified The purpose of an Alias is to be able to refer to other source(s) in the same catalog, perhaps leaving the choice of which target to load up to the user. This source makes no sense outside of a catalog. In this case, the output of the target source is not modified, but this class acts as a prototype 'derived' source for processing the output of some standard driver. After initial discovery, the source's container and other details will be updated from the target; initially, the AliasSource container is not any standard. otherraliasNc tt||||_|p||i|_||_||_d|_dS)a Parameters ---------- target: str Name of the source to load, must be a key in the same catalog mapping: dict or None If given, use this to map the string passed as ``target`` to entries in the catalog metadata: dict or None Extra metadata to associate kwargs: passed on to the target N)superr __init__targetmappingkwargsmetadatasource)selfrrrr __class__s 5lib/python3.11/site-packages/intake/source/derived.pyrzAliasSource.__init__sT k4  ))(333 266"2     cX|jtd|j|j|j|jdd|ji|j|_|jj|_|jj|_|jj |_ |jj |_ dSdS)Nz,AliasSource cannot be used outside a catalogr) cat ValueErrorrrrrrcopy containerpartition_access descriptionrs r _get_sourcezAliasSource._get_source1s 8 MKLL L ; 7=$(4< #<=ddt}dX\XcddDK K05577DM![2DN$(K$@D !#{6D     7 7rc\||jSN)r#rdiscoverr"s rr&zAliasSource.discover;s( {##%%%rc\||jSr%)r#rreadr"s rr(zAliasSource.read?s( {!!!rc^||j|Sr%)r#rread_partition)ris rr*zAliasSource.read_partitionCs* {))!,,,rc\||jSr%)r#r read_chunkedr"s rr-zAliasSource.read_chunkedGs( {'')))rc\||jSr%)r#rto_daskr"s rr/zAliasSource.to_daskKs( {""$$$r)NN)__name__ __module__ __qualname____doc__rversionnamerr#r&r(r*r-r/ __classcell__rs@rr r s  IG D*777&&&"""---***%%%%%%%rr cd|vr&|dd\}}t|fi|}|r||jdi|S|S)N:rr)rsplit cached_cats configure_new)rrr cat_kwargscaturls r get_sourcer@Ssd f}0sA..&//J// 3(s6{(226222 Mrc^|d}t||||i|S)zA target chooser that simply picks the first from the given list This is the default, particularly for the case of only one element in the list r)r@get)targetsrrr>targs rfirstrE]s/ 1:D dCD"!5!5z B BBrc |D]P} t||||i|}||cS#t$rYMwxYwt d)zA target chooser: the first target for which discover() succeeds This may be useful where some drivers are not importable, or some sources can be available only sometimes. z"No targets succeeded at discover())r@rBr& Exception RuntimeError)rCrrr>tss rfirst_discoverablerKgs~  1c6::a#4#4jAAA JJLLLHHH    D  ; < < source, or a fully-qualified dotted string pointing to it target_kwargs: dict of dict with keys matching items of targets cat_kwargs: to pass to intake.open_catalog, if the target is in another catalog container: str (optional) Assumed output container, if known/different from input [Note: the exact form of target_kwargs and cat_kwargs may be subject to change] N)r) rCcallabler_chooser_kwargs_source_params _cat_kwargsr_validate_paramsrr) rrCtarget_chooser target_kwargsr>rrrrs rrzDerivedSource.__init__s& *2>*B*Bc TbHcHc $*   %+  '&DN  (+++++rct|jt|jz tksJ|jD]\}}||jvr ||j|<dS)zBThat all required params are present and that optional types matchN)setrequired_paramsrSoptional_paramsitems)rparvals rrUzDerivedSource._validate_paramssy4'((3t|+<+<<EEEE,2244 ( (HC$,& ($' S! ( (rc||j|j|j|j|_|jdkr|jj|jksJ|jj|jd<|j|jj|_dSdS)z&Pick the source from the given targetsr rN) rPrCrrQrTrRinput_containerrrr"s r_pickzDerivedSource._picks}}T\48T\4K[\\  7 * B<)T-AA A A A"&,"7 h > 4!\3DNNN 4 4r) r0r1r2r3r`rrZr[rErrUrar6r7s@rrMrMwsOIOO/4DUYeitx,,,,,,<(((4444444rrMcBeZdZddgZddiZ fdZdZdZdZxZ S) GenericTransform transformtransform_kwargs allow_daskTct|jd}t|r|nt ||_dS)Nrd)rrUrSrOr _transform)rrdrs rrUz!GenericTransform._validate_paramssI   """L- '/ ':':V)) I@V@VrcF|tS)z0We do not know the schema of a generic transform)rar r"s r _get_schemazGenericTransform._get_schemas xxrc||jdstd|j|jfi|jdS)NrfzGThis transform is not compatible with Daskbecause it has use_dask=Falsere)rjrSrrhrRr/r"s rr/zGenericTransform.to_dasksc |L) kijj jtt|3355ZZFX9YZZZrc||j|jfi|jdSNre)rjrhrRr(rSr"s rr(zGenericTransform.readsD tt|0022WWdlCU6VWWWr) r0r1r2rZr[rUrjr/r(r6r7s@rrcrcs"$67O#T*O WWWWW  [[[ XXXXXXXrrcc4eZdZdZdZdZiZdZdZdZ dZ dS)DataFrameTransformzTransform where the input and output are both Dask-compatible dataframes This derives from GenericTransform, and you must supply ``transform`` and any ``transform_kwargs``. dataframeNc|jI||j|jfi|jd|_|jSrm)_dfrarhrRr/rSr"s rr/zDataFrameTransform.to_dasksR 8 c JJLLL&tt|';';'='=bbN`AabbDHxrc|t|jjdt |jjf|jj|jS)zload metadata only if neededN)dtypeshape npartitionsr)r/r rrdtypeslencolumnsrvrr"s rrjzDataFrameTransform._get_schemas[ DHOD#dh>N:O:O3P^b^f^r~B~KLLL LrcN|Sr%)r/computer"s rr(zDataFrameTransform.reads||~~%%'''r) r0r1r2r3r`rr[rrr/rjr(rrrrorosd "OIO C LLL (((((rroc6eZdZdZdZdZdgZfdZdZxZ S)ColumnszSimple dataframe transform to pick columns Given as an example of how to make a specific dataframe transform. Note that you could use DataFrameTransform directly, by writing a function to choose the columns instead of a method as here. rpryc t||j|itjdi|dS)zx columns: list of labels (usually str) or slice Columns to choose from the target dataframe )rdryreNr)update pick_columnsrr)rryrrs rrzColumns.__init__sD   17UW XXX""6"""""rc(||jdS)Nry)rS)rdfs rrzColumns.pick_columns s$,y)**r) r0r1r2r3r`rrZrrr6r7s@rr}r}sd"OI kO # # # # #+++++++rr}N) functoolsrrrbaserr r r<r@rErKrMrcror}rrrrs$$$$$$$$E%E%E%E%E%*E%E%E%PiimmL)) CCC = = = :4:4:4:4:4J:4:4:4z$X$X$X$X$X}$X$X$XN((((()(((6+++++ +++++r