U ne@sddlZddlZddlmZmZmZmZmZddlm Z ddl m Z ddl m Z ddlmZddlmZddlmZdd lmZeeZGd d d e ZdS) N)CallableIterableOptionalSequenceUnion)Literal)RepresentationMixin) ParslExecutor)ThreadPoolExecutor)ConfigurationError) TaskRecord) MonitoringHubc@seZdZdZejdeeee ee e e de d e d e d e d fee e eeeeeegefe ee eeee e ddddZee edddZddddZdS)ConfigaZ Specification of Parsl configuration options. Parameters ---------- executors : sequence of ParslExecutor, optional List (or other iterable) of `ParslExecutor` instances to use for executing tasks. Default is (:class:`~parsl.executors.threads.ThreadPoolExecutor()`,). app_cache : bool, optional Enable app caching. Default is True. checkpoint_files : sequence of str, optional List of paths to checkpoint files. See :func:`parsl.utils.get_all_checkpoints` and :func:`parsl.utils.get_last_checkpoint` for helpers. Default is None. checkpoint_mode : str, optional Checkpoint mode to use, can be ``'dfk_exit'``, ``'task_exit'``, ``'periodic'`` or ``'manual'``. If set to `None`, checkpointing will be disabled. Default is None. checkpoint_period : str, optional Time interval (in "HH:MM:SS") at which to checkpoint completed tasks. Only has an effect if ``checkpoint_mode='periodic'``. garbage_collect : bool. optional. Delete task records from DFK when tasks have completed. Default: True internal_tasks_max_threads : int, optional Maximum number of threads to allocate for submit side internal tasks such as some data transfers or @joinapps Default is 10. monitoring : MonitoringHub, optional The config to use for database monitoring. Default is None which does not log to a database. retries : int, optional Set the number of retries (or available retry budget when using retry_handler) in case of failure. Default is 0. retry_handler : function, optional A user pluggable handler to decide if/how a task retry should happen. If no handler is specified, then each task failure incurs a retry cost of 1. run_dir : str, optional Path to run directory. Default is 'runinfo'. strategy : str, optional Strategy to use for scaling blocks according to workflow needs. Can be 'simple', 'htex_auto_scale', 'none' or `None`. If 'none' or `None`, dynamic scaling will be disabled. Default is 'simple'. The literal value `None` is deprecated. max_idletime : float, optional The maximum idle time allowed for an executor before strategy could shut down unused blocks. Default is 120.0 seconds. usage_tracking : bool, optional Set this field to True to opt-in to Parsl's usage tracking system. Parsl only collects minimal, non personally-identifiable, information used for reporting to our funding agencies. Default is False. initialize_logging : bool, optional Make DFK optionally not initialize any logging. Log messages will still be passed into the python logging system under the ``parsl`` logger name, but the logging system will not by default perform any further log system configuration. Most noticeably, it will not create a parsl.log logfile. The use case for this is when parsl is used as a library in a bigger system which wants to configure logging in a way that makes sense for that bigger system as a whole. NT rruninfosimple^@F task_exitperiodicdfk_exitmanual) executors app_cachecheckpoint_filescheckpoint_modecheckpoint_periodgarbage_collectinternal_tasks_max_threadsretries retry_handlerrun_dirstrategy max_idletime monitoringusage_trackinginitialize_loggingreturncCst|pg}|stf}||_|||_||_||_|dk rr|dkrZtd |n|dkrrtd ||dkr|dkrd}||_ ||_ ||_ ||_ | |_| |_| |_| |_||_||_| |_dS)NzWThe requested `checkpoint_period={}` will have no effect because `checkpoint_mode=None`rzTRequested checkpoint period of {} only has an effect with checkpoint_mode='periodic'z00:30:00)tupler _executors_validate_executorsrrrloggerdebugformatrrrrrr r!r"r$r%r#)selfrrrrrrrrrr r!r"r#r$r%r.j/mounts/lovelace/software/anaconda3/envs/qiime2-shotgun-2024.2/lib/python3.8/site-packages/parsl/config.py__init__Js< zConfig.__init__)r&cCs|jSN)r()r-r.r.r/rszConfig.executorscsjt|jdkrtddd|jDfddtD}t|dkrftdddd|DdS) Nrz'At least one executor must be specifiedcSsg|] }|jqSr.)label).0er.r.r/ sz.Config._validate_executors..cs$g|]\}}|d|kr|qSr1r.)r3nr4labelsr.r/r5sz&Executors must have unique labels ({})z, cSsg|]}dt|qS)zlabel={})r,repr)r3dr.r.r/r5s)lenrr enumerater,join)r- duplicatesr.r7r/r)s zConfig._validate_executors)NTNNNTrrNrrrNFT)__name__ __module__ __qualname____doc__ typeguard typecheckedrrr boolrstrrrintr Exceptionr floatr r0propertyrr)r.r.r.r/rsV8  6r)loggingrCtypingrrrrrtyping_extensionsr parsl.utilsrparsl.executors.baser parsl.executors.threadsr parsl.errorsr parsl.dataflow.taskrecordr parsl.monitoringr getLoggerr?r*rr.r.r.r/s