a ;gha@s&ddlmZddlmZmZddlZddlZddlZddlmZm Z ddl m Z ddl Z ddlmZmZgZedgdZdd Zd d ZGd d d eZddZd,ddZd-ddZddZd.ddZddZddZd/ddZd0dd ZGd!d"d"eZ ed1d$d%Z!d&d'Z"d(d)Z#d*d+Z$dS)2) namedtuple) ExitStackcontextmanagerN)laxrandom)find_stack_levelidentityCondIndepStackFrame)namedimsizecCsf|ddurb|ddkrD|d|dddi|d\|d<|d <n|d|di|d|d<dS) NvaluetypesamplefnargsZsample_intermediatesTkwargs intermediatesmsgrrb/mounts/lovelace/software/anaconda3/envs/metaDMG/lib/python3.9/site-packages/numpyro/primitives.pydefault_process_messages  rcCs`d}tttD] \}}|||drq2qt|t| ddD]}||qL|S)al Execute the effect stack at a single site according to the following scheme: 1. For each ``Messenger`` in the stack from bottom to top, execute ``Messenger.process_message`` with the message; if the message field "stop" is True, stop; otherwise, continue 2. Apply default behavior (``default_process_message``) to finish remaining site execution 3. For each ``Messenger`` in the stack from top to bottom, execute ``Messenger.postprocess_message`` to update the message and internal messenger state with the site results rstopN) enumeratereversed _PYRO_STACKprocess_messagegetrpostprocess_message)rZpointerhandlerrrr apply_stacks   r"c@s>eZdZdddZddZddZdd Zd d Zd d ZdS) MessengerNcCs<|dur"t|s"tdt|||_tj||gddS)NzJExpected `fn` to be a Python callable object; instead found type(fn) = {}.)updated)callable ValueErrorformatrr functoolsupdate_wrapper)selfrrrr__init__@szMessenger.__init__cCst|dSN)rappendr*rrr __enter__IszMessenger.__enter__cCsT|dur"td|usJtn.|tvrPt|}t|ttD] }tqBdS)N)rpopindexrangelen)r*exc_type exc_value tracebacklocirrr__exit__Ls  zMessenger.__exit__cCsdSr,rr*rrrrr\szMessenger.process_messagecCsdSr,rr;rrrr _szMessenger.postprocess_messagecOsd|jdur,t|dkr|rJ|d|_|S| |j|i|WdS1sV0YdS)Nrr)rr4)r*rrrrr__call__bs   zMessenger.__call__)N) __name__ __module__ __qualname__r+r/r:rr r<rrrrr#?s  r#c Kstjj|d0t|d|fi|d|i}Wdn1sD0Ytjj|dAd(t|d|fi|}Wdn1s0Yt|d|j}t||}t|||} t || S)N)maskZ _observedobsTZ _unobserved)r) numpyrohandlersr@rjnpshape event_dimZreshapewhere deterministic) r rrAobs_maskrZobservedZ unobservedrEZ batch_maskr rrr_masked_observels>6 rJrc Csxt|tsJdt|tjjstd}zddlm}WntyNd}Yn0|dusbt||szddl m} ddl m } Wnty|Yn0t|| jrt 4t jdtd | |jfi|j}Wdq1s0Yn|ts|dur|||d S|S|dur4t|||||d |d Sd ||d ||d |d|dugg|dur`in|d } t| } | dS)a Returns a random sample from the stochastic function `fn`. This can have additional side effects when wrapped inside effect handlers like :class:`~numpyro.handlers.substitute`. .. note:: By design, `sample` primitive is meant to be used inside a NumPyro model. Then :class:`~numpyro.handlers.seed` handler is used to inject a random state to `fn`. In those situations, `rng_key` keyword will take no effect. :param str name: name of the sample site. :param fn: a stochastic function that returns a sample. :param numpy.ndarray obs: observed value :param jax.random.PRNGKey rng_key: an optional random key for `fn`. :param sample_shape: Shape of samples to be drawn. :param dict infer: an optional dictionary containing additional information for inference algorithms. For example, if `fn` is a discrete distribution, setting `infer={'enumerate': 'parallel'}` to tell MCMC marginalize this discrete latent site. :param numpy.ndarray obs_mask: Optional boolean array mask of shape broadcastable with ``fn.batch_shape``. If provided, events with mask=True will be conditioned on ``obs`` and remaining events will be imputed by sampling. This introduces a latent sample site named ``name + "_unobserved"`` which should be used by guides in SVI. Note that this argument is not intended to be used with MCMC. :return: sample from the stochastic `fn`. z,sample_shape needs to be a tuple of integerszIt looks like you tried to use a fn that isn't an instance of numpyro.distributions.Distribution, funsor.Funsor or tensorflow_probability.distributions.Distribution. If you're using funsor or tensorflow_probability, make sure they are correctly installed.r)FunsorN) distributions)TFPDistributionignore)category)rng_key sample_shaper)rPrQinferr) rr rrrr scaleZ is_observedrcond_indep_stackrRr ) isinstancetuplerBrL Distribution TypeErrorZfunsorrK ImportErrorZ%tensorflow_probability.substrates.jaxZ!numpyro.contrib.tfp.distributionsrMwarningscatch_warnings simplefilter FutureWarning __class__ parametersrrJr") r rrArPrQrRrIZ type_errorrKZtfdrM initial_msgrrrrrzs\       6   rc KsVtst|rJd|St|r*dd}nt}d|||f|ddgd}t|}|dS)as Annotate the given site as an optimizable parameter for use with :mod:`jax.example_libraries.optimizers`. For an example of how `param` statements can be used in inference algorithms, refer to :class:`~numpyro.infer.SVI`. :param str name: name of site. :param init_value: initial value specified by the user or a lazy callable that accepts a JAX random PRNGKey and returns an array. Note that the onus of using this to initialize the optimizer is on the user inference algorithm, since there is no global parameter store in NumPyro. :type init_value: numpy.ndarray or callable :param constraint: NumPyro constraint, defaults to ``constraints.real``. :type constraint: numpyro.distributions.constraints.Constraint :param int event_dim: (optional) number of rightmost dimensions unrelated to batching. Dimension to the left of this will be considered batch dimensions; if the param statement is inside a subsampled plate, then corresponding batch dimensions of the parameter will be correspondingly subsampled. If unspecified, all dimensions will be considered event dims and no subsampling will be performed. :return: value for the parameter. Unless wrapped inside a handler like :class:`~numpyro.handlers.substitute`, this will simply return the initial value. zMA callable init_value needs to be put inside a numpyro.handlers.seed handler.c_s |tSr,)prng_key)Zinit_fnrrrrrrszparam..fnparamN)rr rrrr rSrTr )rr%rr")r init_valuerrr`rrrrrbs*  rbcCs$ts|Sd||d}t|}|dS)a Used to designate deterministic sites in the model. Note that most effect handlers will not operate on deterministic sites (except :func:`~numpyro.handlers.trace`), so deterministic sites should be side-effect free. The use case for deterministic nodes is to record any values in the model execution trace. :param str name: name of the deterministic site. :param numpy.ndarray value: deterministic value to record in the trace. rH)rr r r rr")r r r`rrrrrHs  rHcCs,ts|Sd|t|fi|d}t|}|dS)a This primitive is used to store a mutable value that can be changed during model execution:: a = numpyro.mutable("a", {"value": 1.}) a["value"] = 2. assert numpyro.mutable("a")["value"] == 2. For example, this can be used to store and update information like running mean/variance in a neural network batch normalization layer. :param str name: name of the mutable site. :param init_value: mutable value to record in the trace. mutable)rr rrrr r )rrr")r rcr`rrrrre0s recCs"ddddiddd}t||S)z EXPERIMENTAL Inspect the Pyro stack. .. warning:: The format of the returned message may change at any time and does not guarantee backwards compatibility. :returns: A message with mask effects applied. :rtype: dict inspectcSsdS)NTrrrrr^z_inspect..rN)rrrrr r@)r"rrrr_inspectPs ricCs tdS)a Records the effects of enclosing ``handlers.mask`` handlers. This is useful for avoiding expensive ``numpyro.factor()`` computations during prediction, when the log density need not be computed, e.g.:: def model(): # ... if numpyro.get_mask() is not False: log_density = my_expensive_computation() numpyro.factor("foo", log_density) # ... :returns: The mask. :rtype: None, bool, or numpy.ndarray r@)rirrrrget_maskhsrjc CsZ|d}|\}}t|}|durN|dur0tdt}|||\}}t||t||S)az Declare a :mod:`~jax.example_libraries.stax` style neural network inside a model so that its parameters are registered for optimization via :func:`~numpyro.primitives.param` statements. :param str name: name of the module to be registered. :param tuple nn: a tuple of `(init_fn, apply_fn)` obtained by a :mod:`~jax.example_libraries.stax` constructor function. :param tuple input_shape: shape of the input taken by the neural network. :return: a `apply_fn` with bound parameters that takes an array as an input and returns the neural network transformed output array. z$paramsNz3Valid value for `input_shape` needed to initialize.)rbr&rar(partial) r nnZ input_shapeZ module_keyZnn_initZnn_applyZ nn_paramsrP_rrrmodule{s rncsz|durtdtdkrbt||fdd}t|tt|\}}|| dStj ||fddSdS)NzMissing random key to generate subsample indices. Algorithms like HMC/NUTS do not support subsampling. You might want to use SVI or HMCECS instead.cpucsT|}|d}t|dd|}|jt||g|t||g}|dfS)Nrrr)rrandintatrDarrayset)validxZi_p1r9jZrng_keysr rrbody_fns (z_subsample_fn..body_fnF)replace) r&jaxZdefault_backendrsplitrscanrDarangechoice)r subsample_sizerPrxrtrmrrwr _subsample_fns  rcsVeZdZdZdfdd ZeddZfddZed d Zd d Z d dZ Z S)platea Construct for annotating conditionally independent variables. Within a `plate` context manager, `sample` sites will be automatically broadcasted to the size of the plate. Additionally, a scale factor might be applied by certain inference algorithms if `subsample_size` is specified. .. note:: This can be used to subsample minibatches of data: .. code-block:: python with plate("data", len(data), subsample_size=100) as ind: batch = data[ind] assert len(batch) == 100 :param str name: Name of the plate. :param int size: Size of the plate. :param int subsample_size: Optional argument denoting the size of the mini-batch. This can be used to apply a scaling factor by inference algorithms. e.g. when computing ELBO using a mini-batch. :param int dim: Optional argument to specify which dimension in the tensor is used as the plate dim. If `None` (default), the rightmost available dim is allocated. Ncsp||_|dksJd||_|dur4|dkr4td||j|j||\|_|_|jjd|_tt | dS)Nrz size of plate should be positivezdim arg must be negative.) r r r& _subsampler _indicesrErsuperrr+)r*r r rr r^rrr+s zplate.__init__c Csdt|||fddi|dur&||kr&dnt|dgd}t||d}|dd}|dur||jdkrtjd |t|d t d |d }d d|D}|durd}||vr|d8}q|}n ||vsJ||fS)NrrP?)rrr rrr rSrTr rrrz7subsample_size does not match len(subsample), {} vs {}.zJ Did you accidentally use different subsample_size in the model and guide?) stacklevelrTcSsh|] }|jqSrr .0frrr rhz#plate._subsample..r0) rrDr}r"rErZwarnr'r4r) r r rr r subsamplerTZ occupied_dimsZnew_dimrrrrsD   zplate._subsamplecst|jSr,)rr/rr.rrrr/s zplate.__enter__cCs:tdd|D}dg|}|D]}|j||j<q t|S)Ncss|]}|j VqdSr,rrrrr rhz)plate._get_batch_shape..r)maxr r rV)rTZn_dims batch_shaperrrr_get_batch_shapes  zplate._get_batch_shapec Cs0|ddvr$|ddkr tddS|d}t|j|j|j}|||ddkr||}|dj}d|d vr|d d|}d |d d<tt |t |d }||d}t |t |}|d||} |d | |d<|j|jkr,|d durd n|d } | |jr"|j|jnd|d <dS)Nr)rbrrZ control_flowzCannot use control flow primitive under a `plate` primitive. Please move those `plate` statements into the control flow body function. See `scan` documentation for more information.rTrrrQrrrrSrr)NotImplementedErrorr r r rr-rrrr4rZbroadcast_shapesrVexpandr ) r*rrTframeZexpected_shapeZdist_batch_shapeZ overlap_idxZtrailing_shapeZbroadcast_shaperrSrrrrs6         zplate.process_messagecCs|ddvr|jdur|dd}|dur|dks8J|j|}t|d}t|| kr||dkr|||jkr|ddkrd |d |}n d |}td |j|j|j|||j |jkr|d}t ||j |}||d<dS) Nr)rrbrrFrr rrbz$numpyro.param({}, ..., event_dim={})r z$numpyro.subsample(..., event_dim={})zr?__doc__r+ staticmethodrr/rrr __classcell__rrrrrs  %   rr0ccsv|dks JtP}tt|D],\}}td|||||d}||q dVWdn1sh0YdS)a2 Create a contiguous stack of :class:`plate` s with dimensions:: rightmost_dim - len(sizes), ..., rightmost_dim :param str prefix: Name prefix for plates. :param iterable sizes: An iterable of plate sizes. :param int rightmost_dim: The rightmost dim, counting from the right. rz{}_{}rN)rrrrr' enter_context)prefixsizesZ rightmost_dimstackr9r Zplate_irrr plate_stackBs  rcCs0tjj|}|d}t|||ddiddS)z Factor statement to add arbitrary log probability factor to a probabilistic model. :param str name: Name of the trivial sample. :param numpy.ndarray log_factor: A possibly batched log probability factor. NZ is_auxiliaryT)rArR)rBrL distributionZUnitr)r Z log_factorZ unit_distZ unit_valuerrrfactorUs rcCs0tsdSddddddidd}t|}|dS) z A statement to draw a pseudo-random number generator key :func:`~jax.random.PRNGKey` under :class:`~numpyro.handlers.seed` handler. :return: a PRNG key of shape (2,) and dtype unit32. NracSs|Sr,r)rPrrrrgnrhzprng_key..rrP)rrrrr r rd)r`rrrrrabsracCs>ts|St|tr|dksJd|d|id}t|}|dS)a, EXPERIMENTAL Subsampling statement to subsample data based on enclosing :class:`~numpyro.primitives.plate` s. This is typically called on arguments to ``model()`` when subsampling is performed automatically by :class:`~numpyro.primitives.plate` s by passing ``subsample_size`` kwarg. For example the following are equivalent:: # Version 1. using indexing def model(data): with numpyro.plate("data", len(data), subsample_size=10, dim=-data.dim()) as ind: data = data[ind] # ... # Version 2. using numpyro.subsample() def model(data): with numpyro.plate("data", len(data), subsample_size=10, dim=-data.dim()): data = numpyro.subsample(data, event_dim=0) # ... :param numpy.ndarray data: A tensor of batched data. :param int event_dim: The event dimension of the data tensor. Dimensions to the left are considered batch dimensions. :returns: A subsampled version of ``data`` :rtype: ~numpy.ndarray rrrF)rr rr )rrUintr")datarFr`rrrrrxsr)NNrNN)N)N)N)N)r0)% collectionsr contextlibrrr(rZrzrrZ jax.numpynumpyrDrBZ numpyro.utilrrrr rr"objectr#rJrrbrHrerirjrnrrrrrarrrrrs<    - h 9