a 5gh k@sLdZddlmZddlmZmZmZmZmZm Z ddl m Z m Z m Z ddlZddlmZmZddlmZddlZdd l mZmZmZmZmZmZmZmZmZmZm Z m!Z!ddl"Z"dd l#m$Z$gd Z%ed e&eZ'd Z(dZ)e*e&j+Z,ddZ-ddZ.ddZ/ddZ0dddddddZ1ddddddddZ2dddddd d!Z3ddddddd"d#Z4dddd$d%d&Z5ddddd'd(d)Z6dddd$d*d+Z7ddddd'd,d-Z8ddddd.d/d0Z9ddddd.d1d2Z:ddddd.d3d4Z;zdd5lm?Z@e=d7d7d8d9ZAe@e-d:d;dd?ZCe.ZDeAeDZEd@dZ.e7ZFeAeFZGdddd$dAd+Z7eFje7_e5ZHeAeHZIdddd$dBd&Z5eHje5_e1ZJeAeJZKddddddCdZ1eJje1_e3ZLeAeLZMddddddDd!Z3WneNyYn0GdEdFdFejOZPGdGdHdHePZQGdIdJdJePeZRGdKdLdLePZSGdMdNdNeSZTGdOdPdPeSZUGdQdRdReUZVGdSdTdTeUZWGdUdVdVeTZXGdWdXdXeXZYGdYdZdZeXZZGd[d\d\eYZ[Gd]d^d^eYZ\Gd_d`d`eTZ]GdadbdbePZ^dddcdddeZ_dfdgZ`ddhdidjZadkdlZbdmdnZcdZeZfd0e9fd2e:fd+e7fdoZddpdqdrdsdtZedS)ua1 Standard cost functions to minimize for statistical fits. We provide these for convenience, so that you do not have to write your own for standard fits. The cost functions optionally use Numba to accelerate some calculations, if Numba is installed. **There is no need** to set :attr:`iminuit.Minuit.errordef` manually for any of these cost functions. :class:`iminuit.Minuit` automatically uses the correct value, which is provided by each cost function with the attribute ``Cost.errordef``. What to use when ---------------- - Fit a normalised probability density to data - Data are not binned: :class:`UnbinnedNLL` - Data are binned: :class:`BinnedNLL`, also supports histogram of weighted samples - Fit a density to data, density is not normalised - Data are not binned: :class:`ExtendedUnbinnedNLL` - Data are binned: :class:`ExtendedBinnedNLL`, also supports histogram of weighted samples - Fit a template to binned data with bin-wise uncertainties on the template - :class:`Template`, also supports weighted data and weighted template histograms - Fit of a function f(x) to (x, y, yerror) pairs with normal-distributed fluctuations. x is one- or multi-dimensional, y is one-dimensional. - y values contain no outliers: :class:`LeastSquares` - y values contain outliers: :class:`LeastSquares` with loss function set to "soft_l1" - Include constraints from external fits or apply regularisation - :class:`NormalConstraint` Combining cost functions ------------------------ All cost functions can be added, which generates a new combined cost function. Parameters with the same name are shared between component cost functions. Use this to constrain one or several parameters with different data sets and using different statistical models for each data set. Gaussian penalty terms can also be added to the cost function to introduce external knowledge about a parameter. Model parameter limits ---------------------- The Minuit algorithms support box constrains in parameter space. A user-defined model can declare that a parameter is only valid over an interval on the real line with the ``Annotated`` type annotation, see :class:`iminuit.Minuit` for details. A typical example is the sigma parameter of a normal distribution, which must be positive. The cost functions defined here propagate this information to :class:`iminuit.Minuit`. Note: The :class:`Template` declares that the template amplitudes must be non-negative, which is usually the right choice, however, it may be desirable to fit templates which can have negative amplitudes. To achieve this, simply reset the limits with :attr:`iminuit.Minuit.limits` after creating the Minuit instance. User-defined gradients ---------------------- If the user provides a model gradient, the cost functions defined here except :class:`Template` will then also make their gradient available, which is then automatically used by :class:`iminuit.Minuit` (see the constructor for details) to potentially improve the fit (improve convergence or robustness). Note that it is perfectly normal to use Minuit without a user-defined gradient, and Minuit does not always benefit from a user-defined gradient. If the gradient is expensive to compute, the time to converge may increase. If you have trouble with the fitting process, it is unlikely that the issues are resolved by a user-defined gradient. Notes ----- The cost functions defined here have been optimized with knowledge about implementation details of Minuit to give the highest accucary and the most robust results, so they should perform well. If you have trouble with your own implementations, try these. The binned versions of the log-likelihood fits support weighted samples. For each bin of the histogram, the sum of weights and the sum of squared weights is needed then, see class documentation for details.  annotations)describemerge_signaturesPerformanceWarning_smart_sampling_detect_log_spacingis_positive_definite)Model ModelGradient LossFunctionN)NDArray ArrayLike)Sequence) ListTupleUnionr CollectionDictAnyIterableOptionalTypeVarCallablecast)deprecated_parameter) CHISQUARENEGATIVE_LOG_LIKELIHOODchi2multinomial_chi2 poisson_chi2template_chi2_jsctemplate_chi2_datemplate_nll_asyCostCostSumConstant BinnedNLL UnbinnedNLLExtendedBinnedNLLExtendedUnbinnedNLLTemplate LeastSquaresNormalConstraintT??cCs(t|}|dk}t||||<|S)z Evaluate to log(x) for x > 0 and to 0 otherwise. Parameters ---------- x : array Argument. Returns ------- array Elementwise contains log(x) for x > 0 and zero otherwise. r)np zeros_likelog)xrmar8\/mounts/lovelace/software/anaconda3/envs/metaDMG/lib/python3.9/site-packages/iminuit/cost.py log_or_zeros r:cCsttt|t SN)r2sumsortr4 _TINY_FLOATr5r8r8r9 _unbinned_nllsr@cCs|||}||Sr;r8)yyeymzr8r8r9 _z_squareds rEcCs|dur |S|Sr;r8)r5 replacementr8r8r9 _replace_nonesrGrfloat)rArBrCreturncCs4t|||\}}}|jdks"Jtt|||S)a Compute (potentially) chi2-distributed cost. The value returned by this function is chi2-distributed, if the observed values are normally distributed around the expected values with the provided standard deviations. Parameters ---------- y : array-like with shape (N,) Observed values. ye : array-like with shape (N,) Uncertainties of values. ym : array-like with shape (N,) Expected values. Returns ------- float Value of cost function. r)r2 atleast_1dndimr<rErArBrCr8r8r9rsrr)rArBrCgymrIcCsVt||||\}}}}|jdks&J|jdks4Jdtj||||dddS)a Compute gradient of :func:`chi2`. Parameters ---------- y : array-like with shape (N,) Observed values. ye : array-like with shape (N,) Uncertainties of values. ym : array-like with shape (N,) Expected values. gym : array-like with shape (K, N) Gradient of ym with respect to K model parameters. Returns ------- array with shape (K,) Gradient of cost function with respect to model parameters. rZaxis)r2rJrKr<)rArBrCrMr8r8r9 _chi2_gradsrQcCs(t|||}dttd|dS)NrNr)rEr2r<sqrt)rArBrCZz_sqrr8r8r9 _soft_l1_costs rScCsNd|}|||}d|dd}dtj||||ttd|jdS)NrrNgrOrP)r2r<tuplerangerK)rArBrCrMZinv_yerDfr8r8r9_soft_l1_cost_grads rW)nmurIcCs6t||\}}dt|t|t|||S)a/ Compute asymptotically chi2-distributed cost for Poisson-distributed data. See Baker & Cousins, NIM 221 (1984) 437-442. Parameters ---------- n : array-like Observed counts. mu : array-like Expected counts per bin. Returns ------- float Cost function value. Notes ----- The implementation makes the result asymptotically chi2-distributed, which helps to maximise the numerical accuracy for Minuit. If sum(mu) == sum(n), the result is equal to :func:`multinomial_chi2`. rNr2rJr<r:rXrYr8r8r9r!sr!)rXrYgmurIcCs,|jdksJdtjd|||ddS)NrNr0rrPrKr2r<rXrYr\r8r8r9_poisson_chi2_gradsr_cCs.t||\}}dt|t|t|S)a Compute asymptotically chi2-distributed cost for multinomially-distributed data. See Baker & Cousins, NIM 221 (1984) 437-442. Parameters ---------- n : array-like Observed counts. mu : array-like Expected counts. Must satisfy sum(mu) == sum(n). Returns ------- float Cost function value. Notes ----- The implementation makes the result asymptotically chi2-distributed, which helps to maximise the numerical accuracy for Minuit. rNrZr[r8r8r9r sr cCs(|jdksJdtj|||ddS)NrNrOrrPr]r^r8r8r9_multinomial_chi2_grad4sr`)rXrYmu_varrIcCsnt|||\}}}||d}dd||}|t|d||}t|||t|dd|S)a Compute asymptotically chi2-distributed cost for a template fit. J.S. Conway, PHYSTAT 2011, https://doi.org/10.48550/arXiv.1103.0354 Parameters ---------- n : array-like Observed counts. mu : array-like Expected counts. This is the sum of the normalised templates scaled with the component yields. Must be positive everywhere. mu_var : array-like Expected variance of mu. Must be positive everywhere. Returns ------- float Asymptotically chi-square-distributed test statistic. Notes ----- The implementation deviates slightly from the paper by making the result asymptotically chi2-distributed, which helps to maximise the numerical accuracy for Minuit. rNr1r)r2rJrRr!r<)rXrYraZbeta_varpbetar8r8r9r"9s  r"cCsPt|||\}}}|d|}||||t}t|||t|||S)a  Compute asymptotically chi2-distributed cost for a template fit. H.P. Dembinski, A. Abdelmotteleb, https://doi.org/10.48550/arXiv.2206.12346 Parameters ---------- n : array-like Observed counts. mu : array-like Expected counts. This is the sum of the normalised templates scaled with the component yields. mu_var : array-like Expected variance of mu. Must be positive everywhere. Returns ------- float Asymptotically chi-square-distributed test statistic. rN)r2rJr>r!)rXrYrakrcr8r8r9r#_s r#c Csddlm}t|||\}}}|d|d}||}t|t||||||d||td||| S)u Compute marginalized negative log-likelikihood for a template fit. This is the negative logarithm of equation 3.15 of the paper by C.A. Argüelles, A. Schneider, T. Yuan, https://doi.org/10.1007/JHEP06(2019)030. The authors use a Bayesian approach and integrate over the nuisance parameters. Like the other Barlow-Beeston-lite methods, this is an approximation. The resulting likelihood cannot be turned into an asymptotically chi-square distributed test statistic as detailed in Baker & Cousins, NIM 221 (1984) 437-442. Parameters ---------- n : array-like Observed counts. mu : array-like Expected counts. This is the sum of the normalised templates scaled with the component yields. mu_var : array-like Expected variance of mu. Must be positive everywhere. Returns ------- float Negative log-likelihood function value. r)loggammarNr)Z scipy.specialrer2rJr<r4)rXrYraZlgalpharcr8r8r9r${s   (r$)njit)overloadTnumpy)ZnogilcacheZ error_modelalways)inlinecCstSr;)r:r?r8r8r9_ol_log_or_zerosrmcCstSr;)rErLr8r8r9 _ol_z_squaredsrncCs"|jtjtjfvrt|St|Sr;)dtyper2float32float64_unbinned_nll_nb_unbinned_nll_npr?r8r8r9r@scCs6t||\}}|jtjtjfvr,t||St||Sr;)r2rJrorprq_multinomial_chi2_nb_multinomial_chi2_npr[r8r8r9r s cCs6t||\}}|jtjtjfvr,t||St||Sr;)r2rJrorprq_poisson_chi2_nb_poisson_chi2_npr[r8r8r9r!s cCs>t|||\}}}|jtjtjfvr2t|||St|||Sr;)r2rJrorprq_chi2_nb_chi2_nprLr8r8r9rs cCs*|jtjtjfvrt|||St|||Sr;)ror2rprq_soft_l1_cost_nb_soft_l1_cost_nprLr8r8r9rSs c@seZdZUdZdZded<ded<eddZd d Zed d Z ed dZ e j ddZ eddZejddddZdddddZddZddZdddddZdd dd!d"Zed#d$d%d&Ze j d'ddd(d)Ze j d'd dd*d+Ze j d#d$d,d-Zd.S)/r%z@ Base class for all cost functions. :meta private:  _parameters_verbose(Dict[str, Optional[Tuple[float, float]]]r}intr~cCs|S)z; For internal use. :meta private: ) _errordefselfr8r8r9errordefsz Cost.errordefcCstSr;)rrr8r8r9rszCost._errordefcCs|S)z Return number of points in least-squares fits or bins in a binned fit. Infinity is returned if the cost function is unbinned. This is used by Minuit to compute the reduced chi2, a goodness-of-fit estimate. )_ndatarr8r8r9ndatasz Cost.ndatacCs t|jS)z(Return total number of model parameters.)lenr}rr8r8r9nparsz Cost.nparcCstdSr;)NotImplementedrr8r8r9rsz Cost._ndatacCs|jS)zs Access verbosity level. Set this to 1 to print all function calls with input and output. r~rr8r8r9verbosesz Cost.verbosevaluecCs ||_dSr;rrrr8r8r9r$s) parametersrcCs||_||_dSFor internal use.Nr|)rrrr8r8r9__init__(sz Cost.__init__cCs t||Sz{ Add two cost functions to form a combined cost function. Returns ------- CostSum r&)rrhsr8r8r9__add__/sz Cost.__add__cCs t||Srr)rlhsr8r8r9__radd__9sz Cost.__radd__rHargsrIcGs$||}|jdkr t|d||S)z Evaluate the cost function. If verbose >= 1, print arguments and result. Parameters ---------- *args : float Parameter values. Returns ------- float rz->)_valuerprint)rrr6r8r8r9__call__Cs   z Cost.__call__rcGs ||S)aK Compute gradient of the cost function. This requires that a model gradient is provided. Parameters ---------- *args : float Parameter values. Returns ------- ndarray of float The length of the array is equal to the length of args. )_gradrrr8r8r9gradWsz Cost.gradboolrIcCs|S)z4Return True if cost function can compute a gradient.) _has_gradrr8r8r9has_gradisz Cost.has_gradSequence[float]cCsdSr;r8rr8r8r9rnsz Cost._valuecCsdSr;r8rr8r8r9rqsz Cost._gradcCsdSr;r8rr8r8r9rtszCost._has_gradN)__name__ __module__ __qualname____doc__ __slots____annotations__propertyrrrrabcabstractmethodrrsetterrrrrrrrrrr8r8r8r9r%s<        r%cs^eZdZdZdZddfdd ZddZd dd d d Zd d d ddZe ddZ Z S)r'z Cost function that represents a constant. If your cost function produces results that are far away from O(1), adding a constant that brings the value closer to zero may improve the numerical stability. rrHrcs||_tiddS)z!Initialize constant with a value.FN)rsuperrr __class__r8r9rszConstant.__init__cCsdSNrr8rr8r8r9rszConstant._ndatarrcCs|jSr;rrr8r8r9rszConstant._valuercCs tdSr)r2zerosrr8r8r9rszConstant._gradcCsdSNTr8r8r8r8r9rszConstant._has_grad) rrrrrrrrr staticmethodr __classcell__r8r8rr9r'xsr'cseZdZdZdZddfdd Zddd d Zdd d d dZddd ddZddddZ ddZ ddZ ddZ d!ddddd Z ZS)"r&aa Sum of cost functions. Users do not need to create objects of this class themselves. They should just add cost functions, for example:: nll = UnbinnedNLL(...) lsq = LeastSquares(...) ncs = NormalConstraint(...) csum = nll + lsq + ncs CostSum is used to combine data from different experiments or to combine normal cost functions with penalty terms (see NormalConstraint). The parameters of CostSum are the union of all parameters of its constituents. Supports the sequence protocol to access the constituents. Warnings -------- CostSum does not support cost functions that accept a parameter array, because the function signature does not allow one to determine how many parameters are accepted by the function and which parameters overlap between different cost functions. )_items_mapszUnion[Cost, float])itemscsg|_|D]T}t|tr*|j|j7_q t|ttfrR|dkr^|jt|q |j|q t|jdd\}|_t |t dd|jDdS)z Initialize with cost functions. Parameters ---------- *items : Cost Cost functions. May also be other CostSum functions. rTrcss|] }|jVqdSr;)r.0cr8r8r9 z#CostSum.__init__..N) r isinstancer&rrHappendr'rrrrmax)rritemZ signaturesrr8r9rs  zCostSum.__init__rrc#s<t|j|jD](\}}tfdd|D}||fVqdS)Nc3s|]}|VqdSr;r8rirr8r9rrz!CostSum._split..)ziprrrT)rr componentZcmapcomponent_argsr8rr9_splitszCostSum._splitrHrcCs0d}||D]\}}||||j7}q|S)N)rrr)rrr6rrr8r8r9rszCostSum._valuercsZt|j}t|j|jD]:\}}tfdd|D}|||||j7<q|S)Nc3s|]}|VqdSr;r8rrr8r9rrz CostSum._grad..) r2rrrrrrTrr)rrr6rindicesrr8rr9rs  z CostSum._gradrrcCstdd|jDS)Ncss|] }|jVqdSr;)r)rrr8r8r9rrz$CostSum._has_grad..)allrrr8r8r9rszCostSum._has_gradcCstdd|jDS)Ncss|] }|jVqdSr;)rrr8r8r9rrz!CostSum._ndata..)r<rrr8r8r9rszCostSum._ndatacCs |jS)z,Return number of constituent cost functions.)r__len__rr8r8r9rszCostSum.__len__cCs |j|S)z'Get constituent cost function by index.)r __getitem__)rkeyr8r8r9rszCostSum.__getitem__NzDict[int, Dict[str, Any]])rcomponent_kwargsc Csddlm}tdd|D}|}|||d|jd||jd\}}|dur^i}d}t| |D]L\} \} } t | d sqp| | i} | ||| j | fi| |d7}qpdS) a Visualize data and model agreement (requires matplotlib). The visualization is drawn with matplotlib.pyplot into the current figure. Subplots are created to visualize each part of the cost function, the figure height is increased accordingly. Parts without a visualize method are silently ignored. Parameters ---------- args : array-like Parameter values. component_kwargs : dict of dicts, optional Dict that maps an index to dict of keyword arguments. This can be used to pass keyword arguments to a visualize method of a component with that index. **kwargs : Other keyword arguments are forwarded to all components. rpyplotcss|]}t|dVqdS) visualizeN)hasattr)rcompr8r8r9rrz$CostSum.visualize..g?r)numNr) matplotlibrr<ZgcfZ set_figwidthZ get_figwidthZsubplotsnumber enumeraterrgetZscar) rrrpltrXZfig_axrrdrZcargskwargsr8r8r9rs   zCostSum.visualize)N)rrrrrrrrrrrrrrrr8r8rr9r&sr&c@sxeZdZUdZdZded<ddddd d Zed d Zej d ddd ZeddZ e j ddddZ ddZ dS) MaskedCostzV Base class for cost functions that support data masking. :meta private: )_data_mask_maskedOptional[NDArray]rrrr)rdatarcCs&||_d|_|t|||dSr)rr _update_cacher%r)rrrrr8r8r9rszMaskedCost.__init__cCs|jS)z Boolean array, array of indices, or None. If not None, only values selected by the mask are considered. The mask acts on the first dimension of a value array, i.e. values[mask]. Default is None. )rrr8r8r9mask(szMaskedCost.maskzOptional[ArrayLike])rcCs$|dur dnt||_|dSr;)r2asarrayrr)rrr8r8r9r2scCs|jS)zReturn data samples.)rrr8r8r9r7szMaskedCost.datarrcCs||jd<|dSN.)rrrr8r8r9r<s cCs|jt|jd|_dSr)rrGrrrr8r8r9rAszMaskedCost._update_cacheN) rrrrrrrrrrrrr8r8r8r9rs    rc@s>eZdZdZdddddZddZejdddd d Zd S) MaskedCostWithPullszG Base class for cost functions with pulls. :meta private: rrrcCs ||S)a Return studentized residuals (aka pulls). Parameters ---------- args : sequence of float Parameter values. Returns ------- array Array of pull values. If the cost function is masked, the array contains NaN values where the mask value is False. Notes ----- Pulls allow one to estimate how well a model fits the data. A pull is a value computed for each data point. It is given by (observed - predicted) / standard-deviation. If the model is correct, the expectation value of each pull is zero and its variance is one in the asymptotic limit of infinite samples. Under these conditions, the chi-square statistic is computed from the sum of pulls squared has a known probability distribution if the model is correct. It therefore serves as a goodness-of-fit statistic. Beware: the sum of pulls squared in general is not identical to the value returned by the cost function, even if the cost function returns a chi-square distributed test-statistic. The cost function is computed in a slightly differently way that makes the return value approach the asymptotic chi-square distribution faster than a test statistic based on sum of pulls squared. In summary, only use pulls for plots. Compute the chi-square test statistic directly from the cost function. )_pullsrr8r8r9pullsLs!zMaskedCostWithPulls.pullscCst|jjd|jSr;)r2prodrshape_ndimrr8r8r9roszMaskedCostWithPulls._ndatacCsdSr;r8rr8r8r9rrszMaskedCostWithPulls._pullsN) rrrrrrrrrr8r8r8r9rEs #rcseZdZdZdZddddddfd d Zejd d Zejd dZ ddZ ddZ e ddd(ddddddZ ddddd Zdddd!d"Zejdddd#d$Zdd%d&d'ZZS)) UnbinnedCostzE Base class for unbinned cost functions. :meta private: )_model _model_grad_logr rrOptional[ModelGradient]Optional[Sequence[str]])modelrr4rnamecs0||_||_||_tt||t||dSr)rrrrr_model_parameters_norm)rrrrr4rrrr8r9rs zUnbinnedCost.__init__cCsdS)Get probability density model.Nr8rr8r8r9pdfszUnbinnedCost.pdfcCsdS)Get number density model.Nr8rr8r8r9 scaled_pdfszUnbinnedCost.scaled_pdfcCstjSr;)r2infrr8r8r9rszUnbinnedCost._ndatacCs |jjdSN)rrrr8r8r9_npointsszUnbinnedCost._npointsZnbins)binsr2rUnion[int, Sequence[float]])r model_pointsrc sBddlm}tj}|jdkr*tdt|trRt |}j |gR}nt|dkrt |rzt |d|d|}nt |d|d|}j |gR}n"tfdd|d|d\}}tj|||d|dfd\}} d | dd | d d} | d| d} |j| ||d d d |j|d|| d dd S)aX Visualize data and model agreement (requires matplotlib). The visualization is drawn with matplotlib.pyplot into the current axes. Parameters ---------- args : array-like Parameter values. model_points : int or array-like, optional How many points to use to draw the model. Default is 0, in this case an smart sampling algorithm selects the number of points. If array-like, it is interpreted as the point locations. bins : int, optional number of bins. Default is 50 bins. rrr7visualize is not implemented for multi-dimensional datarcsj|gRSr;)rr?rrr8r9rz(UnbinnedCost.visualize..)rrUr1NokfmtC0)fc)rrr2r=rrK ValueErrorrrarrayrr geomspacelinspacerZ histogramerrorbarZ fill_between) rrrrrr5xmrCrXxecxdxr8rr9rs$     " zUnbinnedCost.visualizerHrrcGs||}td||S)a+ Estimate Fisher information for model and sample. The estimated Fisher information is only meaningful if the arguments provided are estimates of the true values. Parameters ---------- *args: float Estimates of model parameters. z ji,ki->jk)_pointwise_scorer2einsumrrgr8r8r9fisher_informations zUnbinnedCost.fisher_informationcGstj|j|S)a Estimate covariance of the parameters with the sandwich estimator. This requires that the model gradient is provided, and that the arguments are the maximum-likelihood estimates. The sandwich estimator is only asymptotically correct. Parameters ---------- *args : float Maximum-likelihood estimates of the parameter values. Returns ------- ndarray of float The array has shape (K, K) for K arguments. )r2linalginvrrr8r8r9 covarianceszUnbinnedCost.covariancecCsdSr;r8rr8r8r9rszUnbinnedCost._pointwise_scorercCs |jduSr;rrr8r8r9rszUnbinnedCost._has_grad)rr)rrrrrrrabstractpropertyrrrrrrrrrrrrr8r8rr9rvs$  3rcseZdZdZdZeddZeddZddd d d d d d ddddfddZdddddZ dddddZ dddddZ dddddZ dddd d!Z ZS)"r)z Unbinned negative log-likelihood. Use this if only the shape of the fitted PDF is of interest and the original unbinned data is available. The data can be one- or multi-dimensional. r8csjrfddSjS)rcstj|Sr;r2exprrrr8r9rrz!UnbinnedNLL.pdf..)rrrr8rr9r s zUnbinnedNLL.pdfcs0tjjjr"fddSfddS)rcstj|Sr;r!rscalerr8r9rrz(UnbinnedNLL.scaled_pdf..csj|Sr;rrr#r8r9rr)r2rrrrrr8r#r9rszUnbinnedNLL.scaled_pdfrFNrr4rrrr rrrr)rrrr4rrcst||||||dS)a Initialize UnbinnedNLL with data and model. Parameters ---------- data : array-like Sample of observations. If the observations are multidimensional, data must have the shape (D, N), where D is the number of dimensions and N the number of data points. pdf : callable Probability density function of the form f(data, par0, [par1, ...]), where data is the data sample and par0, ... are model parameters. If the data are multivariate, data passed to f has shape (D, N), where D is the number of dimensions and N the number of data points. Must return an array with the shape (N,). verbose : int, optional Verbosity level. 0: is no output (default). 1: print current args and negative log-likelihood value. log : bool, optional Distributions of the exponential family (normal, exponential, poisson, ...) allow one to compute the logarithm of the pdf directly, which is more accurate and efficient than numerically computing ``log(pdf)``. Set this to True, if the model returns the logpdf instead of the pdf. Default is False. grad : callable or None, optional Optionally pass the gradient of the pdf. Has the same calling signature like the pdf, but must return an array with the shape (K, N), where N is the number of data points and K is the number of parameters. If `log` is True, the function must return the gradient of the logpdf instead of the pdf. The gradient can be used by Minuit to improve or speed up convergence and to compute the sandwich estimator for the variance of the parameter estimates. Default is None. name : sequence of str or None, optional Optional names for each parameter of the model (in order). Must have the same length as there are model parameters. Default is None. Nrr)rrrrr4rrrr8r9rs.zUnbinnedNLL.__init__rrHrcCs*||}|jrdt|Sdt|S)Ng@ _eval_modelrr2r<r@)rrrVr8r8r9rJs zUnbinnedNLL._valuercCs||}dtj|ddS)Nr(rrPrr2r<rr8r8r9rPs zUnbinnedNLL._gradcCs&||}|jr|S||}||Sr;)_eval_model_gradrr*)rrrrVr8r8r9rTs   zUnbinnedNLL._pointwise_scorecCs$|j}t|j|g|Rd|S)Nr)r_normalize_outputrrrrrr8r8r9r*[szUnbinnedNLL._eval_modelcCs:|jdurtd|j}t|j|g|Rd|j|S)Nno gradient availablemodel gradientrrrr-rrr.r8r8r9r,_s  zUnbinnedNLL._eval_model_gradrrrrrrrrrrrrr*r,rr8r8rr9r)s    0r)cseZdZdZdZeddZeddZddd d d d d d ddddfddZdddddZ dddddZ dddddZ ddddd Z dd!dd"d#Z ZS)$r+z Unbinned extended negative log-likelihood. Use this if shape and normalization of the fitted PDF are of interest and the original unbinned data is available. The data can be one- or multi-dimensional. r8cs$jrfdd}n fdd}|S)rcsj|\}}t||Sr;)rr2r"rrXr5rr8r9fnwsz#ExtendedUnbinnedNLL.pdf..fncsj|\}}||Sr;r%r3rr8r9r4}sr)rr4r8rr9rrs zExtendedUnbinnedNLL.pdfcsjrfddSfddS)zGet density model.cstj|dSNrr!rrr8r9rrz0ExtendedUnbinnedNLL.scaled_pdf..csj|dSr6r%rrr8r9rrr5rr8rr9rs zExtendedUnbinnedNLL.scaled_pdfrFNr&rr rrrr)rrrr4rrcst||||||dS)a` Initialize cost function with data and model. Parameters ---------- data : array-like Sample of observations. If the observations are multidimensional, data must have the shape (D, N), where D is the number of dimensions and N the number of data points. scaled_pdf : callable Density function of the form f(data, par0, [par1, ...]), where data is the sample and par0, ... are model parameters. Must return a tuple (, ). The first value is the density integrated over the data window, the interval that we consider for the fit. For example, if the data are exponentially distributed, but we fit only the interval (0, 5), then the first value is the density integrated from 0 to 5. If the data are multivariate, data passed to f has shape (D, N), where D is the number of dimensions and N the number of data points. verbose : int, optional Verbosity level. 0: is no output (default). 1: print current args and negative log-likelihood value. log : bool, optional Distributions of the exponential family (normal, exponential, poisson, ...) allow one to compute the logarithm of the pdf directly, which is more accurate and efficient than effectively doing ``log(exp(logpdf))``. Set this to True, if the model returns the logarithm of the density as the second argument instead of the density. Default is False. grad : callable or None, optional Optionally pass the gradient of the density function. Has the same calling signature like the density function, but must return two arrays. The first array has shape (K,) where K are the number of parameters, while the second has shape (K, N), where N is the number of data points. The first array is the gradient of the integrated density. The second array is the gradient of the density itself. If `log` is True, the second array must be the gradient of the log-density instead. The gradient can be used by Minuit to improve or speed up convergence and to compute the sandwich estimator for the variance of the parameter estimates. Default is None. name : sequence of str or None, optional Optional names for each parameter of the model (in order). Must have the same length as there are model parameters. Default is None. Nr')rrrrr4rrrr8r9rs3zExtendedUnbinnedNLL.__init__rrHrcCs6||\}}|jr&d|t|Sd|t|SNrNr))rrfintrVr8r8r9rszExtendedUnbinnedNLL._valuercCs||}dtj|ddS)NrOrrPr+rr8r8r9rs zExtendedUnbinnedNLL._gradcCsb||\}}|}|jr6|||ddtjfS||\}}||||ddtjfSr;)r,rrr2Znewaxisr*)rrgintrmrrVr8r8r9rs z$ExtendedUnbinnedNLL._pointwise_scorezTuple[float, float]cCs8|j}|j|g|R\}}t|d|dd}||fS)Nrin second positionmsg)rrr-r)rrrr8rVr8r8r9r*szExtendedUnbinnedNLL._eval_modelTuple[NDArray, NDArray]cCs`|jdurtd|j}|j|g|R\}}t|d|jdd}t|d|j|dd}||fS)Nr/r0zin first positionr<r;r1)rrrr9rr8r8r9r,s  z$ExtendedUnbinnedNLL._eval_model_gradr2r8r8rr9r+hs    5r+cseZdZUdZdZded<ded<ded<d ed <ejZe d d Z d ddddfdd Z dddddZ dddddZ dddddZejdddddZdd d!d"Zdddd#d$Zfd%d&Zddd'd(d)Zddd*d+d,d-Zd.d/ZZS)0 BinnedCostaF Base class for binned cost functions to support histograms filled with weights. Histograms filled with weights are supported by applying the Bohm-Zech transform. The Bohm-Zech approach was further generalized to handle sums of weights which are negative. See Baker & Cousins, NIM 221 (1984) 437-442; Bohm and Zech, NIMA 748 (2014) 1-6; H. Dembinski, M. Schmelling, R. Waldi, Nucl.Instrum.Meth.A 940 (2019) 135-141. Bohm and Zech use the scaled Poisson distribution (SPD) as an approximate way to handle sums of weights instead of Poisson counts. This approach also works for multinomial distributions. The idea of the Bohm and Zech is to use the likelihood for Poisson distributed data also for weighted data. They show that one can match the first and second moment of the compound Poisson distribution for weighted data with a single Poisson distribution with a scaling factor s, that is multiplied with the predicted expectation and the observation. This scaling factor is computed as s = sum(wi) / sum(wi**2), wi are the weights in the current bin. Instead of the Baker & Cousins transformed log-likelihood l(n; mu) for Poisson-distributed data, where n is the observed count and mu is the expectation, we now compute l(sum(w) * s; mu * s), this can be further simplified: l(w * s, mu * s) = 2 * [(w * s) * (log(w * s) - log(mu * s)) - s * mu + s * w] = 2 * s * [w * (log(w) - log(mu)) - mu + w] = s * l(w, mu) For multinomially-distributed data and s = 1, sum(w-mu) = 0, which is why these terms can be omitted in the standard calculation without weights, but in case of weighted counts, sum(s * (w - m)) != 0 and the terms must be kept. The original formulas from Bohm and Zech are only applicable if w >= 0 (with the extra condition that w * log(w) evaluates to 0 for w = 0). One can generalize the formula to w < 0, which is relevant in practice for example in fits of sweighted samples, by computing s = abs(sum(wi)) / sum(wi ** 2) and replacing w * log(w) with 0 for w <= 0. This works, because this extension has the right gradient. The gradient should be equal to hat of the quadratic function s * (w - mu)**2/mu', where mu'=mu but fixed during the gradient computation, see D. Dembinski, M. Schmelling, R. Waldi. The minimum of this quadratic function yields an unbiased estimate of mu, even if some w are negative. Since the quadratic function and the original function have the same gradient, the minima of both functions are the same, and the original function also yields an unbiased estimate. The gradient is not affected by the particular choice of how to handle w * log(w) with w < 0, since this term drops out in the computation of the gradient. Other choices are possible. Our goal was to select an option which keeps the function minimum approximately chi-square distributed, although that property tends to dissolve when negative weights are involved. The minimum can even become negative. :meta private: )_xer _bohm_zech_n _bohm_zech_sz#Union[NDArray, Tuple[NDArray, ...]]r@rrrrArrBcCs|jS)zAccess bin edges.)r@rr8r8r9r%sz BinnedCost.xerr%Union[ArrayLike, Sequence[ArrayLike]])rrXrrc st|tstdt|}t||_|jdkr@ttt||_ nt dd|D|_ t|}|j |jkot|j ddk}|j |jt |krtdt|jdkr|j gn|j D].\}}t||j |dkrtd|d q|rtd nd |_t|||d S) rzxe must be iterablercss|]}t|VqdSr;)rrxeir8r8r9r:rz&BinnedCost.__init__..rrNz4n must either have same dimension as xe or one extraz2n and xe have incompatible shapes along dimension z7, xe must be longer by one element along each dimensionrN)rrr_shape_from_xerrrrrr@rTrKrrrr2rrBrr) rrrXrrrZ is_weightedrrErr8r9r*s$   $ zBinnedCost.__init__rz'Union[NDArray, Tuple[NDArray, NDArray]]rcCs ||S)ai Return the bin-wise expectation for the fitted model. Parameters ---------- args : array-like Parameter values. Returns ------- NDArray Model prediction for each bin. The expectation is always returned for all bins, even if some bins are temporarily masked. )_predrr8r8r9 predictionPszBinnedCost.predictionNonecCs ||S)a Visualize data and model agreement (requires matplotlib). The visualization is drawn with matplotlib.pyplot into the current axes. Parameters ---------- args : sequence of float Parameter values. Notes ----- The automatically provided visualization for multi-dimensional data set is often not very pretty, but still helps to judge whether the fit is reasonable. Since there is no obvious way to draw higher dimensional data with error bars in comparison to a model, the visualization shows all data bins as a single sequence. ) _visualizerr8r8r9rcszBinnedCost.visualizecCsddlm}|\}}||}t|tr0J|jdkr|d}|d}|d}t t |dd}t t | t }n"|j }d|dd|dd}|j|||dd|j||dd d dS) Nrrrrr1r r Tr )fillcolor)rr_n_errrHrrTrreshaper2arangerastyperHrrstairs)rrrrXnerYrrr8r8r9rJxs       zBinnedCost._visualizecCsdSr;r8rr8r8r9rGszBinnedCost._predr>rcCsp|j}|jdur"|}|d}n|d}|dd}|dk}|jdurT|j}tj||<tj||<||fS)Nr1.r.rr)rrBcopyrr2nan)rdrXerrr7r8r8r9rMs       zBinnedCost._n_errcCs"||}|\}}|||Sr;rHrM)rrrYrXrRr8r8r9rs  zBinnedCost._pullscst|j}|jdur||d}|d}t|}|dk}t||||||<t||||<||_|||_n||_dS)NrSrTr) rrrrBr2r3absZmedianrA)rrXvalvarsr7rr8r9rs    zBinnedCost._update_cache)r[rIcCs>|j}|j}|dur||}|j}|dur2||fS|||fSr;rBrrA)rr[r]r7rXr8r8r9 _transformedszBinnedCost._transformedz Tuple[NDArray, NDArray, NDArray])r[r\rIcCsR|j}|j}|dur$||}||}|j}|dur<|||fS|||||dfSr7r^)rr[r\r]r7rXr8r8r9 _transformed2s zBinnedCost._transformed2cCs|jdur|jS|jdS)NrS)rBrrr8r8r9_countss zBinnedCost._counts)rrrrrrrrrXrrrrHrrJrrrGrMrrr_r`rarr8r8rr9r?s* 6 &   r?cseZdZUdZdZded<ded<fddZd d d d d Zd d d ddZd d d ddZ d d d ddZ d d d ddZ ddddZ Z S)BinnedCostWithModelzY Base class for binned cost functions with parametric model. :meta private: ) _xe_shaper _model_xe _model_xm _model_dx _model_lenr _pred_impl np.ndarrayrd"Union[Tuple[int], Tuple[int, ...]]rcc s||_||_|r|rtd|dkr.|j|_n8|dkr@|j|_n&|dkrR|j|_nd|d}t|tt ||||||j dkrt |j f|_ t|j |_|rt|j} | |_|jdd d | |_ntd d |j D|_ td dtj|j ddiD|_|dkrzdd|j D} ddt|j | D} tj| ddi} tj| ddi} t| |_tj| dd|_n|dkrtdt|j |_dS)rz1keywords use_pdf and grad cannot be used togetherZ approximateZ numericalzuse_pdf=zH is not understood, allowed values are '', 'approximate', or 'numerical'rNrr1css|]}t|VqdSr;rrDr8r8r9r rz/BinnedCostWithModel.__init__..cSsg|] }|qSr8flattenrr5r8r8r9 rz0BinnedCostWithModel.__init__..indexingijcSsg|]}t|qSr8)r2diff)rrr8r8r9rprcSs$g|]\}}|ddd|qS)Nrr1r8)rrEZdxir8r8r9rprrrPzDuse_pdf="numerical" is not supported for multidimensional histograms)rrr_pred_approximaterh_pred_numerical _pred_cdfrrrrrrrcrrdr2rsrfrerTvstackmeshgridrrrrg) rrXrrrruse_pdfrr=rrrr8r9rsJ          zBinnedCostWithModel.__init__rrrcCs ||Sr;)rhrr8r8r9rGszBinnedCostWithModel._predcCsf|j|jg|R}t|d|j}|jdkr8||j}t|jD]}tj ||d}qBd||dk<|S)NrrrPr) rrdr-rgrrNrcrUr2rsrrrWrr8r8r9rvs   zBinnedCostWithModel._pred_cdfcCs|j|jg|R}||jSr;)rrerf)rrrAr8r8r9rt+sz%BinnedCostWithModel._pred_approximatecsxddlm}jdksJtjd}tjdD]:}j|}j|d}|fdd||d||<q8|S)Nr)quadrcsj|gRSr;r%r?rr8r9r8rz5BinnedCostWithModel._pred_numerical..)Zscipy.integrater{rr2emptyrgrUrd)rrr{rWrabr8rr9ru/s   z#BinnedCostWithModel._pred_numericalcCsn|j|jg|R}t|d|j|j}|jdkrF||jg|jR}td|jdD]}t j ||d}qV|S)Nr0rrP) rrdr-rrgrrNrcrUr2rsrzr8r8r9 _pred_grad;s zBinnedCostWithModel._pred_gradrrcCs |jduSr;rrr8r8r9rDszBinnedCostWithModel._has_grad)rrrrrrrrGrvrtrurrrr8r8rr9rbs   2   rbcseZdZUdZdZded<ded<ded<d d d d d ddddddfddZdddddZdddddZddddd Z d!d"d#d$Z dd"d%d&Z dddd'd(Z dd)dd*d+Z dddd,d-ZZS).r,u Binned cost function for a template fit with uncertainties on the template. This cost function is for a mixture of components. Use this if the sample originate from two or more components and you are interested in estimating the yield that originates from one or more components. In high-energy physics, one component is often a peaking signal over a smooth background component. A component can be described by a parametric model or a template. A parametric model is accepted in form of a scaled cumulative density function, while a template is a non-parametric shape estimate obtained by histogramming a Monte-Carlo simulation. Even if the Monte-Carlo simulation is asymptotically correct, estimating the shape from a finite simulation sample introduces some uncertainty. This cost function takes that additional uncertainty into account. There are several ways to fit templates and take the sampling uncertainty into account. Barlow and Beeston [1]_ found an exact likelihood for this problem, with one nuisance parameter per component per bin. Solving this likelihood is somewhat challenging though. The Barlow-Beeston likelihood also does not handle the additional uncertainty in weighted templates unless the weights per bin are all equal. Other works [2]_ [3]_ [4]_ describe likelihoods that use only one nuisance parameter per bin, which is an approximation. Some marginalize over the nuisance parameters with some prior, while others profile over the nuisance parameter. This class implements several of these methods. The default method is the one which performs best under most conditions, according to current knowledge. The default may change if this assessment changes. The cost function returns an asymptotically chi-square distributed test statistic, except for the method "asy", where it is the negative logarithm of the marginalised likelihood instead. The standard transform [5]_ which we use convert likelihoods into test statistics only works for (profiled) likelihoods, not for likelihoods marginalized over a prior. All methods implemented here have been generalized to work with both weighted data and weighted templates, under the assumption that the weights are independent of the data. This is not the case for sWeights, and the uncertaintes for results obtained with sWeights will only be approximately correct [6]_. The methods have been further generalized to allow fitting a mixture of parametric models and templates. .. [1] Barlow and Beeston, Comput.Phys.Commun. 77 (1993) 219-228 .. [2] Conway, PHYSTAT 2011 proceeding, https://doi.org/10.48550/arXiv.1103.0354 .. [3] Argüelles, Schneider, Yuan, JHEP 06 (2019) 030 .. [4] Dembinski and Abdelmotteleb, https://doi.org/10.48550/arXiv.2206.12346 .. [5] Baker and Cousins, NIM 221 (1984) 437-442 .. [6] Langenbruch, Eur.Phys.J.C 82 (2022) 5, 393 ) _model_datardrc_implrgz9List[Union[Tuple[NDArray, NDArray], Tuple[Model, float]]]rrirdrjrcNrda)rrmethodrrCz#Collection[Union[Model, ArrayLike]]rrstr)rXrmodel_or_templaterrrcst|}|dkrtdt|}t|} d} ig|_t|D]8\} } t| trt| } | j| kr| j| dks| j dd|krtd| d }| d }n,| j| ks| j |krtd| }| }dt |}||9}||d 9}|j ||fd t jfd | <q>t| trpt| d}t|} |j | | f|D]}||d | d |<qNq>td q>|durtt|krtdfddt|Dttttd}z|||_Wn(tytd|d|Yn0|dkrtjdtd dt||||jdkrPt|jf|_t|j|_n:t dd|jD|_t !ddt j"|jddiD|_t #|j|_$dS)a\ Initialize cost function with data and model. Parameters ---------- n : array-like Histogram counts. If this is an array with dimension D+1, where D is the number of histogram axes, then the last dimension must have two elements and is interpreted as pairs of sum of weights and sum of weights squared. xe : array-like or collection of array-like Bin edge locations, must be len(n) + 1, where n is the number of bins. If the histogram has more than one axis, xe must be a collection of the bin edge locations along each axis. model_or_template : collection of array-like or callable Collection of models or arrays. An array represent the histogram counts of a template. The template histograms must use the same axes as the data histogram. If the counts are represented by an array with dimension D+1, where D is the number of histogram axes, then the last dimension must have two elements and is interpreted as pairs of sum of weights and sum of weights squared. Callables must return the model cdf evaluated as xe. name : sequence of str or None, optional Optional name for the yield of each template and the parameter of each model (in order). Must have the same length as there are templates and model parameters in templates_or_model. Default is None. verbose : int, optional Verbosity level. 0: is no output (default). 1: print current args and negative log-likelihood value. method : {"jsc", "asy", "da"}, optional Which method to use. "jsc": Conway's method [2]_. "asy": ASY method [3]_. "da": DA method [4]_. Default is "da", which to current knowledge offers the best overall performance. The default may change in the future, so please set this parameter explicitly in code that has to be stable. For all methods except the "asy" method, the minimum value is chi-square distributed. rz*at least one template or model is requiredrNrz&shapes of n and templates do not matchrSrTrNrr5rzHmodel_or_template must be a collection of array-likes and/or Model typeszCnumber of names must match number of templates and model parameterscsi|]\}}||qSr8r8)roldnewZ annotatedr8r9 rz%Template.__init__..)Zjscasyhpdrzmethod z$ is not understood, allowed values: rz0key 'hpd' is deprecated, please use 'da' instead)category stacklevelcss|]}t|VqdSr;rlrDr8r8r9rrz$Template.__init__..cSsg|] }|qSr8rmror8r8r9rprz%Template.__init__..rqrr)%rrrFrrrrrrKrrUr2r<rrr rrr"r$r#rKeyErrorwarningswarn FutureWarningrrrrrcrdrTrwrxrrg)rrXrrrrrMrrKrrtttt1t2rVannrdZ known_methodsrrr9rs,           zTemplate.__init__rr>rc Csd}d}d}|jD]\}}t|tjrbt|tjrb||}|||7}||d|7}|d7}qt|tr t|tr ||jg||||R}t|d|j}|j dkr| |j }t |j D]} tj || d}qd||dk<||7}|t|d7}||7}qdsJq||fS)NrrNrrrPgYnF)rrr2ndarrayr rrdr-rgrrNrcrUrsZ ones_like) rrrYrarrrr}rWjr8r8r9rGs,       zTemplate._predrHcCsT||\}}|||\}}}|dk}|||d||d||dS)Nrr)rGr`rrN)rrrYrarXr7r8r8r9rszTemplate._valuercCstdSr;)NotImplementedErrorrr8r8r9r$szTemplate._gradrrcCsdS)NFr8rr8r8r9r'szTemplate._has_gradcCs|jturtStSr;)rr$rrrr8r8r9r*szTemplate._errordefcCs||\}}|t|fS)a= Return the fitted template and its standard deviation. This returns the prediction from the templates, the sum over the products of the template yields with the normalized templates. The standard deviation is returned as the second argument, this is the estimated uncertainty of the fitted template alone. It is obtained via error propagation, taking the statistical uncertainty in the template into account, but regarding the yields as parameters without uncertainty. Parameters ---------- args : array-like Parameter values. Returns ------- y, yerr : NDArray, NDArray Template prediction and its standard deviation, based on the statistical uncertainty of the template only. )rGr2rR)rrrYrar8r8r9rH-szTemplate.predictionrIc Csddlm}|\}}||\}}|jdkr|d}|d}|d}|d}tt|dd}tt| t }n"|j }d|dd|dd}|j |||dddD] } |j |||||| d d qdS) Nrrrrr1r r )FTr )ZbaselinerKrL)rrrMrHrrNr2rOrrPrHrrrQ) rrrrXrRrYmuerrrKr8r8r9rJFs       zTemplate._visualizecCs6||\}}|\}}|||d|ddS)NrNr1rY)rrrYrrXrRr8r8r9r^s zTemplate._pulls)rrrrrrrrGrrrrrHrJrrr8r8rr9r,Hs" 1 ~r,c seZdZdZdZeddZdddddd d d d d dddfddZdddfdd ZdddddZ dddfdd Z Z S)r(a Binned negative log-likelihood. Use this if only the shape of the fitted PDF is of interest and the data is binned. This cost function works with normal and weighted histograms. The histogram can be one- or multi-dimensional. The cost function has a minimum value that is asymptotically chi2-distributed. It is constructed from the log-likelihood assuming a multivariate-normal distribution and using the saturated model as a reference, see :func:`multinomial_chi2` for details. When this class is used with weighted data, we use the Bohm-Zech transform for Poisson-distributed data and the :func:`poisson_chi2` cost function, because :func:`multinomial_chi2` yields biased results for weighted data. The reasoning for this choice is that :func:`multinomial_chi2` and :func:`poisson_chi2` yield the same result for a model which predicts probabilities and expected counts are computed by multiplying the probability with the total number of counts. Thus we can derive :func:`multinomial_chi2` as a special case of :func:`poisson_chi2` in case of unweighted data, but this mathematical equivalence is gone when data are weighted. The correct cost function is then :func:`poisson_chi2`. )_chi2cCs|jS)z Get cumulative density function.r%rr8r8r9cdf}sz BinnedNLL.cdfrNrkrrryrrrCr rrrr)rXrrrrryrc s4t||||||||jdur*t|_nt|_dS)aO Initialize cost function with data and model. Parameters ---------- n : array-like Histogram counts. If this is an array with dimension D+1, where D is the number of histogram axes, then the last dimension must have two elements and is interpreted as pairs of sum of weights and sum of weights squared. xe : array-like or collection of array-like Bin edge locations, must be len(n) + 1, where n is the number of bins. If the histogram has more than one axis, xe must be a collection of the bin edge locations along each axis. cdf : callable Cumulative density function of the form f(xe, par0, par1, ..., parN), where xe is a bin edge and par0, ... are model parameters. The corresponding density must be normalized to unity over the space covered by the histogram. If the model is multivariate, xe must be an array-like with shape (D, N), where D is the dimension and N is the number of points where the model is evaluated. verbose : int, optional Verbosity level. 0: is no output (default). 1: print current args and negative log-likelihood value. grad: callable or None, optional Optionally pass the gradient of the cdf (Default is None). Has the same calling signature like the cdf, but must return an array with the shape (K, N), where N is the number of data points and K is the number of parameters. The gradient can be used by Minuit to improve or speed up convergence. use_pdf: str, optional Either "", "numerical", or "approximate" (Default is ""). If the model cdf is not available, but the model pdf is, this option can be set to "numerical" or "approximate" to compute the integral of the pdf over the bin patch. The option "numerical" uses numerical integration, which is accurate but computationally expensive and only supported for 1D histograms. The option "approximate" uses the zero-order approximation of evaluating the pdf at the bin center, multiplied with the bin area. This is fast and works in higher dimensions, but can lead to biased results if the curvature of the pdf inside the bin is significant. name : sequence of str or None, optional Optional names for each parameter of the model (in order). Must have the same length as there are model parameters. Default is None. N)rrrBr rr!)rrXrrrrryrrr8r9rs5 zBinnedNLL.__init__rrrcs>t|}|j}|dur,|t||}|t|Sr;)rrGrr2r<ra)rrrbr7rr8r9rGs  zBinnedNLL._predrHcCs0||}||\}}||d|dSr)rGr_rrNrrrYrXr8r8r9rs zBinnedNLL._valuec st|}t|}|j}|durft||}|||t|dd|f|d}||}|}t|}||}||} |j}|dur||}| dd|f} |d}|d}| | jdd} |j } | durt ||| S| d} t ||| | S)NrNrr) rrrGrr2r<rarNrrBr`r_) rrZpgrbr7ZpsumrXZntotrYr\r]rr8r9rs.  *     zBinnedNLL._grad) rrrrrrrrrGrrrr8r8rr9r(ds "; r(c sneZdZdZdZeddZdddddd d d d d dddfddZdddddZdddddZ Z S)r*a Binned extended negative log-likelihood. Use this if shape and normalization of the fitted PDF are of interest and the data is binned. This cost function works with normal and weighted histograms. The histogram can be one- or multi-dimensional. The cost function works for both weighted data. The cost function assumes that the weights are independent of the data. This is not the case for sWeights, and the uncertaintes for results obtained with sWeights will only be approximately correct, see C. Langenbruch, Eur.Phys.J.C 82 (2022) 5, 393. The cost function has a minimum value that is asymptotically chi2-distributed. It is constructed from the log-likelihood assuming a poisson distribution and using the saturated model as a reference. r8cCs|jS)zGet integrated density model.r%rr8r8r9 scaled_cdfszExtendedBinnedNLL.scaled_cdfrNrkrrrCr rrrr)rXrrrrryrc st|||||||dS)a Initialize cost function with data and model. Parameters ---------- n : array-like Histogram counts. If this is an array with dimension D+1, where D is the number of histogram axes, then the last dimension must have two elements and is interpreted as pairs of sum of weights and sum of weights squared. xe : array-like or collection of array-like Bin edge locations, must be len(n) + 1, where n is the number of bins. If the histogram has more than one axis, xe must be a collection of the bin edge locations along each axis. scaled_cdf : callable Scaled Cumulative density function of the form f(xe, par0, [par1, ...]), where xe is a bin edge and par0, ... are model parameters. If the model is multivariate, xe must be an array-like with shape (D, N), where D is the dimension and N is the number of points where the model is evaluated. verbose : int, optional Verbosity level. 0: is no output (default). 1: print current args and negative log-likelihood value. grad: callable or None, optional Optionally pass the gradient of the cdf (Default is None). Has the same calling signature like the cdf, but must return an array with the shape (K, N), where N is the number of data points and K is the number of parameters. The gradient can be used by Minuit to improve or speed up convergence. use_pdf: str, optional Either "", "numerical", or "approximate". If the model cdf is not available, but the model pdf is, this option can be set to "numerical" or "approximate" to compute the integral of the pdf over the bin patch. The option "numerical" uses numerical integration, which is accurate but computationally expensive and only supported for 1D histograms. The option "approximate" uses the zero-order approximation of evaluating the pdf at the bin center, multiplied with the bin area. This is fast and works in higher dimensions, but can lead to biased results if the curvature of the pdf inside the bin is significant. name : sequence of str or None, optional Optional names for each parameter of the model (in order). Must have the same length as there are model parameters. Default is None. Nr')rrXrrrrryrrr8r9rs3zExtendedBinnedNLL.__init__rrHrcCs.||}||\}}t|d|dSr)rGr_r!rNrr8r8r9r8s zExtendedBinnedNLL._valuercCs||}||}|j}|dur:||}|dd|f}|d}||jdd}|d}|j}|dur~t|||S|d}t||||S)Nrr)rGrrrNrrarBr_)rrrYr\r7rXr]r8r8r9r=s     zExtendedBinnedNLL._grad) rrrrrrrrrrrr8r8rr9r*s "5r*c szeZdZUdZdZded<ded<ded<d ed <d ed <d ed<eddZejddZeddZ e jddZ eddZ e jddZ eddZ eddZ e jddddZ ddd d d!d"d"d"d dd d d#d$fd%d&Z d'd(ZdBd"d)d*d+d,d-Zd.d/d0d1d2Zd.d/d0d3d4Zd.d/d0d5d6Zd.d/d0d7d8Zd.d9d0d:d;Zd.d/d0dd?d@dAZZS)Cr-a  Least-squares cost function (aka chisquare function). Use this if you have data of the form (x, y +/- yerror), where x can be one-dimensional or multi-dimensional, but y is always one-dimensional. See :meth:`__init__` for details on how to use a multivariate model. )_loss_cost _cost_gradrrrzUnion[str, LossFunction]rz2Callable[[ArrayLike, ArrayLike, ArrayLike], float]rzAOptional[Callable[[NDArray, NDArray, NDArray, NDArray], NDArray]]rr rrrrrcCs.|jdkr|jdddfS|jjd|jS)zGet explanatory variables.rNr)rrr/rr8r8r9r5as zLeastSquares.xcCsL|jdkr"t||jdddf<nt|j|jddd|jf<|dS)Nrr)rrrr/rrr8r8r9r5hs cCs|jdd|jfS)z Get samples.Nrrrr8r8r9rApszLeastSquares.ycCs$t||jdd|jf<|dSr;rrrrrr8r8r9rAuscCs|jdd|jdfS)zGet sample uncertainties.Nrrrr8r8r9yerrorzszLeastSquares.yerrorcCs(t||jdd|jdf<|dSr6rrr8r8r9rscs$tjdkrfddSjSdS)z2Get model of the form y = f(x, par0, [par1, ...]).rcs*t|dkr||Sj|g|RSr6)rr)r5rrr8r9rsz$LeastSquares.model..N)rr}rrr8rr9rs zLeastSquares.modelcCs|jS)zGet loss function.)rrr8r8r9lossszLeastSquares.lossrcsx|_ttrLdkr&t|_t|_qtdkr.z loss must be str or LossFunction) rrrrrrQrrSrWrr )rrr8rr9rs  rrN)rrrrrr)r5rArrrrrrc st|}t|}|jdksJ|jdkr2|jdnd|_||_||_||_t|}t tj g|||R} t t ||| |dS)aw Initialize cost function with data and model. Parameters ---------- x : array-like Locations where the model is evaluated. If the model is multivariate, x must have shape (D, N), where D is the number of dimensions and N the number of data points. y : array-like Observed values. Must have the same length as x. yerror : array-like or float Estimated uncertainty of observed values. Must have same shape as y or be a scalar, which is then broadcasted to same shape as y. model : callable Function of the form f(x, par0, [par1, ...]) whose output is compared to observed values, where x is the location and par0, ... are model parameters. If the model is multivariate, x has shape (D, N), where D is the number of dimensions and N the number of data points. loss : str or callable, optional The loss function can be modified to make the fit robust against outliers, see scipy.optimize.least_squares for details. Only "linear" (default) and "soft_l1" are currently implemented, but users can pass any loss function as this argument. It should be a monotonic, twice differentiable function, which accepts the squared residual and returns a modified squared residual. verbose : int, optional Verbosity level. 0: is no output (default). 1: print current args and negative log-likelihood value. Notes ----- Alternative loss functions make the fit more robust against outliers by weakening the pull of outliers. The mechanical analog of a least-squares fit is a system with attractive forces. The points pull the model towards them with a force whose potential is given by :math:`rho(z)` for a squared-offset :math:`z`. The plot shows the standard potential in comparison with the weaker soft-l1 potential, in which outliers act with a constant force independent of their distance. .. plot:: plots/loss.py rrN)rrKrrrrrr2Z atleast_2dZ column_stackZbroadcast_arraysrrr) rr5rArrrrrrrrr8r9rs5 zLeastSquares.__init__cCs t|jSr;)rrrr8r8r9rszLeastSquares._ndatarz@Tuple[Tuple[NDArray, NDArray, NDArray], Tuple[NDArray, NDArray]])rrrIc sddlm}jdkrtdjj\}}}|j|||ddt|}t |}t |t rzt |} j | gR} n\|dkrt|rt|||} nt|||} j | gR} ntfdd||\} } || | |||f| | ffS) a  Visualize data and model agreement (requires matplotlib). The visualization is drawn with matplotlib.pyplot into the current axes. Parameters ---------- args : array-like Parameter values. model_points : int or array-like, optional How many points to use to draw the model. Default is 0, in this case an smart sampling algorithm selects the number of points. If array-like, it is interpreted as the point locations. rrrrr r csj|gRSr;)rr?rr8r9r rz(LeastSquares.visualize..)rrrrrr/rr2minrrrrrr rrrZplot) rrrrr5rArBZxminZxmaxrrCr8rr9rs$       zLeastSquares.visualizerrrcCs|j|jg|RS)z Return the prediction from the fitted model. Parameters ---------- args : array-like Parameter values. Returns ------- NDArray Model prediction for each bin. )rr5rr8r8r9rH szLeastSquares.predictioncCsP|j}|j}||}|jdurD|j}tj||<tj||<|||Sr;)rArUrrHrr2rV)rrrArBrCr7r8r8r9r' s      zLeastSquares._pullscCsJ|jdkr|jjdn|jjd|j}|j|g|R}t|d|S)Nrrr)rrr/rr-r)rrr5rCr8r8r9rG2 s(zLeastSquares._predcCs`|jdurtd|jdkr(|jjdn|jjd|j}|j|g|R}t|d|j|S)Nr/rrr0)rrrrr/r-rr)rrr5ymgr8r8r9r7 s  (zLeastSquares._pred_gradrHcCs.|jj|jd\}}||}||||Sr;)rr/rrGr)rrrArBrCr8r8r9r> s zLeastSquares._valuecCsL|jdurtd|jj|jd\}}||}||}|||||S)Nzno cost gradient available)rrrr/rrGr)rrrArBrCrr8r8r9rC s    zLeastSquares._gradrrcCs|jduo|jduSr;)rrrr8r8r9rK szLeastSquares._has_grad)r)rrrrrrrr5rrArrrrrrrHrrGrrrrrr8r8rr9r-OsR         $B+ r-cseZdZdZdZddddfdd Zedd Zejd d Zed d Z e jd d Z dddddZ dddddZ ddddZ ddZ ddddZZS)r.a Gaussian penalty for one or several parameters. The Gaussian penalty acts like a pseudo-measurement of the parameter itself, based on a (multi-variate) normal distribution. Penalties can be set for one or several parameters at once (which is more efficient). When several parameter are constrained, one can specify the full covariance matrix of the parameters. Notes ----- It is sometimes necessary to add a weak penalty on a parameter to avoid instabilities in the fit. A typical example in high-energy physics is the fit of a signal peak above some background. If the amplitude of the peak vanishes, the shape parameters of the peak become unconstrained and the fit becomes unstable. This can be avoided by adding weak (large uncertainty) penalty on the shape parameters whose pull is negligible if the peak amplitude is non-zero. This class can also be used to approximately include external measurements of some parameters, if the original cost function is not available or too costly to compute. If the external measurement was performed in the asymptotic limit with a large sample, a Gaussian penalty is an accurate statistical representation of the external result. ) _expected_cov_covinvzUnion[str, Iterable[str]]r)rrerrorcst|tr|fnt|}t|}t||_|jjdkr>td|dkr\t|j|kr\tdt||_t|jt|jkrtd|jjdkr|jdC_n(|jjdkrt |jstdntdt |j|_ t dd |Dd d S) a Initialize the normal constraint with expected value(s) and error(s). Parameters ---------- args : str or sequence of str Parameter name(s). value : float or array-like Expected value(s). Must have same length as `args`. error : float or array-like Expected error(s). If 1D, must have same length as `args`. If 2D, must be the covariance matrix of the parameters. rz)value must be a scalar or one-dimensionalz)size of value does not match size of argsz*size of error does not match size of valuerN*covariance matrix is not positive definitez6covariance matrix cannot have more than two dimensionscSsi|] }|dqSr;r8rrdr8r8r9r rz-NormalConstraint.__init__..FN) rrrTrrrrKrrr rrr)rrrrZtp_argsnargsrr8r9rj s$        zNormalConstraint.__init__cCs|jS)z Get expected covariance of parameters. Can be 1D (diagonal of covariance matrix) or 2D (full covariance matrix). )rrr8r8r9r szNormalConstraint.covariancecCsBt|}|jdkr$t|s$td||jdd<t|j|_dS)NrNr)r2rrKr rrrrr8r8r9r s  cCs|jS)zGet expected parameter values.rrr8r8r9r szNormalConstraint.valuecCs||jdd<dSr;rrr8r8r9r srrHrcCs<||j}|jjdkr*t|d|jStd||j|S)NrNzi,ij,j)rrrKr2r<rrrdeltar8r8r9r s  zNormalConstraint._valuercCs2||j}|jjdkr$d||jSd|j|Sr7)rrrKrr8r8r9r s  zNormalConstraint._gradrrcCsdSrr8rr8r8r9r szNormalConstraint._has_gradcCs t|jSr;)rrrr8r8r9r szNormalConstraint._ndatarc Csddlm}t|}|j}|j}|j}|jdkr.z output of z has shape z, but z is required)rr2rtyperrrrrorrPrHrKrrNrrTr)r5rr=rZ pretty_shaper8r8r9r- s        r-cCs.t|dtr tdd|DSt|dfS)Nrcss|]}t|dVqdS)rNrlrDr8r8r9r rz!_shape_from_xe..r)rrrTr)rr8r8r9rF srFcst|ddt}t|fdd|D}|rt|t|kr\ddt||D}n$t|dkrrtdndd|D}|S) NTrcsi|]}||qSr8r8rrr8r9r rz%_model_parameters..cSsi|]\}}||qSr8r8)rrXZattr8r8r9r rrz8length of name does not match number of model parameterscSsi|] }|dqSr;r8)rrXr8r8r9r r)riternextrrvaluesr)rrrparamsr8rr9r s   r)ZBarlowBeestonLiteZbarlow_beeston_lite_chi2_jscZbarlow_beeston_lite_chi2_hpdZmultinominal_chi2rr)rrIcCsB|tvr:t|\}}tj|d|d|dtdd|StdS)Nz was renamed to z, please import z insteadrN)r)_deprecated_contentrrrAttributeError)rnew_nameobjr8r8r9 __getattr__% s r)fr __future__rutilrrrrr r typingr r r rir2Z numpy.typingrrcollections.abcrZ ABCSequencerrrrrrrrrrrrrZ _deprecatedr__all__rHr/rrZfinfoZtinyr>r:r@rErGrrQrSrWr!r_r r`r"r#r$ZnumbargZnumba.extendingrhZ nb_overloadZjitrmrnrsrrrurtrwrvryrxr{rzModuleNotFoundErrorABCr%r'r&rrrr)r+r?rbr,r(r*r-r.rrr-rFrrrr8r8r8r9sS   8   &-       |41 g}qse