U !`|@sdZddlZddlmZddlmZddlmZddlm Z ddl Z ddl m Z ddlmZdd lmZdd lmZd d lmZmZmZmZmZd d lmZmZd dlmZmZmZm Z d dl!m"Z"d dl#m$Z$d dl%m&Z&m'Z'd dl%m(Z(d dl)m*Z*d dl+m,Z,d dl-m.Z.d dl/m0Z0m1Z1d dl%m2Z2GdddeeeZ3d.ddZ4ddZ5ddZ6d/d d!Z7Gd"d#d#Z8d0d$d%Z9Gd&d'd'eeZ:e2d(d)d*d+d,d-Z;dS)1z'Calibration of predicted probabilities.N) signature)suppress)partial)log)Parallel)expit)xlogy) fmin_bfgs) BaseEstimatorClassifierMixinRegressorMixincloneMetaEstimatorMixin)label_binarize LabelEncoder) check_array column_or_1d deprecated indexable)check_classification_targets)delayed)check_is_fittedcheck_consistent_length)_check_sample_weight)Pipeline)IsotonicRegression) LinearSVC)check_cvcross_val_predict)_deprecate_positional_argsc@sLeZdZdZeddddddddZddd Zd d Zd d ZddZ dS)CalibratedClassifierCVaProbability calibration with isotonic regression or logistic regression. This class uses cross-validation to both estimate the parameters of a classifier and subsequently calibrate a classifier. With default `ensemble=True`, for each cv split it fits a copy of the base estimator to the training subset, and calibrates it using the testing subset. For prediction, predicted probabilities are averaged across these individual calibrated classifiers. When `ensemble=False`, cross-validation is used to obtain unbiased predictions, via :func:`~sklearn.model_selection.cross_val_predict`, which are then used for calibration. For prediction, the base estimator, trained using all the data, is used. This is the method implemented when `probabilities=True` for :mod:`sklearn.svm` estimators. Already fitted classifiers can be calibrated via the parameter `cv="prefit"`. In this case, no cross-validation is used and all provided data is used for calibration. The user has to take care manually that data for model fitting and calibration are disjoint. The calibration is based on the :term:`decision_function` method of the `base_estimator` if it exists, else on :term:`predict_proba`. Read more in the :ref:`User Guide `. Parameters ---------- base_estimator : estimator instance, default=None The classifier whose output need to be calibrated to provide more accurate `predict_proba` outputs. The default classifier is a :class:`~sklearn.svm.LinearSVC`. method : {'sigmoid', 'isotonic'}, default='sigmoid' The method to use for calibration. Can be 'sigmoid' which corresponds to Platt's method (i.e. a logistic regression model) or 'isotonic' which is a non-parametric approach. It is not advised to use isotonic calibration with too few calibration samples ``(<<1000)`` since it tends to overfit. cv : int, cross-validation generator, iterable or "prefit", default=None Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the default 5-fold cross-validation, - integer, to specify the number of folds. - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. For integer/None inputs, if ``y`` is binary or multiclass, :class:`~sklearn.model_selection.StratifiedKFold` is used. If ``y`` is neither binary nor multiclass, :class:`~sklearn.model_selection.KFold` is used. Refer to the :ref:`User Guide ` for the various cross-validation strategies that can be used here. If "prefit" is passed, it is assumed that `base_estimator` has been fitted already and all data is used for calibration. .. versionchanged:: 0.22 ``cv`` default value if None changed from 3-fold to 5-fold. n_jobs : int, default=None Number of jobs to run in parallel. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. Base estimator clones are fitted in parallel across cross-validation iterations. Therefore parallelism happens only when `cv != "prefit"`. See :term:`Glossary ` for more details. .. versionadded:: 0.24 ensemble : bool, default=True Determines how the calibrator is fitted when `cv` is not `'prefit'`. Ignored if `cv='prefit'`. If `True`, the `base_estimator` is fitted using training data and calibrated using testing data, for each `cv` fold. The final estimator is an ensemble of `n_cv` fitted classifer and calibrator pairs, where `n_cv` is the number of cross-validation folds. The output is the average predicted probabilities of all pairs. If `False`, `cv` is used to compute unbiased predictions, via :func:`~sklearn.model_selection.cross_val_predict`, which are then used for calibration. At prediction time, the classifier used is the `base_estimator` trained on all the data. Note that this method is also internally implemented in :mod:`sklearn.svm` estimators with the `probabilities=True` parameter. .. versionadded:: 0.24 Attributes ---------- classes_ : ndarray of shape (n_classes,) The class labels. calibrated_classifiers_ : list (len() equal to cv or 1 if `cv="prefit"` or `ensemble=False`) The list of classifier and calibrator pairs. - When `cv="prefit"`, the fitted `base_estimator` and fitted calibrator. - When `cv` is not "prefit" and `ensemble=True`, `n_cv` fitted `base_estimator` and calibrator pairs. `n_cv` is the number of cross-validation folds. - When `cv` is not "prefit" and `ensemble=False`, the `base_estimator`, fitted on all the data, and fitted calibrator. .. versionchanged:: 0.24 Single calibrated classifier case when `ensemble=False`. Examples -------- >>> from sklearn.datasets import make_classification >>> from sklearn.naive_bayes import GaussianNB >>> from sklearn.calibration import CalibratedClassifierCV >>> X, y = make_classification(n_samples=100, n_features=2, ... n_redundant=0, random_state=42) >>> base_clf = GaussianNB() >>> calibrated_clf = CalibratedClassifierCV(base_estimator=base_clf, cv=3) >>> calibrated_clf.fit(X, y) CalibratedClassifierCV(base_estimator=GaussianNB(), cv=3) >>> len(calibrated_clf.calibrated_classifiers_) 3 >>> calibrated_clf.predict_proba(X)[:5, :] array([[0.110..., 0.889...], [0.072..., 0.927...], [0.928..., 0.071...], [0.928..., 0.071...], [0.071..., 0.928...]]) >>> from sklearn.model_selection import train_test_split >>> X, y = make_classification(n_samples=100, n_features=2, ... n_redundant=0, random_state=42) >>> X_train, X_calib, y_train, y_calib = train_test_split( ... X, y, random_state=42 ... ) >>> base_clf = GaussianNB() >>> base_clf.fit(X_train, y_train) GaussianNB() >>> calibrated_clf = CalibratedClassifierCV( ... base_estimator=base_clf, ... cv="prefit" ... ) >>> calibrated_clf.fit(X_calib, y_calib) CalibratedClassifierCV(base_estimator=GaussianNB(), cv='prefit') >>> len(calibrated_clf.calibrated_classifiers_) 1 >>> calibrated_clf.predict_proba([[-0.5, 0.5]]) array([[0.936..., 0.063...]]) References ---------- .. [1] Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers, B. Zadrozny & C. Elkan, ICML 2001 .. [2] Transforming Classifier Scores into Accurate Multiclass Probability Estimates, B. Zadrozny & C. Elkan, (KDD 2002) .. [3] Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods, J. Platt, (1999) .. [4] Predicting Good Probabilities with Supervised Learning, A. Niculescu-Mizil & R. Caruana, ICML 2005 NsigmoidT)methodcvn_jobsensemblecCs"||_||_||_||_||_dSN)base_estimatorr#r$r%r&)selfr(r#r$r%r&r*2lib/python3.8/site-packages/sklearn/calibration.py__init__s zCalibratedClassifierCV.__init__c stt\jdkr,tddnjg_jdkrtjtr^tjdn tjt t j _ W5QRXjj _ t }tj }t||}t|j j}j|nƈjdddgd d d \t}|j _ tj }tjj} d | kdk rXtsXtj} td | dtjtrnjntjdrjjndrt !fddj Drt"dddt#jd d} j$rt%j&d} | fdd| 'D_nt(} t | j}t)t*| | |j&d}t||}dk rjrj| n | t| |j j}j|S)aFit the calibrated model. Parameters ---------- X : array-like of shape (n_samples, n_features) Training data. y : array-like of shape (n_samples,) Target values. sample_weight : array-like of shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Returns ------- self : object Returns an instance of self. Nr)Z random_stateZprefitcsccsrcooFT) accept_sparseforce_all_finiteZallow_nd sample_weightzSince z^ does not support sample_weights, sample weights will only be used for the calibration itself.n_splitscsg|]}t|kkqSr*)npsum).0Zclass_)n_foldsyr*r+ -sz.CalibratedClassifierCV.fit..z Requesting z.-fold cross-validation but provided less than z! examples for at least one class.)Z classifier)r%c 3s8|]0\}}ttt||jjd VqdS))traintestr#classes supports_swr3N)r_fit_classifier_calibrator_pairrr#classes_)r7r;r<)Xr(r3r)r>r9r*r+ 7sz-CalibratedClassifierCV.fit..) estimatorrAr9r$r#r%)+rrr(rcalibrated_classifiers_r$ isinstancerrrAttributeErrorZn_features_in_r@_get_prediction_methodlen_compute_predictions_fit_calibratorr#appendZ_validate_datarfitr parametersrtype__name__warningswarninthasattrr4r5any ValueErrorrr&rr%splitrrr)r)rAr9r3 pred_method n_classes predictionscalibrated_classifierZlabel_encoder_Zfit_parametersZestimator_namer$ZparallelZthis_estimator method_namer*)rAr(r8r3r)r>r9r+rLs                        zCalibratedClassifierCV.fitcCsft|t|dddgdd}t|jdt|jf}|jD]}||}||7}q<|t|j}|S)aCalibrated probabilities of classification. This function returns calibrated probabilities of classification according to each class on an array of test vectors X. Parameters ---------- X : array-like of shape (n_samples, n_features) The samples. Returns ------- C : ndarray of shape (n_samples, n_classes) The predicted probas. r.r/r0F)r1r2r) rrr5zerosshaperHr@rD predict_proba)r)rAZ mean_probarZprobar*r*r+r^Ss    z$CalibratedClassifierCV.predict_probacCs"t||jtj||ddS)aPredict the target of new samples. The predicted class is the class that has the highest probability, and can thus be different from the prediction of the uncalibrated classifier. Parameters ---------- X : array-like of shape (n_samples, n_features) The samples. Returns ------- C : ndarray of shape (n_samples,) The predicted class. r Zaxis)rr@r5Zargmaxr^)r)rAr*r*r+predictqszCalibratedClassifierCV.predictcCs dddiiS)NZ _xfail_checksZcheck_sample_weights_invariancez8zero sample_weight is not equivalent to removing samplesr*r)r*r*r+ _more_tagss z!CalibratedClassifierCV._more_tags)N)N) rO __module__ __qualname____doc__r r,rLr^rarcr*r*r*r+r!+s( ur!c Cs|dk r*|r*|j||||||dn|||||t|} t|} t| ||| } |dkrjdn||} t|| ||||| d} | S)a Fit a classifier/calibration pair on a given train/test split. Fit the classifier on the train set, compute its predictions on the test set and use the predictions as input to fit the calibrator along with the test labels. Parameters ---------- estimator : estimator instance Cloned base estimator. X : array-like, shape (n_samples, n_features) Sample data. y : array-like, shape (n_samples,) Targets. train : ndarray, shape (n_train_indicies,) Indices of the training subset. test : ndarray, shape (n_test_indicies,) Indices of the testing subset. supports_sw : bool Whether or not the `estimator` supports sample weights. method : {'sigmoid', 'isotonic'} Method to use for calibration. classes : ndarray, shape (n_classes,) The target classes. sample_weight : array-like, default=None Sample weights for `X`. Returns ------- calibrated_classifier : _CalibratedClassifier instance N)r3)rLrHrGrIrJ)rCrAr9r;r<r>r#r=r3rXrWrYswrZr*r*r+r?s$) r?cCs8t|drt|d}nt|dr,t|d}ntd|S)acReturn prediction method. `decision_function` method of `clf` returned, if it exists, otherwise `predict_proba` method returned. Parameters ---------- clf : Estimator instance Fitted classifier to obtain the prediction method from. Returns ------- prediction_method : callable The prediction method. decision_functionr^zF'base_estimator' has no 'decision_function' or 'predict_proba' method.)rSgetattr RuntimeError)clfr#r*r*r+rGs     rGcCs||d}t|dr|j}nt|jdj}|dkrR|jdkr|ddtjf}n4|dkrx|dkr|ddddf}ntd ||S) aReturn predictions for `X` and reshape binary outputs to shape (n_samples, 1). Parameters ---------- pred_method : callable Prediction method. X : array-like or None Data used to obtain predictions. n_classes : int Number of classes present. Returns ------- predictions : array-like, shape (X.shape[0], len(clf.classes_)) The predictions. Note if there are 2 classes, array is of shape (X.shape[0], 1). )rArOr#rhr Nr^zInvalid prediction method: ) rSrOrrMdefaultndimr5newaxisrU)rWrArXrYr[r*r*r+rIs   rIcCst||d}t|}||j}g} t||jD]`\} } |dkrPtdd} n |dkr`t} nt d|d| | |dd| f|| | q4t || ||d } | S) aFit calibrator(s) and return a `_CalibratedClassifier` instance. `n_classes` (i.e. `len(clf.classes_)`) calibrators are fitted. However, if `n_classes` equals 2, one calibrator is fitted. Parameters ---------- clf : estimator instance Fitted classifier. predictions : array-like, shape (n_samples, n_classes) or (n_samples, 1) when binary. Raw predictions returned by the un-calibrated base classifier. y : array-like, shape (n_samples,) The targets. classes : ndarray, shape (n_classes,) All the prediction classes. method : {'sigmoid', 'isotonic'} The method to use for calibration. sample_weight : ndarray, shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Returns ------- pipeline : _CalibratedClassifier instance r=isotonicZclip)Z out_of_boundsr"z8'method' should be one of: 'sigmoid' or 'isotonic'. Got .N)r#r=) rrrL transformr@zipTr_SigmoidCalibrationrUrK_CalibratedClassifier)rkrYr9r=r#r3Y label_encoderpos_class_indices calibrators class_idx this_pred calibratorpipeliner*r*r+rJs&     rJc@s:eZdZdZddddZededdZd d Zd S) rwaPipeline-like chaining a fitted classifier and its fitted calibrators. Parameters ---------- base_estimator : estimator instance Fitted classifier. calibrators : list of fitted estimator instances List of fitted calibrators (either 'IsotonicRegression' or '_SigmoidCalibration'). The number of calibrators equals the number of classes. However, if there are 2 classes, the list contains only one fitted calibrator. classes : array-like of shape (n_classes,) All the prediction classes. method : {'sigmoid', 'isotonic'}, default='sigmoid' The method to use for calibration. Can be 'sigmoid' which corresponds to Platt's method or 'isotonic' which is a non-parametric approach based on isotonic regression. Attributes ---------- calibrators_ : list of fitted estimator instances Same as `calibrators`. Exposed for backward-compatibility. Use `calibrators` instead. .. deprecated:: 0.24 `calibrators_` is deprecated from 0.24 and will be removed in 1.1 (renaming of 0.26). Use `calibrators` instead. r")r#cCs||_||_||_||_dSr')r(r{r=r#)r)r(r{r=r#r*r*r+r,\sz_CalibratedClassifier.__init__zicalibrators_ is deprecated in 0.24 and will be removed in 1.1(renaming of 0.26). Use calibrators instead.cCs|jSr')r{rbr*r*r+ calibrators_esz"_CalibratedClassifier.calibrators_c Cst|j}t|j}t|||}t|j}||jj}t |j d|f}t ||j |jD]0\}} } |dkrz|d7}| | |dd|f<q`|dkrd|dddf|dddf<n |t j|ddddt jf}d||t |<d|d|k|dk@<|S)aCalculate calibrated probabilities. Calculates classification calibrated probabilities for each class, in a one-vs-all manner, for `X`. Parameters ---------- X : ndarray of shape (n_samples, n_features) The sample data. Returns ------- proba : array, shape (n_samples, n_classes) The predicted probabilities. Can be exact zeros. rrlr N?r`grZ| ?)rHr=rGr(rIrrLrsr@r5r\r]rtrur{rar6roZisnan) r)rArXrWrYryrzr_r|r}r~r*r*r+r^ms&    " z#_CalibratedClassifier.predict_probaN) rOrdrerfr,rpropertyrr^r*r*r*r+rw<s   rwc st|}t|}|tt|dk}|jd|}t|j|d|d|dk<d|d|dk<dfdd}fdd}tdt|d|dg}t|||d d }|d|d fS) aNProbability Calibration with sigmoid method (Platt 2000) Parameters ---------- predictions : ndarray of shape (n_samples,) The decision function or predict proba for the samples. y : ndarray of shape (n_samples,) The targets. sample_weight : array-like of shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Returns ------- a : float The slope. b : float The intercept. References ---------- Platt, "Probabilistic Outputs for Support Vector Machines" rrg@csTt|d|d }t|td| }dk rH|S|SdS)Nrr r)rrr6)ABPZlossFruZT1r3r*r+ objectives  z'_sigmoid_calibration..objectivecsVt|d|d }|}dk r2|9}t|}t|}t||gS)Nrr )rr5dotr6array)rrZ TEP_minus_T1PZdAZdB)rrur3r*r+grads  z"_sigmoid_calibration..gradF)ZfprimeZdispr ) rfloatr5r6r]r\rrr ) rYr9r3Zprior0Zprior1rrZAB0ZAB_r*rr+_sigmoid_calibrations   rc@s"eZdZdZdddZddZdS)rvzSigmoid regression model. Attributes ---------- a_ : float The slope. b_ : float The intercept. NcCs6t|}t|}t||\}}t|||\|_|_|S)aFit the model using X, y as training data. Parameters ---------- X : array-like of shape (n_samples,) Training data. y : array-like of shape (n_samples,) Training target. sample_weight : array-like of shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Returns ------- self : object Returns an instance of self. )rrra_b_)r)rAr9r3r*r*r+rLs z_SigmoidCalibration.fitcCst|}t|j||j S)aPredict new data by linear interpolation. Parameters ---------- T : array-like of shape (n_samples,) Data to predict from. Returns ------- T_ : ndarray of shape (n_samples,) The predicted data. )rrrr)r)rur*r*r+ras z_SigmoidCalibration.predict)N)rOrdrerfrLrar*r*r*r+rvs rvFuniform) normalizen_binsstrategycCspt|}t|}t|||r<||||}n |dksT|dkr\tdt|}t|dkr~td|t||ddddf}|dkrt dd|d}t ||d }|d d |d <n$|d krt d d|d}ntdt ||d}tj ||t|d} tj ||t|d} tj |t|d} | dk} | | | | } | | | | }| |fS)a Compute true and predicted probabilities for a calibration curve. The method assumes the inputs come from a binary classifier, and discretize the [0, 1] interval into bins. Calibration curves may also be referred to as reliability diagrams. Read more in the :ref:`User Guide `. Parameters ---------- y_true : array-like of shape (n_samples,) True targets. y_prob : array-like of shape (n_samples,) Probabilities of the positive class. normalize : bool, default=False Whether y_prob needs to be normalized into the [0, 1] interval, i.e. is not a proper probability. If True, the smallest value in y_prob is linearly mapped onto 0 and the largest one onto 1. n_bins : int, default=5 Number of bins to discretize the [0, 1] interval. A bigger number requires more data. Bins with no samples (i.e. without corresponding values in `y_prob`) will not be returned, thus the returned arrays may have less than `n_bins` values. strategy : {'uniform', 'quantile'}, default='uniform' Strategy used to define the widths of the bins. uniform The bins have identical widths. quantile The bins have the same number of samples and depend on `y_prob`. Returns ------- prob_true : ndarray of shape (n_bins,) or smaller The proportion of samples whose class is the positive class, in each bin (fraction of positives). prob_pred : ndarray of shape (n_bins,) or smaller The mean predicted probability in each bin. References ---------- Alexandru Niculescu-Mizil and Rich Caruana (2005) Predicting Good Probabilities With Supervised Learning, in Proceedings of the 22nd International Conference on Machine Learning (ICML). See section 4 (Qualitative Analysis of Predictions). Examples -------- >>> import numpy as np >>> from sklearn.calibration import calibration_curve >>> y_true = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1]) >>> y_pred = np.array([0.1, 0.2, 0.3, 0.4, 0.65, 0.7, 0.8, 0.9, 1.]) >>> prob_true, prob_pred = calibration_curve(y_true, y_pred, n_bins=3) >>> prob_true array([0. , 0.5, 1. ]) >>> prob_pred array([0.2 , 0.525, 0.85 ]) rr z?y_prob has values outside [0, 1] and normalize is set to False.rlzrrg1?zSInvalid entry to 'strategy' input. Strategy must be either 'quantile' or 'uniform'.)Zweights minlength)r) rrminmaxrUr5uniquerHrZlinspaceZ percentileZdigitizeZbincount)Zy_trueZy_probrrrlabelsZ quantilesZbinsZbinidsZbin_sumsZbin_trueZ bin_totalZnonzeroZ prob_trueZ prob_predr*r*r+calibration_curves8C   r)N)N)N)sP                 d :' 5b ?6