U !`cv@sRdZddlZddlmZddlmZddlm Z m Z ddl m Z m Z mZddl mZmZmZddlmZdd lmZmZmZdd lmZdd lmZmZmZmZdd lm Z dd l!m"Z"ddddgZ#d ddZ$d!ddZ%Gdddee e dZ&Gdddee&Z'Gdddee&Z(Gddde e dZ)Gdddeee)Z*Gdddeee)Z+dS)"a This module implements multioutput regression and classification. The estimators provided in this module are meta-estimators: they require a base estimator to be provided in their constructor. The meta-estimator extends single output estimators to multioutput estimators. N)Parallel)ABCMetaabstractmethod) BaseEstimatorcloneMetaEstimatorMixin)RegressorMixinClassifierMixin is_classifier)cross_val_predict) check_array check_X_ycheck_random_state)if_delegate_has_method)check_is_fittedhas_fit_parameter_check_fit_params_deprecate_positional_args)check_classification_targets)delayedMultiOutputRegressorMultiOutputClassifierClassifierChainRegressorChaincKs>t|}|dk r*|j||fd|i|n|j||f||S)N sample_weight)rfit) estimatorXyr fit_paramsr!2lib/python3.8/site-packages/sklearn/multioutput.py_fit_estimator$s r#TcCsl|r t|}|dk rB|dk r0|j||||dqh|j|||dn&|dk r\|j|||dn ||||S)N)classesrr)r$)r partial_fit)rrrr$r first_timer!r!r"_partial_fit_estimator-s  r(c@sNeZdZeeddddZeddddZddd Zd d Z d d Z dS)_MultiOutputEstimatorNn_jobscCs||_||_dSN)rr+selfrr+r!r!r"__init__Csz_MultiOutputEstimator.__init__rcstdddd\jdkr(tddk rDtjdsDtdtd  tjd fd d tj dD_ S) a]Incrementally fit the model to data. Fit a separate model for each output variable. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Data. y : {array-like, sparse matrix} of shape (n_samples, n_outputs) Multi-output targets. classes : list of ndarray of shape (n_outputs,) Each array is unique classes for one output in str/int Can be obtained by via ``[np.unique(y[:, i]) for i in range(y.shape[1])]``, where y is the target matrix of the entire dataset. This argument is required for the first call to partial_fit and can be omitted in the subsequent calls. Note that y doesn't need to contain all labels in `classes`. sample_weight : array-like of shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Only supported if the underlying regressor supports sample weights. Returns ------- self : object FTforce_all_finite multi_output accept_sparserQy must have at least two dimensions for multi-output regression but has only one.Nr5Underlying estimator does not support sample weights. estimators_r*c3sP|]H}ttsj|njdd|fdk r>|ndVqdSr,)rr(r6r.0irr$r'rr.rr!r" xsz4_MultiOutputEstimator.partial_fit..) rndim ValueErrorrrhasattrrr+rangeshaper6)r.rrr$rr!r:r"r&Is      z!_MultiOutputEstimator.partial_fitc stjdstdjdddd\tr object Parameters passed to the ``estimator.fit`` method of each step. .. versionadded:: 0.23 Returns ------- self : object rz0The base estimator should implement a fit methodFTr0rr4Nrr5r*c3s2|]*}ttjdd|ffVqdSr,)rr#rr7rZfit_params_validatedrr.rr!r"r;sz,_MultiOutputEstimator.fit..)r>rr=_validate_datar rr<rrrr+r?r@r6)r.rrrr r!rAr"rs(      z_MultiOutputEstimator.fitcsVt|t|jdstdtdddt|jdfdd|jD}t |j S) aPredict multi-output variable using a model trained for each target variable. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Data. Returns ------- y : {array-like, sparse matrix} of shape (n_samples, n_outputs) Multi-output targets predicted across multiple predictors. Note: Separate models are generated for each predictor. predictz4The base estimator should implement a predict methodFT)r1r3r*c3s|]}t|jVqdSr,)rrC)r8err!r"r;sz0_MultiOutputEstimator.predict..) rr>rr=r rr+r6npZasarrayT)r.rrr!rEr"rCs z_MultiOutputEstimator.predictcCsddiSNmultioutput_onlyTr!r.r!r!r" _more_tagssz _MultiOutputEstimator._more_tags)NN)N) __name__ __module__ __qualname__rrr/rr&rrCrKr!r!r!r"r)@s 6 9r)) metaclasscs@eZdZdZeddfdd Zedd fdd ZZS) ra-Multi target regression This strategy consists of fitting one regressor per target. This is a simple strategy for extending regressors that do not natively support multi-target regression. .. versionadded:: 0.18 Parameters ---------- estimator : estimator object An estimator object implementing :term:`fit` and :term:`predict`. n_jobs : int or None, optional (default=None) The number of jobs to run in parallel. :meth:`fit`, :meth:`predict` and :meth:`partial_fit` (if supported by the passed estimator) will be parallelized for each target. When individual estimators are fast to train or predict, using ``n_jobs > 1`` can result in slower performance due to the parallelism overhead. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all available processes / threads. See :term:`Glossary ` for more details. .. versionchanged:: 0.20 `n_jobs` default changed from 1 to None Attributes ---------- estimators_ : list of ``n_output`` estimators Estimators used for predictions. Examples -------- >>> import numpy as np >>> from sklearn.datasets import load_linnerud >>> from sklearn.multioutput import MultiOutputRegressor >>> from sklearn.linear_model import Ridge >>> X, y = load_linnerud(return_X_y=True) >>> clf = MultiOutputRegressor(Ridge(random_state=123)).fit(X, y) >>> clf.predict(X[[0]]) array([[176..., 35..., 57...]]) Nr*cstj||ddSNr*superr/r- __class__r!r"r/szMultiOutputRegressor.__init__rcstj|||ddS)atIncrementally fit the model to data. Fit a separate model for each output variable. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Data. y : {array-like, sparse matrix} of shape (n_samples, n_outputs) Multi-output targets. sample_weight : array-like of shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Only supported if the underlying regressor supports sample weights. Returns ------- self : object r%N)rRr&)r.rrrrSr!r"r& s z MultiOutputRegressor.partial_fit)N) rLrMrN__doc__rr/rr& __classcell__r!r!rSr"rs -cs\eZdZdZeddfdd Zdfdd Zedd Zd d Z d d Z ddZ Z S)raMulti target classification This strategy consists of fitting one classifier per target. This is a simple strategy for extending classifiers that do not natively support multi-target classification Parameters ---------- estimator : estimator object An estimator object implementing :term:`fit`, :term:`score` and :term:`predict_proba`. n_jobs : int or None, optional (default=None) The number of jobs to run in parallel. :meth:`fit`, :meth:`predict` and :meth:`partial_fit` (if supported by the passed estimator) will be parallelized for each target. When individual estimators are fast to train or predict, using ``n_jobs > 1`` can result in slower performance due to the parallelism overhead. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all available processes / threads. See :term:`Glossary ` for more details. .. versionchanged:: 0.20 `n_jobs` default changed from 1 to None Attributes ---------- classes_ : ndarray of shape (n_classes,) Class labels. estimators_ : list of ``n_output`` estimators Estimators used for predictions. Examples -------- >>> import numpy as np >>> from sklearn.datasets import make_multilabel_classification >>> from sklearn.multioutput import MultiOutputClassifier >>> from sklearn.neighbors import KNeighborsClassifier >>> X, y = make_multilabel_classification(n_classes=3, random_state=0) >>> clf = MultiOutputClassifier(KNeighborsClassifier()).fit(X, y) >>> clf.predict(X[-2:]) array([[1, 1, 0], [1, 1, 1]]) Nr*cstj||ddSrPrQr-rSr!r"r/VszMultiOutputClassifier.__init__c s*tj|||f|dd|jD|_|S)aFit the model to data matrix X and targets Y. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The input data. Y : array-like of shape (n_samples, n_classes) The target values. sample_weight : array-like of shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Only supported if the underlying classifier supports sample weights. **fit_params : dict of string -> object Parameters passed to the ``estimator.fit`` method of each step. .. versionadded:: 0.23 Returns ------- self : object cSsg|] }|jqSr!classes_r8rr!r!r" qsz-MultiOutputClassifier.fit..)rRrr6rX)r.rYrr rSr!r"rZszMultiOutputClassifier.fitcCs*t|tdd|jDs$td|jS)apProbability estimates. Returns prediction probabilities for each class of each output. This method will raise a ``ValueError`` if any of the estimators do not have ``predict_proba``. Parameters ---------- X : array-like of shape (n_samples, n_features) Data Returns ------- p : array of shape (n_samples, n_classes), or a list of n_outputs such arrays if n_outputs > 1. The class probabilities of the input samples. The order of the classes corresponds to that in the attribute :term:`classes_`. .. versionchanged:: 0.19 This function now returns a list of arrays where the length of the list is ``n_outputs``, and each array is (``n_samples``, ``n_classes``) for that particular output. cSsg|]}t|dqS predict_proba)r>rYr!r!r"rZsz7MultiOutputClassifier.predict_proba..z8The base estimator should implement predict_proba method)rallr6AttributeError_predict_probarJr!r!r"r]ts z#MultiOutputClassifier.predict_probacsfdd|jD}|S)Ncsg|]}|qSr!r\rYrEr!r"rZsz8MultiOutputClassifier._predict_proba..)r6)r.rZresultsr!rEr"r`s z$MultiOutputClassifier._predict_probacCsjt|t|j}|jdkr$td|jd|krHtd||jd||}t tj ||kddS)aReturns the mean accuracy on the given test data and labels. Parameters ---------- X : array-like of shape (n_samples, n_features) Test samples y : array-like of shape (n_samples, n_outputs) True values for X Returns ------- scores : float accuracy_score of self.predict(X) versus y rzTy must have at least two dimensions for multi target classification but has only onezCThe number of outputs of Y for fit {0} and score {1} should be same)Zaxis) rlenr6r<r=r@formatrCrFZmeanr^)r.rrZ n_outputs_Zy_predr!r!r"scores   zMultiOutputClassifier.scorecCsddiS)N _skip_testTr!rJr!r!r"rKsz MultiOutputClassifier._more_tags)N) rLrMrNrUrr/rpropertyr]r`rcrKrVr!r!rSr"r%s0 c@s6eZdZeddddddZeddZddZdS) _BaseChainN)ordercv random_statecCs||_||_||_||_dSr,)base_estimatorrgrhri)r.rjrgrhrir!r!r"r/sz_BaseChain.__init__c  sxj||ddd\}}tj}t|ddj_tjtrNt j_jdkrpt t |j d_nNtjt rjdkr| |j d_n$tjtt |j dkrtdfdd t |j dD_jdkr2|ddjf}t|r"tj||fd d }|}nt||f}nbt|rlt|j d |j df}tj||fd d }n(t|j d |j df}t||f}~tjD]\}}|ddj|f} |j|ddd|j d|f| f|jdk r|tjdkr|j d|} tj|ddd| f| jd } t|r`t| d|dd| f<n| |dd| f<qS)aFit the model to data matrix X and targets Y. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The input data. Y : array-like of shape (n_samples, n_classes) The target values. **fit_params : dict of string -> object Parameters passed to the `fit` method of each step. .. versionadded:: 0.23 Returns ------- self : object T)r2r3r3NrZrandomz invalid ordercsg|]}tjqSr!)rrj)r8_rJr!r"rZsz"_BaseChain.fit..Zlil)rbr)rrh)rBrrir rgorder_ isinstancetuplerFZarrayr?r@strZ permutationsortedlistr=r6rhspissparsehstackZtocsrZ lil_matrixzeros enumeraterrar rjZ expand_dims) r.rr[r ri Y_pred_chainX_aug chain_idxrrZcol_idxZ cv_resultr!rJr"rsZ            $  z_BaseChain.fitc Cst|t|dd}t|jdt|jf}t|jD]h\}}|ddd|f}t |r||dkrl|}qt ||f}nt ||f}| ||dd|f<q8t |j }tt|j ||j <|dd|f}|S)aRPredict on the data matrix X using the ClassifierChain model. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The input data. Returns ------- Y_pred : array-like of shape (n_samples, n_classes) The predicted values. TrkrN)rr rFrvr@rar6rwrsrtrurC empty_likermarange) r.rrxrzrprevious_predictionsry inv_orderZY_predr!r!r"rC s   z_BaseChain.predict)rLrMrNrr/rrrCr!r!r!r"rfs GrfcsHeZdZdZfddZedddZedddZd d ZZ S) raLA multi-label model that arranges binary classifiers into a chain. Each model makes a prediction in the order specified by the chain using all of the available features provided to the model plus the predictions of models that are earlier in the chain. Read more in the :ref:`User Guide `. .. versionadded:: 0.19 Parameters ---------- base_estimator : estimator The base estimator from which the classifier chain is built. order : array-like of shape (n_outputs,) or 'random', default=None If None, the order will be determined by the order of columns in the label matrix Y.:: order = [0, 1, 2, ..., Y.shape[1] - 1] The order of the chain can be explicitly set by providing a list of integers. For example, for a chain of length 5.:: order = [1, 3, 2, 4, 0] means that the first model in the chain will make predictions for column 1 in the Y matrix, the second model will make predictions for column 3, etc. If order is 'random' a random ordering will be used. cv : int, cross-validation generator or an iterable, default=None Determines whether to use cross validated predictions or true labels for the results of previous estimators in the chain. Possible inputs for cv are: - None, to use true labels when fitting, - integer, to specify the number of folds in a (Stratified)KFold, - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. random_state : int, RandomState instance or None, optional (default=None) If ``order='random'``, determines random number generation for the chain order. In addition, it controls the random seed given at each `base_estimator` at each chaining iteration. Thus, it is only used when `base_estimator` exposes a `random_state`. Pass an int for reproducible output across multiple function calls. See :term:`Glossary `. Attributes ---------- classes_ : list A list of arrays of length ``len(estimators_)`` containing the class labels for each estimator in the chain. estimators_ : list A list of clones of base_estimator. order_ : list The order of labels in the classifier chain. Examples -------- >>> from sklearn.datasets import make_multilabel_classification >>> from sklearn.linear_model import LogisticRegression >>> from sklearn.model_selection import train_test_split >>> from sklearn.multioutput import ClassifierChain >>> X, Y = make_multilabel_classification( ... n_samples=12, n_classes=3, random_state=0 ... ) >>> X_train, X_test, Y_train, Y_test = train_test_split( ... X, Y, random_state=0 ... ) >>> base_lr = LogisticRegression(solver='lbfgs', random_state=0) >>> chain = ClassifierChain(base_lr, order='random', random_state=0) >>> chain.fit(X_train, Y_train).predict(X_test) array([[1., 1., 0.], [1., 0., 0.], [0., 1., 0.]]) >>> chain.predict_proba(X_test) array([[0.8387..., 0.9431..., 0.4576...], [0.8878..., 0.3684..., 0.2640...], [0.0321..., 0.9935..., 0.0625...]]) See Also -------- RegressorChain : Equivalent for regression. MultioutputClassifier : Classifies each output independently rather than chaining. References ---------- Jesse Read, Bernhard Pfahringer, Geoff Holmes, Eibe Frank, "Classifier Chains for Multi-label Classification", 2009. cs(t||ddt|jD|_|S)aOFit the model to data matrix X and targets Y. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The input data. Y : array-like of shape (n_samples, n_classes) The target values. Returns ------- self : object cSsg|]\}}|jqSr!rW)r8rzrr!r!r"rZsz'ClassifierChain.fit..)rRrrwr6rX)r.rr[rSr!r"rs zClassifierChain.fitrjc Cst|dd}t|jdt|jf}t|jdt|jf}t|jD]|\}}|ddd|f}t|rt ||f}nt ||f}| |dddf|dd|f<| ||dd|f<qJt |j }tt|j ||j <|dd|f} | S)zPredict probability estimates. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Returns ------- Y_prob : array-like of shape (n_samples, n_classes) TrkrNr)r rFrvr@rar6rwrsrtrur]rCr{rmr|) r.rZ Y_prob_chainrxrzrr}ryr~ZY_probr!r!r"r]s  " zClassifierChain.predict_probac Cst|jdt|jf}t|jdt|jf}t|jD]p\}}|ddd|f}t|rtt||f}nt||f}| ||dd|f<| ||dd|f<q>t |j }t t|j ||j <|dd|f} | S)adEvaluate the decision_function of the models in the chain. Parameters ---------- X : array-like of shape (n_samples, n_features) Returns ------- Y_decision : array-like of shape (n_samples, n_classes) Returns the decision function of the sample for each model in the chain. rN)rFrvr@rar6rwrsrtrudecision_functionrCr{rmr|) r.rZY_decision_chainrxrzrr}ryr~Z Y_decisionr!r!r"rs  z!ClassifierChain.decision_functioncCs dddS)NT)rdrIr!rJr!r!r"rKszClassifierChain._more_tags) rLrMrNrUrrr]rrKrVr!r!rSr"r.sb   cs(eZdZdZfddZddZZS)ra: A multi-label model that arranges regressions into a chain. Each model makes a prediction in the order specified by the chain using all of the available features provided to the model plus the predictions of models that are earlier in the chain. Read more in the :ref:`User Guide `. .. versionadded:: 0.20 Parameters ---------- base_estimator : estimator The base estimator from which the classifier chain is built. order : array-like of shape (n_outputs,) or 'random', default=None If None, the order will be determined by the order of columns in the label matrix Y.:: order = [0, 1, 2, ..., Y.shape[1] - 1] The order of the chain can be explicitly set by providing a list of integers. For example, for a chain of length 5.:: order = [1, 3, 2, 4, 0] means that the first model in the chain will make predictions for column 1 in the Y matrix, the second model will make predictions for column 3, etc. If order is 'random' a random ordering will be used. cv : int, cross-validation generator or an iterable, default=None Determines whether to use cross validated predictions or true labels for the results of previous estimators in the chain. Possible inputs for cv are: - None, to use true labels when fitting, - integer, to specify the number of folds in a (Stratified)KFold, - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. random_state : int, RandomState instance or None, optional (default=None) If ``order='random'``, determines random number generation for the chain order. In addition, it controls the random seed given at each `base_estimator` at each chaining iteration. Thus, it is only used when `base_estimator` exposes a `random_state`. Pass an int for reproducible output across multiple function calls. See :term:`Glossary `. Attributes ---------- estimators_ : list A list of clones of base_estimator. order_ : list The order of labels in the classifier chain. Examples -------- >>> from sklearn.multioutput import RegressorChain >>> from sklearn.linear_model import LogisticRegression >>> logreg = LogisticRegression(solver='lbfgs',multi_class='multinomial') >>> X, Y = [[1, 0], [0, 1], [1, 1]], [[0, 2], [1, 1], [2, 0]] >>> chain = RegressorChain(base_estimator=logreg, order=[0, 1]).fit(X, Y) >>> chain.predict(X) array([[0., 2.], [1., 1.], [2., 0.]]) See Also -------- ClassifierChain : Equivalent for classification. MultioutputRegressor : Learns each output independently rather than chaining. c stj||f||S)aFit the model to data matrix X and targets Y. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The input data. Y : array-like of shape (n_samples, n_classes) The target values. **fit_params : dict of string -> object Parameters passed to the `fit` method at each step of the regressor chain. .. versionadded:: 0.23 Returns ------- self : object )rRr)r.rr[r rSr!r"r6szRegressorChain.fitcCsddiSrHr!rJr!r!r"rKMszRegressorChain._more_tags)rLrMrNrUrrKrVr!r!rSr"rsO )N)NNT),rUZnumpyrFZ scipy.sparseZsparsersZjoblibrabcrrbaserrrr r r Zmodel_selectionr Zutilsr rrZutils.metaestimatorsrZutils.validationrrrrZutils.multiclassrZ utils.fixesr__all__r#r(r)rrrfrrr!r!r!r"s@         Lt9