U oe+@sdZddlmZddlZz ddlZWn*ek rNddlmZeddYnXddlm Z e de Gdd d Z d d Z d d Z ddZddZddZddZddZddZddZd%d d!Zed"krdd#lmZedd$dS)&z8Maximum Entropy code. Uses Improved Iterative Scaling. )reduceN)MissingPythonDependencyErrorzQPlease install NumPy if you want to use Bio.MaxEntropy. See http://www.numpy.org/)BiopythonDeprecationWarningzThe 'Bio.MaxEntropy' module is deprecated and will be removed in a future release of Biopython. Consider using scikit-learn instead.c@seZdZdZddZdS) MaxEntropyaHold information for a Maximum Entropy classifier. Members: classes List of the possible classes of data. alphas List of the weights for each feature. feature_fns List of the feature functions. Car data from example Naive Bayes Classifier example by Eric Meisner November 22, 2003 http://www.inf.u-szeged.hu/~ormandi/teaching >>> from Bio.MaxEntropy import train, classify >>> xcar = [ ... ['Red', 'Sports', 'Domestic'], ... ['Red', 'Sports', 'Domestic'], ... ['Red', 'Sports', 'Domestic'], ... ['Yellow', 'Sports', 'Domestic'], ... ['Yellow', 'Sports', 'Imported'], ... ['Yellow', 'SUV', 'Imported'], ... ['Yellow', 'SUV', 'Imported'], ... ['Yellow', 'SUV', 'Domestic'], ... ['Red', 'SUV', 'Imported'], ... ['Red', 'Sports', 'Imported']] >>> ycar = ['Yes','No','Yes','No','Yes','No','Yes','No','No','Yes'] Requires some rules or features >>> def udf1(ts, cl): ... return ts[0] != 'Red' ... >>> def udf2(ts, cl): ... return ts[1] != 'Sports' ... >>> def udf3(ts, cl): ... return ts[2] != 'Domestic' ... >>> user_functions = [udf1, udf2, udf3] # must be an iterable type >>> xe = train(xcar, ycar, user_functions) >>> for xv, yv in zip(xcar, ycar): ... xc = classify(xe, xv) ... print('Pred: %s gives %s y is %s' % (xv, xc, yv)) ... Pred: ['Red', 'Sports', 'Domestic'] gives No y is Yes Pred: ['Red', 'Sports', 'Domestic'] gives No y is No Pred: ['Red', 'Sports', 'Domestic'] gives No y is Yes Pred: ['Yellow', 'Sports', 'Domestic'] gives No y is No Pred: ['Yellow', 'Sports', 'Imported'] gives No y is Yes Pred: ['Yellow', 'SUV', 'Imported'] gives No y is No Pred: ['Yellow', 'SUV', 'Imported'] gives No y is Yes Pred: ['Yellow', 'SUV', 'Domestic'] gives No y is No Pred: ['Red', 'SUV', 'Imported'] gives No y is No Pred: ['Red', 'Sports', 'Imported'] gives No y is Yes cCsg|_g|_g|_dS)zInitialize the class.N)classesalphas feature_fns)selfr -lib/python3.8/site-packages/Bio/MaxEntropy.py__init__ZszMaxEntropy.__init__N)__name__ __module__ __qualname____doc__r r r r r r$s5rcCsdg}t|jt|jkst|jD]<}d}t|j|jD]\}}|||||7}q8||q"|S)zCalculate the log of the probability for each class. me is a MaxEntropy object that has been trained. observation is a vector representing the observed data. The return value is a list of unnormalized log probabilities for each class. )lenrrAssertionErrorrzipappend)me observationscoresklassZlprobfnalphar r r calculateas  rcCsVt||}|d|jd}}tdt|D]$}|||kr,|||j|}}q,|S)z%Classify an observation into a class.r)rrranger)rrrZ max_scorerir r r classifyrs   r cCsRi}tt|D]<}tt|D]*}|||||}|dkr ||||f<q q|S)a>Evaluate a feature function on every instance of the training set and class (PRIVATE). fn is a callback function that takes two parameters: a training instance and a class. Return a dictionary of (training set index, class index) -> non-zero value. Values of 0 are not stored in the dictionary. r)rr)rxsrvaluesrjfr r r _eval_feature_fn|sr%c sit|D]\}}||<q fdd|D}g}t|}|D]<} d} t|D]} | | | || fd7} qP|| |q@|S)zCalculate the expectation of each function from the data (PRIVATE). This is the constraint for the maximum entropy distribution. Return a list of expectations, parallel to the list of features. csg|] }|qSr r ).0yZ class2indexr r sz+_calc_empirical_expects..r) enumeraterrgetr) r!ysrfeaturesindexkeyZys_iZexpectNfeaturesrr r(r _calc_empirical_expectss  r3c Cs`t||||}g}|D]D}d}|D] \\}} } |||| | 7}q&||t|q|S)zCalculate the expectation of each feature from the model (PRIVATE). This is not used in maximum entropy training, but provides a good function for debugging. r)_calc_p_class_given_xitemsrr) r!rr-rp_yxZexpectsr1sumrr#r$r r r _calc_model_expectssr8c Cstt|t|f}t|t|ks*tt||D]6\}}|D]$\\}}} ||||| 7<qDq4t|}tt|D] } t|| } || | || <q|S)zCalculate conditional probability P(y|x) (PRIVATE). y is the class and x is an instance from the training set. Return a XSxCLASSES matrix of probabilities. ) npzerosrrrr5exprr7) r!rr-rprob_yxr1rxr'r$rzr r r r4s  r4cCsFt||f}|D].}|D] \\}}}||||7<qq|S)z/Calculate a matrix of f sharp values (PRIVATE).)r9r:r5)r0Znclassesr-f_sharpr1rr#r$r r r _calc_f_sharps r@cCsd}d}||krd} } |D]N\\} } } || | | t||| | }| |7} | ||| | 7} q || || |} } | | }||8}t||krq|d}qtd|S)z,Solve delta using Newton's method (PRIVATE).rrrz Newton's method did not converge)r5r9r;fabs RuntimeError)r0r1r?Z empiricalr<max_newton_iterationsnewton_convergedeltaitersZf_newtonZ df_newtonrr#r$ZprodZratior r r _iis_solve_deltas & rGc Csdt||||}t|} |dd} tt|D]0} t| || ||| |||} | | | 7<q.| S)zBDo one iteration of hill climbing to find better alphas (PRIVATE).N)r4rrrG) r!rr-r?r e_empiricalrCrDr6r0Z newalphasrrEr r r _train_iiss  rI'h㈵>d绽|=c ss tdtt|kr$td|}} tt|fdd|D} ttt| } t|| | } dgt| } d}||krt|| | | | ||}ddt| |D}tt j |d}|} t }| ||_ |_ |_|dk r||||krq qtd |S) aFTrain a maximum entropy classifier, returns MaxEntropy object. Train a maximum entropy classifier on a training set. training_set is a list of observations. results is a list of the class assignments for each observation. feature_fns is a list of the features. These are callback functions that take an observation and class and return a 1 or 0. update_fn is a callback function that is called at each training iteration. It is passed a MaxEntropy object that encapsulates the current state of the training. The maximum number of iterations and the convergence criterion for IIS are given by max_iis_iterations and iis_converge, respectively, while max_newton_iterations and newton_converge are the maximum number of iterations and the convergence criterion for Newton's method. zNo data in the training set.z2training_set and results should be parallel lists.csg|]}t|qSr )r%)r&rr training_setr r r)6sztrain..rrcSsg|]\}}t||qSr )r9rA)r&r=r'r r r r)KsNzIIS did not converge) ValueErrorrsortedsetr@r3rIrrr9addrrrrrB)rOZresultsrZ update_fnZmax_iis_iterationsZ iis_convergerCrDr!r,r-r?rHrrFZnalphasZdiffrr rNr trainsB    rT__main__) run_doctest)verbose)NrJrKrLrM)r functoolsrwarningsZnumpyr9 ImportErrorZBiorrwarnrrr r%r3r8r4r@rGrIrTr Z Bio._utilsrVr r r r sF    =  $ L