c%,dZddlZddlZddlZddlZddlmZddlZddl m Z m Z ddl m Z ddl mZmZmZddlmZmZmZmZmZmZmZddlmZmZdd lmZejeZ d Z!Gd d ej"Z#Gd dej$ej%Z&dS)a Optimized `Latent Dirichlet Allocation (LDA) `_ in Python. For a faster implementation of LDA (parallelized for multicore machines), see also :mod:`gensim.models.ldamulticore`. This module allows both LDA model estimation from a training corpus and inference of topic distribution on new, unseen documents. The model can also be updated with new documents for online training. The core estimation code is based on the `onlineldavb.py script `_, by Matthew D. Hoffman, David M. Blei, Francis Bach: `'Online Learning for Latent Dirichlet Allocation', NIPS 2010`_. .. _'Online Learning for Latent Dirichlet Allocation', NIPS 2010: online-lda_ .. _'Online Learning for LDA' by Hoffman et al.: online-lda_ .. _online-lda: https://papers.neurips.cc/paper/2010/file/71f6278d140af599e06ad9bf1ba03cb0-Paper.pdf The algorithm: #. Is **streamed**: training documents may come in sequentially, no random access required. #. Runs in **constant memory** w.r.t. the number of documents: size of the training corpus does not affect memory footprint, can process corpora larger than RAM. #. Is **distributed**: makes use of a cluster of machines, if available, to speed up model estimation. Usage examples -------------- Train an LDA model using a Gensim corpus .. sourcecode:: pycon >>> from gensim.test.utils import common_texts >>> from gensim.corpora.dictionary import Dictionary >>> >>> # Create a corpus from a list of texts >>> common_dictionary = Dictionary(common_texts) >>> common_corpus = [common_dictionary.doc2bow(text) for text in common_texts] >>> >>> # Train the model on the corpus. >>> lda = LdaModel(common_corpus, num_topics=10) Save a model to disk, or reload a pre-trained model .. sourcecode:: pycon >>> from gensim.test.utils import datapath >>> >>> # Save model to disk. >>> temp_file = datapath("model") >>> lda.save(temp_file) >>> >>> # Load a potentially pretrained model from disk. >>> lda = LdaModel.load(temp_file) Query, the model using new, unseen documents .. sourcecode:: pycon >>> # Create a new corpus, made of previously unseen documents. >>> other_texts = [ ... ['computer', 'time', 'graph'], ... ['survey', 'response', 'eps'], ... ['human', 'system', 'computer'] ... ] >>> other_corpus = [common_dictionary.doc2bow(text) for text in other_texts] >>> >>> unseen_doc = other_corpus[0] >>> vector = lda[unseen_doc] # get topic probability distribution for a document Update the model by incrementally training on the new corpus .. sourcecode:: pycon >>> lda.update(other_corpus) >>> vector = lda[unseen_doc] A lot of parameters can be tuned to optimize training for your specific case .. sourcecode:: pycon >>> lda = LdaModel(common_corpus, num_topics=50, alpha='auto', eval_every=5) # learn asymmetric alpha from data N) defaultdict)gammalnpsi) polygamma) interfacesutilsmatutils)kullback_leibler hellingerjaccard_distancejensen_shannondirichlet_expectation logsumexpmean_absolute_difference) basemodelCoherenceModel)Callbackc|ttj|t|z |zz}|tdtj|z}| td|z}tj||z d|z tjd|z zz }||z |z }||z|z} t | dkr| }nt d|S)aUpdate a given prior using Newton's method, described in `J. Huang: "Maximum Likelihood Estimation of Dirichlet Distribution Parameters" `_. Parameters ---------- prior : list of float The prior for each possible outcome at the previous iteration (to be updated). N : int Number of observations. logphat : list of float Log probabilities for the current estimation, also called "observed sufficient statistics". rho : float Learning rate. Returns ------- list of float The updated prior. rzupdated prior is not positive)rnpsumrallloggerwarning) priorNlogphatrhogradfcqbdprior updated_priors 6lib/python3.11/site-packages/gensim/models/ldamodel.pyupdate_dir_priorr&rs, RVE]]##c%jj07: ;E Ia ' ''A Yq%  A uqyQURVAE]]23Aqy\A F&L5(M =1 86777 LcneZdZdZejfdZdZdZd dZ d dZ dZ d Z e fd ZxZS) LdaStatezEncapsulate information for distributed computation of :class:`~gensim.models.ldamodel.LdaModel` objects. Objects of this class are sent over the network, so try to keep them lean to reduce traffic. c||d|_tj|||_d|_||_dS)ai Parameters ---------- eta : numpy.ndarray The prior probabilities assigned to each term. shape : tuple of (int, int) Shape of the sufficient statistics: (number of topics to be found, number of terms in the vocabulary). dtype : type Overrides the numpy array default types. FcopydtyperN)astypeetarzerossstatsnumdocsr.)selfr0shaper.s r%__init__zLdaState.__init__sC::e%:00huE222   r'c,d|jdd<d|_dS)zBPrepare the state for a new EM iteration (reset sufficient stats).Nrr2r3r4s r%resetzLdaState.resets AAA r'cb|J|xj|jz c_|xj|jz c_dS)aMerge the result of an E step from one node with that of another node (summing up sufficient statistics). The merging is trivial and after merging all cluster nodes, we have the exact same result as if the computation was run on a single node (no approximation). Parameters ---------- other : :class:`~gensim.models.ldamodel.LdaState` The state object with which the current one will be merged. Nr9)r4others r%mergezLdaState.merges8    u|#   % r'Ncj|J||j}|jdks ||jkrd}n d|z|jz }|xjd|z |zzc_|jdks ||jkrd}n.td|j|d|z|jz }|xj||z|jzz c_||_dS)aOMerge the current state with another one using a weighted average for the sufficient statistics. The number of documents is stretched in both state objects, so that they are of comparable magnitude. This procedure corresponds to the stochastic gradient update from `'Online Learning for LDA' by Hoffman et al.`_, see equations (5) and (9). Parameters ---------- rhot : float Weight of the `other` state in the computed average. A value of 0.0 means that `other` is completely ignored. A value of 1.0 means `self` is completely ignored. other : :class:`~gensim.models.ldamodel.LdaState` The state object with which the current one will be merged. targetsize : int, optional The number of documents to stretch both states to. Nr?z>merging changes from %i documents into a model of %i documents)r3r2rinfo)r4rhotr= targetsizescales r%blendzLdaState.blends$    &J <1  4 dl : 4EE*$t|3E d e++  =A  5u}!< 5EE KKXZ_Zgis t t t*$u}4E te|el22 ! r'cX|J||j}|xj|jz c_||_dS)a)Merge the current state with another one using a weighted sum for the sufficient statistics. In contrast to :meth:`~gensim.models.ldamodel.LdaState.blend`, the sufficient statistics are not scaled prior to aggregation. Parameters ---------- rhot : float Unused. other : :class:`~gensim.models.ldamodel.LdaState` The state object with which the current one will be merged. targetsize : int, optional The number of documents to stretch both states to. N)r3r2)r4rBr=rCs r%blend2zLdaState.blend2s>     &J u|# ! r'c |j|jzS)zGet the parameters of the posterior over the topics, also referred to as "the topics". Returns ------- numpy.ndarray Parameters of the posterior probability over topics. )r0r2r:s r% get_lambdazLdaState.get_lambdasx$+%%r'cDt|S)zGet the log (posterior) probabilities for each topic. Returns ------- numpy.ndarray Posterior probabilities for each topic. )rrIr:s r% get_ElogbetazLdaState.get_Elogbetas%T__%6%6777r'ctt|j|g|Ri|}t|ds1tj|_tjd|j j ||S)aLoad a previously stored state from disk. Overrides :class:`~gensim.utils.SaveLoad.load` by enforcing the `dtype` parameter to ensure backwards compatibility. Parameters ---------- fname : str Path to file that contains the needed object. args : object Positional parameters to be propagated to class:`~gensim.utils.SaveLoad.load` kwargs : object Key-word parameters to be propagated to class:`~gensim.utils.SaveLoad.load` Returns ------- :class:`~gensim.models.ldamodel.LdaState` The state loaded from the given file. r.:dtype was not set in saved %s file %s, assuming np.float64) superr)loadhasattrrfloat64r.loggingrA __class____name__)clsfnameargskwargsresultrSs r%rOz LdaState.loadsq,+x%%*5B4BBB6BBvw'' y:FL LUW]WgWprw x x x r'N)rT __module__ __qualname____doc__rfloat32r6r;r>rErGrIrK classmethodrO __classcell__rSs@r%r)r)s *,$ &&&"%"%"%"%"N""""0 & & &888[r'r)c(eZdZdZddddddddddd d d d d ddd ddejfdZdZdZd,dZ dZ d-dZ d,dZ dZ dZd,dZ d.dZd-dZd/dZd0dZd1dZdZd1dZ d2d#Z d3d$Zd,d%Z d4d'Zd,d(Zd5fd* Zefd+ZxZS)6LdaModelaTrain and use Online Latent Dirichlet Allocation model as presented in `'Online Learning for LDA' by Hoffman et al.`_ Examples ------- Initialize a model using a Gensim corpus .. sourcecode:: pycon >>> from gensim.test.utils import common_corpus >>> >>> lda = LdaModel(common_corpus, num_topics=10) You can then infer topic distributions on new, unseen documents. .. sourcecode:: pycon >>> doc_bow = [(1, 0.3), (2, 0.1), (0, 0.09)] >>> doc_lda = lda[doc_bow] The model can be updated (trained) with new documents. .. sourcecode:: pycon >>> # In practice (corpus =/= initial training corpus), but we use the same here for simplicity. >>> other_corpus = common_corpus >>> >>> lda.update(other_corpus) Model persistency is achieved through :meth:`~gensim.models.ldamodel.LdaModel.load` and :meth:`~gensim.models.ldamodel.LdaModel.save` methods. NdFir symmetricg?r@ 2gMbP?{Gz?c  tj|j|_||_||jt d|jMt dtj||_t|j|_ nNt|jdkr/dt|j z|_ nd|_ |j dkrt dt||_t||_||_| |_| |_||_d|_||_||_| |_||_||_||_||d\|_|_|jj|jfks+JdtA|jj|jfz|| d \|_!|_"|j!j|j fksS|j!j|j|j fks7Jd tA|j!j|j |j|j fztj#||_$| |_%||_&|s*t 'd d|_(d|_)nk|jrtUd  ddl+}|i}tj,di|5}dd l-m.}|/|0|||_(t 1dtA|j(j2z|j(3|j|j||| dt|j(4|_)t 'd|j)dddn #1swxYwYn?#tj$r2}t 6d|tod|zd}~wwxYwtq|j!|j|j f|j|_9|j$:dd|j|j f|j9j;d<tj<t{|j9j;|_>|j!j|jksJ|j>j|jksJ|h|j(du}tj?}|@|||Add|dtj?|z dddSdS) a Parameters ---------- corpus : iterable of list of (int, float), optional Stream of document vectors or sparse matrix of shape (`num_documents`, `num_terms`). If you have a CSC in-memory matrix, you can convert it to a streamed corpus with the help of gensim.matutils.Sparse2Corpus. If not given, the model is left untrained (presumably because you want to call :meth:`~gensim.models.ldamodel.LdaModel.update` manually). num_topics : int, optional The number of requested latent topics to be extracted from the training corpus. id2word : {dict of (int, str), :class:`gensim.corpora.dictionary.Dictionary`} Mapping from word IDs to words. It is used to determine the vocabulary size, as well as for debugging and topic printing. distributed : bool, optional Whether distributed computing should be used to accelerate training. chunksize : int, optional Number of documents to be used in each training chunk. passes : int, optional Number of passes through the corpus during training. update_every : int, optional Number of documents to be iterated through for each update. Set to 0 for batch learning, > 1 for online iterative learning. alpha : {float, numpy.ndarray of float, list of float, str}, optional A-priori belief on document-topic distribution, this can be: * scalar for a symmetric prior over document-topic distribution, * 1D array of length equal to num_topics to denote an asymmetric user defined prior for each topic. Alternatively default prior selecting strategies can be employed by supplying a string: * 'symmetric': (default) Uses a fixed symmetric prior of `1.0 / num_topics`, * 'asymmetric': Uses a fixed normalized asymmetric prior of `1.0 / (topic_index + sqrt(num_topics))`, * 'auto': Learns an asymmetric prior from the corpus (not available if `distributed==True`). eta : {float, numpy.ndarray of float, list of float, str}, optional A-priori belief on topic-word distribution, this can be: * scalar for a symmetric prior over topic-word distribution, * 1D array of length equal to num_words to denote an asymmetric user defined prior for each word, * matrix of shape (num_topics, num_words) to assign a probability for each word-topic combination. Alternatively default prior selecting strategies can be employed by supplying a string: * 'symmetric': (default) Uses a fixed symmetric prior of `1.0 / num_topics`, * 'auto': Learns an asymmetric prior from the corpus. decay : float, optional A number between (0.5, 1] to weight what percentage of the previous lambda value is forgotten when each new document is examined. Corresponds to :math:`\kappa` from `'Online Learning for LDA' by Hoffman et al.`_ offset : float, optional Hyper-parameter that controls how much we will slow down the first steps the first few iterations. Corresponds to :math:`\tau_0` from `'Online Learning for LDA' by Hoffman et al.`_ eval_every : int, optional Log perplexity is estimated every that many updates. Setting this to one slows down training by ~2x. iterations : int, optional Maximum number of iterations through the corpus when inferring the topic distribution of a corpus. gamma_threshold : float, optional Minimum change in the value of the gamma parameters to continue iterating. minimum_probability : float, optional Topics with a probability lower than this threshold will be filtered out. random_state : {np.random.RandomState, int}, optional Either a randomState object or a seed to generate one. Useful for reproducibility. ns_conf : dict of (str, object), optional Key word parameters propagated to :func:`gensim.utils.getNS` to get a Pyro4 nameserver. Only used if `distributed` is set to True. minimum_phi_value : float, optional if `per_word_topics` is True, this represents a lower bound on the term probabilities. per_word_topics : bool If True, the model also computes a list of topics, sorted in descending order of most likely topics for each word, along with their phi values multiplied by the feature length (i.e. word count). callbacks : list of :class:`~gensim.models.callbacks.Callback` Metric callbacks to log and visualize evaluation metrics of the model during training. dtype : {numpy.float16, numpy.float32, numpy.float64}, optional Data-type to use during calculations inside model. All inputs are also converted. NzYat least one of corpus/id2word must be specified, to establish input space dimensionalityzHno word id mapping provided; initializing from corpus, assuming identityrrz6cannot compute LDA over an empty collection (no terms)alphaz6Invalid alpha shape. Got shape %s, but expected (%d, )r0zAInvalid eta shape. Got shape %s, but expected (%d, 1) or (%d, %d)z%using serial LDA version on this nodez8auto-optimizing alpha not implemented in distributed LDA)LDA_DISPATCHER_PREFIX)prefixzlooking for dispatcher at %sF)id2word num_topics chunksizerjr0 distributedz)using distributed version with %i workersz)failed to initialize distributed LDA (%s)r-Y@rh.)chunks_as_numpycreatedztrained z in z.2fs)msg)Brfinfor.rm ValueErrorrrrdict_from_corpuslen num_termsmaxkeysboolrpintrnrodecayoffsetminimum_probability num_updatespasses update_every eval_everyminimum_phi_valueper_word_topics callbacksinit_dir_priorrjoptimize_alphar5strr0 optimize_etaget_random_state random_state iterationsgamma_thresholdrA dispatcher numworkersNotImplementedErrorPyro4getNSgensim.models.lda_dispatcherrkProxylistdebug_pyroUri initialize getworkers Exceptionerror RuntimeErrorr)stategammar2expr expElogbetatimeupdateadd_lifecycle_event)r4corpusrnrmrprorrrjr0rrrrrrrns_confrrrr.rnsrkerr use_numpystarts r%r6zLdaModel.__init__^s^Xe__*   dl k  <  NNe f f f 1&99DL ..DNN    " T\%6%6%8%8!9!99DNNDN >Q  WUVV V ,,j//"  #6  ($!2."*.*=*=eW*M*M' D'zDO#55 p p DDJL\H]H]_c_nGo o p p p'+&9&9#u&E&E#$#x~$.!22 TdhnZ^ZhHi6i T T O  $.$/4> R S T T T"2<@@%. V KK? @ @ @"DODOO" f)*deee V ! G[++7++ ^rRRRRRR&+kk"''AV'2W2WXm2n&o&oDOLL!?#doF^B_B_!_```O.. $ T]#%/'*$/*D*D*F*F&G&GDOKK KT_]]] ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ V V V H#NNN"#NQT#TUUU V dh$.(IQUQ[\\\ !%!2!8!8y4?\`\jJk!l!l #6"7 8I"J"JKKx~++++%3333  t3IIKKE KK K : : :  $ $CtCCu)<CCCC %        s= O"C$O OOOOO P(-PPc4d|dkrjn|dkrjntdd}ttrvdkr[t d|djz tjfd tDj  }nd kr|dkrtd tjfd tDj  }|| z}t d|t|nXdkrgd}tjfdtDj  }|dkr)t d|t|ntd|ddttrtj j }nttjrj d}nmttjt"jfr5tjfdtDj }ntd|z||fS)aQInitialize priors for the Dirichlet distribution. Parameters ---------- prior : {float, numpy.ndarray of float, list of float, str} A-priori belief on document-topic distribution. If `name` == 'alpha', then the prior can be: * scalar for a symmetric prior over document-topic distribution, * 1D array of length equal to num_topics to denote an asymmetric user defined prior for each topic. Alternatively default prior selecting strategies can be employed by supplying a string: * 'symmetric': (default) Uses a fixed symmetric prior of `1.0 / num_topics`, * 'asymmetric': Uses a fixed normalized asymmetric prior of `1.0 / (topic_index + sqrt(num_topics))`, * 'auto': Learns an asymmetric prior from the corpus (not available if `distributed==True`). A-priori belief on topic-word distribution. If `name` == 'eta' then the prior can be: * scalar for a symmetric prior over topic-word distribution, * 1D array of length equal to num_words to denote an asymmetric user defined prior for each word, * matrix of shape (num_topics, num_words) to assign a probability for each word-topic combination. Alternatively default prior selecting strategies can be employed by supplying a string: * 'symmetric': (default) Uses a fixed symmetric prior of `1.0 / num_topics`, * 'auto': Learns an asymmetric prior from the corpus. name : {'alpha', 'eta'} Whether the `prior` is parameterized by the alpha vector (1 parameter per topic) or by the eta (1 parameter per unique term in the vocabulary). Returns ------- init_prior: numpy.ndarray Initialized Dirichlet prior: If 'alpha' was provided as `name` the shape is (self.num_topics, ). If 'eta' was provided as `name` the shape is (len(self.id2word), ). is_auto: bool Flag that shows if hyperparameter optimization should be used or not. Nrerjr0z'name' must be 'alpha' or 'eta'Fzusing symmetric %s at %sr@c3,K|]}djz VdSr@Nrn.0ir4s r% z*LdaModel.init_dir_prior..Cs*GGqS4?*GGGGGGr'r.count asymmetricz.The 'asymmetric' option cannot be used for etac3LK|]}d|tjzz VdSr)rsqrt)rr prior_shapes r%rz*LdaModel.init_dir_prior..Js7RR!SA 4 445RRRRRRr'zusing asymmetric %s %sautoTc3,K|]}djz VdSrrrs r%rz*LdaModel.init_dir_prior..Qs*)\)\A#*?)\)\)\)\)\)\r'z$using autotuned %s, starting with %szUnable to determine proper z value given ''r-r+c3K|]}VdSrZrv)rrrs r%rz*LdaModel.init_dir_prior..\s#%H%He%H%H%H%H%H%Hr'zC%s must be either a np array of scalars, list of scalars, or scalar)rnr{rx isinstancerrrArfromiterranger.rrasarrayndarrayr/numbernumbersReal)r4rnameis_auto init_priorrs`` @r%rzLdaModel.init_dir_priorsH  E 7? @/KK U] @.KK>?? ? eS ! ! k # c 6cDO>STTT[GGGGE+4F4FGGG*K ,& c5=W$%UVVV[RRRRu[?Q?QRRR*K jnn...  4dD)rSrTr{rnrror:s r%__str__zLdaModel.__str__bs@ N # # #T^^^T___djjjRVR`R`R`  r'c||j}tj||_|jj|jksJdS)aPropagate the states topic probabilities to the inner object's attribute. Parameters ---------- current_Elogbeta: numpy.ndarray Posterior probabilities for each topic, optional. If omitted, it will get Elogbeta from state. N)rrKrrrr.)r4current_Elogbetas r% sync_statezLdaModel.sync_stateosR  9#z6688 6"233%333333r'c"d|_d|_dS)zTClear the model's state to free some memory. Used in the distributed implementation.N)rElogbetar:s r%clearzLdaModel.clear~s  r'c ( t|n#t$rt|}YnwxYwt|dkr(tdt||jddt||jf|j d}t|}tj |}|j |j ksJ|j |j ksJ|r!tj |j|j }nd}d }ttjf}tj|j j} t'|D]\} } t| d kr)t)| d d |s d | D} n d | D} tjd | D|j t|  } || ddf}|| ddf}|| ddf}|jdd| f}tj||| z}t/|jD]}|}|j|tj| |z |jzz}t|}tj |}tj||| z}t7||}||jkr|dz }n||| ddf<|j |j ksJ|r/|dd| fxxtj|j| |z z cc<t|dkr/td|t||j|r||jz}|j |j ksJ|j |j ksJ||fS)aGiven a chunk of sparse document vectors, estimate gamma (parameters controlling the topic weights) for each document in the chunk. This function does not modify the model. The whole input chunk of document is assumed to fit in RAM; chunking of a large corpus must be done earlier in the pipeline. Avoids computing the `phi` variational parameter directly using the optimization presented in `Lee, Seung: Algorithms for non-negative matrix factorization" `_. Parameters ---------- chunk : list of list of (int, float) The corpus chunk on which the inference step will be performed. collect_sstats : bool, optional If set to True, also collect (and return) sufficient statistics needed to update the model's topic-word distributions. Returns ------- (numpy.ndarray, {numpy.ndarray, None}) The first element is always returned and it corresponds to the states gamma matrix. The second element is only returned if `collect_sstats` == True and corresponds to the sufficient statistics for the M step. rz/performing inference on a chunk of %i documentsrqrhFr+r-Nrc2g|]\}}t|Srv)rridx_s r% z&LdaModel.inference..s"222FCs3xx222r'cg|]\}}|Srvrvrs r%rz&LdaModel.inference..s---vsAs---r'c3 K|] \}}|V dSrZrv)rrcnts r%rz%LdaModel.inference..s&55vq#s555555r'rz.%i/%i documents converged within %i iterations)rz TypeErrorrrrrrrnr/r.rrr zeros_likerrintegerrweps enumeraterrdotrrrjTrrouter)r4chunkcollect_sstatsr Elogtheta expElogthetar2 converged integer_typesepsilonddocidsctsgammad Elogthetad expElogthetad expElogbetadphinormr lastgamma meanchanges r% inferencezLdaModel.inferences2 JJJJ   KKEEE  u::> X LLJCPUJJ W W W!''i#e**do9VWW^^_c_ipu^vv)%00 vi(( $*,,,,!TZ////  ]4#34:FFFFFF bj* (4:&&*&&% K% KFAs3xx!| .Js1vay-$H$H .22c222-----+55555TZsSVxxXXXC1aaa4[F"1aaa4J(AAA.M+AAAsF3L f]L99GCG4?++  " mbfS7]LN6[6[&[[26:: "z 2 2 & ==G5fiHH  44NIE!E!QQQ$K<4:- - - - Kqqq#v"(=?C'M"J"JJ u::> s LLI9VYZ_V`V`bfbq r r r  . d& &F<4:- - - -{dj((((f}s ..c||j}||d\}}|xj|z c_|xj|jdz c_|j|jksJ|S)awPerform inference on a chunk of documents, and accumulate the collected sufficient statistics. Parameters ---------- chunk : list of list of (int, float) The corpus chunk on which the inference step will be performed. state : :class:`~gensim.models.ldamodel.LdaState`, optional The state to be updated with the newly accumulated sufficient statistics. If none, the models `self.state` is updated. Returns ------- numpy.ndarray Gamma parameters controlling the topic weights, shape (`len(chunk)`, `self.num_topics`). NTrr)rrr2r3r5r.)r4rrrr2s r%do_estepzLdaModel.do_estepsl"  JEuTBB v   Q' {dj(((( r'cdtt|}td|D|z }|j|jksJt |j||||_t dt|j|jj|jksJ|jS)aZUpdate parameters for the Dirichlet prior on the per-document topic weights. Parameters ---------- gammat : numpy.ndarray Previous topic weight parameters. rho : float Learning rate. Returns ------- numpy.ndarray Sequence of alpha parameters. c34K|]}t|VdSrZr)rrs r%rz(LdaModel.update_alpha..s+GGu+E22GGGGGGr'zoptimized alpha %s) floatrzrr.r&rjrrAr)r4gammatrrrs r% update_alphazLdaModel.update_alphas #f++  GGGGGGG!K} ****%dj!WcBB  ($tz*:*:;;;z4:----zr'c8t|jd}td|D|z |jf}|j|jksJt |j||||_|jj|jksJ|jS)aOUpdate parameters for the Dirichlet prior on the per-topic word weights. Parameters ---------- lambdat : numpy.ndarray Previous lambda parameters. rho : float Learning rate. Returns ------- numpy.ndarray The updated eta parameters. rc34K|]}t|VdSrZr)rlambda_s r%rz&LdaModel.update_eta..1s+MM',W55MMMMMMr')rr5rreshaper{r.r&r0)r4lambdatrrrs r% update_etazLdaModel.update_eta s '-" # #MMWMMMMMPQQZZ\`\j[lmm} ****#DHa#>>x~++++xr'c:|t|}td|D}d|zt|z }|||||zz }td|t j| t|||S)aSCalculate and return per-word likelihood bound, using a chunk of documents as evaluation corpus. Also output the calculated statistics, including the perplexity=2^(-bound), to log at INFO level. Parameters ---------- chunk : list of list of (int, float) The corpus chunk on which the inference step will be performed. total_docs : int, optional Number of docs used for evaluation of the perplexity. Returns ------- numpy.ndarray The variational bound score calculated for each word. Nc3*K|]}|D] \}}|V dSrZrv)rdocumentrrs r%rz*LdaModel.log_perplexity..Ms3LL88LLC3LLLLLLLr'r@)subsample_ratiozf%.3f per-word bound, %.1f perplexity estimate based on a held-out corpus of %i documents with %i words)rzrboundrrArexp2)r4r total_docs corpus_wordsr  perwordbounds r%log_perplexityzLdaModel.log_perplexity9s$  $UJLLuLLLLL  *SZZ7zz%zII__kMkl  t "'<-00#e**l   r'c  jj|j}|j}|j}|j}| j} t|} nC#t$r6t dtd|D} YnwxYw| dkrt ddSt| j jxj| z c_|r/d} |dkr| dz } n| d z } t| |jzz} nd } | } t| |pdjzz}t#d| | z }td | j|| | ||| ||zd krt d fd}jrBt+j}|t/t0_t5|D]_jr@tdjjjn*t;jjjj j!}d}d}tEj#|| j!}tI|D]\}}|t|z }|r1|| ks|dz|jzzdkr%|| jrKtd|zt|z| j&|nktd|zt|z| '||}j(r)||d}~|r|dz|jzzdkrˉjr3tdj*}+||dk~jr:tdjjn*t;jjjj j!}d}|| krtYdjrO|-}|.D]%\}}j|/|&|r`jr3tdj*}+||dk~d}adS)ac Train the model with new documents, by EM-iterating over the corpus until the topics converge, or until the maximum number of allowed iterations is reached. `corpus` must be an iterable. In distributed mode, the E step is distributed over a cluster of machines. Notes ----- This update also supports updating an already trained model (`self`) with new documents from `corpus`; the two models are then merged in proportion to the number of old vs. new documents. This feature is still experimental for non-stationary input streams. For stationary input (no topic drift in new documents), on the other hand, this equals the online update of `'Online Learning for LDA' by Hoffman et al.`_ and is guaranteed to converge for any `decay` in (0.5, 1]. Additionally, for smaller corpus sizes, an increasing `offset` may be beneficial (see Table 1 in the same paper). Parameters ---------- corpus : iterable of list of (int, float), optional Stream of document vectors or sparse matrix of shape (`num_documents`, `num_terms`) used to update the model. chunksize : int, optional Number of documents to be used in each training chunk. decay : float, optional A number between (0.5, 1] to weight what percentage of the previous lambda value is forgotten when each new document is examined. Corresponds to :math:`\kappa` from `'Online Learning for LDA' by Hoffman et al.`_ offset : float, optional Hyper-parameter that controls how much we will slow down the first steps the first few iterations. Corresponds to :math:`\tau_0` from `'Online Learning for LDA' by Hoffman et al.`_ passes : int, optional Number of passes through the corpus during training. update_every : int, optional Number of documents to be iterated through for each update. Set to 0 for batch learning, > 1 for online iterative learning. eval_every : int, optional Log perplexity is estimated every that many updates. Setting this to one slows down training by ~2x. iterations : int, optional Maximum number of iterations through the corpus when inferring the topic distribution of a corpus. gamma_threshold : float, optional Minimum change in the value of the gamma parameters to continue iterating. chunks_as_numpy : bool, optional Whether each chunk passed to the inference step should be a numpy.ndarray or not. Numpy can in some settings turn the term IDs into floats, these will be converted back into integers in inference, which incurs a performance hit. For distributed computing it may be desirable to keep the chunks as `numpy.ndarray`. Nz4input corpus stream has no len(); counting documentsc3K|]}dVdS)rNrv)rrs r%rz"LdaModel.update..s"..!A......r'rz-LdaModel.update() called with an empty corpusonlinerz (single-pass)z (multi-pass)batchzrunning %s LDA training, %s topics, %i passes over the supplied corpus of %i documents, updating model once every %i documents, evaluating perplexity every %i documents, iterating %ix with a convergence threshold of %frfzxtoo few updates, training might not converge; consider increasing the number of passes or iterations to improve accuracycBtzjz z SrZ)powr)rorrpass_r4sr%rzLdaModel.update..rhos&v~)9I)EFOO Or'zinitializing %s workersF)as_numpyr.)r z5PROGRESS: pass %i, dispatching documents up to #%i/%iz%PROGRESS: pass %i, at document #%i/%iTzFreached the end of input; now waiting for all remaining jobs to finishzinitializing workerszIinput corpus size changed during training (don't use generators as input))0rrrrrrrrzrrrrminrorr3rr|rArnrr set_modelrrmetricsrrr;r)r0r2r5r.rgrouperrrputjobrrrgetstatedo_mstepr on_epoch_enditemsappend)r4rrorrrrrrrrr lencorpus updatetype updateafter evalafterupdates_per_passrcallbackr=dirtyreallenchunkschunk_norrcurrent_metricsmetricvaluers` ``` @r%rzLdaModel.updateVs#h  JE  ![F  ![F  -,L  )J  )J  3"2O /F II / / / NNQ R R R..v.....III / >  NNJ K K K F  7It~66I i'  $!J{ ... o- i)G))STTKK J#K JO!t#F#RSS q)k"9::  ?  J     f $r )  NN]    P P P P P P P P P > -//H   t $ $ $&t,,DL6]]F F E P 5tGGG%%dj1111 4:+<+BDJOOEG]69VZV`aaaF#,V#4#4) ") "%3u::%EGy$8Ex!|PZ]a]lPl>mqr>rE'')'DDD?9KKOx)3c%jj@) O**51111KK?x)3c%jj@)"]]5%88F*9))&##%%888 "X\lT_6T$UYZ$Z "; $lmmm $ 8 8 : :MM##%% :::X $:;;;--dj9999 (4:3D3JDJ W W!E)# p"#nooo~ 7"*"7"7">">%4%:%:%<%<77MFEL(//6666 ?7KK hiii O4466E cceeUEAI666MF F sA=BBcbtd|j}|j|||j}|||dt||}t d|||j r-| |j ||s|xj |jz c_ dSdS)aMaximization step: use linear interpolation between the existing topics and collected sufficient statistics in `other` to update the topics. Parameters ---------- rho : float Learning rate. other : :class:`~gensim.models.ldamodel.LdaModel` The model whose sufficient statistics will be used to update the topics. extra_pass : bool, optional Whether this step required an additional pass over the corpus. zupdating topicsztopic diff=%f, rho=%fN)rrrrKrEr print_topicsrravelrArrrIrr3)r4rr= extra_passprevious_Elogbetardiffs r%rzLdaModel.do_msteps  &'''!J3355 e$$$:2244 ())) !'(9(?(?(A(ACSCYCYC[C[\\ +T3777   : OODJ1133S 9 9 9 .    -     . .r'c  d}|j}t| t|D]K\}}||jzdkrt d||||g\}} n||}t| |j|jksJ j|jksJ|t fd|Dz }|tj |j |z zz }|tj t|t|j z z }|ttj |j ttj |z z }M||z}|tj |j |z zz }|tj t|t|j z z }tj|j dkr|j |jz} ntj |j } |tj t| ttj |dz z }|S)aEstimate the variational bound of documents from the corpus as E_q[log p(corpus)] - E_q[log q(corpus)]. Parameters ---------- corpus : iterable of list of (int, float), optional Stream of document vectors or sparse matrix of shape (`num_documents`, `num_terms`) used to estimate the variational bounds. gamma : numpy.ndarray, optional Topic weight variational parameters for each document. If not supplied, it will be inferred from the model. subsample_ratio : float, optional Percentage of the whole corpus represented by the passed `corpus` argument (in case this was a sample). Set to 1.0 if the whole corpus was passed.This is used as a multiplicative factor to scale the likelihood appropriately. Returns ------- numpy.ndarray The variational bound score calculated for each document. r8rzbound: at document #%iNc 3vK|]3\}}|tddt|fzzV4dSrZ)rr)ridrrrs r%rz!LdaModel.bound..bsO__PWPRTWyhqqq#b''z6J)JKKK______r'r)rrIrrrorrrr.rrrjrr0ndimr{) r4rrr score_lambdarrrrsum_etarrs @@r%r zLdaModel.bound<sM**''))(11'' K KFAs4>!Q& : 5q999 " NNC511 q.v66J<4:- - - -#tz1 1 1 1 S_____[^_____ _E RVTZ&0J>?? ?E RVGFOOgdj.A.AABB BE WRVDJ//00726&>>3J3JJ JEE   G+x7888 ((748+<+<<=== 748   ! 'h/GGfTX&&G ((726'13E3E+F+FFGGG r'Tc |dks |jkrj}t|}nt|j}jdjt jzz}ttj |}|d|dz|| dzdz}g}j } |D]} | |  z tj |d} fd| D |rd d D || f|r(td | j|  |S) a*Get a representation for selected topics. Parameters ---------- num_topics : int, optional Number of topics to be returned. Unlike LSA, there is no natural ordering between the topics in LDA. The returned topics subset of all topics is therefore arbitrary and may change between two LDA training runs. num_words : int, optional Number of words to be presented for each topic. These will be the most relevant words (assigned the highest probability for each topic). log : bool, optional Whether the output is also logged, besides being returned. formatted : bool, optional Whether the topic representations should be formatted as strings. If False, they are returned as 2 tuples of (word, probability). Returns ------- list of {str, tuple of (str, float)} a list of topics, each represented either as a string (when `formatted` == True) or word-probability pairs. rg-C6?NTreversec:g|]}j||fSrvrm)rr9r4topic_s r%rz(LdaModel.show_topics..s)EEEt|B'4EEEr'z + c3*K|]\}}d||fzVdS)z %.3f*"%s"Nrv)rkvs r%rz'LdaModel.show_topics..s/#L#LTQK1a&$8#L#L#L#L#L#Lr'ztopic #%i (%.3f): %s)rnrrrjrrandrzrr argsortrrIrjoinr"rrA) r4rn num_wordslog formatted chosen_topics sort_alpha sorted_topicsshowntopicrbestnrDs ` @r% show_topicszLdaModel.show_topicszs2 > _Z4?: _J!*--MMZ99Jft/@/E/Ec$*oo/V/V&VVJ!!1*!=!=>>M)*::?*:;mZK[\L\L]L]>^^M %%'' N NA1XFfjjll*F$VYEEEEEEEEEuEEEF M#L#LV#L#L#LLL LL!V % % % N 2Atz!}fMMM r'cHfd||DS)aKGet the representation for a single topic. Words here are the actual strings, in constrast to :meth:`~gensim.models.ldamodel.LdaModel.get_topic_terms` that represents words by their vocabulary ID. Parameters ---------- topicid : int The ID of the topic to be returned topn : int, optional Number of the most significant words that are associated with the topic. Returns ------- list of (str, float) Word - probability pairs for the most relevant words generated by the topic. c4g|]\}}j||fSrvrC)rr9r/r4s r%rz'LdaModel.show_topic..s)___ib%b!5)___r')get_topic_terms)r4topicidtopns` r% show_topiczLdaModel.show_topics2"`___4;O;OPWY];^;^____r'cz|j}||ddddfz S)zGet the term-topic matrix learned during inference. Returns ------- numpy.ndarray The probability for each word in each topic, shape (`num_topics`, `vocabulary_size`). r)axisN)rrIr)r4topicss r% get_topicszLdaModel.get_topicss=&&((  **111d7333r'c||z tj|d}fd|DS)a<Get the representation for a single topic. Words the integer IDs, in constrast to :meth:`~gensim.models.ldamodel.LdaModel.show_topic` that represents words by the actual strings. Parameters ---------- topicid : int The ID of the topic to be returned topn : int, optional Number of the most significant words that are associated with the topic. Returns ------- list of (int, float) Word ID - probability pairs for the most relevant words generated by the topic. Tr@c$g|] }||f Srvrv)rrrRs r%rz,LdaModel.get_topic_terms..s"333ceCj!333r')r^rr rI)r4rXrYrSrRs @r%rWzLdaModel.get_topic_termss\"!!'* # d;;;3333U3333r'u_massc Jt|||||||}|} g} D]=tj|d} fd| D} | | >t | | } t| ddS)aGet the topics with the highest coherence score the coherence for each topic. Parameters ---------- corpus : iterable of list of (int, float), optional Corpus in BoW format. texts : list of list of str, optional Tokenized texts, needed for coherence models that use sliding window based (i.e. coherence=`c_something`) probability estimator . dictionary : :class:`~gensim.corpora.dictionary.Dictionary`, optional Gensim dictionary mapping of id word to create corpus. If `model.id2word` is present, this is not needed. If both are provided, passed `dictionary` will be used. window_size : int, optional Is the size of the window to be used for coherence measures using boolean sliding window as their probability estimator. For 'u_mass' this doesn't matter. If None - the default window sizes are used which are: 'c_v' - 110, 'c_uci' - 10, 'c_npmi' - 10. coherence : {'u_mass', 'c_v', 'c_uci', 'c_npmi'}, optional Coherence measure to be used. Fastest method - 'u_mass', 'c_uci' also known as `c_pmi`. For 'u_mass' corpus should be provided, if texts is provided, it will be converted to corpus using the dictionary. For 'c_v', 'c_uci' and 'c_npmi' `texts` should be provided (`corpus` isn't needed) topn : int, optional Integer corresponding to the number of top words to be extracted from each topic. processes : int, optional Number of processes to use for probability estimation phase, any value less than 1 will be interpreted as num_cpus - 1. Returns ------- list of (list of (int, str), float) Each element in the list is a pair of a topic representation and its coherence score. Topic representations are distributions of words, represented as a list of pairs of word IDs and their probabilities. )modelrtexts dictionary window_size coherencerY processesT)rYrAc:g|]}|j|fSrvrC)r_idr4rRs r%rz'LdaModel.top_topics..s)HHH3c DL$56HHHr'c|dS)Nrrv)tups r%z%LdaModel.top_topics..s SVr')keyrA)rget_coherence_per_topicr^r rIr"zipsorted)r4rrfrgrhrirYrjcmcoherence_scores str_topicsrSbeststr scored_topicsrRs` @r% top_topicszLdaModel.top_topicssHvUz#yt    5577 __&& ' 'E$UtDDDEHHHHH%HHHG   g & & & &J(899 m););TJJJJr'c*|jtd||j}t|d}tj|\}}|r t ||}|j|fi|S||g|\}} |dt|dz } fdt| D} |s| Sg} g} |D]\}}g}g}td|j D]Z}| |||krF| | |||f| || ||f[| ||ft|d}d |D}| ||f| | | fS) aGet the topic distribution for the given document. Parameters ---------- bow : corpus : list of (int, float) The document in BOW format. minimum_probability : float Topics with an assigned probability lower than this threshold will be discarded. minimum_phi_value : float If `per_word_topics` is True, this represents a lower bound on the term probabilities that are included. If set to None, a value of 1e-8 is used to prevent 0s. per_word_topics : bool If True, this function will also return two extra lists as explained in the "Returns" section. Returns ------- list of (int, float) Topic distribution for the whole document. Each element in the list is a pair of a topic's id, and the probability that was assigned to it. list of (int, list of (int, float), optional Most probable topics per word. Each element in the list is a pair of a word's id, and a list of topics sorted by their relevance to this word. Only returned if `per_word_topics` was set to True. list of (int, list of float), optional Phi relevance values, multiplied by the feature length, for each word-topic combination. Each element in the list is a pair of a word's id and a list of the phi values between this word and each topic. Only returned if `per_word_topics` was set to True. N:0yE>)rrrrrc*g|]\}}|k ||fSrvrv)rrX topicvaluers r%rz0LdaModel.get_document_topics..Ms>   &9gz00 j !   r'Tr@cg|] }|d S)rrvrxs r%rz0LdaModel.get_document_topics..fs===aQqT===r') rr|r is_corpusdict_applyrrrrrnr"rs)r4bowrrrrrrXrphis topic_distdocument_topics word_topicword_phi word_typeweight phi_values phi_topictopic_idsorted_phi_values topics_sorteds ` r%get_document_topicszLdaModel.get_document_topicss"<  ;"&": !"5t<<  9 $ 8  1488"OC00 6  1 /$7"3F 4;v0000 0nncU?nKK t1XE!H -     =Fz=R=R     #" " !$ : : IvJI!!T_55 L L>),0AAL%%tH~i'@(&KLLL$$hXy0I%JKKK OOY 2 3 3 3!'z4 @ @ @ ==+<===M   y-8 9 9 9 9 H44r'cb||j}t|d}t|tr'|j|gdd}g}t d|jD]A}|j|||kr(| ||j||fB|S)aGet the most relevant topics to the given word. Parameters ---------- word_id : int The word for which the topic distribution will be computed. minimum_probability : float, optional Topics with an assigned probability below this threshold will be discarded. Returns ------- list of (int, float) The relevant topics represented as pairs of their ID and their assigned probability, sorted by relevance to the given word. Nr{r) rr|rrrmdoc2bowrrnrr")r4word_idrvaluesrs r%get_term_topicszLdaModel.get_term_topicsks"  ;"&": !"5t<< gs # # <l**G955a8;Ga11 O OH)'26II O x)9()CG)LMNNN r'r cttttd}||vrSdd|D} t d| tj s't dj ||}  } } | j d| j d}} d}fdt| D}fd t|D}|d kr||} } |rB| |ks Jd tj| }|rtj| t }n5tj| |f}|rtj| |ft }tj|j D]}|d}|r|}n|d }| | || |||<|r||||z}||||}t!|dt't)||}t!|dt't)||}||g||<|rAtjtj|dkr|tj|z}||fS)aCalculate the difference in topic distributions between two models: `self` and `other`. Parameters ---------- other : :class:`~gensim.models.ldamodel.LdaModel` The model which will be compared against the current object. distance : {'kullback_leibler', 'hellinger', 'jaccard', 'jensen_shannon'} The distance metric to calculate the difference with. num_words : int, optional The number of most relevant words used if `distance == 'jaccard'`. Also used for annotating topics. n_ann_terms : int, optional Max number of words in intersection/symmetric difference between topics. Used for annotation. diagonal : bool, optional Whether we need the difference between identical topics (the diagonal of the difference matrix). annotation : bool, optional Whether the intersection or difference of words between two topics should be returned. normed : bool, optional Whether the matrix should be normalized or not. Returns ------- numpy.ndarray A difference matrix. Each element corresponds to the difference between the two topics, shape (`self.num_topics`, `other.num_topics`) numpy.ndarray, optional Annotation matrix where for each pair we include the word from the intersection of the two topics, and the word from the symmetric difference of the two topics. Only included if `annotation == True`. Shape (`self.num_topics`, `other_model.num_topics`, 2). Examples -------- Get the differences between each pair of topics inferred by two models .. sourcecode:: pycon >>> from gensim.models.ldamulticore import LdaMulticore >>> from gensim.test.utils import datapath >>> >>> m1 = LdaMulticore.load(datapath("lda_3_0_1_model")) >>> m2 = LdaMulticore.load(datapath("ldamodel_python_3_5")) >>> mdiff, annotation = m1.diff(m2) >>> topic_diff = mdiff # get matrix with difference for each topic pair from `m1` and `m2` )r r jaccardr z, c3@K|]}d|VdS)z`{}`N)formatrs r%rz LdaModel.diff..s."N"N6==#3#3"N"N"N"N"N"Nr'z!Incorrect distance, valid only {}z*The parameter `other` must be of type `{}`rNcRg|]#}d|D$S)ch|]\}}|Srvrvrwrs r% z+LdaModel.diff...sNNNVaqNNNr'rYrZ)rrRrKr4s r%rz!LdaModel.diff..s9kkkSXNNtu9'M'MNNNkkkr'cRg|]#}d|D$S)ch|]\}}|Srvrvrs r%rz+LdaModel.diff...sOOOVaqOOOr'rr)rrRrKr=s r%rz!LdaModel.diff..s<lllTYOOu'7'7I'7'N'NOOOlllr'rzgBoth input models should have same no. of topics, as the diagonal will only be valid in a square matrixr-rr{)r r r r rJr}rxrrrSrTr^r5rrr1rndindexsymmetric_differencerrzabsr|)r4r=distancerK n_ann_termsdiagonal annotationnormed distances valid_keys distance_funcd1d2t1_sizet2_sizeannotation_terms fst_topics snd_topicszrRtopic1topic2 pos_tokens neg_tokenss`` ` r%r6z LdaModel.diffs.^!1"',    9 $ U"N"NY^^=M=M"N"N"NNNJ@GG SSTT T%00 aIPPQUQ^__`` `!(+ ""E$4$4$6$6B8A; kkkkk\abi\j\jkkk lllll]bcj]k]klll y  ,B  Lg% H HH H H H!!A A#%8G4#@#@#@ '7+,,A L#%8Wg,>d#K#K#K Z(( C CE1XF "q$}RZF<|j)|jjtj|dg|Ri|d|vr-tj|jtj|d|H|rFt |tr|g}d|D}thdt|z}ngd}dd g}t |j tr |j d ks`_ for an example on how to work around these issues. See Also -------- :meth:`~gensim.models.ldamodel.LdaModel.load` Load model. Parameters ---------- fname : str Path to the system file where the model will be persisted. ignore : tuple of str, optional The named attributes in the tuple will be left out of the pickled model. The reason why the internal `state` is ignored by default is that it uses its own serialisation rather than the one provided by this method. separately : {list of str, None}, optional If None - automatically detect large numpy/scipy.sparse arrays in the object being stored, and store them into separate files. This avoids pickle memory errors and allows `mmap`'ing large arrays back on load efficiently. If list of str - this attributes will be stored in separate files, the automatic check is not performed in this case. *args Positional arguments propagated to :meth:`~gensim.utils.SaveLoad.save`. **kwargs Key word arguments propagated to :meth:`~gensim.utils.SaveLoad.save`. N.staterm.id2wordcg|]}||Srvrvres r%rz!LdaModel.save..Fs---A1-a---r'>rrmr)rrrmrr2rrrjr0cg|]}||Srvrvrs r%rz!LdaModel.save..Zs55515!555r')ignore separately)rsaversmart_extensionpicklermrrrsetrjrrrzr5r"r0rNrc)r4rVrrrWrXseparately_explicitrSs r%rz LdaModel.savesMX : U DJOE1%BB TT T T TV T T T F " Q Lu'>> from gensim.test.utils import datapath >>> >>> fname = datapath("lda_3_0_1_model") >>> lda = LdaModel.load(fname, mmap='r') mmapNrz+random_state not set so using default valuer.rMrz failed to load state from %s: %srz-failed to load id2word dictionary from %s: %s)getrNrcrOrPrrrrRrrrQr.rArSrTrr)rrospathisfileunpicklerm) rUrVrWrXrY state_fnamer id2word_fnamerSs r%rOz LdaModel.load`s< FD11v*x%%*5B4BBB6BB v~.. K"'"8">">F  OI J J Jvw'' y:FL LUW]WgWprw x x x+E8<<  P#=FtFFFvFFFLL P P P O> Q O O O O O O O O P-eZ@@ 7>>- ( ( c c!& !>!> c c c OQ^`abbbbbbbb c s0C.. D8DDE** F4FFrZ)F) NNNNNNNNF)Nr@)rfrfFT)rf)NNNNrarbrc)NNF)r rdrfFTT)rN) rTr[r\r]rr^r6rrrrrrrrrrrr rTrZr^rWryrrr6rrr_rOr`ras@r%rcrc<s  B#sD"d11"CPR4"DD!&$bj oooobQ#Q#Q#f     4 4 4 4 hhhhT242:AEKO5:AAAAF!.!.!.!.F<<<<|5555n````& 4 4 44444,PT:<2K2K2K2KhTX,1O5O5O5O5b@BEEIk#k#k#k#Z````.OaOaOaOaOaOab>>>>[>>>>>r'rc)'r]rRrrr collectionsrnumpyr scipy.specialrrrgensimrrr gensim.matutilsr r r r rrr gensim.modelsrrgensim.models.callbacksr getLoggerrTrr&SaveLoadr)TransformationABCBaseTopicModelrcrvr'r%rsSSj ######&&&&&&&&######..........43333333,,,,,,  8 $ $$$$N`````u~```Fcccccz+Y-Ecccccr'