cModZddlZddlZddlZddlZddlm Z ddl m Z ddl m Z ddl m Z ddlmZddlmZmZdd lmZejeZdd Zeejd kZGd de jejZdS)a Online Non-Negative Matrix Factorization. Implementation of the efficient incremental algorithm of Renbo Zhao, Vincent Y. F. Tan et al. `[PDF] `_. This NMF implementation updates in a streaming fashion and works best with sparse corpora. - W is a word-topic matrix - h is a topic-document matrix - v is an input corpus batch, word-document matrix - A, B - matrices that accumulate information from every consecutive chunk. A = h.dot(ht), B = v.dot(ht). The idea of the algorithm is as follows: .. code-block:: text Initialize W, A and B matrices Input the corpus Split the corpus into batches for v in batches: infer h: do coordinate gradient descent step to find h that minimizes (v - Wh) l2 norm bound h so that it is non-negative update A and B: A = h.dot(ht) B = v.dot(ht) update W: do gradient descent step to find W that minimizes 0.5*trace(WtWA) - trace(WtB) l2 norm Examples -------- Train an NMF model using a Gensim corpus .. sourcecode:: pycon >>> from gensim.models import Nmf >>> from gensim.test.utils import common_texts >>> from gensim.corpora.dictionary import Dictionary >>> >>> # Create a corpus from a list of texts >>> common_dictionary = Dictionary(common_texts) >>> common_corpus = [common_dictionary.doc2bow(text) for text in common_texts] >>> >>> # Train the model on the corpus. >>> nmf = Nmf(common_corpus, num_topics=10) Save a model to disk, or reload a pre-trained model .. sourcecode:: pycon >>> from gensim.test.utils import datapath >>> >>> # Save model to disk. >>> temp_file = datapath("model") >>> nmf.save(temp_file) >>> >>> # Load a potentially pretrained model from disk. >>> nmf = Nmf.load(temp_file) Infer vectors for new documents .. sourcecode:: pycon >>> # Create a new corpus, made of previously unseen documents. >>> other_texts = [ ... ['computer', 'time', 'graph'], ... ['survey', 'response', 'eps'], ... ['human', 'system', 'computer'] ... ] >>> other_corpus = [common_dictionary.doc2bow(text) for text in other_texts] >>> >>> unseen_doc = other_corpus[0] >>> vector = Nmf[unseen_doc] # get topic probability distribution for a document Update the model by incrementally training on the new corpus .. sourcecode:: pycon >>> nmf.update(other_corpus) >>> vector = nmf[unseen_doc] A lot of parameters can be tuned to optimize training for your specific case .. sourcecode:: pycon >>> nmf = Nmf(common_corpus, num_topics=50, kappa=0.1, eval_every=5) # decrease training step size The NMF should be used whenever one needs extremely fast and memory optimized topic model. N)halfnorm) interfaces)matutils)utilsTransformedCorpus) basemodelCoherenceModel)solve_hc|ttt|dd|S)N.)tuplemapintsplit)versionprefixs 1lib/python3.11/site-packages/gensim/models/nmf.py version_tuplerss/ S'--,,WfW566 7 77)rceZdZdZ d#dZd$dZd$dZd%dZd&dZd&dZ d'dZ d(dZ d(dZ dZ dZd)dZdZd$dZd Zed!Zd(d"ZdS)*NmfzOnline Non-Negative Matrix Factorization. `Renbo Zhao et al :"Online Nonnegative Matrix Factorization with Outliers" `_ Nd?{Gz?-C6?2MbP? Tc||_||_||_||_||_||_||_| |_| |_| |_ | |_ | |_ tj ||_d|_|jtj||_t#|j|_d|_d|_d|_d|_t.j|_d|_|||dSdS)aE Parameters ---------- corpus : iterable of list of (int, float) or `csc_matrix` with the shape (n_tokens, n_documents), optional Training corpus. Can be either iterable of documents, which are lists of `(word_id, word_count)`, or a sparse csc matrix of BOWs for each document. If not specified, the model is left uninitialized (presumably, to be trained later with `self.train()`). num_topics : int, optional Number of topics to extract. id2word: {dict of (int, str), :class:`gensim.corpora.dictionary.Dictionary`} Mapping from word IDs to words. It is used to determine the vocabulary size, as well as for debugging and topic printing. chunksize: int, optional Number of documents to be used in each training chunk. passes: int, optional Number of full passes over the training corpus. Leave at default `passes=1` if your input is an iterator. kappa : float, optional Gradient descent step size. Larger value makes the model train faster, but could lead to non-convergence if set too large. minimum_probability: If `normalize` is True, topics with smaller probabilities are filtered out. If `normalize` is False, topics with smaller factors are filtered out. If set to None, a value of 1e-8 is used to prevent 0s. w_max_iter: int, optional Maximum number of iterations to train W per each batch. w_stop_condition: float, optional If error difference gets less than that, training of ``W`` stops for the current batch. h_max_iter: int, optional Maximum number of iterations to train h per each batch. h_stop_condition: float If error difference gets less than that, training of ``h`` stops for the current batch. eval_every: int, optional Number of batches after which l2 norm of (v - Wh) is computed. Decreases performance if set too low. normalize: bool or None, optional Whether to normalize the result. Allows for estimation of perplexity, coherence, e.t.c. random_state: {np.random.RandomState, int}, optional Seed for random generator. Needed for reproducibility. N) num_topicsid2word chunksizepasses_kappaminimum_probability _w_max_iter_w_stop_condition _h_max_iter_h_stop_condition eval_every normalizerget_random_state random_statev_maxdict_from_corpuslen num_tokensAB_Ww_stdnpinf_w_error_hupdate)selfcorpusr&r'r(r)kappar+ w_max_iterw_stop_condition h_max_iterh_stop_conditionr0r1r3s r__init__z Nmf.__init__sv% "  #6 %!1%!1$"!2<@@ < : 1&99DLdl++    KK       rc|jj}||j}|r-||dddz S|S)aGet the term-topic matrix learned during inference. Parameters ---------- normalize: bool or None, optional Whether to normalize the result. Allows for estimation of perplexity, coherence, e.t.c. Returns ------- numpy.ndarray The probability for each word in each topic, shape (`num_topics`, `vocabulary_size`). Nr)axis)r:Tr1sumreshape)rAr1 dense_topicss r get_topicszNmf.get_topicssWwy  'I  J,"2"2"2":":"B"B2q"I"II Irc.|||SN)get_document_topics)rAbowepss r __getitem__zNmf.__getitem__s''S111rFc|j}tjjjd}jD] }||dkz } |jjdz}|dks |jkrj}t |}nRt|j}ttj |} | d|dz| | dzdz}g} |} |D]} | | tj |d } fd| D|rd d D| | f|r#td | || | S) aGet the topics sorted by sparsity. Parameters ---------- num_topics : int, optional Number of topics to be returned. Unlike LSA, there is no natural ordering between the topics in NMF. The returned topics subset of all topics is therefore arbitrary and may change between two NMF training runs. num_words : int, optional Number of words to be presented for each topic. These will be the most relevant words (assigned the highest probability for each topic). log : bool, optional Whether the result is also logged, besides being returned. formatted : bool, optional Whether the topic representations should be formatted as strings. If False, they are returned as 2 tuples of (word, probability). normalize: bool or None, optional Whether to normalize the result. Allows for estimation of perplexity, coherence, e.t.c. Returns ------- list of {str, tuple of (str, float)} a list of topics, each represented either as a string (when `formatted` == True) or word-probability pairs. Nrrr r1Treversec:g|]}j||fSr').0idrAtopics r z#Nmf.show_topics../s)CCCrdl2&b 2CCCrz + c"g|] \}}d||fz S)z %.3f*"%s"r\)r^kvs rraz#Nmf.show_topics..1s%#K#K#KTQK1a&$8#K#K#Krztopic #%i (%.3f): %s)r1r<zerosr:shaper&rangeminlistrargsortrPraveljoinappendloggerinfo)rAr& num_wordslog formattedr1sparsityrow chosen_topics sorted_topicsshowntopicsibestnr`s` @r show_topicszNmf.show_topicss6  'I8DGM!,--7 # #C  "HHDGM!$$ > Z4?: J!*--MMZ99J !1(!;!;<.Os9   E\" u %   rrX)r1get_topic_terms)rAtopicidtopnr1s` r show_topiczNmf.show_topic9s]&  'I    !11'4.ps"333ceCj!333r)r:r1rMrrj)rArrr1rzr`s @rrzNmf.get_topic_termsUss&7 #  'I  ! UYY[[ E d;;;3333U3333ru_massrKc Jt|||||||}|} g} D]=tj|d} fd| D} | | >t | | } t| ddS)aGet the topics sorted by coherence. Parameters ---------- corpus : iterable of list of (int, float) or `csc_matrix` with the shape (n_tokens, n_documents) Training corpus. Can be either iterable of documents, which are lists of `(word_id, word_count)`, or a sparse csc matrix of BOWs for each document. If not specified, the model is left uninitialized (presumably, to be trained later with `self.train()`). texts : list of list of str, optional Tokenized texts, needed for coherence models that use sliding window based (i.e. coherence=`c_something`) probability estimator . dictionary : {dict of (int, str), :class:`gensim.corpora.dictionary.Dictionary`}, optional Dictionary mapping of id word to create corpus. If `model.id2word` is present, this is not needed. If both are provided, passed `dictionary` will be used. window_size : int, optional Is the size of the window to be used for coherence measures using boolean sliding window as their probability estimator. For 'u_mass' this doesn't matter. If None - the default window sizes are used which are: 'c_v' - 110, 'c_uci' - 10, 'c_npmi' - 10. coherence : {'u_mass', 'c_v', 'c_uci', 'c_npmi'}, optional Coherence measure to be used. Fastest method - 'u_mass', 'c_uci' also known as `c_pmi`. For 'u_mass' corpus should be provided, if texts is provided, it will be converted to corpus using the dictionary. For 'c_v', 'c_uci' and 'c_npmi' `texts` should be provided (`corpus` isn't needed) topn : int, optional Integer corresponding to the number of top words to be extracted from each topic. processes : int, optional Number of processes to use for probability estimation phase, any value less than 1 will be interpreted as num_cpus - 1. Returns ------- list of (list of (int, str), float) Each element in the list is a pair of a topic representation and its coherence score. Topic representations are distributions of words, represented as a list of pairs of word IDs and their probabilities. )modelrBtexts dictionary window_size coherencer processesT)rrZc:g|]}|j|fSr\r])r^_idrAr`s rraz"Nmf.top_topics..s)HHH3c DL$56HHHrc|dSNrr\)tups rz Nmf.top_topics..s SVr)keyrZ)r get_coherence_per_topicrPrrjrmzipsorted)rArBrrrrrrcmcoherence_scores str_topicsrzbeststr scored_topicsr`s` @r top_topicszNmf.top_topicsrsNvUz#yt    5577 __&& ' 'E$UtDDDEHHHHH%HHHG   g & & & &J(899 m););TJJJJrc||j}t|d}t|tr'|j|gdd}g}|j|}||j}|r/|dkr||z}td|j D]'}||}||kr| ||f(|S)a:Get the most relevant topics to the given word. Parameters ---------- word_id : int The word for which the topic distribution will be computed. minimum_probability : float, optional If `normalize` is True, topics with smaller probabilities are filtered out. If `normalize` is False, topics with smaller factors are filtered out. If set to None, a value of 1e-8 is used to prevent 0s. normalize: bool or None, optional Whether to normalize the result. Allows for estimation of perplexity, coherence, e.t.c. Returns ------- list of (int, float) The relevant topics represented as pairs of their ID and their assigned probability, sorted by relevance to the given word. N:0yE>r) r+max isinstancestrr'doc2bowr:r1rMrgr&rm)rAword_idr+r1values word_topicstopic_id word_coefs rget_term_topicszNmf.get_term_topicss*  ;"&": !"5t<< gs # # <l**G955a8;Ggg&  'I  -**Q. - ;??,, ,Ka11 5 5H#H-I// 5 x3444 rc|jtdtj|\}}|rt }|j|fi|St j|g|j}| ||j tj }||j }|r|} | r|| z}fdt|dddfDS)a0Get the topic distribution for the given document. Parameters ---------- bow : list of (int, float) The document in BOW format. minimum_probability : float If `normalize` is True, topics with smaller probabilities are filtered out. If `normalize` is False, topics with smaller factors are filtered out. If set to None, a value of 1e-8 is used to prevent 0s. normalize: bool or None, optional Whether to normalize the result. Allows for estimation of perplexity, coherence, e.t.c. Returns ------- list of (int, float) Topic distribution for the whole document. Each element in the list is a pair of a topic's id, and the probability that was assigned to it. Nr)r+)r4c.g|]\}}r|k ||fSr\r\)r^rprobar+s rraz+Nmf.get_document_topics..sG   U& +02E*E %L   rr)r+rr is_corpusdict_applyr corpus2cscr7 _solveprojr:r<r=r1rM enumerate) rArTr+r1rrBkwargsrdhthe_sums ` rrSzNmf.get_document_topicss,  ;"&": !"5t<<"OC00 6  1.ABBBF4;v0000 0  t 7 7 OOAtwbfO 5 5  'I  eeggG W     '!!!Q$00    rctj||j|jzz |_tj|jtj|j|jf|j z|_ tj |j|jf|_ tj |j|jf|_ dS)zInfer info from the first batch and initialize the matrices. Parameters ---------- v : `csc_matrix` with the shape (n_tokens, chunksize) Batch of bows. )sizer3N)r<sqrtmeanr7r&r;absrrvsr3r:rer8r9)rArds r_setupz Nmf._setupsWQVVXX4?)JKLL & Jlot7dFW   4?DO<==4?DO<==rc |jj}d}t|j|jjD]D\}}|t jt j|||z z }Et j|S)Nr) r:rLrr?r<rMsquaredotr)rArdWtl2doc doc_topicss rl2_normz Nmf.l2_normst WY "13 22 @ @OC "&C*..*<*<$<>>?? ?BBwr{{rc|j}|j}tj}t t jjjrj d}n; t}n*#t$rt dYnwxYw|t|j}t||pd|z}|dkrtddSt t"jjrjdkrt)dt dj||dn||d}t-|D]}t t jjjr/fd t-dj djD} nt/jj} t3| D]\} } t t jjjr=| ddj| j df} | j d} nDj| t;j| j } t| } tj |r#t d || |z| zn#t d || |z| z|j!"| #| j!j$j% _$j$}|r[| dz|z|ks | dz|zdkrCt d&| 'dxj(|dz zc_(xj(|)|j*z c_(xj(|zc_(xj+|dz zc_+xj+| )|j*z c_+xj+|zc_+,|dz }t dj-dS)aTrain the model with new documents. Parameters ---------- corpus : iterable of list of (int, float) or `csc_matrix` with the shape (n_tokens, n_documents) Training corpus. Can be either iterable of documents, which are lists of `(word_id, word_count)`, or a sparse csc matrix of BOWs for each document. If not specified, the model is left uninitialized (presumably, to be trained later with `self.train()`). chunksize: int, optional Number of documents to be used in each training chunk. passes: int, optional Number of full passes over the training corpus. Leave at default `passes=1` if your input is an iterator. eval_every: int, optional Number of batches after which l2 norm of (v - Wh) is computed. Decreases performance if set too low. Nrz input corpus stream has no len()rz(Nmf.update() called with an empty corpusz0Corpus is an iterator, only `passes=1` is valid.zzrunning NMF training, %s topics, %i passes over the supplied corpus of %s documents, evaluating L2 norm every %i documentszunknown number ofc 3vK|]3}dd|tjd|jzfV4dSr)rhrfr()r^col_idxrBrAs r zNmf.update..`sc  111gc&,q/7T^;S&T&TTTU r) num_termsz"PROGRESS: pass %i, at document #%iz%PROGRESS: pass %i, at document #%i/%i)rr4z L2 norm: %sz W error: %s).r)r0r<r=rscipysparsecsc csc_matrixrfr6 TypeErrorrnrorhr(warning collectionsabcIterator ValueErrorr&rgrgrouperrr3 permutationshufflerrr7isinfr:rrr?r4r print_topicsr8rrLr9_solve_wr>)rArBr(r)r0 lencorpus evalafterchunk_overall_idxpass_r chunk_idxchunkrd chunk_lenrs`` rr@z Nmf.update's(  ![F  )JF fel.9 : : @ QII @KK  @ @ @ >????? @  7It~66I JO!y#@AA >  NNE F F F F fko6 7 7 QDK!O QOPP P  & OVI%\%8%8S\^g   6]]@ :@ :E&%,"2"=>> @ Q Q@@  -??$-g$6$63 :3 : 5fel&6&ABB +aaa!2!>!>u{1~!N!NNOA ! II%--e444 +"&/A !$E I8I&& KK<y94y@ KK?y94y@) 7#KKNNN//!TWtz/RRG)Y]i$?9$L)R[^_R_cmQmqrQr)KK t||A???%%a(((+a//!%%**$+++a//!%%**$++ !Q&! M4=9999g3 :@ :@ :sA$$$B  B c0fd}jtjjz }t jD]}tdj j j}xj ||j z zzc_ ||}j tjkr7tj|j z j z jkr |_ dS|_ dS)z Update W.cdtjd|jztjdjjz S)z7An optimized version of 0.5 * trace(WtWA) - trace(WtB).g?zij,ij)r<einsumr:r9)WArAs rerrorzNmf._solve_w..errors77B88829WdgW[W];^;^^ ^rz w_error: %sN)r*r<linalgnormr8rgr,rndebugr>r:rr9 _transformr=rr-)rAreta iter_numberrerror_s` rrz Nmf._solve_ws _ _ _ _ _kBINN46222 !122 # #K LL 6 6 6TV$$B GGsb46k* *GG OO   U2YYF & FFT]2dmCDDtG]] !' "DMM# # #rc t|||fi|S)aApply the transformation to a whole corpus and get the result as another corpus. Parameters ---------- corpus : iterable of list of (int, float) or `csc_matrix` with the shape (n_tokens, n_documents) Training corpus. Can be either iterable of documents, which are lists of `(word_id, word_count)`, or a sparse csc matrix of BOWs for each document. If not specified, the model is left uninitialized (presumably, to be trained later with `self.train()`). chunksize : int, optional If provided, a more effective processing will performed. Returns ------- :class:`~gensim.interfaces.TransformedCorpus` Transformed corpus. r)rArBr(rs rrz Nmf._applys&!vyCCFCCCrctj|jd|j|jtjtjd|j|j}tj|d||xj|zc_dS)zApply boundaries on W.r)outzij,ij->jrN)r<clipr:r4rrmaximum)rAsumsqs rrzNmf._transformsm DJDG4444 *dgtw??@@ 5!'''' 5rctr$|j|jjStjj||SrR) OLD_SCIPYrLrrrr)densers r_dense_dot_csczNmf._dense_dot_cscs>  ;EIIeg&&) )<*..uc:: :rc|j\}}|||_n |j||_|jd}||f}| |j|krtj|}|j} | |} d} t|jD]} t d| | | |} |j |jtj}t#|| | ||j}||z}| r"tj| |z |jkrn|} |S)a]Update residuals and representation (h) matrices. Parameters ---------- v : scipy.sparse.csc_matrix Subset of training corpus. W : ndarray Dictionary matrix. h : ndarray Representation matrix. v_max : float Maximum possible value in matrices. Nrz h_error: %s)rfr4rr<rerLrrgr.rnrrr3rr&astypeint32r r*rr/)rArdWrr4mn batch_sizehshaperWtWh_errorrWtvrrs rrzNmf._solveprojsQw1  !DJJ Z !DJWQZ Z  !6) !  A SffQii !122  K LL 0 0 0%%b!,,C+77HHOOPRPXYYKQS+t{CCF aKF 26'F"233d6LL GGr)NrNrrrrr r!r"r#r$TNrR)r$r$FTN)r$N)NNNrrrK)NN)NNN)__name__ __module__ __qualname____doc__rHrPrVr{rrrrrSrrr@rrr staticmethodrrr\rrrrzs Z Z Z Z x,2222AAAAF    84444:KO:<5K5K5K5Kn,,,,\<@&*/ / / / b>>>*w:w:w:w:r###8DDDD*;;\; 000000rr)r )rcollections.abcrloggingnumpyr< scipy.sparser scipy.statsrgensimrrrgensim.interfacesr gensim.modelsr r gensim.models.nmf_pgdr getLoggerrrnr __version__rTransformationABCBaseTopicModelrr\rrrsJ^^B //////33333333))))))  8 $ $8888 M%+ , , 7 U U U U U * & (@U U U U U r