cMdZddlmZddlZ ddlZn#e$rddlZYnwxYwddlm Z ddl m Z ddl m Z ddlmZddlmZed ZGd d ZdS) a This module integrates Spotify's `Annoy `_ (Approximate Nearest Neighbors Oh Yeah) library with Gensim's :class:`~gensim.models.word2vec.Word2Vec`, :class:`~gensim.models.doc2vec.Doc2Vec`, :class:`~gensim.models.fasttext.FastText` and :class:`~gensim.models.keyedvectors.KeyedVectors` word embeddings. .. Important:: To use this module, you must have the ``annoy`` library installed. To install it, run ``pip install annoy``. )absolute_importN)utils)Doc2Vec)Word2Vec)FastText) KeyedVectorszNAnnoy not installed. To use the Annoy indexer, please run `pip install annoy`.c@eZdZdZddZejfdZdZdZ dZ dS) AnnoyIndexeracThis class allows the use of `Annoy `_ for fast (approximate) vector retrieval in `most_similar()` calls of :class:`~gensim.models.word2vec.Word2Vec`, :class:`~gensim.models.doc2vec.Doc2Vec`, :class:`~gensim.models.fasttext.FastText` and :class:`~gensim.models.keyedvectors.Word2VecKeyedVectors` models. Ncd|_d|_||_||_|r|rt |jt r |jj}n`t |jttfr |jj }n2t |jtfr|j}ntd| | |j|jdSdSdS)a Parameters ---------- model : trained model, optional Use vectors from this model as the source for the index. num_trees : int, optional Number of trees for Annoy indexer. Examples -------- .. sourcecode:: pycon >>> from gensim.similarities.annoy import AnnoyIndexer >>> from gensim.models import Word2Vec >>> >>> sentences = [['cute', 'cat', 'say', 'meow'], ['cute', 'dog', 'say', 'woof']] >>> model = Word2Vec(sentences, min_count=1, seed=1) >>> >>> indexer = AnnoyIndexer(model, 2) >>> model.most_similar("cat", topn=2, indexer=indexer) [('cat', 1.0), ('dog', 0.32011348009109497)] NzGOnly a Word2Vec, Doc2Vec, FastText or KeyedVectors instance can be used)indexlabelsmodel num_trees isinstancerdvrrwvr ValueError_build_from_modelget_normed_vectors index_to_key vector_size)selfrrkvs 9lib/python3.11/site-packages/gensim/similarities/annoy.py__init__zAnnoyIndexer.__init__.s0   "  ]Y ]$*g.. lZ]DJ8(<== lZ]DJ88 lZ !jkkk  " "2#8#8#:#:BOR^ \ \ \ \ \ ] ] ] ]c|j||jj|j|jd}t j|dzd5}tj |||ddddS#1swxYwYdS)aSave AnnoyIndexer instance to disk. Parameters ---------- fname : str Path to output. Save will produce 2 files: `fname`: Annoy index itself. `fname.dict`: Index metadata. protocol : int, optional Protocol for pickle. Notes ----- This method saves **only the index**. The trained model isn't preserved. )frr .dictwb)protocolN) r saverrrr ropen_pickledump)rfnamer!dfouts rr"zAnnoyIndexer.saveWs" *(t~QUQ\ ] ] Z . . 5$ LD8 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5s A22A69A6c0|dz}tj|rtj|std|d|d ddlm}n#t $rtwxYwtj |d5}tj | }dddn #1swxYwY|d |_ ||d d |_|j||d |_dS)aaLoad an AnnoyIndexer instance from disk. Parameters ---------- fname : str The path as previously used by ``save()``. Examples -------- .. sourcecode:: pycon >>> from gensim.similarities.index import AnnoyIndexer >>> from gensim.models import Word2Vec >>> from tempfile import mkstemp >>> >>> sentences = [['cute', 'cat', 'say', 'meow'], ['cute', 'dog', 'say', 'woof']] >>> model = Word2Vec(sentences, min_count=1, seed=1, epochs=10) >>> >>> indexer = AnnoyIndexer(model, 2) >>> _, temp_fn = mkstemp() >>> indexer.save(temp_fn) >>> >>> new_indexer = AnnoyIndexer() >>> new_indexer.load(temp_fn) >>> new_indexer.model = model rzCan't find index files 'z' and 'z)' - unable to restore AnnoyIndexer state.r AnnoyIndexrbNrrangularmetricr )ospathexistsIOErrorannoyr+ ImportError_NOANNOYrr#r$loadsreadrr loadr )rr& fname_dictr+rr's rr9zAnnoyIndexer.loadms_8W_ u%% "'..*D*D n5nnnnn   ( ( ( ( ( ( (   N Z D ) ) (Q affhh''A ( ( ( ( ( ( ( ( ( ( ( ( ( ( (;Z#y999  k sA""A4 'B??CCc ddlm}n#t$rtwxYw||d}t |D]\}}|||||j||_||_ dS)Nrr*r-r.) r4r+r5r6 enumerateadd_itembuildrr r )rvectorsr num_featuresr+r vector_numvectors rrzAnnoyIndexer._build_from_models  ( ( ( ( ( ( (   N  < :::"+G"4"4 / / J NN:v . . . . DN###  s cj||d\fdttDS)atFind `num_neighbors` most similar items. Parameters ---------- vector : numpy.array Vector for word/document. num_neighbors : int Number of most similar items Returns ------- list of (str, float) List of most similar items in format [(`item`, `cosine_distance`), ... ] T)include_distancescRg|]#}j|d|dz z f$S))r ).0i distancesidsrs r z-AnnoyIndexer.most_similar..s7UUUSV$a)A,*:&:;UUUr)r get_nns_by_vectorrangelen)rrB num_neighborsrJrKs` @@r most_similarzAnnoyIndexer.most_similars] 55 MT6;;YVUUUUUU3s88__UUUUr)NN) __name__ __module__ __qualname____doc__rrPICKLE_PROTOCOLr"r9rrQrrr r &s']']']']R$)#85555,+"+"+"Z   VVVVVrr )rU __future__rr0cPickler$r5picklegensimrgensim.models.doc2vecrgensim.models.word2vecrgensim.models.fasttextr gensim.modelsrr6r rWrrr`s!  '&&&&& ))))))++++++++++++&&&&&& ;g h hVVVVVVVVVVVVVVVVVVVVs