§ Øõc¯<ãó—dZGd„d¦«ZedkrgddlZe¦«Zejdd…D]KZee¦«5ZeD] Z e e e ¦«¦«Œ! ddd¦«n#1swxYwYŒJdSdS)aaPorter Stemming Algorithm This is the Porter stemming algorithm, ported to Python from the version coded up in ANSI C by the author. It may be be regarded as canonical, in that it follows the algorithm presented in [1]_, see also [2]_ Author - Vivake Gupta (v@nano.com), optimizations and cleanup of the code by Lars Buitinck. Examples -------- .. sourcecode:: pycon >>> from gensim.parsing.porter import PorterStemmer >>> >>> p = PorterStemmer() >>> p.stem("apple") 'appl' >>> >>> p.stem_sentence("Cats and ponies have meeting") 'cat and poni have meet' >>> >>> p.stem_documents(["Cats and ponies", "have meeting"]) ['cat and poni', 'have meet'] .. [1] Porter, 1980, An algorithm for suffix stripping, http://www.cs.odu.edu/~jbollen/IR04/readings/readings5.pdf .. [2] http://www.tartarus.org/~martin/PorterStemmer có~—eZdZdZd„Zd„Zd„Zd„Zd„Zd„Z d„Z d „Zd „Zd„Z d„Zd „Zd„Zd„Zd„Zd„Zd„Zd„ZdS)Ú PorterStemmera1Class contains implementation of Porter stemming algorithm. Attributes -------- b : str Buffer holding a word to be stemmed. The letters are in b[0], b[1] ... ending at b[`k`]. k : int Readjusted downwards as the stemming progresses. j : int Word length. có0—d|_d|_d|_dS)NÚé)ÚbÚkÚj©Úselfs ú5lib/python3.11/site-packages/gensim/parsing/porter.pyÚ__init__zPorterStemmer.__init__.s€ØˆŒØˆŒØˆŒˆˆócóv—|j|}|dvrdS|dkr|dkp| |dz ¦«SdS)aåCheck if b[i] is a consonant letter. Parameters ---------- i : int Index for `b`. Returns ------- bool Examples -------- .. sourcecode:: pycon >>> from gensim.parsing.porter import PorterStemmer >>> p = PorterStemmer() >>> p.b = "hi" >>> p._cons(1) False >>> p.b = "meow" >>> p._cons(3) True ÚaeiouFÚyréT©rÚ_cons)rÚiÚchs rrzPorterStemmer._cons3sS€ð4ŒVAŒYˆØ ˆ=ð Ø5Ø Š9ð 3Ø˜’6Ð2 §¢¨A°©EÑ!2Ô!2Ð2Ð2Øˆtrcó2—d} ||jkrdS| |¦«sn|dz }Œ)|dz }d} ||jkr|S| |¦«rn|dz }Œ)|dz }|dz } ||jkr|S| |¦«sn|dz }Œ)|dz }Œd)a©Calculate the number of consonant sequences between 0 and j. If c is a consonant sequence and v a vowel sequence, and <..> indicates arbitrary presence, gives 0 vc gives 1 vcvc gives 2 vcvcvc gives 3 Returns ------- int The number of consonant sequences between 0 and j. Examples -------- .. sourcecode:: pycon >>> from gensim.parsing.porter import PorterStemmer >>> p = PorterStemmer() >>> p.b = "aobm" >>> p.j = 11 >>> p._m() 2 rTr)r r)rrÚns rÚ_mzPorterStemmer._mTsó€ð8 ˆð Ø4”6Šzð ØqØ—:’:˜a‘=”=ð ØØ ‰FˆAð ð ˆQ‰ˆØ ˆð ð Øt”v’:ðØHØ—:’:˜a‘=”=ðØØQ‘ð ð ‰FˆAØ ‰FˆAð Øt”v’:ðØHØ—z’z !‘}”}ðØØQ‘ð ð ‰FˆAð rcóf‡—tˆfd„t‰jdz¦«D¦«¦«S)aÜCheck if b[0: j + 1] contains a vowel letter. Returns ------- bool Examples -------- .. sourcecode:: pycon >>> from gensim.parsing.porter import PorterStemmer >>> p = PorterStemmer() >>> p.b = "gnsm" >>> p.j = 3 >>> p._vowelinstem() False >>> p.b = "gensim" >>> p.j = 5 >>> p._vowelinstem() True c3óB•K—|]}‰ |¦«V—ŒdS©N)r)Ú.0rrs €rú z-PorterStemmer._vowelinstem..¡s-øèè€Ð@Ð@¨t—z’z !‘}”}Ð@Ð@Ð@Ð@Ð@Ð@rr)ÚallÚranger r s`rÚ_vowelinstemzPorterStemmer._vowelinstemŠs9ø€õ.Ð@Ð@Ð@Ð@e°D´F¸Q±JÑ.?Ô.?Ð@Ñ@Ô@Ñ@Ô@Ð@Ð@rcóv—|dko3|j||j|dz ko| |¦«S)a5Check if b[j - 1: j + 1] contain a double consonant letter. Parameters ---------- j : int Index for `b` Returns ------- bool Examples -------- .. sourcecode:: pycon >>> from gensim.parsing.porter import PorterStemmer >>> p = PorterStemmer() >>> p.b = "real" >>> p.j = 3 >>> p._doublec(3) False >>> p.b = "really" >>> p.j = 5 >>> p._doublec(4) True rrr)rr s rÚ_doubleczPorterStemmer._doublec£s7€ð81ŠuÐE˜œ œ d¤f¨Q°©U¤mÒ3ÐE¸¿ º À1¹ ¼ ÐErcóº—|dksE| |¦«r0| |dz ¦«s| |dz ¦«sdS|j|dvS)a[Check if b[j - 2: j + 1] makes the (consonant, vowel, consonant) pattern and also if the second 'c' is not 'w', 'x' or 'y'. This is used when trying to restore an 'e' at the end of a short word, e.g. cav(e), lov(e), hop(e), crim(e), but snow, box, tray. Parameters ---------- i : int Index for `b` Returns ------- bool Examples -------- .. sourcecode:: pycon >>> from gensim.parsing.porter import PorterStemmer >>> p = PorterStemmer() >>> p.b = "lib" >>> p.j = 2 >>> p._cvc(2) True >>> p.b = "dll" >>> p.j = 2 >>> p._cvc(2) False >>> p.b = "wow" >>> p.j = 2 >>> p._cvc(2) False érFÚwxy)rr)rrs rÚ_cvczPorterStemmer._cvcÁsg€ðD ˆqŠ5ð ˜Ÿ š 1™ œ ð ¨¯ª°A¸±EÑ):Ô):ð À$Ç*Â*ÈQÐQRÉUÑBSÔBSð Ø5ØŒvaŒy Ð%Ð%rcóî—|d|j|jkrdSt|¦«}||jdzkrdS|j|j|z dz|jdz…|krdS|j|z |_dS)a¶Check if b[: k + 1] ends with `s`. Parameters ---------- s : str Returns ------- bool Examples -------- .. sourcecode:: pycon >>> from gensim.parsing.porter import PorterStemmer >>> p = PorterStemmer() >>> p.b = "cowboy" >>> p.j = 5 >>> p.k = 2 >>> p._ends("cow") True éÿÿÿÿFrT)rrÚlenr )rÚsÚlengths rÚ_endszPorterStemmer._endsçs†€ð0 ˆRŒ5D”F˜4œ6”NÒ"ð Ø5ÝQ‘”ˆØT”V˜a‘ZÒ ð Ø5ØŒ6$”&˜6‘/ AÑ% d¤f¨q¡jÐ0Ô1°QÒ6ð Ø5Ø”˜&‘ˆŒØˆtrcó|—|jd|jdz…|z|_t|j¦«dz |_dS)zbAppend `s` to `b`, adjusting `k`. Parameters ---------- s : str Nr)rr r*r©rr+s rÚ_settozPorterStemmer._setto s8€ð”˜˜œ !™˜Ô$ qÑ(ˆŒÝT”V‘”˜q‘ˆŒˆˆrcód—| ¦«dkr| |¦«dSdS)Nr)rr0r/s rÚ_rzPorterStemmer._rs3€Ø7Š7‰9Œ9qŠ=ð ØKŠK˜‰NŒNˆNˆNˆNð ð rcóp—|j|jdkrz| d¦«r|xjdzc_nT| d¦«r| d¦«n)|j|jdz dkr|xjdzc_| d¦«r,| ¦«dkr|xjdzc_dSdS| d ¦«s| d ¦«r5| ¦«r|j|_| d¦«r| d¦«dS| d ¦«r| d¦«dS| d¦«r| d¦«dS| |j¦«r+|j|jdz dvr|xjdzc_dSdS| ¦«dkr5| |j¦«r| d¦«dSdSdSdSdS)aÙGet rid of plurals and -ed or -ing. caresses -> caress ponies -> poni ties -> ti caress -> caress cats -> cat feed -> feed agreed -> agree disabled -> disable matting -> mat mating -> mate meeting -> meet milling -> mill messing -> mess meetings -> meet r+Ússesr%ÚiesrrÚeedrÚedÚingÚatÚateÚblÚbleÚizÚizeÚlszÚeN) rrr-r0rr!r r#r'r s rÚ_step1abzPorterStemmer._step1absR€ð,Œ6$”&Œ>˜SÒ ð ØzŠz˜&Ñ!Ô!ð Ø”˜!‘”Ø—’˜EÑ"Ô"ð Ø—’˜CÑ Ô Ð Ð Ø”˜œ ™ Ô# sÒ*ð Ø”˜!‘”Ø:Š:eÑÔð !ØwŠw‰yŒy˜1Š}ð Ø”˜!‘”ð ð àjŠj˜ÑÔð ! $§*¢*¨UÑ"3Ô"3ñ !¸×9JÒ9JÑ9LÔ9Lñ !Ø”VˆDŒFØzŠz˜$ÑÔð !Ø—’˜EÑ"Ô"Ð"Ð"Ð"Ø—’˜DÑ!Ô!ð !Ø—’˜EÑ"Ô"Ð"Ð"Ð"Ø—’˜DÑ!Ô!ð !Ø—’˜EÑ"Ô"Ð"Ð"Ð"Ø—’˜tœvÑ&Ô&ð !Ø”6˜$œ& 1™*Ô%¨UÐ2ð ØF”F˜a‘KF”FFFð ð à—’‘”˜a’ð ! D§I¢I¨d¬fÑ$5Ô$5ð !Ø—’˜CÑ Ô Ð Ð Ð ð !ð !ð !ð !ð !ð !ð !ð !rcó˜—| d¦«r2| ¦«r |jd|j…dz|_dSdSdS)zATurn terminal 'y' to 'i' when there is another vowel in the stem.rNr)r-r!rrr s rÚ_step1czPorterStemmer._step1cFsW€à:Š:c‰?Œ?ð +˜t×0Ò0Ñ2Ô2ð +Ø”V˜G˜TœV˜G”_ sÑ*ˆDŒFˆFˆFð +ð +ð +ð +rcóè—|j|jdz }|dkrZ| d¦«r| d¦«d+S| d¦«r| d¦«d+Sd+S|dkrZ| d¦«r| d ¦«d+S| d ¦«r| d¦«d+Sd+S|dkr.| d ¦«r| d¦«d+Sd+S|dkrÞ| d¦«r| d¦«d+S| d¦«r| d¦«d+S| d¦«r| d¦«d+S| d¦«r| d¦«d+S| d¦«r| d¦«d+Sd+S|dkr†| d¦«r| d¦«d+S| d¦«r| d¦«d+S| d¦«r| d¦«d+Sd+S|dkr²| d¦«r| d¦«d+S| d¦«r| d ¦«d+S| d!¦«r| d"¦«d+S| d#¦«r| d¦«d+Sd+S|d$kr†| d%¦«r| d¦«d+S| d&¦«r| d ¦«d+S| d'¦«r| d¦«d+Sd+S|d(kr,| d)¦«r| d*¦«d+Sd+Sd+S),z®Map double suffices to single ones. So, -ization ( = -ize plus -ation) maps to -ize etc. Note that the string before the suffix must give _m() > 0. rÚaÚationalr:ÚtionalÚtionÚcÚenciÚenceÚanciÚancer@Úizerr>ÚlÚblir<ÚalliÚalÚentliÚentÚeliÚousliÚousÚoÚizationÚationÚatorr+ÚalismÚivenessÚiveÚfulnessÚfulÚousnessÚtÚalitiÚivitiÚbilitiÚgÚlogiÚlogN©rrr-r2©rrs rÚ_step2zPorterStemmer._step2Ks€ðŒVD”F˜Q‘JÔ ˆØ Š9ð3 ØzŠz˜)Ñ$Ô$ð Ø—’˜‘”Ø—’˜HÑ%Ô%ð Ø—’˜‘”ð ð à 3ŠYð. ØzŠz˜&Ñ!Ô!ð Ø—’˜‘”Ø—’˜FÑ#Ô#ð Ø—’˜‘”ð ð à 3ŠYð) ØzŠz˜&Ñ!Ô!ð Ø—’˜‘”ð ð à 3ŠYð& ØzŠz˜%Ñ Ô ð Ø—’˜‘”ð—’˜FÑ#Ô#ð Ø—’˜‘ ” Ø—’˜GÑ$Ô$ð Ø—’˜‘”Ø—’˜EÑ"Ô"ð Ø—’˜‘”Ø—’˜GÑ$Ô$ð Ø—’˜‘”ð ð à 3ŠYð ØzŠz˜)Ñ$Ô$ð Ø—’˜‘”Ø—’˜GÑ$Ô$ð Ø—’˜‘”Ø—’˜FÑ#Ô#ð Ø—’˜‘”ð ð à 3ŠYð ØzŠz˜'Ñ"Ô"ð Ø—’˜‘ ” Ø—’˜IÑ&Ô&ð Ø—’˜‘”Ø—’˜IÑ&Ô&ð Ø—’˜‘”Ø—’˜IÑ&Ô&ð Ø—’˜‘”ð ð à 3ŠYð ØzŠz˜'Ñ"Ô"ð Ø—’˜‘ ” Ø—’˜GÑ$Ô$ð Ø—’˜‘”Ø—’˜HÑ%Ô%ð Ø—’˜‘”ð ð à 3ŠYð ØzŠz˜&Ñ!Ô!ð Ø—’˜‘”ð ð ð ð rcóÒ—|j|j}|dkr†| d¦«r| d¦«dS| d¦«r| d¦«dS| d¦«r| d¦«dSdS|dkr.| d ¦«r| d¦«dSdS|d krZ| d¦«r| d¦«dS| d¦«r| d¦«dSdS|d kr,| d¦«r| d¦«dSdSdS)z=Deal with -ic-, -full, -ness etc. Similar strategy to _step2.r@ÚicateÚicÚativerÚalizerRrÚicitirOÚicalr`r+ÚnessNrirjs rÚ_step3zPorterStemmer._step3‰s‚€à ŒVD”FŒ^ˆØ Š9ð ØzŠz˜'Ñ"Ô"ð Ø—’˜‘ ” Ø—’˜GÑ$Ô$ð Ø—’˜‘”Ø—’˜GÑ$Ô$ð Ø—’˜‘ ” ð ð à 3ŠYð ØzŠz˜'Ñ"Ô"ð Ø—’˜‘ ” ð ð à 3ŠYð ØzŠz˜&Ñ!Ô!ð Ø—’˜‘ ” Ø—’˜EÑ"Ô"ð Ø—’˜‘”ð ð à 3ŠYð ØzŠz˜&Ñ!Ô!ð Ø—’˜‘”ð ð ð ð rcó¶—|j|jdz }|dkr| d¦«sdSnþ|dkr.| d¦«s| d¦«sdSnÊ|dkr| d ¦«sdSn«|d kr| d¦«sdSnŒ|dkr.| d ¦«s| d¦«sdSnX|dkr]| d¦«rn;| d¦«rn$| d¦«rn | d¦«rn÷dS|dkrB| d¦«r|j|jdvrnÅ| d¦«rn¯dS|dkr| d¦«sdSn|dkr-| d¦«s| d¦«sdSn\|dkr| d¦«sdSn>|dkr| d ¦«sdSn |d!kr| d"¦«sdSndS| ¦«dkr|j|_dSdS)#z2Takes off -ant, -ence etc., in context vcvc.rrErRNrIrMrKr@ÚerrrnrOÚableÚiblerÚantÚementÚmentrTrXÚionÚstÚour+Úismrbr:ÚitiÚurWÚvr^Úzr>)rrr-r rrjs rÚ_step4zPorterStemmer._step4ŸsÙ€à ŒVD”F˜Q‘JÔ ˆØ Š9ð2 Ø—:’:˜dÑ#Ô#ð Øñ à 3ŠYð/ Ø—:’:˜fÑ%Ô%ð ¨d¯jªj¸Ñ.@Ô.@ð ØùØ 3ŠYð, Ø—:’:˜dÑ#Ô#ð Øñ à 3ŠYð) Ø—:’:˜dÑ#Ô#ð Øñ à 3ŠYð& Ø—:’:˜fÑ%Ô%ð ¨d¯jªj¸Ñ.@Ô.@ð ØùØ 3ŠYð# ØzŠz˜%Ñ Ô ð ÙØ—’˜GÑ$Ô$ð ÙØ—’˜FÑ#Ô#ð ÙØ—’˜EÑ"Ô"ð ØàØ 3ŠYð ØzŠz˜%Ñ Ô ð T¤V¨D¬F¤^°tÐ%;ð ØØ—’˜DÑ!Ô!ð ØðØ 3ŠYð Ø—:’:˜eÑ$Ô$ð Øð à 3ŠYð Ø—:’:˜eÑ$Ô$ð ¨T¯ZªZ¸Ñ->Ô->ð ØøØ 3ŠYð Ø—:’:˜eÑ$Ô$ð Øð à 3ŠYð Ø—:’:˜eÑ$Ô$ð Øð à 3ŠYð Ø—:’:˜eÑ$Ô$ð Øð ð ˆFØ7Š7‰9Œ9qŠ=ð Ø”VˆDŒFˆFˆFð ð rcó—|jx}|_|j|dkrH| ¦«}|dks|dkr(| |dz ¦«s|xjdzc_|j|jdkrD| |j¦«r,| ¦«dkr|xjdzc_dSdSdSdS)z?Remove a final -e if _m() > 1, and change -ll to -l if m() > 1.r@rrON)rr rrr'r#)rrrEs rÚ_step5zPorterStemmer._step5ØsÛ€à”VÐˆˆDŒFØŒ6!Œ9˜Òð Ø—’‘ ” ˆAØ1Šuð ˜˜ašð ¨¯ ª °!°a±%Ñ(8Ô(8ð Ø”˜!‘”ØŒ6$”&Œ>˜SÒ ð T§]¢]°4´6Ñ%:Ô%:ð ¸t¿wºw¹y¼yÈ1º}ð ØˆFŒFa‰KˆFŒFˆFˆFð ð ð ð ð ð rcó˜—| ¦«}t|¦«dz }|dkr|S||_||_| ¦«| ¦«| ¦«| ¦«| ¦«| ¦«|jd|jdz…S)aStem the word `w`. Parameters ---------- w : str Returns ------- str Stemmed version of `w`. Examples -------- .. sourcecode:: pycon >>> from gensim.parsing.porter import PorterStemmer >>> p = PorterStemmer() >>> p.stem("ponies") 'poni' rN) Úlowerr*rrrArCrkrtr„r†)rÚwrs rÚstemzPorterStemmer.stemâs¦€ð. GŠG‰IŒIˆÝ‰FŒFQ‰JˆØŠ6ð ØˆHðˆŒØˆŒà Š ‰ŒˆØŠ‰ŒˆØŠ‰ Œ ˆ ØŠ‰ Œ ˆ ØŠ‰ Œ ˆ ØŠ‰ Œ ˆ ØŒvkt”v ‘zkÔ"Ð"rcój‡—d ˆfd„| ¦«D¦«¦«S)aÖStem the sentence `txt`. Parameters ---------- txt : str Input sentence. Returns ------- str Stemmed sentence. Examples -------- .. sourcecode:: pycon >>> from gensim.parsing.porter import PorterStemmer >>> p = PorterStemmer() >>> p.stem_sentence("Wow very nice woman with apple") 'wow veri nice woman with appl' ú c3óB•K—|]}‰ |¦«V—ŒdSr)rŠ©rÚxrs €rrz.PorterStemmer.stem_sentence..%s-øèè€Ð:Ð:¨˜Ÿ š !™œÐ:Ð:Ð:Ð:Ð:Ð:r)ÚjoinÚsplit)rÚtxts` rÚ stem_sentencezPorterStemmer.stem_sentences4ø€ð.xŠxÐ:Ð:Ð:Ð:¨c¯iªi©k¬kÐ:Ñ:Ô:Ñ:Ô:Ð:rcó ‡—ˆfd„|D¦«S)aStem documents. Parameters ---------- docs : list of str Input documents Returns ------- list of str Stemmed documents. Examples -------- .. sourcecode:: pycon >>> from gensim.parsing.porter import PorterStemmer >>> p = PorterStemmer() >>> p.stem_documents(["Have a very nice weekend", "Have a very nice weekend"]) ['have a veri nice weekend', 'have a veri nice weekend'] có:•—g|]}‰ |¦«‘ŒS©)r“rŽs €rú z0PorterStemmer.stem_documents..>s'ø€Ð4Ð4Ð4¨!×"Ò" 1Ñ%Ô%Ð4Ð4Ð4rr–)rÚdocss` rÚstem_documentszPorterStemmer.stem_documents'sø€ð.5Ð4Ð4Ð4¨tÐ4Ñ4Ô4Ð4rN)Ú__name__Ú __module__Ú__qualname__Ú__doc__r rrr!r#r'r-r0r2rArCrkrtr„r†rŠr“r™r–rrrr!s:€€€€€ðððððð ðððB4ð4ð4ðlAðAðAð2FðFðFð<$&ð$&ð$&ðL ð ð ðD !ð !ð !ðððð,!ð,!ð,!ð\+ð+ð+ð ;ð;ð;ð|ððð,7ð7ð7ðrððð*#ð*#ð*#ðX;ð;ð;ð25ð5ð5ð5ð5rrÚ__main__rNr)rrršÚsysÚpÚargvÚfÚopenÚinfileÚlineÚprintr“r–rrúr§s$ðððð<]5ð]5ð]5ð]5ð]5ñ]5ô]5ð]5ð@ˆzÒð-Ø€J€J€Jàˆ ‰Œ€Aà ŒXabbŒ\ð-ð-ˆØ ˆT!‰WŒWð -˜Øð -ð -Øa—o’o dÑ+Ô+Ñ,Ô,Ð,Ð,ð -ð -ð -ð -ñ -ô -ð -ð -ð -ð -ð -ð -øøøð -ð -ð -ð -øð -ð-ð -ð-sÁ$A1Á1A5 Á8A5