QdM dZddlZddlZddlZddlZddlZddlZddlZddlm Z m Z ddl m Z m Z ddlmZmZddlmZmZGdde ZGd d e ZGd d ZGd deZGddeZdS)ai NLTK Twitter client This module offers methods for collecting and processing Tweets. Most of the functionality depends on access to the Twitter APIs, and this is handled via the third party Twython library. If one of the methods below returns an integer, it is probably a `Twitter error code `_. For example, the response of '420' means that you have reached the limit of the requests you can currently make to the Twitter API. Currently, `rate limits for the search API `_ are divided into 15 minute windows. N)TwythonTwythonStreamer) TwythonErrorTwythonRateLimitError)BasicTweetHandler TweetHandlerI) credsfromfile guess_pathc8eZdZdZdZdZdZdZdZd d Z d S) Streamerz Retrieve data from the Twitter Streaming API. The streaming API requires `OAuth 1.0 `_ authentication. cRd|_d|_tj|||||dS)NT)handler do_continuer__init__selfapp_key app_secret oauth_tokenoauth_token_secrets :lib/python3.11/site-packages/nltk/twitter/twitterclient.pyrzStreamer.__init__0s<  ':{4F     c||_dS)zr Register a method for handling Tweets. :param TweetHandlerI handler: method for viewing Nrrrs rregisterzStreamer.register8  rcD|jrk|jUd|vrO|jxjdz c_|j||j|_dSdSt d||jdS)z8 :param data: response from Twitter API Ntextz$No data handler has been registered.)rrcounterhandle ValueError disconnect on_finish)rdatas r on_successzStreamer.on_success@s   %|'T>>L((A-((L''---'+|'?'?'A'AD$$$"> !!GHHH OO    L " " $ $ $ $ $rc$t|dS)z :param status_code: The status code returned by the Twitter API :param data: The response from Twitter API N)print)r status_coder&s ron_errorzStreamer.on_errorPs krc|jrY |jn5#tjj$r}|t d|Yd}~Od}~wwxYw|jWdSdS)z: Wrapper for 'statuses / sample' API call NError (stream will continue): )rstatusessamplerequests exceptionsChunkedEncodingErrorr))res rr/zStreamer.sampleXs    $$&&&&&;   =>1>>???      s#AAAenc|jrz |dkr|dkrd}t||j|||n5#tjj$r}|td|Yd}~pd}~wwxYw|jxdSdS)z: Wrapper for 'statuses / filter' API call r4z+Please supply a value for 'track', 'follow'trackfollowlangNr-)rr#r.filterr0r1r2r))rr8r9r:msgr3s rr;zStreamer.filteris  B;;6R<<GC$S//) $$5d$KKKK&;   =>1>>???      s:AA6A11A6N)r4r4r5) __name__ __module__ __qualname____doc__rrr'r+r/r;rrr r (s}   %%% "rr cLeZdZdZdZdZddZddZ dd Zd Z ddZ d S)Queryz2 Retrieve data from the Twitter REST API. cRd|_d|_tj|||||dS)a :param app_key: (optional) Your applications key :param app_secret: (optional) Your applications secret key :param oauth_token: (optional) When using **OAuth 1**, combined with oauth_token_secret to make authenticated calls :param oauth_token_secret: (optional) When using **OAuth 1** combined with oauth_token to make authenticated calls NT)rrrrrs rrzQuery.__init__s3 w KASTTTTTrc||_dS)z Register a method for handling Tweets. :param TweetHandlerI handler: method for viewing or writing Tweets to a file. Nrrs rrzQuery.registerrrTcd|D|r#tdtd|dfdtdtdD}fd|D}tj|S) a Given a file object containing a list of Tweet IDs, fetch the corresponding full Tweets from the Twitter API. The API call `statuses/lookup` will fail to retrieve a Tweet if the user has deleted it. This call to the Twitter API is rate-limited. See for details. :param ids_f: input file object consisting of Tweet IDs, one to a line :return: iterable of Tweet objects in JSON format c:g|]}||SrA)strip).0lines r z)Query.expand_tweetids..s%6666tzz||666rzCounted z Tweet IDs in .c*g|]}||dzS)drA)rIiidss rrKz)Query.expand_tweetids..s&GGG!SQW%GGGrrrNc3DK|]}|VdS))idN) lookup_status)rIchunkrs r z(Query.expand_tweetids..s4NN5$,,,66NNNNNNr)r)lenrange itertoolschain from_iterable)rids_fverbose id_chunkschunked_tweetsrPs` @rexpand_tweetidszQuery.expand_tweetidss76666  ? =SXX==U=== > > >HGGGuQC#/F/FGGG NNNNINNN,,^<<`_ for more information about admissible search parameters. :param str keywords: A list of query terms to search for, written as a comma-separated string :param int limit: Number of Tweets to process :param str lang: language :param int max_id: id of the last tweet fetched :param int retries_after_twython_exception: number of retries when searching Tweets before raising an exception :rtype: python generator rbrrNrecent)qcountr: result_typer.z7No Tweets available through REST API for those keywordsNr rRF)rlrmr:rcrnzWaiting for 15 minutes -iz Fatal error in Twython request -z)No more Tweets available through rest api) rrrcsearchminrVr)r!rrtimesleepr) rrarbr:rcretries_after_twython_exceptioncount_from_queryresultsrmresultretriesmcountr3s rrdzQuery.search_tweetss.| :-5999DL  "(DL  kk#c5//("G +,,EzzOPPP$ ")*"5eai"@"F"JDL !*-    $$)$$<++--66FF7 && S%*:":;;++ <. ( &)   444555 7###   <<<===2g==G1     +,,EzzABBB  %  #**"5eai"@"F"JDL !*-    $$)$$<++--66FF7E&&&&&&s$77D// F9&E$$ F1FFc fd|DS)a Convert a list of userIDs into a variety of information about the users. See . :param list userids: A list of integer strings corresponding to Twitter userIDs :rtype: list(json) c<g|]}|S))user_id) show_user)rIuseridrs rrKz+Query.user_info_from_id..%s'EEE6v..EEErrA)ruseridss` ruser_info_from_idzQuery.user_info_from_ids FEEEWEEEErfalsect||||}|D]}|j|dS)a Return a collection of the most recent Tweets posted by the user :param str user: The user's screen name; the initial '@' symbol should be omitted :param int limit: The number of Tweets to recover; 200 is the maximum allowed :param str include_rts: Whether to include statuses which have been retweeted by the user; possible values are 'true' and 'false' ) screen_namerm include_rtsN)get_user_timelinerr")rrrbrr&items r user_tweetszQuery.user_tweets'sY%%#5k&   & &D L   % % % % & &r)T)rNr5)rNr5Nr)r) r=r>r?r@rrr_rhrdrrrArrrCrC{s U U U====6!!!!. () VVVVp F F F&&&&&&rrCc2eZdZdZdZ d d ZdS) TwitterzH Wrapper class with restricted functionality and fewer options. ct|_tdi|j|_t di|j|_dS)NrA)r _oauthr streamerrCqueryrs rrzTwitter.__init__=s?#oo  //4;// ))T[)) rr4TrNNr5Fc .|r|} d} nd} |} |rt|| | } nt|| | || } |rt|} n|r|} d} nd} |} t|| | || } |r`|j| |dkr!|dkr|jdS|j|||dS|j| |dkrtd|j|||dS) an Process some Tweets in a simple manner. :param str keywords: Keywords to use for searching or filtering :param list follow: UserIDs to use for filtering Tweets from the public stream :param bool to_screen: If `True`, display the tweet texts on the screen, otherwise print to a file :param bool stream: If `True`, use the live public stream, otherwise search past public Tweets :param int limit: The number of data items to process in the current round of processing. :param tuple date_limit: The date at which to stop collecting new data. This should be entered as a tuple which can serve as the argument to `datetime.datetime`. E.g. `date_limit=(2015, 4, 1, 12, 40)` for 12:30 pm on April 1 2015. Note that, in the case of streaming, this is the maximum date, i.e. a date in the future; if not, it is the minimum date, i.e. a date in the past :param str lang: language :param bool repeat: A flag to determine whether multiple files should be written. If `True`, the length of each file will be set by the value of `limit`. Use only if `to_screen` is `False`. See also :py:func:`handle`. :param gzip_compress: if `True`, output files are compressed with gzip. N)rbupper_date_limitlower_date_limit)rbrrre gzip_compressrjr4r7z1Please supply at least one keyword to search for.)rbr:) TweetViewer TweetWriterrrr/r;rr#rh) rrar9 to_screenstreamrb date_limitr:rerrrrs rrfzTwitter.tweetsBsV  *) #  # )   !!1!1GG "!1!1+ G  !...GG .#- #'  #' #- !!1!1+ G  L M " "7 + + +2~~&B,, $$&&&&& $$8F$NNNNN J   ( ( (2~~ !TUUU ))(%d)KKKKKr) r4r4TTrNNr5FF)r=r>r?r@rrfrArrrr8sj*** ^L^L^L^L^L^LrrceZdZdZdZdZdS)rz4 Handle data by sending it to the terminal. cp|d}t||||jrdSdS)z Direct data to `sys.stdout` :return: return ``False`` if processing should cease, otherwise return ``True``. :rtype: bool :param data: Tweet object returned by Twitter API rN)r)check_date_limitdo_stop)rr&rs rr"zTweetViewer.handlesEF| d  d### <  F  rc6td|jddSNzWritten z Tweets)r)r!rs rr%zTweetViewer.on_finishs$ ..../////rN)r=r>r?r@r"r%rArrrrs<   00000rrcFeZdZdZ d dZdZd Zd Zd Zd Z dS)rz. Handle data by writing it to a file. Nrf twitter-filesFc||_t||_||_||_||_d|_tj ||||dS)a The difference between the upper and lower date limits depends on whether Tweets are coming in an ascending date order (i.e. when streaming) or descending date order (i.e. when searching past Tweets). :param int limit: number of data items to process in the current round of processing. :param tuple upper_date_limit: The date at which to stop collecting new data. This should be entered as a tuple which can serve as the argument to `datetime.datetime`. E.g. `upper_date_limit=(2015, 4, 1, 12, 40)` for 12:30 pm on April 1 2015. :param tuple lower_date_limit: The date at which to stop collecting new data. See `upper_data_limit` for formatting. :param str fprefix: The prefix to use in creating file names for Tweet collections. :param str subdir: The name of the directory where Tweet collection files should be stored. :param bool repeat: flag to determine whether multiple files should be written. If `True`, the length of each file will be set by the value of `limit`. See also :py:func:`handle`. :param gzip_compress: if `True`, output files are compressed with gzip. N) fprefixr subdirrtimestamped_filefnamereoutputrr)rrbrrrrrers rrzTweetWriter.__init__scL  (( ***,,   tU,<>NOOOOOrc^|j}|j}|r3tj|stj|tj||}d}tj |}|j rd}nd}|d|d|}|S)zD :return: timestamped file name :rtype: str z %Y%m%d-%H%M%Sz.gzr4rLz.json) rrospathexistsmkdirjoindatetimenowstrftimer)rrrrfmt timestampsuffixoutfiles rrzTweetWriter.timestamped_files ,  !7>>&)) !    VW--%))++44S99   FFF55Y55V55rc|jrX|jr tj|jd|_nt|jd|_t d|jtj|}|jr1|j |dz dn|j |dz| ||j rdSd|_dS)z Write Twitter data as line-delimited JSON into one or more files. :return: return `False` if processing should cease, otherwise return `True`. :param data: tweet object returned by Twitter API wz Writing to  zutf-8NF) startinguprgzipopenrrr)jsondumpswriteencoderr)rr& json_datas rr"zTweetWriter.handles ? .! 4"i C88 "4:s33 , ,, - - -Jt$$   0 K  y4/77@@ A A A A K  i$. / / / d### <  Frcztd|jd|jr|jdSdSr)r)r!rclosers rr%zTweetWriter.on_finishsL ..../// ; K         rc|jdkrtj|S|jrdS|j|jkr|dS)NFT)rerrrr!rb _restart_filers rrzTweetWriter.do_continue"sW ;%   ,T22 2 < 5 <4: % %    trc||||_d|_d|_dS)NTr)r%rrrr!rs rrzTweetWriter._restart_file0s7 **,,  r)rNNrfrFF) r=r>r?r@rrr"r%rrrArrrrs ,P,P,P,P\*   4     rr)r@rrrXrrrqr0twythonrrtwython.exceptionsrrnltk.twitter.apirrnltk.twitter.utilr r r rCrrrrArrrs    ,,,,,,,,BBBBBBBB========77777777PPPPPPPPfz&z&z&z&z&Gz&z&z&zhLhLhLhLhLhLhLhLV00000-0000yyyyy-yyyyyr