Merge pull request #87 from taspinar/tweet_html

Tweet html

Author: taspinar

HISTORY.rst

@@ -3,6 +3,10 @@
 ## 0.x.x
 
 TBD
+## 0.5.0 ( 2018-01-11 )
+### Added
+- Added the html code of a tweet message to the Tweet class as one of its attributes
+
 ## 0.4.2 ( 2018-01-09 )
 ### Fixed
 - Fixed backward compatability of the new --lang parameter by placing it at the end of all arguments.

README.md

@@ -31,6 +31,7 @@ Per Tweet it scrapes the following information:
 + Tweet-id
 + Tweet-url
 + Tweet text
++ Tweet html
 + Tweet timestamp
 + No. of likes
 + No. of replies

README.rst

@@ -31,7 +31,7 @@ for anyone looking for older past data to make a model from. With
 TwitterScraper there is no such limitation.
 
 Per Tweet it scrapes the following information: + Username and Full Name
-+ Tweet-id + Tweet-url + Tweet text + Tweet timestamp + No. of likes +
++ Tweet-id + Tweet-url + Tweet text + Tweet html + Tweet timestamp + No. of likes +
 No. of replies + No. of retweets
 
 2. Installation and Usage

changelog.txt

@@ -3,7 +3,11 @@
 ## 0.x.x
 
 TBD
-## 0.4.2 ( 2018-01-9 )
+## 0.5.0 ( 2018-01-11 )
+### Added
+- Added the html code of a tweet message to the Tweet class as one of its attributes
+
+## 0.4.2 ( 2018-01-09 )
 ### Fixed
 - Fixed backward compatability of the new --lang parameter by placing it at the end of all arguments.
 

setup.py

@@ -8,7 +8,7 @@
 
 setup(
     name='twitterscraper',
-    version='0.4.2',
+    version='0.5.0',
     description='Tool for scraping Tweets',
     url='https://github.com/taspinar/twitterscraper',
     author=['Ahmet Taspinar', 'Lasse Schuirmann'],

twitterscraper/tweet.py

@@ -6,7 +6,7 @@
 
 @generate_ordering('timestamp', 'id', 'text', 'user', 'replies', 'retweets', 'likes')
 class Tweet:
-    def __init__(self, user, fullname, id, url, timestamp, text, replies, retweets, likes):
+    def __init__(self, user, fullname, id, url, timestamp, text, replies, retweets, likes, html):
         self.user = user
         self.fullname = fullname
         self.id = id
@@ -16,6 +16,7 @@ def __init__(self, user, fullname, id, url, timestamp, text, replies, retweets,
         self.replies = replies
         self.retweets = retweets
         self.likes = likes
+        self.html = html
 
     @classmethod
     def from_soup(cls, tweet):
@@ -36,6 +37,7 @@ def from_soup(cls, tweet):
             likes = tweet.find(
                 'span', 'ProfileTweet-action--favorite u-hiddenVisually').find(
                     'span', 'ProfileTweet-actionCount')['data-tweet-stat-count'] or '0',
+            html=tweet.find('p', 'tweet-text') or "",
         )
 
     @classmethod