61 lines
1.8 KiB
Markdown
61 lines
1.8 KiB
Markdown
## Twitter scraper
|
|
Scrape user's tweets :D
|
|
|
|
## Usage:
|
|
|
|
### Unauthenticated
|
|
Example:
|
|
```
|
|
scraper = TweetScraper()
|
|
tweets = scraper.get_tweets_anonymous("<user_id>")
|
|
```
|
|
|
|
This will only allow use of the anonymous user tweets method, other methods will fail.
|
|
|
|
The anonymous method returns a list of tweets from the user as viewed from a logged-out session. It will only return 100 tweets (not necessarily the most recent)
|
|
|
|
|
|
### Authenticated
|
|
Example:
|
|
```
|
|
dotenv.load_dotenv()
|
|
|
|
auth_token = os.environ["AUTH_TOKEN"]
|
|
csrf_token = os.environ["CSRF_TOKEN"]
|
|
|
|
scraper = TweetsScraper(auth_token, csrf_token)
|
|
|
|
user_id = scraper.get_id_from_screen_name("pobnellion")
|
|
user_tweets = scraper.get_tweets(user_id, 100)
|
|
```
|
|
|
|
Allows you to get tweets as a logged in user. Twitter only makes the 2000 ish most recent tweets available, but that should be more than enough.
|
|
|
|
You can either directly pass in the user id to `get_tweets()`, or use `get_id_from_screen_name()` to get the id if you don't have it.
|
|
|
|
To use dotenv, include a `.env` file in the directory with the following contents (no quotes around the values):
|
|
```
|
|
AUTH_TOKEN=<auth token>
|
|
CSRF_TOKEN=<csrf token>
|
|
```
|
|
|
|
You can find your auth and csrf tokens in twitter's cookies (F12 in your browser > storage tab > cookies)
|
|
The auth token cookie is called `auth_token` and the csrf token is called `ct0`
|
|
|
|
### Tweet object
|
|
Contains the text of the tweet, along with the timestamp and some stats (like count, repost count, views, etc)
|
|
|
|
#### Fields:
|
|
- id : tweet id
|
|
- views : view count
|
|
- text : tweet content
|
|
- likes : like count
|
|
- replies : reply count
|
|
- retweets : retweet count
|
|
- quotes : quite tweet count
|
|
- date : post date
|
|
|
|
Printing a tweet object results in an overview:
|
|
|
|
`L:52 RT:2 Q:1 R:3 V:1032 2025-01-20T01:53:57+00:00 Example tweet text`
|