Twitter scraper

Scrape user's tweets :D

Usage:

Unauthenticated

Example:

scraper = TweetScraper()
tweets = scraper.get_tweets_anonymous("<user_id>") 

This will only allow use of the anonymous user tweets method, other methods will fail.

The anonymous method returns a list of tweets from the user as viewed from a logged-out session. It will only return 100 tweets (not necessarily the most recent)

Authenticated

Example:

dotenv.load_dotenv()

auth_token = os.environ["AUTH_TOKEN"]
csrf_token = os.environ["CSRF_TOKEN"]

scraper = TweetsScraper(auth_token, csrf_token)

user_id = scraper.get_id_from_screen_name("pobnellion")
user_tweets = scraper.get_tweets(user_id, 100)

Allows you to get tweets as a logged in user. Twitter only makes the 2000 ish most recent tweets available, but that should be more than enough.

You can either directly pass in the user id to get_tweets(), or use get_id_from_screen_name() to get the id if you don't have it.

To use dotenv, include a .env file in the directory with the following contents (no quotes around the values):

AUTH_TOKEN=<auth token>
CSRF_TOKEN=<csrf token>

You can find your auth and csrf tokens in twitter's cookies (F12 in your browser > storage tab > cookies) The auth token cookie is called auth_token and the csrf token is called ct0

Tweet object

Contains the text of the tweet, along with the timestamp and some stats (like count, repost count, views, etc)

Fields:

• id : tweet id
• views : view count
• text : tweet content
• likes : like count
• replies : reply count
• retweets : retweet count
• quotes : quite tweet count
• date : post date

Printing a tweet object results in an overview:

L:52 RT:2 Q:1 R:3 V:1032 2025-01-20T01:53:57+00:00 Example tweet text