Shipped Something New 👀

Okkayyy so shipped a new component in Agentgenesis after a looooong time 😭✨

This time it’s the TweetScraper Tool — a handy little tool you can use to scrape tweets from accounts and also extract text from specific tweets. 🐦📤

Internally, I’ve used some magical Apify actors for this. And if you’re wondering...

---

What Even Is Apify?

So Apify is basically a marketplace of services (called actors) where people upload scrapers and automation tools, and you can use them — some are paid, some are freemium.

Now after some digging, I figured out two 🔥 actors that:

• can extract tweets from user accounts,

• extract text from individual tweets,

• and... they’ve been working consistently for a long time, super stable,

• cheap af — you can probably do most of your scraping with just the free $5 monthly credits you get.

---

Isn’t This Just a Wrapper?

Yeah, technically this is a wrapper around those Apify actors... BUT here's why it’s more than that:

• You don’t need to spend hours figuring out which actor to use.

• You don’t need to learn how to plug in and integrate with Apify.

• You don’t even need to worry about stability or pricing.

All that headache is gone. Wrapped up nicely inside this TweetScraperTool class. It’s expandable, time-saving, and makes your life easier. So yeah — a wrapper, but one that actually slaps. 💥

---

How To Use 🛠️

Read the docs once on how to set it up:
👉 Installation Docs

---

Scraping Single Tweet Text

You can scrape a tweet using the tweet ID:

const singleTweetData = await tweetScraper.scrapeTweetText({
  tweetId: "1944299394032882148",
});

Or with the tweet URL:

const singleTweetData = await tweetScraper.scrapeTweetText({
  tweetUrl: "https://x.com/kirat_tw/status/1943192256497762309",
});

Both options work fine — whether you provide the ID or full URL. 🙌

---

Scraping Multiple Latest Tweets from an Account

If you wanna scrape recent tweets from a user:

const latestTweets = await tweetScraper.scrapeTweets("https://x.com/mannupaaji", 5);

Parameters:

• accountUrl — URL of the Twitter account

• count — (optional) how many tweets to fetch (defaults to 10)

• sinceDate — (optional) start date to fetch tweets from (defaults to 2024-03-05)

Wanna try the sinceDate param in action? Let's say you want to scrape 5 tweets from 2 days ago, you can do:

const twoDaysAgo = new Date();
twoDaysAgo.setDate(twoDaysAgo.getDate() - 2);

const tweets = await tweetScraper.scrapeTweets("https://x.com/mannupaaji", 5, twoDaysAgo);

This will start fetching tweets posted from 2 days earlier — super handy when you’re working with fresh timelines or filtering recent buzz. 🔍

---

So That’s It 😎

Copy-paste, experiment, extend the code... go build something cool.
And hey —

• Tag me on Twitter 🐤

• Drop a ⭐ on GitHub 🚀

Happy Tweet Scraping!

By Abir!