Hello, and good morning.
I'm using NodeXL Pro to import content from Twitter (using search network option). I'm trying to import results for a very popular hashtag in Brazil, that certainly has had more than 18 thousand tweets last week (the hashtag in question is "#justiça",
the name of a tv show and also the portuguese word for justice). It has reached trending topics last friday.
When I hit import though, only 787 tweets were imported. I know for a fact this quantity does not correspond to the reality. What I expected to happen was for NodeXL Pro to import the last 18 thousand tweets available on Twitter API (that lasts for at least
7 days, if I'm not wrong, or 30 if using firehose). I have used NodeXL in other ocasions for that same purpose and suceeded.
Could anyone please explain what is going on and if there is any way to fix it?
Sep 5 at 5:06 PM
Thank you for the interest in NodeXL!
Twitter limits all data access through its API.
Twitter will not deliver more than 18,000 tweets via the REST API in any query.
Twitter will not deliver more than 7 or 8 days of data.
NodeXL Pro provides access to the full Twitter API and is limited to 18,000 tweets or 7-8 days - which ever comes first.
When you perform the same query multiple times, you may get very different results. Just as you can never enter the same river twice, Twitter may have varying levels of activity or server load that leads them to deliver different sets and volumes of tweets.
Only a commercial relationship with one of the Twitter data resellers can get closer to a "complete" data set.
Note, different platforms have different approaches to data access limits. For example, Facebook provides access to data based on a rate limit but does not impose a historical limit - you can collect Page and Group data back to the dawn of time (or 2007).
You can enter advanced "date range" operators in the search term box in the Import from Twitter Search Network dialog box. For example, if you enter this into the search term box:
NodeXL since:2016-09-03 until:2016-09-04
...then you'll get the tweets between those two days. You can read about these and other operators by clicking the "Advanced search help" link in the dialog box.
However, Twitter severely limits how far back you can go. Here is what Twitter says about that:
"The Search API is not complete index of all Tweets, but instead an index of recent Tweets. At the moment that index includes between 6-9 days of Tweets."
(The "Search API" is what NodeXL uses to get tweets from Twitter.) The unfortunate result is that you will not be able to get the June-through-August tweets that you want. There is no way to do that in NodeXL.