follower sets & the Twitter API

Jul 3, 2014 at 5:52 AM
Hi, I'm trying to collect the follow data for a single Twitter account. That account has 23K followers and 98 following. Ideally I'd surface mutual follows as well, but minimally I'd like to collect the entire set of followers for this one target account. Right now, my imports are only returning 2500 followers or so.

I've previously (under the prior version of the API) used NodeXL to collect larger follower data, running it overnight so that it paused but continued when it hit the rate limit. I'm not getting any pausing though -- it's just stopping after a single cycle and presenting me with the data it gathered then.

Is this something I can fix/improve?

Thanks for your help!

~Amy
Jul 3, 2014 at 5:06 PM
Hello, Amy:

Which version of NodeXL are you using? You can tell by going to NodeXL, Help, About in the Excel ribbon.

I ask because the Twitter importers have been overhauled in recent versions of NodeXL.

-- Tony
Jul 3, 2014 at 5:53 PM
Hi Tony,

I'm using 1.0.1.331.

~Amy
Jul 3, 2014 at 6:31 PM
Hello, Amy:

In newer versions of NodeXL, including the one you have, NodeXL asks Twitter only for each user's 2,000 newest friends. There is a "more about this option" help link in the Import from Twitter Users Network dialog box that explains this. I'll include the help text in my next post.

We made this change because asking for too many friends was unreliable. It worked for some people, but others reported having Twitter abruptly stop in the middle of the network, for no apparent reason. Also, asking for too many friends can result in graphs that are too large for NodeXL to handle.

If you really need the old behavior, you can uninstall your version of NodeXL ( http://nodexl.codeplex.com/discussions/522821 ) and install version 1.0.1.251 instead ( https://nodexl.codeplex.com/releases/view/117300 ). If you do this, please note that we are no longer making updates to that version. All program updates are now made to what's called "NodeXL Excel Template 2014," which is at version 1.0.1.331 as of today.

-- Tony
Jul 3, 2014 at 6:32 PM
Importing the network can take a long time when you select this option.

Due to Twitter rate limiting, NodeXL can import the friends and followers of only fifteen users every fifteen minutes. If you specify more than fifteen usernames (or you specify a Twitter List that has more than fifteen users), NodeXL has to pause before Twitter will allow it to continue importing the network.

If you specify 120 users, for example, and you ask for friends and followers, it will take at least two hours for NodeXL to import the network.

Because it takes so long to import friends and followers, NodeXL doesn't attempt to import all of them. Instead, it asks Twitter for each user's 2,000 newest friends and followers. If a user has 4,000 friends, for example, NodeXL will import only her 2,000 newest friends.
Jul 18, 2014 at 10:48 PM
Twitter indeed allows collecting 75,000 follower IDs immediately, you need to wait 15 minutes for another 75K. DD-CSS lets you download 75,000 follower IDs as a csv file for now, unlimited follower download will hopefully be added soon.
Jul 19, 2014 at 3:55 AM
Edited Jul 19, 2014 at 4:03 AM
Aren't you talking about getting the follower IDs for just one user? We need to get the follower IDs for hundreds, if not thousands of users. Twitter's "GET followers/ids" API lets you get follower IDs for fifteen users before rate limits kick in, at which point you have to wait about fifteen minutes to get follower IDs for fifteen more users.

-- Tony
Jul 19, 2014 at 5:16 AM
Hi Tony, yes you're right I was addressing followers of single user. Thanks for pointing that out. But it looks like Amy is also looking for that, quoting her question:
I'm trying to collect the follow data for a single Twitter account