EventGraphs

Apr 24, 2013 at 11:47 PM
Hello,

I'm trying to use NodeXL to create an EventGraph from a Twitter hashtag. The problem is NodeXL seems to allow me to import either the follower edges or the Tweet/mentions/replied edges but not both.

What I'm trying to do is something similar to this: http://www.nodexlgraphgallery.org/Pages/Graph.aspx?graphID=4287

What am I missing?

-Bryan
Apr 25, 2013 at 7:12 PM
Bryan:

If you go to NodeXL, Data, Import, From Twitter Search Network in recent versions of NodeXL, there are options to add an edge for each replies-to relationship, mentions relationship, and follows relationship. The options are checkboxes, so you can check some or all of the options.

Please clarify what's happening for you. Are you using a different item on the Import menu, or are you not seeing the options I mentioned in the Import from Twitter Search Network dialog box, or are you specifying the options but not getting the expected edges?

-- Tony
Apr 26, 2013 at 10:06 PM
Tony,

Thanks for the response. I'm seeing the options and checking the boxes, but not getting all of the edges. In my most recent attempt I checked all the options and got follows, tweets and mentions only. If I do not check the follows relationship box, I get tweets, mentions and replies. The problem seems to be when I try to retrieve everything.

-Bryan
Apr 27, 2013 at 3:18 AM
Bryan:

Please tell me the following information, and I'll try to reproduce the problem on my own computer.
  1. The version of NodeXL you are using. You can find out by going to NodeXL, Help, About in the ribbon.
  2. All of your settings in the Import from Twitter Search Network dialog box.
-- Tony
Apr 29, 2013 at 4:44 PM
Tony,

I'm using version 1.0.1.238

My settings are:

Vertex contains: #prepcom

Add an edge for:

"Replies to"
"Mentions"
Tweet that is not a "replies-to" or "mentions"
Follows relationship
(All boxes checked)

Limit to 200

No extra columns

I have a Twitter account and I have authorized NodeXL

Just ran it again this morning (Monday) and still have the same issue; the relationship column on the edges worksheet only records Followed or Tweet.

-Bryan
Coordinator
Apr 29, 2013 at 5:11 PM
Hello!

Thanks for the interest in the NodeXL project.

See:

http://nodexlgraphgallery.org/Pages/Graph.aspx?graphID=4383

for a sample map for your query.

You can use the settings in that workbook to create new maps.

I suspect the issue is that the limit of 200 is insufficient to capture many follows edges. Increasing that limit may capture more.

Please note, however, that after May 7th, 2013 the Twitter 1.0 API will be turned off and replaced with API 1.1.

Twitter API 1.1 has positive and negative aspects. On the plus side, you will get more tweets for a query more quickly than before. On the minus side, the "follows"edge is about to become significantly more difficult to extract from Twitter. This may make your efforts in this thread moot, so I thought you should know!

Regards,
Marc
Apr 29, 2013 at 5:54 PM
Bryan:

Thanks for the details. I'll try your settings later today.

However, the version of NodeXL you are using includes the latest Twitter changes, which (among other things) severely affect how long it takes to get follows relationships. With 200 tweets, I would expect it to take over 10 hours to get all the follows relationships, yet you said you tried your settings again just this morning. Are you not getting a bunch of "Reached Twitter rate limits. Pausing until..." messages at the bottom of the Import from Twitter Search Network dialog box?

-- Tony
Coordinator
Apr 29, 2013 at 6:03 PM
I am still using the 1.0 interface, but I do not expect that to be available in about two weeks.

As you note, the 1.1 API makes the time to do the Follows query so long that a single map is likely to take a day or more. NodeXL will patiently wait that long, but you may not be as willing to wait.
Apr 29, 2013 at 10:30 PM
@Tony,

Yes, I am getting a lot of "Reached Twitter rate limits." This morning's run resulted in 199 edges and 26 nodes, and took about an hour.

@Marc

Thanks much for the fantastic map! I'll definitely include it for the write-up I'll be doing for WMDJunction.com.

For me, the minuses of the new API definitely outweigh the pluses. I boosted the tweet number to 1000 last week and it took about a 1/2 day to capture everything. It still misses a lot of relationships though. Fortunately, my team is using a Google spreadsheet to archive all the tagged tweets (5000+ and growing), so I think I'll be able to clean them up to use later. That's a lot more work and not as simple as NodeXL's search feature, but beggars can't be choosers.

-Bryan
Apr 29, 2013 at 10:41 PM
Edited Apr 29, 2013 at 10:42 PM
Bryan:

My earlier calculations were off; it should take a few hours to get the followers for 200 tweeters.

I didn't want to wait that long during my tests, though, so I installed version 1.0.1.238 and used the following settings:
  1. Add a vertex for each person whose recent tweet contains "#prepcom".
  2. Add an edge for each "Replies-to", "Mentions", "Tweet that is not a replies-to or mentions", and "Follows".
  3. Limit to 30 tweets. (You used 200.)
The results, after one 15-minute pause, were that I got 99 edges, of which one was a Mentions, 29 were Tweet, and 69 were Followed. (I'm referring here to the values in the Relationship column on the Edges worksheet.)

I then unchecked the "Follows" edge option and got 30 edges, of which one was a Mentions and 29 were Tweet.

I then checked only the "Follows" edge option and got 75 "Followed" edges.

My conclusion: Everything worked as expected. There were a few more Followed in the final test, but that was about an hour after the first test. People were tweeting during that time, and so the list of people on the Vertices worksheet--and the corresponding followed edges--changed during my tests.

I'm not certain what's happening in your case, but one possibility is that the tweeting going on during your own tests, which must have been several hours long, made any comparisons invalid. As far as I can tell, the checkboxes are working properly. Let me know if you observe otherwise.

-- Tony
Apr 30, 2013 at 5:42 PM
Tony,

Thank you for your extensive testing and follow up. Maybe the issue is just the time lag. The conference is due to end later this week, and I will be able to perform some additional tests of the import function versus the data in my tweet archive. I'll let you know if I see any differences.

Again, I really appreciate your responsiveness and help in resolving this.

-Bryan