NodeXL Forum Rss Feed

New Post: Making Twitter Network with Twitter API

owlmail0420 — Sun, 19 May 2013 03:22:20 GMT

Hi, everyone! Can I ask one question?

I have a big big problem with making tiwtter network.

I am doing my class assignment with NodeXL book, and I am stuck in Figure 10.17 on page 157-158.

I couldn't find Twitter replies/metions API. I clicked URL which book suggests on page 157, but it said "could not authenticate you."

So, my request is... how can I get replies and mentions from Twitter and put these things to my network? ;)

Ah, I am using the latest version! and it takes too long time to collect twitter network ;0;

New Post: NodeXLControl Vertices with same location when animating graph.

TonyDaviesHP — Mon, 13 May 2013 17:56:13 GMT

On 12/05/2013 18:47, tcap479 wrote:

From: tcap479

I tried what you're doing and my guess about the cause of the problem turns out to be correct.

I'm not sure what the long-term fix for this should be. For now, you can use either of the workarounds I suggested. They worked when I tried them.

-- Tony

Cheers, will do!

New Post: NodeXLControl Vertices with same location when animating graph.

TonyDaviesHP — Mon, 13 May 2013 17:39:43 GMT

On 09/05/2013 16:04, tcap479 wrote:

From: tcap479

Tony D:

Thanks for all the details, which are very useful. You've given me the information I'll need to reproduce the problem when I get back to a development computer this weekend.

We don't use the control in this manner in our own application, and no one else has reported doing so, either. I mention that only to explain why such an obvious bug has survived for so long.

In the meantime, I think I know what's going on. The Fruchterman-Reingold layout algorithm requires that vertex locations be initially randomized. NodeXL does this automatically the first time it lays out the graph. It then marks the graph with a "graph has been laid out at least once" metadata key, which it uses to avoid randomizing the vertices again if you lay out the graph again. That way the graph can be laid out iteratively, with the previously laid out locations used as the starting point for the next iteration.

You, however, are adding vertices between iterations, and the locations of those additional vertices never get randomized. The added vertices all have default locations of 0,0, and Fruchterman-Reingold doesn't properly lay out vertices with the same locations. (That's why they have to be randomized.)

So as a temporary workaround--or maybe a permanent one, I'm not sure yet--assign random locations to your new vertices before you add them to the graph. Or as a simpler technique, assign Vertex.ID to the X- and Y-coordinates of each added vertex. IDs are unique, so that should keep the vertices from having the same initial locations.

An alternative workaround is to remove the "graph has been laid out at least once" marker from the graph before you add more vertices. That will cause all vertex locations to be randomized with each iteration. You would do that by calling LayoutMetadataUtil.MarkGraphAsNotLaidOut(nodeXLControl.Graph).

In any case, let me know what happens.

-- Tony

Tony,

Again, many thanks for your swift response. Unfortunately I am on leave at present so will not be able to test your suggested fixes until next week but will try then as soon as I return.

Regards

Tony D

New Post: NodeXLControl Vertices with same location when animating graph.

tcap479 — Sun, 12 May 2013 17:47:18 GMT

I tried what you're doing and my guess about the cause of the problem turns out to be correct.

I'm not sure what the long-term fix for this should be. For now, you can use either of the workarounds I suggested. They worked when I tried them.

-- Tony

New Post: NodeXLControl Vertices with same location when animating graph.

tcap479 — Thu, 09 May 2013 15:04:39 GMT

Tony D:

Thanks for all the details, which are very useful. You've given me the information I'll need to reproduce the problem when I get back to a development computer this weekend.

We don't use the control in this manner in our own application, and no one else has reported doing so, either. I mention that only to explain why such an obvious bug has survived for so long.

In the meantime, I think I know what's going on. The Fruchterman-Reingold layout algorithm requires that vertex locations be initially randomized. NodeXL does this automatically the first time it lays out the graph. It then marks the graph with a "graph has been laid out at least once" metadata key, which it uses to avoid randomizing the vertices again if you lay out the graph again. That way the graph can be laid out iteratively, with the previously laid out locations used as the starting point for the next iteration.

You, however, are adding vertices between iterations, and the locations of those additional vertices never get randomized. The added vertices all have default locations of 0,0, and Fruchterman-Reingold doesn't properly lay out vertices with the same locations. (That's why they have to be randomized.)

So as a temporary workaround--or maybe a permanent one, I'm not sure yet--assign random locations to your new vertices before you add them to the graph. Or as a simpler technique, assign Vertex.ID to the X- and Y-coordinates of each added vertex. IDs are unique, so that should keep the vertices from having the same initial locations.

An alternative workaround is to remove the "graph has been laid out at least once" marker from the graph before you add more vertices. That will cause all vertex locations to be randomized with each iteration. You would do that by calling LayoutMetadataUtil.MarkGraphAsNotLaidOut(nodeXLControl.Graph).

In any case, let me know what happens.

-- Tony C

New Post: NodeXLControl Vertices with same location when animating graph.

TonyDaviesHP — Thu, 09 May 2013 10:27:55 GMT

Tony,

Thanks for you quick response.

You have paraphrased the problem correctly: I am adding nodes and edges to the graph and redrawing at which point multiple vertices end up with the same location.

I have tried nodeXLControl.Layout = new Smrf.NodeXL.Layouts.FruchtermanReingoldLayout(); but this does not change the outcome.

To add some more detail; only the last set of vertices to be added are co-located at each redraw, previous duplications are resolved - for example:

At the first pass one vertex (1) is added

At the second pass, four further vertices (2-5) with edges to vertex 1 are added and all have the same location. This is evidenced both visually - they overlap - and by looking at the X and Y location co-ordinates which are identical.

At the third pass, 4 further vertices (6-9) are added with edges to vertices 2 and these will all have the same location. The vertices 2-5 will now have discreet locations.

And so on, leaving the last set of vertices added co-located

Here are the location data for a typical example...

ID: 1, X co-ord: 779.6434, Y co-ord: 261.4258
ID: 2, X co-ord: 585.4472, Y co-ord: 229.007
ID: 3, X co-ord: 399.4764, Y co-ord: 305.0099
ID: 4, X co-ord: 369.0031, Y co-ord: 136.486
ID: 5, X co-ord: 788, Y co-ord: 230.753
ID: 6, X co-ord: 282.7024, Y co-ord: 340.5359
ID: 7, X co-ord: 282.7024, Y co-ord: 340.5359
ID: 8, X co-ord: 282.7024, Y co-ord: 340.5359
ID: 9, X co-ord: 267.7145, Y co-ord: 149.6625
ID: 10, X co-ord: 267.7145, Y co-ord: 149.6625
ID: 11, X co-ord: 267.7145, Y co-ord: 149.6625
ID: 12, X co-ord: 267.7145, Y co-ord: 149.6625
ID: 13, X co-ord: 267.7145, Y co-ord: 149.6625
ID: 14, X co-ord: 267.7145, Y co-ord: 149.6625
ID: 15, X co-ord: 267.7145, Y co-ord: 149.6625
ID: 16, X co-ord: 267.7145, Y co-ord: 149.6625
ID: 17, X co-ord: 267.7145, Y co-ord: 149.6625
ID: 18, X co-ord: 148.3842, Y co-ord: 41.90063
ID: 19, X co-ord: 267.7145, Y co-ord: 149.6625
ID: 20, X co-ord: 267.7145, Y co-ord: 149.6625
ID: 21, X co-ord: 16.57254, Y co-ord: 155.1314
ID: 22, X co-ord: 16.57254, Y co-ord: 155.1314
ID: 23, X co-ord: 16.57254, Y co-ord: 155.1314
ID: 24, X co-ord: 16.57254, Y co-ord: 155.1314
ID: 25, X co-ord: 16.57254, Y co-ord: 155.1314
ID: 26, X co-ord: 16.57254, Y co-ord: 155.1314
ID: 27, X co-ord: 16.57254, Y co-ord: 155.1314
ID: 28, X co-ord: 16.57254, Y co-ord: 155.1314
ID: 29, X co-ord: 16.57254, Y co-ord: 155.1314
ID: 30, X co-ord: 12, Y co-ord: 12.9282
ID: 31, X co-ord: 12, Y co-ord: 12.9282
ID: 32, X co-ord: 12, Y co-ord: 12.9282
ID: 33, X co-ord: 12, Y co-ord: 12.9282
ID: 34, X co-ord: 16.57254, Y co-ord: 155.1314
ID: 35, X co-ord: 16.57254, Y co-ord: 155.1314

Sorry this is so wordy; a picture would, as they say, be worth a thousand!

Regards

Tony D

New Post: Average Geodesic Distance

tcap479 — Thu, 09 May 2013 00:25:00 GMT

Alice:

It doesn't, and I'm not familiar with that particular metric. We may have to add it to our feature request list.

Good luck in your poly sci class. (That's what we called it when I was in school, anyway.)

-- Tony

New Post: Average Geodesic Distance

alice_research — Wed, 08 May 2013 18:40:33 GMT

Thanks so much--that info is really helpful for interpreting the statistic and also gives me an idea of how to recalculate the statistic so that the denominator does not include self-loops (as I am not using those in my dataset).

Does NodeXL calculate statistical significance, for example to compare a statistic between two groups? I have not been able to discover how to find statistical significance in social network analysis, so any help would be fantastic.

Thanks again for all this help. My Political Science professor plans to implement NodeXL in classes and this will all be a great help.

New Post: NodeXLControl Vertices with same location when animating graph.

tcap479 — Wed, 08 May 2013 14:25:32 GMT

Tony D:

I'm not sure I understand the problem you're describing. You're adding vertices and edges to the graph (but not removing any), calling DrawGraph(true), and you find that after the graph is drawn, multiple vertices end up at the exact same vertex locations? Are you determining this visually or by programmatically looking at the Vertex.Location property?

I'm not near a development computer and so I can't try your technique right now to determine if there might be a bug somewhere, but as an experiment, try inserting this line before you redraw the graph:

nodeXLControl.Layout = new Smrf.NodeXL.Layouts.FruchtermanReingoldLayout();

Does that change the results?

-- Tony C

New Post: NodeXLControl Vertices with same location when animating graph.

TonyDaviesHP — Wed, 08 May 2013 08:04:12 GMT

Hello,

I have written a C# Windows Form based application to display the memberOf links in active directory. The user wants to be able to show the hierarchy a level at a time and this is accomplished by creating additional vertices and edges and calling DrawGrapth(true) to re-render the graph for each iteration (a sort of step-by-step animation).
However, when I use this mechanism multiple vertices are created with the same location co-ordinates at each iteration. Does anyone have any idea why this might be happening and what can be done about it?

New Post: How could we extract user/list network from public discussion forum

romani — Tue, 07 May 2013 20:52:01 GMT

Thank you very much for the help. I really appreciate it.

Best regards,
Romani

New Post: twitter rate limit and vertex relationship??

kyoki5290 — Tue, 07 May 2013 14:32:11 GMT

Hi Marc and Tony,

Thank you so much for all your help and input! Tony you were right, I unchecked the edge option for relationships and I got all the data I wanted right away. Also the relationship column now tells me whether it was a tweet, mention, etc., and that's what I was looking for. Before, I was just getting "follows" and for the "relationship date" I was just getting that exact date and time when the data was imported--so all the data had the same date. Now I got all the correct info. Thanks again!

Marc, thanks a bunch for the graph! Looks great so far and am looking forward to really examining it!

Will let you know if I have any more questions.
Thanks to both for such quick replies.
-Aliya

New Post: twitter rate limit and vertex relationship??

tcap479 — Tue, 07 May 2013 02:28:30 GMT

Kyoki:

You are using a recent version of NodeXL that severely limits some of the information you can get from Twitter in a reasonable period of time. This is due to changes imposed by Twitter on programs like NodeXL. A discussion about this issue can be found at http://nodexl.codeplex.com/discussions/437841.

The limit you are running into has to do with "following" relationships. In the Import from Twitter Search Network dialog box, uncheck the edge option for such relationships and you will no longer encounter this 15-minute rate limit. Of course, you won't get followed edges either, but there is nothing NodeXL can do about that.

If you want "mentions" edges, "replies to" edges, and edges that aren't mentions or replies-to (which NodeXL arbitrarily calls "Tweet" edges), then check the corresponding checkboxes for those edges as well. Those types of edges are not rate-limited.

Note that the graph Marc created for you was done with an earlier version of NodeXL that was not subject to the same severe rate limits as recent versions, and so he was able to get followed edges for you. If you try the same thing, it could take hours or even days to get the same network.

-- Tony

New Post: Activate NodeXL Ribbon methods by VBA for Excel

tcap479 — Tue, 07 May 2013 02:09:04 GMT

I tried to do that once and failed. The custom tab that NodeXL adds to the Ribbon is inaccessible from VBA, as far as I can tell.

-- Tony

New Post: twitter rate limit and vertex relationship??

MarcSmith — Tue, 07 May 2013 01:54:10 GMT

The map of the Twitter search network query for constitution pbs is here:
http://nodexlgraphgallery.org/Pages/Graph.aspx?graphID=4532

New Post: twitter rate limit and vertex relationship??

MarcSmith — Tue, 07 May 2013 01:06:24 GMT

Hello!

It could be that you have selected the "Add an edge for each: Follows relationship (slower)."

That "slower" is an understatement.

Which version of the application are you running? Are you logged in when you query?

Regards,
Marc

New Post: twitter rate limit and vertex relationship??

kyoki5290 — Mon, 06 May 2013 22:00:59 GMT

Hi, I am trying to configure some data from twitter on posts that mention, include, tweet or retweet posts with the keywords "constitution pbs". I have done a manual search on twitter and know that there are hundreds of such posts (some of them being mine). However, when I try to import data from twitter, I am almost immediately given the "rate limit exceeded" warning and have to wait 15 minutes.

After that, I only have partial information which gives me data to about 50 rows (my tweets do not show up on the data for some reason). I know there is tons of data but how can I import all of it? I have read various forums on this issue but haven't found a solution. After I click "import from twitter search" a window pops up with many options. I have tried various options but they all yield the same result--very little data.

Also, in the relationships column, I only see "following" concerning the first and second vertex. The first time I attempted this I got proper data which included "mentioned, tweeted" and etc. How can I get that data back?

Hope this wasn't too confusing and thanks for all the help!

New Post: How could we extract user/list network from public discussion forum

MarcSmith — Mon, 06 May 2013 21:51:49 GMT

The details of an edge list and how to import them into NodeXL is discussed in the book, Analyzing social media networks with NodeXL.

The practical issues of collecting data from the message boards you listed involve some basic programming. You will need to "scrape" these sites. A programming language like Python can be useful for this process.

Regards,
Marc

New Post: How could we extract user/list network from public discussion forum

romani — Mon, 06 May 2013 21:47:07 GMT

Dear Marc,

Thank you very much for your help. I really appreciate it. I would be grateful if you just explain me how can I get reply networks in message boards.
Also how can I creat an edge list? Could you please introduce me some related sources.
We would like to extract data for example from the following forums:

http://www.ummah.com/forum/forum.php
http://www.allaturkaa.de/forum/index.php?page=Index&s=63f9e984148c281b0a9d53b9da66732919e26c2c
http://hyvers.hyves.nl/forum/
http://forums.marokko.nl/

I am looking forward to hearing from you.
Best regards,
Romani

New Post: Color for Vertices?

MarcSmith — Mon, 06 May 2013 21:41:23 GMT

Could it be that you have also created groups for this network?

If so, try NodeXL > Analysis > Groups > Group Options >

Set the "What colors should be used for the groups' vertices?" to "The colors specified in the Vertex Color column on the Vertices worksheet."

Do the same for shape, if you'd like to set that attribute at the vertex rather than group level.

Regards,
Marc