community detection

Oct 4, 2013 at 8:08 AM
Hi
when I was running community detection algorithm "Clauset-Newman-Moore" in Node XL
, I faced with this error:
"---------------------------

NodeXL

An unexpected problem occurred. If it occurs again, please copy the details to the clipboard by typing Ctrl-C, then post the details to http://www.codeplex.com/NodeXL/Thread/List.aspx.



Details:



[IOException]: A problem occurred while calling the executable that calculates SNAP graph metrics. Details:

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.





at Smrf.NodeXL.Algorithms.GraphMetricCalculatorBase.CalculateSnapGraphMetrics(IGraph oGraph, SnapGraphMetrics eSnapGraphMetrics, List1 oGroups)

at Smrf.NodeXL.Algorithms.GraphMetricCalculatorBase.CalculateSnapGraphMetrics(IGraph oGraph, SnapGraphMetrics eSnapGraphMetrics)

at Smrf.NodeXL.Algorithms.ClusterCalculator.TryCalculateClustersSnap(IGraph oGraph, SnapGraphMetrics eSnapGraphMetric, BackgroundWorker oBackgroundWorker, ICollection
1& oGraphMetrics)

at Smrf.NodeXL.Algorithms.ClusterCalculator.TryCalculateGraphMetrics(IGraph graph, BackgroundWorker backgroundWorker, ICollection`1& graphMetrics)

at Smrf.NodeXL.ExcelTemplate.ClusterCalculator2.TryCalculateGraphMetrics(IGraph graph, CalculateGraphMetricsContext calculateGraphMetricsContext, GraphMetricColumn[]& graphMetricColumns)

at Smrf.NodeXL.ExcelTemplate.GraphMetricCalculationManager.CalculateGraphMetricsAsyncInternal(CalculateGraphMetricsAsyncArgs oCalculateGraphMetricsAsyncArgs, BackgroundWorker oBackgroundWorker, DoWorkEventArgs oDoWorkEventArgs)

at Smrf.NodeXL.ExcelTemplate.GraphMetricCalculationManager.BackgroundWorker_DoWork(Object sender, DoWorkEventArgs e)

at System.ComponentModel.BackgroundWorker.OnDoWork(DoWorkEventArgs e)

at System.ComponentModel.BackgroundWorker.WorkerThreadStart(Object argument)

OK

"
I run this algorithm on network with 800000 edge.
can anyone help me in this regard?
Coordinator
Oct 4, 2013 at 1:13 PM
This looks like a out of memory error.

How much RAM is in the system?

Is it possible to reduce the size of the data set and try again?

Marc

Sent from my iPhone

Oct 5, 2013 at 4:21 PM
Hi Marc
thank you for your reply
My laptop's RAM is 4GB

can you please say that in what number of edges, Node XL faces with out of memory problem?
Coordinator
Oct 5, 2013 at 4:27 PM
Hard to provide a fixed volume.

I can say 4gb 75is a very small amount of RAM for such a large network.

Over 10k edges is 'big'.

I recommend either 8 or even 24gb of RAM to handle such a large graph.

Marc

Sent from my iPhone

Oct 5, 2013 at 6:09 PM
Edited Oct 5, 2013 at 7:44 PM
This does look like a memory problem, and adding more memory to the computer might help, but I have to point out that NodeXL simply isn't suited to a graph with 800,000 edges. Even if adding more memory gets you past the error message you reported, you will likely run into severe performance problems elsewhere.

I generally tell people that NodeXL works well for a few thousand edges and vertices. Some people with hefty computers and a lot of patience have successfully worked with a few tens of thousands, but 800,000 is pretty much out of the question.

-- Tony
Oct 6, 2013 at 6:41 AM
I selected 100000 edges and the program run successfully.
I have another question: can you suggest a Social Network Analysis software that can process large amount of data. As you know the data extracting from social networks are huge. Furthermore, if we want to sample a small part of a network, we have to use community detection algorithm.
Coordinator
Oct 6, 2013 at 1:41 PM
If you have programming skills you can try using the SNAP library that NodeXL uses. When accessed directly (without the interface overhead of NodeXL) the SNAP library is capable of mega-scale analysis of networks. See: http://snap.stanford.edu

Gephi is also capable of handling larger data sets than NodeXL: see http://gephi.org

-
Marc
Oct 6, 2013 at 7:35 PM
Yes, SNAP is a good choice.
thanks alot