Problem with showing graph with around 280000 edges

Mar 1, 2015 at 12:25 PM

NodeXL

An unexpected problem occurred. If it occurs again, please copy the details to the clipboard by typing Ctrl-C, then post the details to http://www.codeplex.com/NodeXL/Thread/List.aspx.



Details:



[OutOfMemoryException]: Exception of type 'System.OutOfMemoryException' was thrown.



at System.RuntimeType.InvokeDispMethod(String name, BindingFlags invokeAttr, Object target, Object[] args, Boolean[] byrefModifiers, Int32 culture, String[] namedParameters)

at System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParams)

at System.RuntimeType.ForwardCallToInvokeMember(String memberName, BindingFlags flags, Object target, Int32[] aWrapperTypes, MessageData& msgData)

at Microsoft.Office.Interop.Excel.Range.get_Value(Object RangeValueDataType)

at Smrf.AppLib.ExcelUtil.GetRangeValues(Range range)

at Smrf.NodeXL.ExcelTemplate.VertexWorksheetPopulator.ReadEdgeTable(ListObject oEdgeTable, HashSet`1 oUniqueVertexNames)

at Smrf.NodeXL.ExcelTemplate.VertexWorksheetPopulator.PopulateVertexWorksheet(Workbook workbook, Boolean activateVertexWorksheetWhenDone)

at Smrf.NodeXL.ExcelTemplate.WorkbookReader.ReadWorkbookInternal(Workbook workbook, ReadWorkbookContext readWorkbookContext)

at Smrf.NodeXL.ExcelTemplate.WorkbookReader.ReadWorkbook(Workbook workbook, ReadWorkbookContext readWorkbookContext)

at Smrf.NodeXL.ExcelTemplate.TaskPane.ReadWorkbook(Boolean bLayOutGraph)
Coordinator
Mar 2, 2015 at 12:25 AM
Hello!

Thank you for the interest in NodeXL!

The error message you are getting is caused by a lack of RAM.

A network with 280,000 edges can easily be too large to handle inside NodeXL.

Your machine should have as much RAM as possible (minimum 16GB) to attempt to analyze a network of that size.

NodeXL tends to handle 1K to 10K edges on most 8GB systems.

There are several other tools that may perform better against the scale of your data. One is to use the network calculation package NodeXL uses, SNAP (http://snap.stanford.edu). When called from a program or the command line, SNAP may be able to handle a larger volume of data than it can when called within the context of NodeXL. Other tools, like Gephi or Pajek may also be able to handle that volume.

For working with large data within NodeXL, we would recommend slicing, filtering, or compressing your network to get it into a manageable size. You can take smaller time periods (or randomly sample). You can merge and count duplicate edges to reduce the edge detail while retaining the aggregate connection details. You could cluster and then render the network of just the cluster to cluster connections.

Good luck with your data set!
Regards,
Marc
Mar 2, 2015 at 3:35 AM
Dear Marc,
Thank you so much for your prompt response.
My PC has 12GB of RAM, and I was waiting for almost a day for NodeXL to show up the graph. Finally, the Graph is shown perfectly.
Next, I want to calculate the graph metrics and get this bug:


NodeXL

An unexpected problem occurred. If it occurs again, please copy the details to the clipboard by typing Ctrl-C, then post the details to http://www.codeplex.com/NodeXL/Thread/List.aspx.



Details:



[OutOfMemoryException]: Exception of type 'System.OutOfMemoryException' was thrown.



at System.RuntimeType.InvokeDispMethod(String name, BindingFlags invokeAttr, Object target, Object[] args, Boolean[] byrefModifiers, Int32 culture, String[] namedParameters)

at System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParams)

at System.RuntimeType.ForwardCallToInvokeMember(String memberName, BindingFlags flags, Object target, Int32[] aWrapperTypes, MessageData& msgData)

at Microsoft.Office.Interop.Excel.Range.get_Value(Object RangeValueDataType)

at Smrf.AppLib.ExcelUtil.GetRangeValues(Range range)




at Smrf.NodeXL.ExcelTemplate.VertexWorksheetPopulator.ReadEdgeTable(ListObject oEdgeTable, HashSet`1 oUniqueVertexNames)

at Smrf.NodeXL.ExcelTemplate.VertexWorksheetPopulator.PopulateVertexWorksheet(Workbook workbook, Boolean activateVertexWorksheetWhenDone)

at Smrf.NodeXL.ExcelTemplate.WorkbookReader.ReadWorkbookInternal(Workbook workbook, ReadWorkbookContext readWorkbookContext)

at Smrf.NodeXL.ExcelTemplate.WorkbookReader.ReadWorkbook(Workbook workbook, ReadWorkbookContext readWorkbookContext)

at Smrf.NodeXL.ExcelTemplate.TaskPane.ReadWorkbook(Boolean bLayOutGraph)

Is that also because the scalability of NiodeXL for my case?

Coordinator
Mar 2, 2015 at 4:01 AM
Hello!

That is also an Out of Memory error.

A large graph requires many N^2 calculations that can quickly consume all available memory.

Consider configuring NodeXL so that it skips calculating some or all of the network metrics.

You may also want to use a simple shape in place of a image file to represent each vertex.

These steps can sometimes allow a large network to be partially analyzed within the limits imposed by NodeXL.

Regards,

Marc