ENTRIES TAGGED "social graph"
Graph data is an area that has attracted many enthusiastic entrepreneurs and developers
The popular open source project GraphLab received a major boost early this week when a new company comprised of its founding developers, raised funding to develop analytic tools for graph data sets. GraphLab Inc. will continue to use the open source GraphLab to “push the limits of graph computation and develop new ideas”, but having a commercial company will accelerate development, and allow the hiring of resources dedicated to improving usability and documentation.
While social media placed graph data on the radar of many companies, similar data sets can be found in many domains including the life and health sciences, security, and financial services. Graph data is different enough that it necessitates special tools and techniques. Because tools were a bit too complex for casual users, in the past this meant graph data analytics was the province of specialists. Fortunately graph data is an area that has attracted many enthusiastic entrepreneurs and developers. The tools have improved and I expect things to get much easier for users in the future. A great place to learn more about tools for graph data, is at the upcoming GraphLab Workshop (on July 1st in SF).
Data wrangling: creating graphs
Before you can take advantage of the other tools mentioned in this post, you’ll need to turn your data (e.g., web pages) into graphs. GraphBuilder is an open source project from Intel, that uses Hadoop MapReduce1 to build graphs out of large data sets. Another option is the combination of GraphX/Spark described below. (A startup called Trifacta is building a general-purpose, data wrangling tool, that could help as well. )
A disk-based, single-node, graph analytics system that scales to massive graphs
Designed specifically to run on a single computer with limited memory1 (DRAM), since its release a few months ago GraphChi has been used to analyze graphs with billions of edges. Running on a single machine means deployment and debugging are simpler. In addition it is no longer necessary to find (optimal) graph partitions that minimize communication between compute nodes – the starting point for many distributed graph computations.
The stated goal of GraphChi is to “Compute on graphs with billions of edges, in a reasonable time, on a single PC.” One way to define “reasonable amount of computation time” is to compare against the results produced by other graph processing systems. That’s exactly what GraphChi’s creators did in a recent paper. They found that GraphChi compared favorably to graph analytics packages such as Pegasus and Stanford GPS. While GraphChi was 2-3X slower2 in some cases, it is easier to deploy, easier to debug, and way more energy efficient. Read more…
A Facebook app organizes your friends via shared interests and experiences.
This week's visualization clusters your Facebook friends based on shared education, location, occupation, and interests.
Tony Hirst's Facebook visualization shows "social interest positioning."
Tony Hirst used Google Refine and Gephi to reveal the likes his Facebook friends have in common.
Pinboard founder questions the social graph, Cloudera and Kaggle raise money for big data.
In this week's data news, Pinboard founder Maciej Ceglowski challenges the notion of a "social graph," Cloudera and Kaggle raise money for big data, and the Supreme Court looks at GPS and privacy issues.
Work on data projects that matter, data journalism, and a social graph of the Marvel universe.
This week's big data news includes a call for Data Without Borders, data journalism catches the Knight Foundation's attention, IBM's new big data appliance, and a social graph built around the Marvel universe.