Pew Research Center national survey

Rebecca J. Rosen at The Atlantic took a look at legal scholar Daniel J. Solove’s argument that we should care about the government’s collection of our data, but not for the reasons one might think — the collection itself, he argues, isn’t as troubling as the fact that they’re holding the data in perpetuity and that we don’t have access to it. Rosen quotes Solove:

“The NSA program involves a massive database of information that individuals cannot access. … This kind of information processing, which forbids people’s knowledge or involvement, resembles in some ways a kind of due process problem. It is a structural problem involving the way people are treated by government institutions. Moreover, it creates a power imbalance between individuals and the government. … This issue is not about whether the information gathered is something people want to hide, but rather about the power and the structure of government.”

In a similar vein, Moxie Marlinspike at Wired tackled the response from some that the NSA’s data gathering efforts aren’t worrisome if individuals have nothing to hide. He quotes from <a href="“>James Duane, a professor at Regent Law School and former defense attorney, and from Supreme Court Justice Breyer to make the point that federal criminal laws span 50 titles of the United States Code in 27,000 pages and that the actual total number of laws and regulations isn’t exactly known, creating the problem of knowing what you may or may not need to hide. Marlinspike writes:

“If the federal government had access to every email you’ve ever written and every phone call you’ve ever made, it’s almost certain that they could find something you’ve done which violates a provision in the 27,000 pages of federal statues or 10,000 administrative regulations. You probably do have something to hide, you just don’t know it yet.”

Marlinspike also argues that having something to hide is an important part of our growth as a society—the recent legal victories in same-sex marriage and legalization of marijuana “would probably not have been possible without the ability to break the law,” he writes. Furthermore, he argues, a dystopian world where law enforcement is 100% effective opens wide avenues for abuse of power: “…if everyone’s every action were being monitored, and everyone technically violates some obscure law at some time, then punishment becomes purely selective,” Marlinspike writes. “Those in power will essentially have what they need to punish anyone they’d like, whenever they choose, as if there were no rules at all.” You can read his full piece at Wired.

Mark Jaquith writes on Medium that based on news reports of the PRISM program, we don’t actually know what the NSA is doing or how they’re gathering data, making it impossible for citizens to judge whether or not to be outraged. Jaquith notes the troubling lack of technical details in Glenn Greenwald and Ewan MacAskill’s account of the PRISM program at The Guardian and their follow-up article on Edward Snowden, who leaked the story. Jaquith points out the the authors — not Snowden — described the program as “[allowing] the agency to directly and unilaterally seize the communications off the companies’ servers.” Likewise, he notes, the Washington Post reported that “[f]rom inside a company’s data stream the NSA is capable of pulling out anything it likes.”

All of which, if true, calls for outrage. But on the other side, a report at The New York Times tells a slightly different story. Jaquith quotes from the piece:

“But instead of adding a back door to their servers, the companies were essentially asked to erect a locked mailbox and give the government the key, people briefed on the negotiations said. … The data shared in these ways, the people said, is shared after company lawyers have reviewed the FISA request according to company practice. It is not sent automatically or in bulk, and the government does not have full access to company servers.”

Jaquith points out this is “indirect and moderated” access — in direct opposition to the access Greenwald and MacAskill described as “direct and unilateral.” “The difference between these two explanations isn’t some nuanced distinction that only tech geeks should care about,” Jaquith says. “This is the difference between companies voluntarily giving the government direct and unilateral access to arbitrary customer data and companies merely complying with the law in a technically efficient way that doesn’t change the nature of the data received by the government.” He notes that the only way Greenwald and MacAskill can be correct at this point, without offering further corroborating evidence from Snowden, is if everyone — all the companies involved, the sources for The New York Times, the NSA, and the U.S. President — are lying, which he allows isn’t impossible, but stresses that the technical details do matter. “There is no aspect of this story more important,” he says, “than finding out which account is accurate.” You can read his full report at Medium.

Facebook opens an Open Compute data center

Facebook opened its first data center in Europe this week in Luleå, Sweden, housed only with its Open Compute servers. In a post announcing the launch, the company described the data center as “likely to be one of the most efficient and sustainable” centers in the world and explained that the equipment is powered by 100% renewable, locally generated hydro-electric power — power so reliable, they’ve been able to “reduce the number of backup generators required at the site by more than 70 percent.”

Jon Brodkin notes at Ars Technica that the data center’s power usage effectiveness (PUE) rating is an impressive 1.07 and that Facebook plans to post near real-time PUE data for this center, as it does for its US data centers. Brodkin reports that Facebook’s next goal is to provide companies with an alternative to Cisco and other network vendors by releasing an Open Compute design for a top-of-rack switch that will work with any networking software.

Local police now “databanking” DNA

In a post at The New York Times, Joseph Goldstein took a look at the growing DNA-gathering practices of local law enforcement agencies. Instead of waiting for state and federal agencies, local agencies are developing their own DNA databases, and their methods are causing some concern. Goldstein writes:

“These local databases operate under their own rules, providing the police much more leeway than state and federal regulations. And the police sometimes collect samples from far more than those convicted of or arrested for serious offenses — in some cases, innocent victims of crimes who do not necessarily realize their DNA will be saved for future searches.”

Barry Scheck, a co-director of the Innocence Project, told Goldstein that they’ve warned local law enforcement that the public would be “disturbed” when these “rogue, unregulated” databases came to light. Goldstein reports that DNA samples are being taken from people “on the mere suspicion of a crime” and entered into a database regardless of whether or not the subject was charged or found guilty. Samples also are gathered from people to rule them out of a crime — say, a homeowner burglary victim — but then kept on file. Goldstein notes that the Supreme Court’s recent decision in Maryland v. King was the first to address this sort of DNA “databanking” and that it could serve to accelerate the practice. “While that decision said nothing explicit about the authority of local law enforcement to keep DNA databases,” Goldstein reports, “it could well encourage local jurisdictions to push ahead, several experts said.”

Tip us off

News tips and suggestions are always welcome, so please send them along.

Related:

• The NSA is spying on us
• Court case sheds light on FBI stingray surveillance
• FBI documents “strongly suggest” they read citizens emails sans warrants
• What Facebook’s Open Compute Project means
• More Strata Week coverage

O’Reilly Strata Conference — Strata brings together the leading minds in data science and big data — decision makers and practitioners driving the future of their businesses and technologies. Get the skills, tools, and strategies you need to make data work.

Strata Rx Health Data Conference: September 25-27 | Boston, MA
Strata + Hadoop World: October 28-30 | New York, NY
Strata in London: November 15-17 | London, England

Click here for the full visualization.

Users can zoom in and select a particular station to pull up detailed information on how many bikes and spaces are available. The screenshot below shows 32 bikes and 3 spaces available at Lafayette St. and Jersey St. at the time of writing.

Click here for the full visualization.

The bar across the top of the map also links to real-time bike share information in other cities around the world, including London, Toronto, Washington D.C., and Milan. A full list of all available cities can be viewed in the drop-down list in the upper right corner.

In a blog post, O’Brien explains that the data is updated automatically every two to 10 minutes, noting that data generally comes from the bike share provider’s website or their official API, but he also gathers data from third-party data collectors, such as citybik.es.

Hat tip to Susi at Velojoy for highlighting O’Brien’s work.

More visualizations:

• Stravinsky’s “The Rite of Spring”
• Hospital procedure charges across the U.S., compared
• CIA rendition flights of terror suspects
• Real-time Wikipedia edits
• Building collapse rescue efforts

There has been much coverage recently of the Internet of Things, connecting everything from washers and dryers to thermostats to cars to the Internet. Wearable sensors — things like FitBit and health-care-related sensors that can be printed onto fabric or even onto human skin — are also in the spotlight.

Kevin Fitchard reports at GigaOm that researchers at CEA-Leti and three French universities believe these areas are not mutually exclusive and have launched a project around wireless body area networks called CORMORAN. The group believes that one day soon our bodies will be constantly connected to the Internet via sensors and transmitters that “can be used to form cooperative ad hoc networks that could be used for group indoor navigation, crowd-motion capture, health monitoring on a massive scale and especially collaborative communications,” Fitchard writes. He takes a look at some of the benefits and potential applications of such a collaborative network — location-based services would be able to direct users to proper gates or trains in busy airports and train stations, for instance — and some of the pitfalls, such as potential security and privacy issues. You can read his full report at GigaOm.

In related news, wearable sensors — and even our bodies — may not only be used to connect us to a network, but also to identify us as well. Casey Johnston at Ars Technica reports this week on two password protection systems that Motorola presented at All Things Digital’s D11 conference: electronic tattoos and consumable pills that would replace the security tokens from the two-factor authenticator.

The tattoos, Johnston reports, aren’t branded onto skin, but are more like flexible stickers, “islands of high-performance silicon connected by accordion-like structures,” that move with a person’s body and remain adhered to the skin. Johnston does point out, though, that the long-term plan likely is to embed the silicon and wires into the skin “to make the user a proper bionic human.”

The pills are just that — a pill that the user swallows that “turns one’s entire body into an authenticator,” Johnston writes. Regina Dugan, senior vice president of the Advanced Technology and Projects group at Motorola Mobility explained that the pills use stomach acid to power a switch to “[create] an 18-bit EKG-like symbol in your body, and your body becomes the authenticator.” You can read Johnston’s full report at Ars Technica.

The NSA is spying on us

Headline news this week surrounded the U.S. National Security Agency. First, Glenn Greenwald posted a report at The Guardian publishing a leaked copy of a court order that “requires Verizon on an ‘ongoing, daily basis’ to give the NSA information on all telephone calls in its systems, both within the US and between the US and other countries.”

The order covers “all call detail records or ‘telephony metadata,’” Greenwald reports, including telephone numbers, location data, call duration and time of call, and unique identifiers — but it doesn’t cover conversation content. Because the information is classified as “metadata,” or transactional information, as opposed to “communications,” it does not require a warrant, Greenwald explains.

Jane Mayer at the New Yorker took a close look at this “metadata” to see just how bad it could be. Susan Landau, a mathematician and former Sun Microsystems engineer, told her that people don’t understand — “[metadata] is more intrusive than content.” With the amount of information that can be gathered, she explained, “you know exactly what is happening — you don’t need the content.” You can read Mayer’s in-depth report at the New Yorker.

Megan Garber at The Atlantic put together a FAQ on the situation, covering exactly what data is being gathered, who has access to it, what they’re doing with it (“as far as we know”), why it doesn’t violate the Fourth Amendment, if the surveillance is limited to Verizon customers, and much more. You can read the FAQ at The Atlantic.

An additional NSA surveillance program came to light this week as well. Glenn Greenwald and Ewen MacAskill at The Guardian report they received a 41-slide PowerPoint presentation describing a program called PRISM that allows the NSA direct access to the servers at nine U.S. Internet companies: Microsoft, Yahoo, Google Facebook, PalTalk, YouTube, Skype, AOL and Apple, and according to Greenwald’s and MacAskill’s report, “[t]he program is continuing to expand, with other providers due to come online.” According to their report Dropbox is “described as ‘coming soon.’”Harrison Weber

Victor Luckerson at Time broke down the program particulars, noting that the program was established in 2007 with Microsoft and that the program allows NSA officials direct access to 10 types of data: email messages, instant messages, videos, photos, stored data, voice chats, file transfers, video conferences, log-in times and profile details on social networks.

In an in-depth report at the Washington Post, Barton Gellman and Laura Poitras quote from a statement issued by Director of National Intelligence James R. Clapper: “information collected under this program is among the most important and valuable foreign intelligence information we collect, and is used to protect our nation from a wide variety of threats. The unauthorized disclosure of information about this important and entirely legal program is reprehensible and risks important protections for the security of Americans.” The Washington Post also published a selection of the slides with annotations.

Philip Rucker, Sean Sullivan and and Aaron Blake report at the Washington Post that President Obama defended the program, saying that such programs “make a difference in our capacity to anticipate and prevent possible terrorist activity” and that “[y]ou can’t have 100 percent security and also then have 100 percent privacy and zero inconvenience.”

Harrison Weber at The Next Web notes (as do the reports at the Washington Post and The Guardian) that the major Internet companies named as participants have denied involvement in and knowledge of the PRISM program. Weber points out that the NSA wouldn’t have given the Internet companies the program’s name, and that “those who receive national security letters are prevented from discussing their existence by law.”

Temporal cloaks hide telecommunication data

Zeeya Merali reports at Scientific American that ultrasecure communications may soon be at hand via a temporal cloak. “Electrical engineers have used lasers to create a cloak that can hide communications in a ‘time hole’,” Merali writes, “so that it seems as if they were never sent.” The first working temporal cloak was built last year by a team at Cornell, but the cloaking windows opened too rarely to hide data at telecommunication speeds. Merali describes how Joseph Lukens, an electrical engineer at Purdue University and author of this latest study, solved the problem:

“To speed up the cloaking rate, Lukens and his colleagues exploited a wave phenomenon that was first discovered by British inventor Henry Fox Talbot in 1836. When a light wave passes through a series of parallel slits called a diffraction grating, it splits apart. The rays emanating from the slits combine on the other side to create an intricate interference pattern of peaks and troughs. Talbot discovered that this pattern repeats at regular intervals, creating what is now known as a Talbot carpet. There is also a temporal version of this effect in which you manipulate light over time to generate regular periods with zero light intensity, says Lukens. Data can be then be hidden in these holes in time.”

When testing the cloak, the research team determined the cloak was able to hide data added at a rate of 12.7 gigabits per second. You can read more about the research and watch a video showing how it works at Scientific American. You can also access the team’s published paper at Nature.

Tip us off

News tips and suggestions are always welcome, so please send them along.

Related:

• The power of the Internet, wielded by machines and things
• Industrial Internet, the machines are talking (Free Radar Report)
• Court case sheds light on FBI stingray surveillance
• Data brokers know more about us than we know
• More Strata Week coverage

O’Reilly Strata Conference — Strata brings together the leading minds in data science and big data — decision makers and practitioners driving the future of their businesses and technologies. Get the skills, tools, and strategies you need to make data work.Strata Rx Health Data Conference: September 25-27 | Boston, MA
Strata + Hadoop World: October 28-30 | New York, NY
Strata in London: November 15-17 | London, England

Tsioulcas notes that for the past 40 years, Malinowski has been honing his “Music Animation Machine” from a “20-foot printed scroll to the software and iPad apps he’s created.” You can read Tsioulcas’ interview with Malinowski — in which he talks about the benefits of visualizing music, his methods, and what he learns in the process — at NPR.

Related:

• The future of classical music

More visualizations:

• Hospital procedure charges across the U.S., compared
• CIA rendition flights of terror suspects
• Real-time Wikipedia edits
• Building collapse rescue efforts
• Every recorded U.S terror attack 1970-2011

Google urged users this week to take more care in creating passwords. In a post on the Google Blog, Google Software Engineer Diana Smetters offered some guidelines, including using a different password for each online account, keeping them in a safe place, creating a recovery option and making them hard to guess. Smetters suggests using a mix of letters and numbers and avoiding basing passwords on common phrases.

Though industry experts generally applauded Google’s efforts to increase consumer awareness, most agreed the company could do more. Seth Rosenblatt reports at CNET that industry experts Alex Salazar and Mary Landesman feel Google should be pressuring developers and companies to improve their security practices.

Landesman noted, for instance, that using spaces in passwords makes them stronger, but most sites don’t let you do that. Salazar outlined three steps Google could take to make the web safer for consumers: pressure companies to require consumers to choose passwords that are easy to remember but hard to break; be a stronger two-factor authentication advocate; and to publish guidelines for developers — and to do a better job of stressing the importance of protecting your customers. Landesman pointed out that often, blame for password breaches is misplaced on users. “[Password security] is tilted against the user,” she said.

Salazar and Landesman make a valid point. Dan Goodin reports this week on the ease of hacking passwords in a post at Ars Technica. He recounts the exercise Ars deputy editor Nate Anderson undertook in March to crack a list of more than 16,000 cryptographically hashed passcodes; with no password cracking training whatsoever, he was able to decipher nearly half of them. Since then, they’ve given the list to cracking experts to see how the passwords fared.

“The list contained 16,449 passwords converted into hashes using the MD5 cryptographic hash function,” Goodin noted, which is the way most websites store passwords. Their top cracker was able to decipher 90% of passwords on the list, and even their weakest cracker — “who used the least amount of hardware, devoted only one hour, used a tiny word list, and conducted an interview throughout the process” — deciphered 62% of the passwords. You can read the full account of the exercise, including methods and tools used by the cracker team, at Ars Technica.

As sensors become more prolific, a new breed of tiny computers is emerging

Wired’s Bryan Gardiner put together an interactive story this week looking at the various kinds of sensors, what they track, and how the data is used to benefit humans in various ways. Readers can hover over the arrows placed in the images to reveal a detail shot of the sensor being highlighted, along with information about the type of sensor employed, the data it gathers, and how it’s used. The example below shows how sensors are being used to aid us in our everyday activities — sensors placed behind athletes’ ears can help detect concussions.

Click here to access the full story at Wired.

Click here to access the full story at Wired.

His presentation also includes the worldwide sensor network Deep-Ocean Assessment and Reporting of Tsunamis (DART) Buoy Array and a network of magnetometer sensors in San Francisco helping drivers find parking spaces. You can see his full story at Wired.

And as sensors need to go more and more places to gather important data — even inside the human body — the computers and processors are going to need to shrink quite a bit in size. Tom Simonite at MIT Technology Review reports this week on a new computer chip, the KLO2 chip, being developed by Freescale. “The genesis of the chip,” Simonite writes, “was a customer asking for help creating a wireless device small enough to be easily swallowed and cheap enough to be considered ‘digestible.’” According to the chip’s specs, it measures in at 1.9 mm by 2.0 mm, which contains, Simonite notes, “memory, RAM, a processor, and more.” You can read Simonite’s full report, including how Freescale is overcoming packaging issues involved in bringing sensors and other components together in such a small space, at MIT Technology Review.

Calling out the big data skeptics

GigaOm’s Derrick Harris called out the big data skeptics this week. Highlighting several noted skeptics from Mike Loukides to Nassim Taleb to Gartner’s Hype Cycle dropping big data into the “Trough of Disillusionment,” Harris declared, “I call B.S. on all of it.”

Harris points out that the benefits of big data are directly related to the expectations going in, that critics declaring that big data isn’t what it’s cracked up to be are setting up a strawman “because no one should think it’s magic to begin with.” He addresses the issues of correlation versus causation, biases of web-derived data, focusing on insights to the exclusion of other big data benefits such as automation, and the importance of approaching big data solutions with a plan and managing — even setting — expectations. “Big data will never equal perfect data, but it can definitely point us in the right direction,” Harris writes. “I suggest not throwing the baby away with the bathwater.” You can read his full piece at GigaOm.

Tip us off

News tips and suggestions are always welcome, so please send them along.

Related:

• A different take on data skepticism
• Data skepticism
• Rising security challenges in a world of connected things
• A step toward personal data control
• More Strata Week coverage

O’Reilly Strata Conference — Strata brings together the leading minds in data science and big data — decision makers and practitioners driving the future of their businesses and technologies. Get the skills, tools, and strategies you need to make data work.

Strata Rx Health Data Conference: September 25-27 | Boston, MA
Strata + Hadoop World: October 28-30 | New York, NY
Strata in London: November 15-17 | London, England

Click here for the full interactive visualization.

Users can enter a city or ZIP code to isolate an area or click on the map to zoom in and select specific hospitals. Clicking on a specific hospital reveals detailed data on procedure costs:

Click here for the full interactive visualization.

The Washington Post’s visualization compares average costs by state against national averages:

Click here for the full interactive visualization.

The Washington Post also has made the data available on a per-hospital basis, allowing users to enter their local hospital names to reveal detailed procedure cost data:

Click here for The Washington Post’s hospital search page.

Hat tip to Nathan Yau for highlighting the visualizations.

More visualizations:

• CIA rendition flights of terror suspects
• Real-time Wikipedia edits
• Building collapse rescue efforts
• Every recorded U.S terror attack 1970-2011
• Commuting Paris

Intel is looking to take the lead in what it has dubbed the “data economy,” helping consumers and individuals realize and retain more value from their personal data. Antonio Regalado and Jessica Leber report at MIT Technology Review that the the world’s largest computer chip maker has launched a “Data Economy Initiative.” Ken Anderson, a cultural anthropologist who is in charge of the project, described the initiative to them as “a multiyear study whose goal is to explore new uses of technology that might let people benefit more directly, and in new ways, from their own data.”

As part of the initiative, Intel is funding hackathons to encourage developers to experiment with personal data in new ways, Regalado and Leber note. “[Intel] has also paid for a rebellious-sounding website called We the Data,” they report, “featuring raised fists and stories comparing Facebook to Exxon Mobil.” You can read their full report, including a close look at the ecosystem that may be giving rise to a new “data economy” at MIT Technology Review.

In related news, Christopher Mims reports at Quartz that results from Safe Shepherd’s millions of searches have shown that on average, each American has 12.5 personal records freely available and accessible online. Safe Shepherd is a company that searches public records on behalf of its users and clients and displays them in a dashboard.

Anyone can use their service for free to see what records are out there — Mims reports that a Safe Shepherd search showed 29 records that included his data on people-search sites alone; my own search showed 63, though some of those records weren’t mine. Users can then buy a membership for $13.95 per month to have the records expunged, according to the Safe Shepherd site. Mims notes, however, that Safe Shepherd can’t help with such problems as identity theft. You can read his full report at Quartz.

Sears extends its data reach into data centers

Rich Miller reports at Data Center Knowledge this week that Sears is getting into the data center business. The company has created Ubiquity Critical Environments, a new unit of Sears Holdings that will be responsible for identifying and repositioning appropriate store locations to be converted into data centers. Miller notes that “Sears Holdings has one of the largest real estate portfolios in the country, with 3,200 properties spanning 25 million square feet of space.”

Miller reports that Ubiquity will be able to choose from properties of both closed and open stores, and that the first conversion project will be a Sears store on the south side of Chicago that is slated to close in June. There’s the question of Sears’ mall properties as well. Ubiquity’s chief operating officer Sean Farney told Miller that though a mall location isn’t ideal for a data center at this point, he does see potential for disaster recovery space and convenient locations for wireless towers. Sears’ foray into data centers isn’t as much of a stretch as it might seem — the retail giant recently made a name for itself in Hadoop with the launch of its MetaScale data-management system.

Revolutionizing data publishing with open source tools

James Smith, a web developer at the Open Data Institute (ODI), presents an argument this week that open source tools could revolutionize open data publishing, much like they did with open source software. “Data is different from code in many ways, but the open source development model has a lot we can learn from,” Smith writes. “When I can fork your open dataset, fix errors, and easily get my changes integrated back in, we all win.”

Smith is fully committed to experimenting with the possibilities. He’s built a Git Data Viewer to help make Git repositories more user-friendly, and he’s uploaded a MOD dataset to GitHub for developers to tinker, where users can view an open pull request to improve a dataset file format. “I’m already learning that while many operations on datasets match well with Git,” he writes, “some do not.” He stresses that this is an experiment that might not work, but invites developers to join in.

Tip us off

News tips and suggestions are always welcome, so please send them along.

Related:

• Dueling views on data center efficiency
• Data centers running at the speed of light
• Curiosity turned loose on GitHub data
• A step toward personal data control
• More Strata Week coverage

O’Reilly Strata Conference — Strata brings together the leading minds in data science and big data — decision makers and practitioners driving the future of their businesses and technologies. Get the skills, tools, and strategies you need to make data work.

Strata Rx Health Data Conference: September 25-27 | Boston, MA
Strata + Hadoop World: October 28-30 | New York, NY
Strata in London: November 15-17 | London, England

Click here for the full visualization.

In a post at The Guardian’s Datablog, James Ball notes that interactive visualization is the result of three years of research and includes more than 11,000 rows of data on confirmed and suspected rendition flights. The complexity of the data led the Datablog team to develop a guide on how to use the visualization.

First, the guide helps users choose what to look at. The default view, pictured in the screenshot above, displays information on 1,500 flights noted as significant within the data. The guide suggests narrowing this down; the visualization is broken down into circuits of flights:

“Circuits can include original journeys from America, R&R stops in the Caribbean, refuelling stops, and the central rendition journeys themselves.

“The menu on the left-hand side of the graphic gives a range of ways the information can be narrowed down: trips which only take in certain airports can be picked, or particular companies, or particular individuals known to have been targets of rendition. The date range can also be selected using the sliding toggles.”

The guide also defines the color coding — simple flights (where researchers had no strong evidence a suspect was on board) are grey while paler red lines mark “highly suspicious” flights (those with strong evidence of an on-board suspect), for instance. Users are also guided through methods of gathering data on specific flights. Additional information, supporting data and research also can be found on The Rendition Project’s website.

More visualizations:

• Visualization of the Week: Real-time Wikipedia edits
• Building collapse rescue efforts
• Visualization of the Week: Every recorded U.S terror attack 1970-2011
• Commuting Paris
• Visualization of the Week: A day in the life of a bus line

Google introduced a new version of Google maps at Google I/O this week that learns from each use to customize itself to individual users, adapting based on user clicks and searches. A post on the Google blog outlines the updates, which include recommendations for places you might enjoy (based upon your map activity), ratings and reviews, integrated Google Earth, and tours generated from user photos, to name a few.

Leo Mirani at The Atlantic says the update “fixes the one thing that has always been wrong with maps” — namely, neutrality. Though maps are typically viewed as neutral objects, Mirani argues, they’re “about as impartial as journalism.” He writes:

“Is the hot dog vendor who stands on a street corner as worthy of inclusion as the bank on the same corner? That decision still lay with mapmakers — in this case a giant internet company. Google’s solution to the problem was to remove itself from the equation.”

Mirani notes that “mental maps,” maps created based on how a person sees the world, is nothing new, but Google has managed to provide the mental map as a service to millions of users.

In a post at The Atlantic Cities, Emily Badger argues that this level of individual customization comes at a cost — “An algorithm that knows you too well,” she writes, “does a terrible job of telling you things you don’t already know.” Badger relates the new maps to Eli Pariser’s “filter bubble” concept: as the algorithms get to know you, you increasingly get content that leans toward your established point of view, preventing you from broadening your experiences. Badger also notes the increasing issue with “the inequality of information online…rendering some real-world people and places virtually invisible” and wonders if the new maps won’t exacerbate the problem. You can read her full report at The Atlantic Cities.

App development trends lean toward predictive, intelligent service offerings

MIT Technology’s Tom Simonite took a look at the growing app development trend to provide users with personalized information, service connections, and recommendations before even being prompted or searched. Noting the departure of this trend from typical “dumb” computers and software that waited for human operator interaction, Simonite looks at such apps as Google Now, which aims to predict a user’s actions in order to provide appropriate assistance, and the Osito iPhone app, which similarly predicts actions to offer helpful information but also provides actionable assistance, such as a button to call a cab when a user’s flight reminder pops up.

Bit.ly chief data scientist Hilary Mason told Simonite that Google Now is far from perfect in the usefulness of the information it provides, but she uses it anyway and finds the technology important “because it’s the first time Google has taken all they know about us to make a product that makes our lives better.” You can read Simonite’s full report at MIT Technology Review.

In related news, Google announced new services at its I/O conference this week that will help developers build apps that can track users as well as Google does — and without draining a user’s battery. Jessica Leber reports at MIT Technology Review that the service will allow developers to build apps that tap into the accelerometer of a user’s device instead of the “power-hungry” GPS sensor to determine whether a person is driving, walking or cycling; the apps would run Google algorithms, Leber notes, that would “learn over time whether a person is stuck in traffic or just out for an evening stroll.” The new services also include the ability for developers to create geofences to trigger actions based on a user’s location. You can read Leber’s full report at MIT Technology Review.

The New Yorker gets an anonymous inbox to protect sources

The New Yorker launched Strongbox this week, a new tool for people to share files, information, and messages with The New Yorker staff with an increased level of anonymity. In a post announcing the launch, Amy Davidson explains that with the way Strongbox is set up, they won’t be able to tell where a piece of information came from, and thus won’t be able to tell anyone where it came from, providing better protection for their sources.

The Strongbox site explains further that the system is only accessible through the Tor network and that when a user submits a message or file, no I.P. address is recorded and no information about a user’s browser, computer or operating system is gathered. Strongbox also doesn’t include any third-party content or use cookies. Users are given a randomly generated, unique code name so that New Yorker staff have a way to contact them if necessary via a message left in Strongbox.

Davidson notes that the Strongbox tool was developed by Aaron Swartz and Kevin Poulsen, and is based on their underlying code dubbed DeadDrop, which will be open source. You can read the development story and history in a post Poulsen wrote for the New Yorker.

Tip us off

News tips and suggestions are always welcome, so please send them along.

Related:

• Google unveils its Knowledge Graph
• Google’s Rich Snippets and the Semantic Web
• Predictive data analytics is saving lives and taxpayer dollars in New York City
• Why you can’t really anonymize your data
• More Strata Week coverage

O’Reilly Strata Conference — Strata brings together the leading minds in data science and big data — decision makers and practitioners driving the future of their businesses and technologies. Get the skills, tools, and strategies you need to make data work.

Strata Rx Health Data Conference: September 25-27 | Boston, MA
Strata + Hadoop World: October 28-30 | New York, NY
Strata in London: November 15-17 | London, England

Click here for the full visualization.

On the Wikipedia Recent Changes Map, edits can be viewed on 11 Wikipedia language versions, including English, German, Russian and Japanese. Every time an unregistered user edits a Wikipedia entry, his or her IP address is recorded and translated into an approximate location. The about section of the visualization explains that registered users don’t have associated IP information, so registered user edits are not shown on this map.

In a blog post about the map visualization, LaPorte and Hashemi note that the map was built with several libraries and services, including d3, DataMaps, and freegeoip.net, and that the “map listens to live feeds of Wikipedia revisions, broadcast using wikimon.” The map’s code is open source and available on GitHub.

Hat tip to Megan Geuss at Ars Technica and to Emily Badger at The Atlantic for highlighting LaPorte’s and Hashemi’s work.

More visualizations:

• Building collapse rescue efforts
• Visualization of the Week: Every recorded U.S terror attack 1970-2011
• Commuting Paris
• Visualization of the Week: A day in the life of a bus line
• Block-level electricity use in Los Angeles

O’Reilly Strata Conference — Strata brings together the leading minds in data science and big data — decision makers and practitioners driving the future of their businesses and technologies. Get the skills, tools, and strategies you need to make data work.

Strata Rx Health Data Conference: September 25-27 | Boston, MA
Strata + Hadoop World: October 28-30 | New York, NY
Strata in London: November 15-17 | London, England