What inspired you to become a code artist?

Scott Murray

Scott Murray: I had designed websites for a long time, but several years ago was frustrated by web browsers’ limitations. I went back to school for an MFA to force myself to explore interactive options beyond the browser. At MassArt, I was introduced to Processing, the free programming environment for artists. It opened up a whole new world of programmatic means of manipulating and interacting with data — and not just traditional data sets, but also live “data” such as from input devices or dynamic APIs, which can then be used to manipulate the output. Processing let me start prototyping ideas immediately; it is so enjoyable to be able to build something that really works, rather than designing static mockups first, and then hopefully, one day, invest the time to program it. Something about that shift in process is both empowering and liberating — being able to express your ideas quickly in code, and watch the system carry out your instructions, ultimately creating images and experiences that are beyond what you had originally envisioned.

Why did you decide to write Interactive Data Visualization for the Web?

Scott Murray: D3.js is the most powerful tool for creating visualizations on the web, hands down. Yet when I tried to use it for a project in late 2011, there wasn’t a lot of useful information available on how to use it. A few early members of the community had written tutorials, but they assumed a level of JavaScript familiarity that I didn’t have. The official D3 documentation is excellent, but not necessarily accessible to beginners. So I set about banging my head against the wall for a few weeks, struggling to learn the peculiarities of D3, and taking notes every time I encountered a challenging concept or had a revelation. Once the project was done, I revisited my notes and starting writing tutorials that would introduce each of those challenging concepts to beginners — those new to JavaScript, and even HTML and CSS. (I’m seeing more people drawn to JavaScript via D3, just as my first experiences with JavaScript were thanks to jQuery.) My goal was to spare others the frustrations I experienced. Within months, the tutorials on my site were getting a lot of traffic, and people wrote in to request more tutorials: on the basics, on interactivity, on mapping. So I expanded the tutorials into a full-length book.

Who do you envision reading this book? What will they learn after reading your book?

Scott Murray: This book is for anyone interested in learning how to use D3 to create and publish visualizations on the web: journalists, designers, data scientists, statisticians, students, researchers, and would-be mapmakers. The book includes an introductory chapter on web fundamentals — HTML, CSS, JavaScript — so it’s very accessible to people new to web development, and even programming generally. There are also more than 100 code examples that accompany the book, so it’s easy to follow along and tweak the examples as you learn. In the end, anyone who works through the book will have a grip on all the basic concepts of D3. You’ll be able to make charts and graphs, even highly customized geographic maps with data overlays. And hopefully you’ll be inspired to learn more, experiment, and share your creations with the datavis community.

Any words of advice for an aspiring code artist?

Scott Murray: Start making things now. I get to work with students, and I see them get stuck in their heads, trying to plan out every last detail before they start working on a project. This results in the project never starting at all, or being finished in a very rushed fashion. People (myself included) often have a tendency to over-think things, especially intimidating projects that will involve learning something new, or trying something we’ve never tried. While thinking is good, over-thinking prevents us from doing. And, in reality, doing is a critical part of thinking — you can’t really separate the two. So I suggest getting comfortable with not knowing what you’re doing before you do it. Just start making things now, today, even if you feel underprepared or like you don’t have all the answers you need yet. Guess what? No one has all the answers (even though we pretend to). We’re all just here figuring this stuff out as we go. So don’t over-think it, start producing projects, and get those projects out in the world. You’ll learn what you need to learn along the way.

This interview was edited and condensed.

Related: 

• Slides from Scott Murray’s D3.js tutorial at Strata Santa Clara 2013
• Engaging Audiences with Data Visualization – Webcast
• Interactive Data Visualization for the Web – Book
• Data Journalism Handbook
• Movers and shakers on the data journalism front

What inspired both of you to write your book?

Alistair Croll

A big part of the inspiration came from our work with Year One Labs, an early stage accelerator that we co-founded with two other partners in 2010. We implemented a Lean Startup program that we put the startups through and provided them with up to 12 months of hands-on mentorship. We saw with these companies as well as others that we’ve worked on ourselves, advised and invested in, that they struggled with what to measure, how to measure it, and why to measure certain things.

The core principle of Lean Startup is build, measure, and learn. While most entrepreneurs understand the “build” part since they’re often technical founders that are excellent at building stuff, they had a hard time with the measure and learn parts of the cycle. Lean Analytics is a way of codifying that further, without being overly prescriptive. We hope it provides a practical and deeper guide to implementing Lean Startup principles successfully and using analytics to genuinely affect your business.

What are some of the unique aspects to using analytics in a startup environment?

Benjamin Yoskovitz

One of the biggest challenges with using analytics in a startup environment is the vast amount of unknowns that a startup faces. Startups don’t really know what they are at the beginning. In fact, they shouldn’t even be building a product to solve a problem. In many ways they’re building products to learn what to build. Learning in an environment of risk and uncertainty is hard. So tracking things is also hard. Startups are also heavily influenced by what they see around them. They see companies that seem to be growing really quickly, the latest hottest trend, competition and so on. Those influences can negatively affect a startup’s focus and the rigorous approach needed to find true insight and grow a real business. Lean Analytics is meant to poke a hole in an entrepreneur’s reality distortion field, and encourage…or force! … a level of focus and attention that can cut out the noise and help founders move as quickly as possible without doing so blindly.

What defines a good metric?

Good metrics have a few qualities. For starters, a good metric should be a ratio or rate. It makes the number easier to compare. You want to avoid absolute numbers that always go up and to the right. Those are typically vanity metrics.

A good metric has to be incredibly easy to understand. You should be able to tell anyone the number and they can instantly understand what you’re doing and why.

A good metric, ultimately, has to change the way you behave. Or at least provide the opportunity for you to change. If you’re tracking a number and can’t figure out how changes in that number whether it be up, down, or sideways, would impact how you behave and what you do, then it’s a bad number. It probably isn’t worth tracking and certainly not worth focusing on. Good metrics are designed to improve decision making.

What are the stages of lean analytics?

We’ve defined five stages of Lean Analytics: Empathy, Stickiness, Virality, Revenue and Scale. We believe all startups go through these stages in this order, although we’ve certainly seen exceptions. And we’ve defined these stages as a way of focusing on a startup’s lifecycle and how the metrics change as a startup moves from one stage to the next. We’ve also created gates through which a startup goes to help it decide whether it’s ready to move to the next stage.

Empathy is all about “getting out of the building” and identifying problems worth solving. It’s about key insights that you’ll learn from interviewing customers, which guides you to a solution. The metrics you track here are largely qualitative, but you may also start to look at levels of interest you can drive to a website or landing page and early conversion. Basically, you have to answer the question: Does anyone really care about what I’m doing?

Stickiness is about proving that people use your product which early on is a Minimum Viable Product or MVP and that people remain engaged. You’re going to track the percent of active users, frequency of use, and try to qualitatively understand if you’re providing the value you promised to customers.

Virality is about figuring out and growing your acquisition channels. Now that you have a product that’s working reasonably well with early adopters, how do you grow the list of users and see if they too become active and engaged? The metric to track here is viral coefficient which in a perfect world is above 1, meaning that every active user invites one other user that becomes active, in which case you can grow quite quickly, but it’s not the only metric that matters. You want to track actions within your application or product that are designed to encourage virality. This might be invites or shares. You have to look at the difference between inherent and artificial virality as well. Ultimately you get through this stage when you’ve proven that you can acquire users reasonably well, and you see scalable opportunities to do so going forward.

Revenue is about providing the fundamentals of the business model. Prior to getting to this stage you may have been charging money, but you weren’t focused on fine tuning that aspect of the business. And you were properly spending money to acquire customers but not really focusing on whether the economics made sense. Now you have to prove the economics. So you look at things like the Customer Lifetime Value and compare that to the Customer Acquisition Cost. You might look at the Customer Acquisition Payback which is how long does it take a customer to payback the acquisition cost you made to bring them in. You’re likely going to look at conversion from free to paid, especially if you are building a freemium business. You’re also going to look at churn or how many people abandon your product or service. To get through this stage you need to have a reasonably well-oiled financial machine that makes sense.

Scale is about growing the business as big as possible. You know you can acquire customers, you know a good enough percentage will stick around and pay, and you know the economics make sense. So now you have to grow. Depending on your business you’ll be looking at different channels such as partners or growing a bigger sales team, APIs for developing an ecosystem, business development opportunities and so on. You may expand into new markets, develop secondary, or ancillary products as well.

The book is filled with case studies. How did both of you decide which case studies to include in the book and why?

It wasn’t a complicated process. Many of the case studies came from people we knew and  who were leveraging Lean Startup and analytics in a meaningful way. Some of them came from our own experience. We felt it was important to share those as well. As we developed the framework for the book, such as tackling different business models, the Lean Analytics stages, etc., we looked for great examples that could speak to each of the key points we were making. We talk a great deal in the book about The One Metric That Matters. This basically means: focus on one metric only, at any given time. This came from our experience but also from talking to a lot of other people. Then we picked a couple of great stories or case studies that reflected the importance of the concept.

It was important for us to have real world examples of all types of companies whether they were big, small, successful, less so, early stage, late stage, etc., so there would be variety, but also because we know these examples resonate a great deal with people. We know that people are looking for “proof” that Lean works and that a focus on analytics matters; hopefully we’ve been able to provide that in the book.

This interview was edited and condensed.

Reading Barlow on a Sunday afternoon

During a break in between offsite meetings that Edd and I were attending the other day, he asked me, “did you read the Barlow piece?”

“Umm, no.” I replied sheepishly. Insert a sidelong glance from Edd that said much without saying anything aloud. He’s really good at that.

In my utterly meager defense, Mike Loukides is the editor on Mike Barlow’s Real-Time Big Data Analytics: Emerging Architecture. As Loukides is one of the core drivers behind O’Reilly’s book publishing program and someone who I perceive to be an unofficial boss of my own choosing, I am not really inclined to worry about things that I really don’t need to worry about. Then I started getting not-so-subtle inquiries from additional people asking if I would consider reviewing the manuscript for the Strata community site. This resulted in me emailing Loukides for a copy and sitting in a local cafe on a Sunday afternoon to read through the manuscript.

Since I, ahem, hadn’t exactly been paying the closest attention to the history behind the piece, I wasn’t certain what to expect.

While I was reading through the manuscript and coming across points such as “for some, real-time big data analytics (RTBDA) is a ticket to improved sales, higher profits and lower marketing costs. To others, it signals the dawn of a new era in which machines begin to think and respond more like humans,” I realized that Barlow was providing distilled insight and context for managers looking to understand what “real-time” means and how it may impact the overall architecture of a data system.

In the data industry we often bandy about terms such as “real-time data analytics.”  A lot. Yet, we often don’t pause to provide context around the ever evolving definition of “real-time.” Why? Well, likely because data innovations are moving at the speed of light and we may figure, “why worry about what we don’t have to?” Then, it may take something like a sidelong glance from a colleague to remind us that it isn’t about “worrying” per se … it is about being connected to the various stakeholders within our community.

In Real-Time Big Data Analytics: Emerging Architecture, Barlow takes the time to discuss what real-time means, what an architecture for a real-time data stack looks like, and what the different phases of real-time are. Also, as Barlow correctly reminds us, ”focusing on the stakeholders and their needs is important because it reminds us that the RTBDA technology exists for a specific purpose: creating value from data.”

If you are interested in reading Barlow’s ebook for yourself, you can find it here.

As I have referred to before, I just love thought provoking and candid discussions.

I first heard about the event when Kathy Yu, Alistair Croll, and I met at the SF Ferry Building to talk about Strata over breakfast. I’m not a morning person. It takes a few moments for the caffeine to take effect. Alistair is the opposite. I don’t know if Alistair had his dose of caffeine earlier that day or if he just generates his own energy. Whatever it is, it enables him to chair Strata, run his own business, keep up with his precocious two-year-old daughter, and co-author the forthcoming Lean Analytics. Yet, that morning, I was half-tuning Alistair out while I was sipping on my coffee and taking a picture of my crispy caramelized waffle. Yes, I’m that person. But when Alistair started talking about data warfare, he had my full attention. As we rely more upon data, we become more vulnerable to various attacks. It is important for us to learn more about what the potential attack vectors could be and how to defend against them. The speakers at the upcoming Strata Online Conference on data warfare will get us all thinking about this.

The speakers and the topics of their sessions include:

• Alistair Croll — Stacks get Hacked
• Christina Gagnier — Sex. Drugs. Rock. And CODE: Hacking Cybersecurity
• Jo Prichard — Crowdsourcing large scale identity theft and fraud to make bucket loads of easy money
• Duncan Ross and Fran Bennett — Using data for EVIL: a beginners guide
• Joseph Turian — Black-hat Data Science
• Vishwanath Ramarao — What to do when your Machine Learning gets attacked

See what I mean about thought provoking and candid discussions?

I’ve already signed up for it. If you would like to know more or register for this free Strata Online Conference on data warfare, please visit here.

It made a significant difference.

Thinking about math from the perspectives of research design methodology and how data can be used to manipulate people made quite an impact on my teenage worldview. This experience also reinforced my belief that education is what you decide it will be. There is always more than one way to learn and education doesn’t necessarily have to happen in a physical classroom. Growing up in the San Francisco Bay Area where friends and loved ones decided to forgo traditional higher ed completely to start their own companies or immediately work in jobs in technology also contributed to this belief.

While full time students who are looking at a career in data science may have the time to do seemingly nutty things like take overlapping math classes, this is not something that most people with full time jobs are able to do. When people with full time jobs ask me about what they need to do to move into data science, I probe them about the kind of job in data science they want and about their analytical and empathy skills. Then, I immediately follow up with “So, how are your math skills?.” Interestingly enough, I get a lot people saying how they don’t have time to physically go into a classroom or that it has been, like, forever since they’ve used statistics and/or linear algebra for data analysis. Even more interesting is how often people don’t realize just how many resources are available to learn math outside of the physical-attendance-in-a-classroom-model.

Huh.

Typically when I get resistance from people about brushing up on their math skills, I tell them to review the jobs sections on Kaggle, Twitter, LinkedIn, and heck even the CIA; subscribe to Indeed or the like for various data-related job alerts; and then take a moment to absorb what the most common requirements are for the type of job in data science they want. I tell people to do this because, naturally, I have already done so and I know exactly what the answer is.

Yup, it is math.

If you weren’t a math major or do not use math in your everyday job, don’t freak out.

Thankfully, there are a multitude of resources available to learn or brush up on math skills.

I’ve included a few of my recommendations in this post. I will leave it to you to pick which individual math resources apply to your specific situation.  You’ll know exactly which math skills you need to brush up on after reviewing the most common job requirements for the data science job you want.

Yet, at an absolute minimum, you will need to know statistics and that should just be the beginning.

MIT OpenCourseWare (MIT OCW) — MIT OCW provides a comprehensive list of free online undergraduate and graduate courses in math including linear algebra, statistics, and algorithms. I have been using this site for years as a reference for many topics.

Coursera — Coursera offers free online courses from well-regarded universities. Courses are always being added. Yet, depending on your current math skills, courses on mathematical thinking, calculus, linear algebra, and algorithms may be helpful.

Khan Academy — Khan Academy has an extensive library of math-related videos that may assist students of all ages.

Mathbabe aka Cathy O’Neil’s Recommendations — While there is a little overlap between my and Ms. O’Neil’s recommendations, I wanted to include her recommendations because they are extensive.

iTunes U — Downloading videos to your device for audio playback during commutes, doing the dishes, etc. helps with reinforcement with what you are reading and may also provide context. I often play videos from iTunes U as well as O’Reilly’s Strata Conferences while I am cooking. iTunes U offers lectures and classes on statistics, algorithms, and more.

Don’t forget! If you have any ideas or suggestions for a post on strata.oreilly.com, please feel free to let me know at pitchstrata@oreilly.com.

Strata Conference Santa Clara — Strata Conference Santa Clara, being held Feb. 26-28, 2013 in California, gives you the skills, tools, and technologies you need to make data work today.Save 20% on registration with the code STRATA20

Thanksgiving 2012

Over Thanksgiving, Richie and Violet asked me if I preferred the iPhone or the Galaxy SIII. I have both. It is a long story. My response was, “It depends.” Richie, who would probably bleed Apple if you cut him, was very unsatisfied with my answer. Violet was more diplomatic. Yet, it does depend. It depends on what the user wants to use the device for.

I say, “It depends” a lot in my life.

Both in the personal life and the work life … well, because it really is all one life isn’t it?  With my work over the past decade or so, I have been obsessive about being user-focused. I spend a lot of time thinking about whom a product, feature, or service is for and how they will use it. Not how I want them to use it — how they want to use it and what problem they are trying to solve with it.

Before I joined O’Reilly, I was obsessively focused on the audience for my data analysis. “C” level execs look for different kinds of insights than a director of engineering. A field sales rep looks for different insights than a software developer. Understanding more about who the user or audience was for a data project enabled me to map the insights to the user’s role, their priorities, and how they wanted to use the data. Because, you know what isn’t too great? When you spend a significant amount of time working on something that does not get used or is not what someone needed to help them in their job.

Writing Strata piece on a Saturday afternoon

If there were a Data Analysis Anonymous support group, I’d bet that one of the top challenges discussed would be dealing with spending so much time, resources, and err … funding on unused data projects. This also crosses over to other roles within multiple industries. Just think about how many products, services, and additional features have been launched into the market and no one uses them. Each unused feature or product may represent hundreds, if not thousands, of human work hours. Wasted.

Since I’ve joined O’Reilly, a variation of the question “how do we help people become data scientists?” has come up every day. As the Strata editor, this is a question I should be thinking about every day … even at 12:48 AM staring at my ceiling or writing a Strata piece on a Saturday afternoon at a local cafe. My response often is, unsurprisingly, “it depends.” There is no single path to becoming a data scientist. Saying that there is only one path to becoming a data scientist is like saying that all product directors started their careers with Ph.Ds in computer science and electrical engineering. Ummmm. Yeah. So not the case.

At a very broad level, everyone interested in careers in data science will need to be familiar with some math, programming, tools, design, analysis … and wait for it … empathy. As in, empathy for the users of your data projects. Ooooh, I can already envision the hatertude that is going to fill my inbox with my empathy recommendation. Please feel free to bring it on. You can reach me here.

Anyway.

How deep you need to go into each category depends on your background (i.e., quant, qualitative analyst, designer, software engineer, student, etc.)  and what kind of work you want to do (i.e, open source, startups, government, corporate, etc.).

Working at the office

O’Reilly has a data science starter kit that is a great bundle that provides insight into the broad technology categories. In the future, I’ll provide additional suggestions on the types of resources users can reference to help them with their path toward learning more about data science, and if they want, becoming a data scientist. Within the Strata community site, I’ll be seeking to answer questions like:

• “I’m currently a quant that works a lot with mySQL and am interested in data science. Now what?”
• “I am a software developer. Do I really need to learn any more math? Seriously?”
• “I’m currently a graphic designer. What should I learn about data science in order to bring additional meaning to my design?”
• “I think I want to get my Ph.D in math. Probably statistics. What else should I think about while I complete my studies if I want to be a data scientist when I grow up?”
• “I am a business intelligence analyst that works primarily with Excel. What other skills do I need to become a data scientist?”

These won’t be the only questions. I’ll also be seeking to provide insights to even more questions from many different types of users who are interested in data science. Keep a lookout for future postings from me and friends of O’Reilly that will provide more detailed recommendations. While I plan to cover quite a wide range of topics within the Strata community, insight into the multiple types of user-centric learning journeys needs to be addressed.

If you have any ideas or suggestions on which learning journeys should be written about first, please feel free to let me know at pitchstrata@oreilly.com.

Strata Conference Santa Clara — Strata Conference Santa Clara, being held Feb. 26-28, 2013 in California, gives you the skills, tools, and technologies you need to make data work today.

Save 20% on registration with the code STRATA20

I decided to take a chance, not take another break just then, and read the lengthy note. I didn’t need that caffeine hit after all. Apparently, neither did half a dozen other editors.

The note was about ethics.

In a previous life, I worked in the competitive intelligence field. I remember participating in a friendly confab at an industry event and then someone mentioned the word “e-t-h-i-c-s”. It was rather fascinating to see how that word elicited stoic faces.  No one wanted to be the first person to say anything on that topic. Now when working at ORM, mention the word “ethics!” and folks are not shy about saying exactly what they think. Not. At. All.

During the discussion, Ethics of Big Data by Kord Davis, came up.  While I was not the editor on this book, I did read it when I was in New York. It made my list of recommended books for people looking to jump into the world of big data. Why? Because I remembered the stoic poker faces from my previous life in competitive intelligence.

Sometimes people are willing to debate ethics head-on. For example, when I was listening to Fred Trotter’s session at StrataRX, I had to keep myself from shouting “RIGHT ON!” when he talked about moral compasses, ethics, and patient data. I was decidedly less restrained at DataGotham though. I didn’t keep myself from laughing out loud when I was in the audience and listening to Joseph Turian’s session:

Also, I couldn’t help but smile at the passionate perspectives when I was reading through the ORM editors’ thread about ethics.

Yet, sometimes, when facing the stoic poker faces … you need tools to help bring about the discussion. Ethics of Big Data is one of those tools.  If you are navigating a corporate culture that is supremely riddled with layers upon layers of complex politics and is not open to head-on passionate debate on ethics, then Kord’s insights and approach will help. After I finished the book, I wished that I had something like this years ago.

I should warn you that this is not the last time that I will discuss or publish on ethics. We have also published perspectives about ethics within the Strata community site before. Yet, ethics is not a topic to be confined to one perspective, approach, definition, or a handful of posts. Ethics is an ongoing dialog that is extremely important for us to have, especially for those of us that work with data and make decisions on what to do with it.

Don’t forget to contact us at pitchstrata@oreilly.com if you have a tip or an idea that you’d like to write about for the Strata blog.

Strata Conference Santa Clara— Strata Conference Santa Clara, being held Feb. 26-28, 2013 in California, gives you the skills, tools, and technologies you need to make data work today.Save 20% on registration with the code STRATA20

When I told some of my friends and family that I was joining O’Reilly Media as an editor focusing on ORM’s Strata practice area, their responses reflected the diversity of my loved ones.

I’ve paraphrased some of the best ones here:

• “That is great! I have a bunch of their books. Everyone I know has the animal books.”
• “Bill O’Reilly owns a media company?”
• “I don’t get you techie people. Didn’t you already do a bunch of weird ninja-y data type stuff?”
• “Congrats! I have a lot of respect for ORM.”
• “… wait a sec, didn’t you STOP being a Java editor years ago to go work at an assessment data startup? ”

Sigh.

The people in my life have a few things in common.  They are smart, articulate, really truly not afraid to say what they think, and seek to be the change they wish to see in the world.  We don’t always agree [massive understatement]. Yet, our motivations are the same.

Why am I telling you this?

I believe that at our core, no matter how different we may seem, we do not actively seek to harm. Yet, everyone that works with data already has or will be facing certain choices on what to do with data. Choices that are obviously for good or for evil. Choices that are neither completely for good or completely for evil. Choices that we are reluctant to discuss because we do not want to implicate ourselves or the companies we work for. Yet, just because we are reluctant to discuss them does not mean we are not facing these challenges.

If you have the courage to speak out regarding the real everyday challenges that you experience while working with data, then I want to listen. If you have discovered solutions to these everyday challenges, then I want to publish your insight. If you engage in anything I publish, whether you agree or disagree, have suggestions for how things could be different or better, then please say something.

You can reach me at pitchstrata@oreilly.com.

I seek to support ORM’s vision to be a platform for the ideas of innovators.  I hope to publish multiple perspectives and voices in order to provoke discourse and enable innovation with data. I, obviously, cannot do this alone.  I am fortunate to join the likes of Tim, Jim, Edd, Mike, Julie, Alistair, Mac, Andy, Roger, Ben, Alex, Jenn, and many friends of O’Reilly who have already provided insights and enabled discussions about what is happening in “big” data.

Interested in joining us in the data revolution? Or being the change you wish to see in the world? Then contact us with your idea at pitchstrata@oreilly.com.

Strata Conference Santa Clara— Strata Conference Santa Clara, being held Feb. 26-28, 2013 in California, gives you the skills, tools, and technologies you need to make data work today.Save 20% on registration with the code STRATA20