How to Use Public Data Sources: U.S. Census

One of the key skills of a Data Scientist is the ability to incorporate publicly available data into analyses to generate more accurate predictions or supplement an analysis in some way. The U.S. Census Bureau is one very rich data source. 

Datasets such as the Decennial Census, the American Community Survey, and the Current Population Survey contain a wealth of information for Data Scientists to use in predictive modeling or other quantitative analysis. 

Until recently, these rich data sources were very difficult to access. The American Fact Finder interface was notoriously cumbersome to work with. However, recently, the census bureau released a set of APIs that make it much easier to access data. 

Here are a few resources to help get you started:

• Request a developer key for the U.S. Census API
• Sample calls to U.S. Census API to help you format your first calls.
• Additional tutorial for working with the U.S. Census API

PS - I'm on Snapchat talking about Data Science (and other things). Join the conversation!

Data Science Lite

Summary:

• Data Science principles can be scaled to benefit any company - meeting them where they are on the analytical continuum and improving analytical rigor and sophistication.
• "Data Science Lite" employs all of the same skills as full-fledged Data Science.
• The data science and analytics community must be willing to meet companies where they are to help them make meaningful advancements for their business. 

Data Science is for Every Company

By now you have undoubtedly heard of the concepts of Data Science. You have likely heard success stories of companies that are employing big data and machine learning (ML) algorithms to advance their business. What if your organization's data isn't "big?" What if you aren't quite ready to build and deploy a living, breathing ML algorithm? What if you are still making the case to your company's leadership that there is value in using data in more creative ways and are not ready for a big investment?

I argue that data science is still for you! The same skills and techniques that highly sophisticated companies are using can easily be scaled to benefit organizations that aren't as far along the analytical continuum to nudge them up in their level of analytical rigor and sophistication. 

What you need is what I like to call: Data Science Lite.

What is Data Science Lite?

You have probably seen some version of this image, which has been used to describe the unique skill set of a Data Scientist. Data Scientists have:

• Technical & programming skills that allow them to work directly with raw data and databases.
• Quantitative & analytical skills that allow for the application of statistical techniques to data.
• Subject matter expertise that allows the findings of analyses to be applied within the business context.

Data Science 'Lite' uses all of the same skills and principles, but they are scaled down to work for companies who might not be as far along the analytical continuum. For example, if the left-hand column of this table doesn't sound like your organization, the right-hand column might:

* For a number of reasons, I wouldn't recommend keeping your data in an Access database, but a Data Scientist can work with it there and help you move it to a more stable platform. 

** R appears in both columns because it can be used in a variety of ways ranging from very sophisticated to more basic.

The argument I'm making is that any company can benefit from Data Science techniques. Making informed, data-driven decisions benefits all companies, even if you aren't ready to make a huge investment. We in the data science and analytics community should be committed to helping all companies move along the analytical continuum. We should be willing to meet companies where they are and set up a plan for making impactful, yet realistic progress toward more rigorous sophisticated data analysis. 

At Decision Analytics, we are committed to doing exactly that. Want to learn more? We would love to chat. Drop us a line and we can help you harness the power of your data and make it work for you, no matter where you're currently at on the analytical continuum. 

SQL Hack

We’ve all been in a situation where we need to write a large block of semi-repetitive code. In this video, I’ll show you how to create a large block of semi-repetitive code in little to no time, without writing much code at all.

Data Science: It's Not About the Sex Appeal

Hot or Not?

In 2012, Harvard Business Review called Data Science:

"the sexiest job of the 21st century."

They cited the complex skill-set, high demand, and scarcity of data scientists as some of the drivers of this trend. Contrast that with a recent Forbes article whose take-home point is that the sexy days of data science are numbered. They claim that automation and a new crop of professionally-trained candidates will fundamentally change the landscape of this field.

"...the data scientist shortage has been overblown."

These two articles certainly are not the only ones of their kind. And, as I read through article after article, I started to feel like my profession, the one I have loved and worked in for nearly a decade now, was suddenly the subject of a tabloid “Hot or Not” list. As this debate rages on, we in the data science community are hard at work, doing what we love to do: being curious, learning and applying new skills, asking new questions, finding answers. The answer to the question of whether Data Scientists are “hot or not” is irrelevant. I’ll let you in on a little secret:

we're not in it for the sex appeal.

Why do we choose this as a career?

I recently reached out to a dozen of my colleagues in the data science community to ask them why they choose to be in this profession. It probably won’t come as a shock that not a single one of them said they did it because it was “sexy.” So, what are the reasons? Why do we choose this career? Here’s a summary of what I heard from my colleagues, coupled with my own reasons for loving my job.

We love to help people make better decisions. Nothing is more cringe-worthy to a Data Scientist than hearing that a company made a big, expensive decision without any data to guide them.

We are creative. We love making something new out of nothing. My colleague, Nick Ryberg, said it best, “There’s a really nice intersection between the 5-year-old inside of me who, more than anything, simply loves connecting lego blocks in new and interesting ways.”

We are endlessly curious. We feel compelled to explore the unknown and to push the boundaries of what we currently understand. This curiosity doesn’t stop when we’ve answered a question - each answer leads to a new set of follow-up questions.

We are lifelong learners. There is always a new language, tool, or technique to learn. The rise of free or low-cost online learning communities has created a virtual playground for us, and we’re taking full advantage of it.  

Staying Power

In my opinion, there will always be a high demand for Data Scientists, although likely not in their current form. To say otherwise is to assume that we will someday know everything, that we will stop learning and growing, and that we will lose our curiosity. I’m much more optimistic than that. Here’s why:

The volume, complexity, and breadth of data is not decreasing. In fact, it is exponentially increasing.

There will always be new questions that need to be answered. As business gets more complex and competition grows, so does our need for answers. Additionally, future work builds off of the work we’re doing today - that is the nature of the scientific method.

Automation has always been present. Truly innovative people use technology and automation to their advantage. Automation can either be viewed as a threat or as a tool to help us accomplish even more. The complex work we can tackle today is possible because technology and automation have freed up time and resources that can be deployed onto new, complex problems.

There will always be something new to learn. Data Scientists are not the type of people who rest on their laurels. Rather, the skills and knowledge that we have today are merely a foundation for the skills we’ll learn next.

In the end, the question of whether the field of data science is sexy is irrelevant. We don’t do this job for the sex appeal. We do it because it’s in our nature to ask, explore, create, and learn. Call me an optimist, but I believe there will always be a huge need for that - it just might look different than it does today.

How about you? Why do you choose to be a Data Scientist?

Make Your Own Fire: Expand Your Portable Skillset

Let me ask you a question, and be honest with yourself. When was the last time you learned something truly new? Something outside of your current skillset or comfort zone? Something difficult that took real grit to learn? Something that pushed the boundaries of your abilities in a new direction?

Many times, people focus on learning new skills to help them excel in their current job. In this post, I’m going to walk through 3 reasons why I believe this is the wrong approach. Instead, I argue that you should view your arsenal of skills as your own, independent, portable toolkit. These are the skills and knowledge that you can take with you, regardless of your current job, and they set the limits of the opportunities you can pursue. The wider your arsenal of portable skills, the more opportunities you can chase after.  

Reason #1: Your Current Job Might be Temporary

I likely don’t have to tell you that the job market can be very volatile. Year-to-date, in 2015 alone, there have been more than 44 million job separations, over 16 million of which were due to layoffs or discharges (Source: Bureau of Labor Statistics JOLT). On the other side of the coin, there were more than 45 million hires during this same time period.

Now, more than ever, people are professional nomads. They move around and follow opportunity where it arises. Additionally, companies must constantly pivot to stay competitive, which can mean big changes in their workforce to abandon old goals and processes (and the people whose roles supported them) in favor of new ones. The job you are in today and the skills you need to excel at it could very well be temporary.

Reason #2: Innovation Changes Which Skills are “Necessary”

Even if you do remain stable in the same job, your role and the skills necessary to accomplish it are likely to change. One of the biggest disruptions in the modern business environment was the personal computer. It’s hard to believe, but IBM released the Personal Computer 5150 in 1981, a mere 34 years ago. By 1989, 1 in 3 people were using a PC at work. In just 8 years, it became much more difficult to get a good, corporate job without being “a computer person.”

Today, when you show up on your first day at a new job, not only are computer skills a requirement, but you’re likely not given any training at all about how to use the computer that is sitting on your new desk. Skills that were once “cutting edge” are now baseline, necessary, and assumed.

Reason #3: You Could Miss Out on Something You Love

This one is a bit personal for me - you won’t know whether you love doing something until you try it for the first time. Ten years ago, if you would have told me that some of my most enjoyable days at work would be spent programming, I would have laughed in your face. However, although it wasn’t required of me at the time, I initially learned to program because I was curious about it and thought it might make it easier for me to get at and work with raw data.

Now, flashing forward, nothing beats the feeling you get when you create something out of nothing. You know the look on Tom Hanks’ character’s face when he makes fire for the first time in the movie Castaway? That’s me every time I program something that I have never programmed before. Every. Stinking. Time. Nothing beats that feeling, and I would have missed out on it had I not discovered my love for programming.

New Years’ Resolution: Widen Your Landscape of Opportunity

As 2015 comes to a close and we all look forward to 2016, I’d encourage each of you to set the goal to acquire 1 truly new skill next year. MOOC’s like Coursera, Khan Academy, Code Academy, Lynda.com, and Udemy have made it easier than ever to learn from some of the best instructors in the world.

Need Some Inspiration?

Here are some questions that can help you get started. Happy fire-starting!

• What outside department do you often work with (Finance, Marketing, IT, etc.)? What new skill would help you work more effectively with that department?
• Who is your professional idol? What skill or knowledge do they have that you don’t have?
• What skills do college and graduate programs in your field now offer that weren’t offered when you were in school?
• What manual process takes up the most time in your day or frustrates you the most? Is there software or tools to help alleviate that burden?
• What job would you love to apply for, but you don’t feel quite qualified to take? What missing skill or knowledge is holding you back?
• Which one of your friends or family members has a job that sounds really enjoyable to you? What is one baseline skill they would suggest you learn if you wanted to break into that field?