Why I’m Not Impressed With Your Big Data
by Tiemo Winterkamp
Musings on the challenges of big data in a year of serious hype
There’s a reason you haven’t heard more than a handful of big data success stories in 2012. Handling big data correctly is hard, requires huge infrastructure and resource investments, and may not be worth it…yet. According to one survey in November 2012, 60% of businesses said it’s too early to tell if their big data project was successful and produced a proper ROI. It seems that so much of the hype around big data is focused on the technologies you need to buy and the talent you need to acquire (data scientist is the latest fad title), and not on what’s most important: what you can do with the data, what value you can extract, what business decisions you can speed up or improve with all that data.
With companies jumping on the big data bandwagon to the tune of $28 billion this year, it’s time to discuss why it might be best to ignore the hype for now and focus on reaping insight from the data you have already. Here’s why I’m not impressed with your big data:
You don’t actually have big data.
The marketing hype can lead you to believe you have a “big data problem” when you really don’t. Using the terminology incorrectly has the potential to harm your business, causing you to invest in unnecessary infrastructure when you may be able to leverage what you already have in place. Even Microsoft and Yahoo! have made this mistake…
Teradata PARTNERS User Group Conference 2012 Recap
by Roland Hoelscher
I just got back from the Teradata PARTNERS Conference in Washington D.C. – once again a a great event for learning from experts in the industry, listening to real-world examples on challenges with managing and leveraging huge data volumes, and networking with our fellow Teradata partners and customers alike.
It was my second consecutive year at the event, and what struck me most this year was that the topics have clearly shifted from managing big data to leveraging big data. Obviously, data volumes are exploding due to social media and clickstream data, sensor data and other sources and will only continue to grow. This year’s conference, however, was all about Analytics – how to use those data to drive business benefits. And there were great examples given at the conference.
In one of his presentations, Stephen Brobst, CTO of Teradata, described the benefits of collecting weather data around retail stores to determine whether conditions have a significant impact on food consumption in the store (e.g. the deli section). He said combining external weather forecasts with internal operational data and analytical information allows stores to adjust staffing and supplies for a huge impact on the bottom line.
Shaun Connolly, Program Director of Global Industry Solution at Teradata, described an example of how FedEx was able to save $60 million in staffing per year…
Careers in Business Intelligence: What Makes BI an Attractive Field
by Monique MorganAlong with the demand for big data, better data, and the need for greater insight into company operations comes the need for analytic professionals who can effectively leverage this data to maximize business benefits. But it’s not always easy to find and hire these types of people. Within the past few years, we’ve seen titles such as BI Analyst, Data Analyst, Data Scientist, and Big Data Engineer emerge in job listings as companies seek out much-needed expertise to wrangle their growing amounts of information. As a matter of fact, McKinsey and Company’s often-quoted 2011 report on big data predicted that by 2018, the US alone faces a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts to analyze big data and make decisions based on their findings. The bright side of this situation is that job seekers with analytic talent and business acumen have a tremendous opportunity to make a positive impact for business teams. BI is a growing field that shows no signs of slowing down, so let’s take a closer look at what makes it attractive to job seekers as well as students deciding what course of study to pursue.
Invest in Good Data Before Big Data
by Dwight deVera
Big data is without a doubt 1 of the top 5 BI trends of 2012. The hype around big data has driven many companies to hoard massive amounts of structured and unstructured information in the hope of unearthing useful insight that will help them gain competitive advantage. Admittedly, there is significant value to be extracted from your company’s growing vault of data; however it is data quality – not necessarily quantity – that is your company’s biggest asset. So here are 3 reasons why you should devote more of your IT budget to data quality:
1) Because good data quality sets the stage for sound business decisions.
Sensible business decisions should be based on accurate, timely information coupled with the necessary analysis. Decision-makers need to be equipped with facts in order to plan strategically and stay ahead of the competition – and facts are entirely based on having correct data. Though it’s not as “sexy” as big data, mobile BI, or cloud, data quality should be the foundation of all of these other initiatives.
Admittedly, achieving data quality is tough. Gartner analyst Bill Hostmann says, “Regardless of big data, old data, new data, little data, probably the biggest challenge in BI is data quality.” It crosses department lines (both IT and business users must take responsibility), and processes that have multiple levels of responsibility often suffer from the “everyone and no one is responsible” conundrum. It’s also a complex process that requires laying out common definitions (what is a customer, what are our conventions for company names – Inc. or no Inc. – for example), performing an initial data cleanse, and then keeping things tidy through ongoing data monitoring, ETL, and other technologies.
But ensuring that your data is timely, accurate, consistent, and complete means users will trust the data, and ultimately, that’s the goal of the entire exercise if you see this first reason as the most important. Trusting the data means being able to trust the decisions that are based on the data. Clean up the data you have in place, then you can move on to a strategy that incorporates additional sources of big data.
2) Because you have to.
Predictive analytics involves making predictions about the future or setting potential courses of action by analyzing past data. A 2012 benchmark study by