Sixty-four percent of organizations are already investing in or plan to invest in big data soon according to a recent Gartner report.* That equates to a huge number of individuals who now have to research how to embark on a big data deployment. The prevalence and benefits of big data analytics are undeniable, but there are some considerations to keep in mind before jumping in:
1) Identify a specific business need
Big data projects reap the most benefits when they address specific business needs. Having a use case in mind will help determine what data you need to analyze – social, machine or transactional data. Gartner recommends researching use cases and success stories in other industries; why not get inspired by what’s worked for others? Gartner analyst Doug Laney recently shared examples of big data at work in various industries: using big data analytics, the department store Macy’s was able to adjust prices in near real time for 73 million items based on demand and inventory; Wal-Mart was able to optimize search results and increase web checkouts by 10 – 15%; and American Express used sophisticated predictive models to analyze historical transactions and forecast potential churn. Once you’ve identified the analytic need not met by “small” data analysis, you have the first green light for considering big data technology.
Unearthing previously unimaginable insights from massive data sets is the premise of all the big data hype. Over the past few years as more and more stories come out about how companies are finding competitive advantages in their data, big data has moved beyond the buzz. Enterprises are deploying big data projects at a faster rate every year, and even more plan to do so within the next 2 years.
The extent to which a company can take advantage of big data analysis is determined by the amount of resources and infrastructure it has available. The good news is that now the barriers to entry have been lowered, making it possible for more organizations to meet their goals to transform operations with insights gained from big data. Here are three approaches that companies of any size can take based on their particular situation.
One thing to note is that these are underlying infrastructure approaches, and that you’ll still need an analytic engine like arcplan on top in order to interact with, visualize and distribute your insights.
Lots of resources and lots of infrastructure
Before big data was “big data,” Teradata was the only game in town. They’ve been at it for so long and their functionality is so robust – some of their capabilities are second to none. Now other vendors like SAP (with HANA) and Kognitio have their own massively parallel analytic databases. They offer robust processing and querying power on multiple machines simultaneously, enable near real-time MDX (Multidimensional Expressions, for OLAP querying) and SQL (Structured Query Language, the standard way to ask a database a question) queries, and in the case of SAP HANA and Kognitio, are fully in-memory. Not surprisingly, Teradata and SAP HANA come at a high price, but for that high price, the insights you achieve can be very near the speed of thought.
Why It’s a Bad Idea to Build a Business Intelligence Platform From Scratch
A friend of mine is a Python developer for a billion-dollar corporation. His team is building a custom call center reporting app that connects to the company’s cloud data storage via APIs. I’ve seen some of the application and while it’s impressive for a custom system, it’s mostly tables of numbers with the occasional pie chart. This is after 8 months of work, and the only people accessing the system are a select few big data scientists.
Believe it or not, a number of companies are doing this kind of in-house development of analytical platforms. All the hype surrounding big data has them convinced that they’re missing out on the action. Consequently, companies large and small are devoting huge amounts of time, money and human resources to developing custom business intelligence systems for big data (Google BigQuery, Hadoop, etc.) reporting rather than simply choosing a platform that already exists and is proven to work in a similar environment.
At the heart of this trend is a desire for big data to have a greater impact in the organization. Since it’s usually small teams of data scientists who are dealing with big data, their impact and effectiveness is equivalent to a small drop in a much larger body of water – their ripple effect throughout the organization is often minimal and short-lived. To extend the reach of big data in the company and get important insights out to a greater number of decision makers, a BI platform is a necessary next step – one that leverages big data insights in easily-digestible executive reports and dashboards.
Some companies are going down the road of custom BI platform development, but their efforts are no match for solutions like arcplan that are already available. Below is a list of what you’d need to do to build a BI platform from scratch. You’ll quickly see why the effort and expense aren’t worthwhile.
Business Intelligence as the Gateway to Big Data
Recorded Date: August 28, 2013
Speakers: Dwight deVera, arcplan Senior VP; Tom Veith, Senior Solutions Manager
About this webinar:
You’ve heard the hype around big data, and maybe you’ve put some thought into how it could impact your business. After all, the promise of big data is accessing hidden insights, discovering new approaches, and making better decisions. But how do you begin developing a technology approach that’s practical and doesn’t require a massive investment of money, time and resources?
The answer is to leverage business intelligence platforms that can handle huge data volumes, provide real-time access and enable data exploration. This webinar serves as a primer on how to practically use big data and BI together. Investments in big data usually allow a group of data scientists to deliver their results to a small community of business end users. To get beyond small communities and have an enterprise impact, you’ll need business intelligence – the mechanism to scale your big data initiative across the enterprise.
In this webinar, we:
- Lay out the big data approaches you can take based on your available resources and infrastructure, and the benefits and challenges of each approach
- Explain the benefits of utilizing existing BI platforms for big data analysis and visualization
- Demonstrate big data and BI in action on Teradata and Google BigQuery
This isn’t a webinar for IT professionals only. We break the concepts down in a way that makes sense for everyone.