Big Data and the future of decision making

As individuals, it is estimated that we make about 35,000 conscious decisions daily. Get up with the alarm or press snooze? Walk or drive to work? Make that large up-front investment for your business now or hold off until next year? The act of decision making is complex and yet it is about to get even more sophisticated due to the technological advances underway.

One of the biggest game changers is access to data like never before. Now consumers are able to order from across the globe and have goods arrive within the week, when previously, options were limited by proximity. To help consumers make these purchase decisions, there has been a surge of online review forums tracking consumer sentiment data. In fact, according to a survey by BrightLocal, nearly nine out of ten consumers say they check reviews on websites like TripAdvisor or Yelp! before deciding where to spend their money.

In response, businesses have had to become more strategic when it comes to reaching their audiences, and this is where the increasing importance of analytics comes into play. Today, business analysts have access to an ever-expanding amount of Big Data—the growing mass of information created by applications, devices, point-of-sale systems, social networks and more.

If used correctly, Big Data can provide organizations with valuable information to shape their businesses. This idea is exciting to business executives and as a result, the collection of Big Data has proliferated across industries in hopes of increasing sales and improving margins.

However, many are discovering that deriving value from this data still proves challenging in practice. According to a recent study by 451 Research, 85% of organizations say they are not yet able to exploit Big Data for competitive advantage.

The problem with Big Data

Why the gap? Why do organizations have such big hopes for the implantation of large-scale data collection, but come up so short?

The answer here is that it is extremely difficult to pull value from Big Data if you do not know the specific question you are trying to answer. Organizations must be able to isolate the data they need from the bulk of it, otherwise they are essentially trying to “boil the ocean” when that ocean is growing larger every moment. If the data set is too large, decision makers run the risk of unintentionally selectively pulling or eliminating values that may result in misleading correlations.

Lightning strikes—but not often

Without proper analytic tools to sort and manage the data coming in, companies are relying on strokes of luck to lead them to success. For example, Pfizer beat the odds with their drug UK92480, which was a washout during trials to treat angina in the 1990s. Just as the company was getting ready to give up on the drug, someone noticed a strange, yet consistent side effect being reported by trial participants. Now drug UK92480 is marketed under a different name: Viagara.

This is one of the biggest commercial drug success stories in history and the possibility of such a fortuitous accident has encouraged businesses to save every scrap of data they have in hopes of lucking out. However, they are more likely to grow their success by using science and analytics to make informed, strategic decisions based on their collected data.

Harnessing Big Data through investments in analytic technology

As technology improves, consumer demands increase and industries continue to become more complex, the demand for data scientists to interpret and propagate the masses of collected data will also continue to grow. Yet according to a McKinsey study, by 2018 only 200,000 data scientists will be available to fill more than 490,000 jobs. This means that the act of business decision making and interpreting data will have to continue to adapt and evolve through the adoption of highly sophisticated software and processes, in conjunction with a fierce battle for talent.

To make world-class decisions, you must start with a focus on the decisions—not on the data. Five strategies to strengthen your organizations focus in the right areas are:

1. Capture subject matter expertise. Organizations will benefit from gathering the expertise and perspectives of subject matter experts, codifying these perspectives and incorporating them into a decision process. Unfortunately most organizations are either unequipped to do so or are unware it is possible.
2. Accelerate intelligent solution creation. Only 16% of the average IT budget goes to fund innovation, with the rest devoted to maintaining legacy platforms built to solve yesterday’s problems. This can delay the creation of innovative decision management applications for months and sometimes years. With the appropriate technology, the time needed to create intelligent solutions can be shortened to a few weeks.
3. Speed insight to execution. It is assumed that the faster organizations can deliver new applications, the better when, in fact, decreasing the time it takes to update and improve analytic models can be even more important. Organizations are discovering that change management isn’t just an important attribute of a successful decision management solution—it’s a hallmark.
4. Build institutional memory. When decision makers are promoted or leave, they tend to take with them valuable knowledge. Once they are gone, their replacements don’t know why certain decisions were made or what logic drove certain actions. The ability to capture that knowledge and create a real knowledge-management infrastructure has become mission critical.
5. Increase analytic accessibility. Experience has taught us that providing an open framework to easily add powerful and advanced analytics is both unique and tremendously valuable. In fact, it may be a game changer for the analytics industry at large because it holds the key to democratizing analytics and decision science. By quickly and easily leveraging powerful and flexible analytic IP anyone, can quickly plug in analytic modules into a business flow or decision orchestration.

Ultimately, analytics-powered decisions will help improve many of the 245 trillion decisions being made around the world daily. The ways companies choose to use these breakthroughs in technology may be one of the most important decisions they ever make.

This article originally appeared in the October 2017 issue of Direct Marketing.