Big Data: Quantum Change in Targeting Effectiveness
Big data is driving a revolution in targeting by making possible real-time micro-segmentation that continuously takes a large number of signals into account leading to significantly better predictions of what consumers want than previously possible.
How Big Data drives Micro-Segmentation?
Generate Rules From Data
Machine-learning makes it possible to generate complex multi-factor rules from data automatically. This makes it possible to construct sophisticated targeting models with minimal human effort that continuously adapt to changing data.
Effective Man+Machine Collaboration
Humans are good at expressing simple rules that are hard for machine learning and machine learning is good at deriving complex multi-factor rules that are beyond human reasoning.
Exploit Comprehensive Signals
Big data technologies make it possible to exploit large numbers of signals from unstructured and structured data.
Adapt in Real-Time
Targeting takes advantage of consumer actions as they happen, such as clicks, check-ins, searches, and purchases.
Previous Technology Generation: User Attribute Models
Targeted marketing based on user attribute models came of age in the 1980s with marketers working with statisticians to build predictive models of consumer behavior. Typically, a statistician would extract consumer attributes and transactions from a database, analyze the data to select, for example, a dozen attributes that showed correlation with purchases, and build a predictive model using tools such as SAS or SPSS. These models created substantial business value compared to the prior approach of demographic selection. For example, a retailer could restrict mailings of expensive catalogs to consumers who have a high likelihood of purchase rather than simply mail to everyone or select customers based on criteria like zip code and income level alone.
Previous Technology Generation: Rule-Based Segments
As data warehousing came of age in the 1990s, it led to the next technology generation of rule-based segments. It became possible to collect and store historical data on transactions as well as record and analyze other structured data on consumer interactions such as store visits or clicks. This led to development of rule-based targeting systems that are implemented using SQL. Rule-based targeting systems provide a business-user friendly rule authoring system that allows refinement and testing of rules. The same systems are also used by statisticians to express rules found using tools such as SAS and SPSS.
Drawbacks of Previous Technology Generations
Expensive and Slow
It can take several weeks of human effort to hypothesize and validate rule, and more complex rules take even longer. Further, the world changes constantly and consumers are subjected to new influences through entertainment, news, local and national events, or new products and promotions. This requires that rules be constantly maintained and adapted to a changing world. In practice, this often means that marketers settle for a few simple rules that have a reasonable chance of stability over time.
Given the difficulty of handling multiple factors, people focus on a small number of “primary” signals. This leads to loss of targeting effectiveness. For example, cross-channel signals are typically ignored as are signals that are critical for small segments of the population but “on the average” are of low value. The new Big Data approach allows one to integrate a large number of signals, each of which adds to the effectiveness of the model.
No Real-Time Adaptation
Traditional data approaches are based on offline methods and are built on technologies such as relational databases and statistical modeling tools that do not support any real-time processing.
Experience has shown that there are large numbers of multi-factor correlations that apply to small populations of consumers. For example, there may be millions of significant correlations in data for 10 million consumers and we may find micro-segments in which a few correlations hold in a statistically significant way. The traditional approach that relies on large segments based on a few attributes has low effectiveness due to the inability to account for micro-segments.