Akio Morita, co-founder of Sony, is often quoted as saying, “We don’t ask people what they want; they don’t know.” People usually interpret this to mean that Morita-san did not value consumer insights, and they use it as a reason for not doing research or ignoring its results.

But nothing could be further from the truth.  Morita-san was a diligent student of consumers; he simply rejected traditional research methodologies, preferring to observe people and engage in informal discussions with them throughout the course of his day.  He would translate the insights he gleaned into product ideas and strategies that created some of the greatest technological breakthroughs of our time including the Walkman and the VCR.

In the same way, marketers should have a natural curiosity about consumers and actively pursue consumer understanding – not data that assuages a risk-averse CEO, or results that document performance for the record, not even analyses that purportedly assesses the ROI of certain marketing programs.  Instead marketers should seek out true insights that lead to proprietary perspectives on market opportunities and to new ideas that spark creativity and innovation.

The problem is, the role of data is misunderstood.

Many business leaders expect research to tell them what to do.  They want to data that makes it explicitly clear which new product idea is the winner, exactly how much they should raise prices, or a precise ROI calculation on a marketing investment. Not only is this an unrealistic expectation, it’s also a lazy approach.  Steve Jobs, another technology pioneer known to disdain market research, got it right when he said, “It isn’t the consumers’ job to know what they want.” It’s the marketer’s job to interpret consumer understanding into ideas of what consumers want.

People also expect research to predict the future.  Although great strides in database capabilities have been made and significant advances in predictive technologies are underway, most organizations don’t currently have the data or the capabilities to generate truly predictive data.  So they must rely on data that only tells them about the past.  Marketers must then translate what happened in the past into future projections. Like it or not, this requires creative thinking, assumptions, and leaps of faith.

Data definitely can inform these efforts.  Research, particularly anthropologically-based methods, helps marketers understand consumers’ lives and their needs and desires.  And prototype testing shows whether and how innovations fit with the way people currently live, think, and act.  This kind of understanding can lead to hypotheses about how people will react to new offerings.

Database analytics yield rich behavioral profiles and customer valuations.  These should guide targeting strategies and marketing investments.

And, research if designed well can also indicate how to communicate with consumers effectively – which messages trigger interest, how resonance varies between groups, how people interpret different communications approaches.  But importantly, data shouldn’t dictate creative strategy. By definition, creativity is original not derived.

In the end, the role of research is to inform.  The role of a marketer is to develop ideas.  That’s why business should be “data-informed, idea-led.”

Traditionally, the marketing research process starts with a business question that leads to a survey project.  Results come later, often too late.  The project results might be enriched with other information to help strengthen the insight and to tell the story, but “The project” is the starting point. Speed to learning is often in the slow lane.

Now we live in a world where there is a river of information that pre-dates the marketing question.  This river of information flows continuously and is fed by digital tributaries from social media, search, navigation pathways that wind their ways through owned media sites, etc.  This river is an incredibly rich source of marketing insights that is naturally occurring, in people’s own words, about what they care about.

So the marketing research flip is this: instead of starting with “The Project” and then adding in other stuff, we should start with the river and add in survey projects as needed.

What are some of the nuggets of gold we might find in the river?

• Certain consumer groups using our product differently than we ever imagined.  This was documented by the ARF listening playbook for Hennessy who found that people on Blackplanet.com were talking about the brand differently and using it more for mixing, changing Hennessy’s thinking tremendously about their own brand.
• Search leading to new predictive power.  Google economists presented compelling evidence last year of the forecasting value of search terms at predicting auto sales down to the brand level.  It makes sense; if more people are searching for your brand, it is a very good sign for near-term sales.
• Website metrics about traffic, which pages are viewed, etc. tell you a lot about how people engage with your brand.
• Emerging vocabulary.  When you measure brand performance based on attribute lists, you are using language that trails the marketplace, as it is the researcher’s vocabulary.  The challenge is to create listening platforms that create fast “sense and respond” systems to emerging social changes.
• Digital behavioral response to advertising or events.  Conversation in social media, search, visitation to owned media, the viral effects of sharing all can spike in response to either great advertising or to events (like acceleration problems).  I remember the NY Times created a tool to graphically show tweets minute by minute and by geography during the super bowl in 2009 (click on talking about ads on the left and play with the timeline).  The word response was dramatic.

Now, when you commit to wading into the river, you are committing to synthesis of these different information feeds into common themes that will hopefully result in the “aha” moment.  Also, because the river is continuously flowing, the speed to learning is faster than waiting for project results.

This new vision of how market research should operate it is not mine alone; it is the collective view of the leadership of the ARF Research Transformation Super-Council which includes leaders from Unilever, J&J, Kraft, Colgate, Kantar, IPSOS, McKinsey, Cambiar, Cambridge Group, MTVN and others.

On October 28^th, the new vision for research transformation and insights value creation will be unveiled by the ARF.  In this brave new marketing world where the consumer is totally empowered by digital life, there has never been a better time for the insights and research profession to up its game and have decisive impact on marketing action.

“Have senior, seasoned impact players on the front line (who are experts doing what the client can’t do vs. what the client doesn’t have time to do) focused on deploying their creativity on ideas that drive client’s business. Be focused on value creation, embrace and use data and analytics to elevate work, and be collaborative.”  

–Tony Palmer, chief marketing officer of Kimberly-Clark quoted in Ad Age May, 2010

Actually, Mr. Palmer was referring to K/C’s relationship with its advertising agencies but the statement could easily apply to what a progressive research/insights team wants from its research partners (take a minute and reread it with that in mind).

At the annual ARF Rethink conference, Susan Wagner, Vice President, Global Strategic Insights, Johnson & Johnson Group of Consumer Companies called for this level of servicing and said she would pay more for it.  Stan Stanunathan, VP Marketing Strategy and Insights , The Coca-Cola Company said he would be interested in creating leveraged compensation models where the research agency gets a bonus for superior business outcomes (which is starting to happen with ad agency compensation approaches).

Could the evolution of advertising agencies provide footprints in the snow for the evolution of marketing research firms?  If so, let’s examine ad agency evolution.

About 15 years ago, at super-agency conglomerates like WPP, the media parts from the different agencies were consolidated while the creative agencies were kept distinct.  The reason this made sense, as Erwin Ephron explained to me, is that media buying is about scale much more than the creative side is.  One could imagine thinking that media was like the factory and the creative side was where catching lightening in a bottle happens.  So an advertiser might find a super-agency, rotate among the creative teams until they get what they are looking for, and keep the media buying consistent.

Now a funny thing happened in the advertising industry; I’m not sure that these two halves of the business evolved as anticipated.  Today, media agencies are not a commodity service; they have highly differentiated offerings based on buying power, approaches, tools, and ability to impact media strategy which is becoming increasingly central to brand strategy. Media agencies are often the lead agency, have created planner teams (like creative agencies) and are even building their own creative teams supposedly to communicate more seamlessly with the creative agencies chosen by the advertiser.  In other words, the media agencies have evolved such that they are also in the value creation business.

Here’s the analogy to super-research agencies.  Online panels…the “factory” that executes the research is built on scale like media buying and the research account teams offering strategic thinking and branded solutions …or should be…are analogous to the creative agencies (splicing in Tony’s quote)”…deploying their creativity on ideas that drive client’s business. Be focused on value creation, embrace and use data and analytics to elevate work, and be collaborative.”

In fact, the consolidation of online research “factories” is happening.  For example, Kantar has built and acquired research agencies accounting for billions of dollars of billings which are served by a consolidated online research panel, Lightspeed.  We also see the emergence of companies that are basically production companies like GMI and Peanut labs.

If the big research agencies were to split their businesses this way, I’d say that both pieces have impressive business opportunities.  The marketing research agency’s commercial model is still mostly based on running research projects or programs efficiently and then analyzing the results from those programs.  However, in the ARF Insights Value Creation Model, we see the opportunity for “synthesis” to add tremendous value.  Right now, I’d say that consultants, account planners, and buyer-side insights teams are much more in the “synthesis” business than research agency account teams, but if research agencies  cultivate new skills in their account teams and ratchet up their offerings they can compete by providing a whole new level of business impact. There an opportunity to create a new type of research and insights firm with a high level of expertise at finding the “so what”.

Panel operations “companies” would have three great opportunities.  Improve online research data trustworthiness, develop strategies for obtaining representative data from hard to research consumer groups, and offer effective and efficient access to these capabilities to the broad research community.

At the ARF Research Transformation initiative, research firms are having this much needed conversation collaboratively with their clients about the future of the research ecosystem.

“Inspiring better business futures, by enabling organizations to bring value to daily lives, through continuous learning, listening and translating people and markets.”

It’s about being an agent of change for the whole organization.  Inspiring better business futures is what we do.  Research and insights is just our fastball. As Donna Goldfarb from Unilever said when asked what her job is, “I sell soap”.

Then we heard a sobering factoid from McKinsey; 70% of all transformation efforts FAIL.

So, how do we make sure that our Research transformation initiative is a lasting metamorphosis rather than something that reverts back to old beliefs and ways of working?

One of the great insights from the meeting came from Guarav Bhatnagar, Assoc. Partner, McKinsey who said that most transformation initiatives are impelled by an “oh sh*t” motivation which, he notes, is not sustainable.  As soon as things start getting a little better, the sense of urgency diminishes to the point that the transformation mandate loses its steam.  So while this is a great way to create a call to action, a transformation must also find the “oh wow” for the whole organization.

The original motivator for the ARF research transformation initiative was from an ARF meeting in July 2008 about listening methods which quickly become a discussion about how the research function is not making the impact on organizations that it should be.  At that meeting, leaders said things like:

• “We have lost the capacity to listen for the unexpected”
• “Surveys are torture”
• And the famous Kim Dedeker (then at P&G) remark; “Research as we know it will be on life support by 2012’. 

Clearly, we had our “oh crap” call to action.  To make our transformation sustainable, we must find the “oh wow” and we need to find it for the organization, not just for the research profession to make our transformation sustainable.  Guarav also noted organizations don’t transform, unless people transform.

This perspective ties perfectly into the July 2009 ARF workshop where we thought of the research function as if it were a brand and sought to find a new brand narrative.  The thinking from the book Primal Branding (Pat Hanlon author) is that brands are belief systems.  For the brand “research”, those beliefs must become corporate culture, permeating the whole organization.

Brand narratives should be about the “oh wow” and changing the belief systems of people, meeting the McKinsey acid test for lasting transformations.  Here are selected passages from the Research brand narrative created at the ARF:

Before digital media and the long-tail of choices, things were different. ..Today, marketing organizations must commit top to bottom to becoming fast learning organizations and immediately sense seismic trembles in people’s needs, preferences, attitudes, changing behavioral patterns in social and media communities, and on and on.  Inspired by human insights, today’s business leaders must create their own future.

…In a great leap forward, the Research/Insights function serves as an agent of change for creating the fast-learning organization.  We bring the voice of the human into the boardroom. We use proven methods for surveys and for “listening” to social media, search, shopper databases, digital behaviors, customer care, continuous and other naturally occurring data.  We also take advantage of new methods such as biometrics, communities, virtual environments…to help us uncover the next marketplace move, even when consumers cannot yet articulate the changes themselves.

Then, we translate for the organization the meaning of what we are learning about human beings and markets into lucid foresight that tells us where to play–and where to win.  Research used to be about the what (data), then about the so what? (analysis). Today, we go to the now what? (strategy, action), and we find ourselves accountable for the business results of what we recommend. 

…Our team needs to continually retool, going beyond business metrics that merely quantify–and commit to listening for the unexpected (which is where change and innovation and leapfrogging your competition comes from). .. the enterprise must realize that funding consumer and market learning is a critical investment, not a discretionary expense. 

We need to create a new breed of “researcher”: inquisitive and courageous by nature, analytical by training. They are passionate about understanding consumers, translating insights into business opportunities, leveraging social sciences and analytic skills, and then using storytelling to communicate these insights in unforgettable ways.

Above all, they bring energy and insights into the boardroom.

While some decision makers may prefer to trust their own instinct, and some traditional researchers are more comfortable crunching numbers than “listening” to consumers, they are today’s minority and against great odds.  We must understand that the big innovative successes will always be driven by human and market insights and that this is the new path forward…

The rituals of how we work together are changing and must continue to change.  We are brought into big issues from the beginning, we help frame the challenges and opportunities, we take stands based from our perspective of human behavior…We build windows, not walls. We create sparks of insight … that light the way to growth. Forward.

Via The ARF Research Transformation Super Council, industry leadership will be creating the roadmap to this vision.

All layers of the wedding cake are needed to for research to have an impact on creating a better business future.  Check out what happens if any one of the layers is missing by viewing the short slideshow:

Of course, easier said than done!  What are the skills and mindset of the successful researcher of the future?  What are the tools that are needed in light of the new findings from anthropology, behavioral economics, and anthropology about how humans process information, recognize, and decide?  What are the processes and structure of the organization that has committed to a culture of bringing the voice of humans into every marketing decision?  What are the metrics of success? 

Led by some of the biggest global marketers, consultants, and research agencies, the ARF Research Transformation Super-Council is committed to developing the roadmap for how organizations can transform to this new model and culture.

For more on the ARF Research Transformation vision, click here to view the youtube video, “inspiring better brands”.