Skip to main content
July 14, 2023

How much is enough? Data, insights, and the Goldilocks equation

How to ensure your data is ‘just right’ for insight gathering.
data for medical affairs

We recently partnered with Reuters to explore how medical affairs teams work with insights. Our insights management survey yielded some interesting results, but none more so than this: 40% of respondents said they collect so much data it’s difficult to know what to focus on, while almost the same proportion (38%) stated they don’t generate enough data to extract the insights they need.

Same industry, same profession, but the exact opposite problem.

What’s equally fascinating is that this leaves just 22% of med affairs teams in what we could call the ‘Goldilocks Zone,’ with a volume of data that’s ‘just right’ for insight gathering. But what does it mean to have too much or too little data? What might be causing the problem, and how can medical affairs teams gather the correct amount in the future?

Questions about insights management? Our buyer’s guide has answers.

Too much data for insight gathering

For 40% of our survey respondents, proliferating data might be too much of a good thing. A surfeit of information can quickly become unmanageable and overwhelming, leading to ‘analysis paralysis’ and an inability to extract valuable insights.

However, for many med affairs teams, the issue is as much about data quality as it is about data quantity. Large volumes of data are often unstructured, incomplete, inconsistent, or inaccurate – usually due to data collection or integration issues. Any conclusions from such unreliable data will only lead to erroneous or misleading insights.

“The life sciences data landscape is growing in complexity and scale, with organizations generating increasingly high volumes of data. Yet, these data are still stored and searched using outdated methods. As a result, there are vast amounts of unstructured data stored in various siloed locations.” – Drug Discovery World

Data at volume becomes a resource issue, too. Many small to midsize life science companies lack the infrastructure to store and manage large quantities of data, let alone the computational resources and processing power to analyze it effectively. Maintaining data privacy, security, and compliance becomes increasingly complex at scale, too.

Lastly, ‘noise’ becomes a real issue in larger datasets. Irrelevant information can hide meaningful patterns and correlations, obscuring the insights you seek and slowing information gathering.

Too little data for insight gathering

The problem of too little data is perhaps more obvious. Simply put, working from too small a data set means you likely lack the information required to reach reliable conclusions.

Insufficient data makes creating robust, accurate data models all but impossible. The model will either fit the noise in the data over any valuable nuggets of information or be far too simplistic to offer anything of use. Equally, a small sample size reduces statistical power – limiting the ability to spot relationships between variables in the data.

In smaller data sets, discrepancies and outliers carry more weight. Erroneous data points can mask important patterns and relationships or invalidate your findings. Small data sets also fail to paint a complete picture – overlooking conditions or events that occur infrequently within the population, such as rare diseases.

According to McKinsey, “Lack of high-quality data sources and data integration… is cited by both pharma and medtech data and analytics (DnA) leaders as a top issue. In addition to accessing existing data, DnA leaders are increasingly interested in supporting decision-making by generating new data from business operations, such as new digital channels for HCP engagement.”

The importance of insights management

So, data sets that fall to either side of our nominal ‘Goldilocks Zone’ can be equally damaging for med affairs teams. But why do 78% of companies fail to capture the data they need to extract valuable insights? Other results from our survey may shed some light on the problem.

Let’s start with the fact that just 61% of respondents use technology to collect data and generate insights – a little over half of companies. And those that do use a variety of methods:

  • Data visualization products (58%)
  • Analytics software (49%)
  • Spreadsheets (37%)

This speaks to the fact that even those companies that have embraced technology for data capture and insight gathering haven’t yet adopted a unified strategy. With multiple systems, products, and approaches at their disposal, it’s hardly any wonder that data remains siloed, unstructured, or incomplete.

Here’s one last statistic from our study. 85% of respondents say that a single, organization-wide system helps them generate and share insights more effectively.

Medical affairs is seeking an end-to-end data capture and insights management approach. An insights management platform streamlines data collection, organization, and analysis, generating the valuable insights you need to make informed strategic decisions easier. Is an insights management platform the right move to help your business find the ‘Goldilocks Zone’? Read our complete insights management platform buyer’s guide.

More insights, direct to your inbox.

Get our newsletter.