We recently asked readers to pose their research questions on the B2B Marketing LinkedIn group.  One question came back more than once.  “In a B2B context how can I ensure a survey is reliable?”.  A big question but let me give it a shot.

Reliability boils down to whether the right questions have been asked to enough of the right people.

Survey questions need to be unambiguous, unbiased and asked in a standardised, objective format.   Beware of three classic mistakes here:

  • The ‘double barrelled’ question including two mutually exclusive items, e.g. ‘how satisfied are you with the quality and speed of response?’
  • The leading question containing implicit assumptions, e.g. ‘how much do you expect material costs to increase?’
  • The weighted scale which biases towards a favourable answer, e.g. ‘Are you extremely, very, quite or not at all satisfied?’

The next challenge is to ensure the survey has accessed those whose opinions really matter.  Ask yourself:

  • Have the major organisations in the market been surveyed?
  • Are key sub-segments of the market represented relative to their importance?
  • Can you be sure that individuals surveyed are really budget holders or influencers?

And finally the perennial challenge.  How many interviews is enough?  One approach is to use a test of statistical precision known as the ‘margin of error’.  This calculation indicates the degree to which answers might vary were the same survey to be repeated 100 times, e.g. a 3% margin indicates that in 95 out of 100 identical surveys the result would vary by no more or less than 3%.  Aiming for a low margin of error is the ideal approach but is not always practical in B2B surveys where there is a small pool of potential participants to draw on who are not always accessible.  An alternative then is to use the margin of error alongside four rules of thumb when specifying a target number of interviews:

  • What level of precision is adequate to make decisions?
  • Will additional interviews add sufficient value to justify additional cost?
  • Will it be possible to analyse differences between key segments?
  • Is the ideal number of interviews actually practical?

Find out more about Circle Research’s B2B research services here.

Enjoyed this post?  Subscribe and receive new posts by RSS


Leave a Reply

Your email address will not be published. Required fields are marked *

Recent posts

Inside-out brand research

How can we ensure actionable results?

The British Research Barometer: An insight into the insight industry


B2B marketing

Business trends

Employee research

Generate and test new B2B product and service concepts

Identify the optimum B2B brand positioning

Identify the optimum B2B pricing model and point

Improve customer satisfaction and loyalty

Map the B2B buying journey and drivers of supplier choice

Map the B2B market structure and profile competitors

Market research design and methods

News about Circle Research

Segment based on behaviours, needs and attitudes

Support B2B thought leadership & PR

Track and manage B2B brand perceptions


About Andrew

Andrew has specialised in B2B research for over a decade and co-founded Circle Research in 2006. He is a columnist for B2B Marketing Magazine, a regular contributor to Research Live and frequent speaker at leading events such as the B2B Leaders Forum, Customer Experience Live and the Social Media World Forum. Andrew is a Chartered Member of the MRS, teaches the MRS B2B research course and holds an MA in Psychology from Aberdeen University alongside an MSc in Marketing from Strathclyde University.

Follow us

© Circle Research 2018 | Registered in England: 5669149