+44 (0) 20 7960 3802

How to tell if a B2B survey is reliable

By Andrew Dalglish -

We recently asked readers to pose their research questions on the B2B Marketing LinkedIn group.  One question came back more than once.  “In a B2B context how can I ensure a survey is reliable?”.  A big question but let me give it a shot.

Reliability boils down to whether the right questions have been asked to enough of the right people.

Survey questions need to be unambiguous, unbiased and asked in a standardised, objective format.   Beware of three classic mistakes here:

  • The ‘double barrelled’ question including two mutually exclusive items, e.g. ‘how satisfied are you with the quality and speed of response?’
  • The leading question containing implicit assumptions, e.g. ‘how much do you expect material costs to increase?’
  • The weighted scale which biases towards a favourable answer, e.g. ‘Are you extremely, very, quite or not at all satisfied?’

The next challenge is to ensure the survey has accessed those whose opinions really matter.  Ask yourself:

  • Have the major organisations in the market been surveyed?
  • Are key sub-segments of the market represented relative to their importance?
  • Can you be sure that individuals surveyed are really budget holders or influencers?

And finally the perennial challenge.  How many interviews is enough?  One approach is to use a test of statistical precision known as the ‘margin of error’.  This calculation indicates the degree to which answers might vary were the same survey to be repeated 100 times, e.g. a 3% margin indicates that in 95 out of 100 identical surveys the result would vary by no more or less than 3%.  Aiming for a low margin of error is the ideal approach but is not always practical in B2B surveys where there is a small pool of potential participants to draw on who are not always accessible.  An alternative then is to use the margin of error alongside four rules of thumb when specifying a target number of interviews:

  • What level of precision is adequate to make decisions?
  • Will additional interviews add sufficient value to justify additional cost?
  • Will it be possible to analyse differences between key segments?
  • Is the ideal number of interviews actually practical?

Find out more about Circle Research’s B2B research services here.

Enjoyed this post?  Subscribe and receive new posts by RSS

 

Leave a Reply

Your email address will not be published. Required fields are marked *