The shortlist is the last decision, not the first. What to settle before you contact anyone.

Key takeaways

1. Most bad research engagements are scoped badly, not staffed badly. The failure usually happens before an agency is chosen, when the brief describes a method rather than the decision the study needs to inform.
2. A DIY survey tool is genuinely sufficient for some work. The line is not budget. It is whether you need to generalise from the results to a population you cannot reach yourself.
3. Seniority is the variable that separates providers most and appears in proposals least. Ask who designs the questionnaire and who writes the analysis, and get names rather than roles.
4. Two proposals that look identical usually are not. Base size, sample source, subgroup support and analysis depth vary enormously between quotes for the same brief, and none of it is visible in the headline price.

Choosing a research agency is treated as a procurement exercise: assemble a shortlist, send the same brief to three providers, compare the responses, pick one. It is an orderly process and it frequently produces a study that nobody uses.

The reason is that the most consequential decisions have already been made by the time the shortlist exists. What question the research answers, which audience it speaks to, which subgroups it must support, how the findings will be used. Those decisions determine whether the study is worth commissioning. The choice of provider determines how well it is executed, which matters, but it is the second question rather than the first.

This piece works in that order. What to settle before you contact anyone, when you do not need an agency at all, what actually distinguishes providers, and how to read two proposals that look the same.

Why the shortlist is the wrong place to start

A research brief should name a decision. Not a topic, not a method, a decision.

"We want to understand our brand health" is a topic. "We need to know whether our declining consideration is a relevance problem or a visibility problem, because the two require different budgets and we are setting that budget in November" is a decision. The second brief produces a study that gets used. The first produces a report that gets circulated.

The distinction matters commercially because it determines what the study has to be able to do. A decision brief tells you which subgroups need a defensible base, how precise the estimates must be, and what would count as a clear answer. Those requirements drive the design, and the design drives the cost. Start from the topic instead and you get a standard study priced against a standard template, which may or may not answer your question.

Three things worth settling internally before any provider conversation:

The decision the research informs, and when it gets made. If the answer arrives after the budget is locked, the study was expensive entertainment.

Which cuts of the data you will present. National only, or by state, by segment, by customer and non-customer? Every cut you intend to act on has to be designed for. Adding one after fieldwork is not possible.

What you already know and how you know it. Existing data often answers part of the question, and the research only needs to cover the rest.

A provider who is any good will drag you through this anyway. Arriving with it done means the first conversation is about design rather than discovery, and it makes competing proposals genuinely comparable. If you do want a starting point for who to approach, we have written separately about the leading market research companies in Australia, but it is worth doing the work above first.

When is a DIY survey tool actually enough?

Often, and more often than agencies like to admit.

Survey platforms are excellent at what they are built for. If you want structured feedback from people you already have a relationship with, a customer list, an event audience, a user base, a platform will do it well and there is no reason to pay an agency for it. Satisfaction after a service interaction, feature preferences among existing users, feedback following a conference: all reasonable DIY work.

The line is not budget or sophistication. It is whether you need to generalise from your respondents to a population you cannot reach yourself.

The moment the question becomes "what does the Australian market think", rather than "what do our customers think", three problems appear that a survey tool does not solve.

You need people who are not yours. Your customer list contains only people who already chose you. It cannot tell you why the market did not, which is usually the commercially important question. Reaching non-customers means buying sample, and sample quality varies enormously in ways that are invisible in the final data. A study built on poor sample produces confident numbers that are simply wrong, which is worse than no study, and we have written separately about how badly this can go without anyone noticing.

You need the results weighted. Raw responses from a panel do not match the population. Correcting for that is routine work, and skipping it is the most common reason DIY brand studies produce numbers that contradict everything else the business knows.

You need the questions not to lead. Questionnaire design is the part that looks easiest and is not. Small changes in wording and order move results by more than the effects most brands are trying to detect. The risk is not that you get no answer. It is that you get a clean, plausible answer that is an artefact of how you asked.

The practical test: if you would present the finding to a board as representative of a market, it needs to actually be representative, and that is the work an agency is doing. If the finding is about your own customers and will be read as such, a survey tool is fine.

There is a middle path worth knowing about. Some organisations run their own operational surveys continuously and commission one properly designed study a year for the market-level questions. That is usually a better allocation than either extreme.

What separates one research agency from another

Four things, none of which is usually visible in a credentials deck.

Who actually does the work. In larger agencies the senior person who wins the project is frequently not the person who designs the questionnaire or writes the analysis. That is not automatically a problem, but it should be a stated fact. Ask directly: who writes the questionnaire, who runs the analysis, who presents the findings, and are those the same people in the room now?

Where the sample comes from. Panel providers differ in recruitment, incentive structure and fraud controls. An agency should be able to name its sample source and explain how it screens for inattentive and duplicate respondents. Vagueness here is a genuine warning sign, because sample is the input everything else depends on.

Whether the questionnaire is custom or a template. Syndicated and semi-syndicated products reuse a standard instrument across clients, which is what makes them cheaper. That is a legitimate trade and sometimes the right one. But a standard questionnaire cannot ask your question, and if your brief contains anything specific to your category or your strategic situation, a template will quietly drop it.

What happens after the data arrives. The difference between a deck of charts and a document that changes a decision is analytical time. Ask to see an example of a report, redacted if necessary. The gap between providers is more visible in a sample report than in any capability statement.

What is deliberately not on this list: size, client logos, and years in business. None of them predicts whether your particular study will be well designed. A large agency with a distinguished client list will still hand your project to whoever is available.

The questions that reveal the most

Five, in rough order of how much they tell you.

"Who specifically will design the questionnaire and write the analysis?" Names, not roles.

"What sample source will you use, and how do you screen it?" A good answer is specific and slightly technical. A poor one is reassuring and general.

"Which subgroups will this design support at the base size you have quoted?" This tests whether they have read your brief or priced a template.

"What would you change about our brief?" The most useful question on the list. A provider who has nothing to suggest either has not engaged with it or is not willing to argue with you, and both are expensive over the life of a program.

"What would make you tell us not to run this study?" Research that should not be commissioned is common. Anyone who cannot describe a situation where they would say so is selling rather than advising.

How to compare two proposals that look the same

Fix the specification yourself, then ask everyone to price it.

Send the same numbers to each provider: audience definition, base size, the subgroups that must be reportable, questionnaire length, number of waves, and the deliverables you need. Without that, you are comparing quotes built on different assumptions, and the cheapest one is usually cheapest because it assumed less. Published ranges for a standard Australian tracker give you a reference point before any quote arrives.

Four things to line up once the responses arrive:

Base size and how it is distributed. A quote for n=800 split across two audiences is a different study from n=800 in one.

What is one-off and what recurs. Design and setup are largely first-wave costs. A second wave should be meaningfully cheaper. If it is not, ask why.

Who is in the analysis line. Senior hours and junior hours cost differently and produce differently.

What the number excludes. Additional markets, translation, extra subgroups and incentive top-ups for hard-to-reach audiences are the usual omissions, and they surface after the contract is signed. Hard-to-reach B2B samples in particular can move a total substantially, and a proposal that treats them as trivial has probably not costed them.

If one quote is dramatically cheaper than the others for the same fixed specification, the difference is almost always in sample quality or analysis depth. Both are invisible until the study is finished, which is when it is too late to do anything about it.

What this comes down to

The provider decision is the last and smallest of the decisions that determine whether research pays for itself. A well-briefed study run by a competent agency will beat a badly briefed study run by an excellent one, every time, because the second study answers a question nobody needed answered.

So the sequence is: settle the decision the research informs, decide whether the question is about your own customers or about a market, specify the design yourself to the point where quotes are comparable, and only then compare providers on seniority, sample and analysis rather than on size or reputation.

The single question worth carrying into every conversation is what commercial decision changes depending on the result. If a provider cannot articulate that back to you, or if you cannot articulate it to them, the study is not ready to be commissioned regardless of who runs it. The metrics themselves are the easy part. Knowing which decision they are meant to govern is the work.

Frequently asked questions

Should we run a competitive pitch or approach one provider directly?

A pitch is worth the effort when the program is large, multi-year, or when you genuinely do not know what good looks like in this category. For a single annual tracker, the cost of running a formal pitch, in your time and theirs, often exceeds what it saves. A middle option works well: fix your specification, send it to two or three providers, and treat the quality of their questions about your brief as the main selection criterion rather than the price.

Does it matter if an agency has not worked in our category?

Less than most buyers assume, and it cuts both ways. Category familiarity speeds up questionnaire design and helps interpret findings. It also imports assumptions, and an agency that has run the same study for four of your competitors may reproduce a standard view of the category rather than testing it. What matters more is whether they have handled your structural situation before: your audience type, your purchase cycle, your kind of decision.

How long should a brand tracking project take?

For a single-wave study, typically six to ten weeks from brief to final report. Roughly two weeks on design and questionnaire development, one to two weeks in field, and two to four weeks on analysis and reporting. Compressed timelines are usually achieved by shortening the analysis phase, which is the part that determines whether the study changes anything. If you need results by a fixed date, work backwards and brief early rather than compressing the end.

If you are scoping a brand research program and want a second opinion on the brief before you take it to market, that conversation is worth having whoever you end up commissioning. We will tell you if the study you are describing is not the one you need.

Schedule a free 30-minute consultation to talk through your brief.

Tom Morris is Managing Director of Brand Health, an Australian brand research consultancy specialising in custom brand tracking. Connect on LinkedIn.