About the figures here
Agency costs and timelines below are directional ranges from public market norms, not a quote — they vary widely by scope and firm. The pipeline’s per-run cost is measured. Treat the comparison as a shape, not a spec.
01What you’re actually buying from an agency
A market-research engagement buys you three things: access to people (panels, recruited interviewees), the labour of synthesising what they say, and a credible name on the cover of the deck. For a board presentation or a category-defining bet, that bundle is worth its four- or five-figure invoice and its multi-week timeline.
For most earlier decisions — should I build this, who hurts most, what do they already pay — you don’t need the panel or the cover page. You need the evidence. And the evidence is mostly people already talking in public, for free.
02Agency engagement vs a DIY pipeline run
The pipeline cost is the measured AI classification cost for roughly 300 threads. Agency figures are directional market ranges, not a quote.
03What you give up — honestly
You lose the recruited panel, so you can’t reach people who don’t post publicly, and you lose a senior researcher’s judgment in the synthesis. Forum data also skews toward the frustrated and the engaged — it’s a real bias you have to read around.
What you keep is more useful than it sounds: speed, a cost low enough to run the study five times instead of once, and full traceability — every line in the output links back to the thread it came from, so anyone can check the work instead of trusting a summary.
04When DIY is the right call
Use the cheap, fast version for the decisions you make often and early: idea validation, audience discovery, pain mapping, competitive teardown. Run it, act, and re-run when the picture changes.
Save the agency for when you genuinely need a recruited panel or an external name to sign off a high-stakes bet. The mistake is paying agency prices for a question a quick self-serve run would have answered well enough to move on.
See exactly what a run produces
No black box — the full six-step run, the scoring schema, and what a run costs are documented end to end.
Read the methodology →05A DIY research workflow that holds up
Frame the question as a claim
Write the decision as a falsifiable sentence — “teams switching CRMs lose data they care about” — not a vague topic. A claim tells you what evidence would confirm or kill it, which keeps the whole study from drifting.
Decide what actually needs a panel
Be honest about the question. Sizing a market or testing a price point may genuinely need recruited respondents. Knowing who hurts, why, and what they already pay rarely does — that evidence is already public.
Gather real public discussion
Pull unprompted threads where your audience already complains, compares tools, and asks for help. You want how they describe the problem in their own words, not how they answer a prompt you wrote.
Extract structured signal
Classify each thread into a fixed schema — pain intensity, willingness to pay, tools named, sentiment — so the findings tally across hundreds of posts instead of staying as one-off anecdotes you cannot compare.
Verify against the sources
For every conclusion, open the threads behind it. If a ranked, sourced report links each claim to the post it came from, this is a spot-check, not a rebuild — and it is the step that separates research from a hunch.
Act, then re-run
Make the call the study supports, ship something, and re-run the same claim later. Because the run is fast and repeatable, you can watch the signal move over weeks instead of paying for a fresh study each time.
06How to keep DIY research honest
Doing it yourself removes the agency’s guardrails, so you have to supply them. These are the pitfalls that quietly invalidate a self-run study, and the checks that keep it defensible:
- Selection bias — if you only read the threads that agree with you, you have built a case, not a study. Define the search before you look, and keep the off-target results in view.
- Anecdote as data — one vivid quote feels like proof and is not. Count how many threads show a pattern before you call it a finding, and say plainly how many you looked at.
- No source links — a conclusion nobody can trace is just your opinion in a nicer font. Every claim should link to the thread behind it so a sceptic can check the work.
- Forgetting the bias of the medium — public forums skew toward the frustrated and the engaged. Read around it, and never present qualitative signal as if it were representative of the whole market.
- Unstable method — if you cannot describe how you gathered and scored the evidence, you cannot re-run it or defend it. Write the claim, the schema, and the steps down so the next run is comparable to this one.
Frequently asked questions
Yes, for most early decisions. Idea validation, audience discovery, and pain mapping rely on evidence people already share in public, so the work is gathering and structuring that discussion rather than recruiting respondents. Frame the question as a claim, classify real threads into a fixed schema, and link every finding to its source. Sizing and statistical testing are where you may still want a panel.
A traditional agency engagement typically runs into four or five figures and takes weeks, because you are paying for recruited panels, senior synthesis, and a credible name on the deck. A self-run study over public discussion costs a fraction of that and turns around in an afternoon. The trade is speed and repeatability against the reach of a recruited panel.
Usually not for the questions startups ask most. Should we build this, who hurts most, what do they already pay — these are answerable from unprompted public threads, fast and cheaply enough to re-run as the picture changes. Save the agency for a high-stakes bet that genuinely needs a recruited panel or an external name to sign it off.
It is reliable for qualitative signal if you keep it honest. Define the search before you look to avoid cherry-picking, count threads instead of leaning on one quote, and link every claim to its source so the work is checkable. It is not representative of the whole market — public forums skew toward the engaged — so read around that bias and do not over-claim.
A repeatable pipeline over public discussion replaces most of what an agency does for early decisions. Instead of a panel and a deck, you get a ranked, sourced report of real threads scored by pain, willingness to pay, sentiment, and tools — each linked to the post it came from. It will not replace a recruited panel for sizing, but it answers the build-or-not questions well.