Playbooks · 4 min read
Before You Measure ROI, Ask What the Number Would Change
Two posts on this site already cover this ground properly and you should probably be reading one of them instead.
Working out the ROI by hand has the actual arithmetic — the four numbers you can count, the one you cannot, and the by-hand matching exercise that produces a defensible figure. Writing an honest ROI report with no attribution has the document shape for when somebody else is going to read it.
This post exists for the thing neither of them asks, which is whether you should be doing this at all.
The three questions before the spreadsheet
Measurement has a cost. For this channel it is an afternoon of exporting leads and cross-checking them against your own sales record by hand, because there is no attribution, no conversion tracking and no join between the two. That afternoon is worth spending or it is not, and three questions settle it.
- What decision would this number change? Write the decision down before you start. If you cannot name one, you are measuring for reassurance, which is a real motive and a bad use of an afternoon.
- Could you act on either answer? If the number came back bad, what would you actually do differently — turn it off, rewrite the templates, change the offer? If the honest answer is "carry on", the measurement is decorative.
- Is the measurement cheaper than the decision? Spending a day to decide about something that costs nothing to run is a bad trade, and the running cost here is genuinely often nothing.
A number nobody will act on is not evidence. It is a way of feeling responsible.
The asymmetry that makes this channel unusual
Most ROI arguments are hard on both sides. Here, one side is trivially exact and the other is not available at all, which is worth knowing before you design anything.
The cost side is precise to the rupee. Templated replies are free and unlimited — not free on a trial, free. A credit is spent only when the model writes a new reply, one credit per reply, and the ledger has one row for each. The free tier is 100 credits with no card and packs start at ₹499. For an inbox that is mostly four repeated questions, the running cost is zero and stays zero.
The return side is testimony. Nothing here sees an order, a payment or your website. Any revenue figure attached to this channel is an inference, including the ones produced by tools that display them confidently.
Which means the real cost line is not credits at all. It is the half day spent building and testing, and the hour a month spent re-reading templates so they do not go stale quoting last season's price.
What to do instead of measuring, most of the time
For most accounts, the honest answer to "what is the ROI" is that the cost is near zero and the question is therefore not very interesting. What is interesting is whether the thing is working, which is a different question with a cheaper answer.
Three columns, monthly, in a spreadsheet you own: leads captured, leads that turned into customers, and one sentence about anything you had to fix. Fifteen minutes a month. Six months of it tells you more than any single measurement exercise, because it shows direction rather than a snapshot.
When it genuinely is worth the afternoon
Three situations, and outside them it usually is not.
A client is paying for it. Then the measurement is part of the deliverable and the reporting post is the one to follow.
You are deciding whether to keep doing something that costs real time. Not credits — hours. If somebody on your team spends six hours a week in the inbox, that is worth knowing the return on.
You are about to spend money on the strength of it. Ads, a bigger team, a different tool. A decision with a real price deserves a real count first.
The number you will not get, in any of those cases
The counterfactual. Some share of the people who bought after an automated reply would have messaged anyway, waited, and bought regardless. The automation answered them faster; it did not create them, and no experiment available to you separates the two groups.
So the defensible claim stays conditional: if even a modest share of these conversations would otherwise have gone unanswered, the setup paid for itself. That sentence survives being asked for a source. What the three data stores can and cannot tell you is where to look before you promise anybody more than that.
