A customer who has bought from you for years sees an ad and places a new order. Another person discovers the brand through a campaign and buys for the first time.
It is a legitimate question when you have to decide where the next budget goes. Not every business has the data for an extensive experiment; it is worth finding out which method can clarify enough for your decision.
Both orders may appear in an advertising report. To understand advertising’s effect, we also need to consider what would have happened without it.
It is a difficult question: we cannot simultaneously observe the same person at the same time in both situations. However, we can build comparisons that help estimate the difference.
Credit in the report and additional effect
Attribution assigns credit to eligible interactions according to defined rules. Incrementality asks what changed because of the advertising.
That distinction does not mean excluding existing customers or dismissing remarketing. Ads can introduce them to a product, reactivate interest or encourage another purchase. Assess that role in context.
What the logic of an experiment looks like
One approach compares a group eligible to see the tested advertising with a control group that does not see it. The groups should be built so that the experimental method minimizes relevant differences between them.
What difference does the ad make?
Method diagram. No measured results or numerical estimate.
Exposed group
- Eligible for the advertising tested.
- Measure the agreed outcome.
Control group
- Without the advertising being tested.
- Measure the same outcome.
Compare the groups using the agreed experimental method.
Interpret the difference alongside the uncertainty in the estimate.
| Element | Why it matters |
|---|---|
| How groups are split | Helps avoid selecting people who were already more likely to buy |
| The same period | Reduces confusion with seasonal or market changes |
| Outcome defined in advance | Clarifies whether we track orders, value or another action |
| Volume and uncertainty | Shows how accurate the estimate can be |
Google describes treatment and control groups in its Conversion Lift documentation. The tool is not available to every account; eligibility and conditions need checking for the specific case. Google Ads: about Conversion Lift
This diagram shows the logic of the method. It does not include a measured result from a NOT MORE. campaign.
Why comparing two months does not settle the question
Advertising stops and sales fall. Seasonality, availability, promotions or other changes may also have played a part. Advertising resumes and sales rise; those same factors can still affect what we observe.
Comparing periods helps form hypotheses. A simple “before and after” comparison does not isolate cause. The more important the decision, the more carefully we should choose the evaluation method.
Two geographical areas are not automatically comparable because they are similar in size. Their histories, offers, competitors and the experimental design matter too.
What we do when an experiment is not feasible
Some businesses have low volumes, long sales cycles or insufficient resources for a precise experiment. In those situations, work with the available evidence, state the assumptions and keep the limitations clear.
We can analyze trends in new and existing customers, the changes in the offer and the history of the channels. These observations inform the decision, without being presented as exact causal evidence.
Sometimes the right question is whether a more sophisticated measurement would change the decision enough to justify its cost and time.
Can you evaluate the ad’s contribution if you have few sales?
You can get useful signals, but the precision of a causal estimate may be limited. At low volume, a few orders can change the result considerably. Before a test, we define the change we want to detect, the right comparison, the time frame and the risk of a wrong conclusion.
A sample size calculator needs relevant assumptions. For a test about purchases, the number of impressions is not enough if the assumed conversion rate and the unit being compared are missing. More impressions shown to the same people do not automatically mean more independent observations.
What do you do until you have enough data? You check measurement, the quality of inquiries, where customers come from and alternative explanations. You can make a cautious decision to continue, with an investment limit. Such a decision has to be presented with its uncertainty, without claiming you have proven the additional sales generated by the ad.
How we use the question in collaboration
At NOT MORE., we distinguish what the systems report from what we can demonstrate about a campaign’s effect. The method should be proportionate to the investment, available data and decision at stake.
A good result is more valuable when we understand its meaning. A recognized limit helps us choose the next step with clearer expectations.
Three useful questions
- What does attribution tell us, and what have we checked beyond it?
- What other factors could explain the change?
- Is an experiment feasible, and would it change our decision?
What decision would you make if you understood the campaign contribution?
Tell us which activity you are considering keeping, reducing or expanding, and what data you have. We can discuss what can realistically be learned and whether further analysis would change the decision.