How we reconcile three measurement methods without averaging them.
A diagnostic is the first phase of a Signal Rift engagement — four weeks, your actual spend, our reconciliation methodology. Below is what week one looks like when we run it: three methods, one divergence, and the decision rule we use to resolve it.
Numbers below are representative. During a diagnostic, we run this against your actual spend data in week one.
Assume a retail portfolio running $4.1M on paid channels in a given week. We run three measurement methods in parallel. They agree on the total but disagree on where credit belongs. Attribution says retargeting drove 22% of conversions. MMM says 9%. The most recent geo holdout test said 14%.
Across the portfolio, this is a $1.4M/month allocation question. If Attribution is right, the team is under-spending retargeting by six figures. If MMM is right, retargeting is overcredited and pulling budget away from prospecting. If the geo test is right, the truth is somewhere in between — but closer to MMM than to Attribution.
Averaging the three produces a number that no method generates and no one can defend in a board meeting. We don't average. We treat the three estimates as a distribution over plausible truths and simulate allocation decisions across the full range.
CREDIT90% CI
We carry the disagreement into the decision.
Each method's estimate is a posterior distribution, not a point. We keep all three distributions alive in the allocation simulation. The question we answer is: which allocation performs best across every plausible version of reality the three methods suggest, weighted by how much we trust each method for this channel?
The chart shows three posteriors for retargeting credit. The shaded bands are 90% credible intervals. The dashed vertical line is where the weighted average would fall — and why we don't use it. The weighted average sits in a region where all three methods assign low probability. It is the least likely truth.
Running a test costs money. Not running it costs money. We calculate both.
When the three methods disagree beyond a threshold, we design an experiment that would resolve the specific disagreement. Then we calculate Value of Information: how much the answer is worth, weighed against the cost of running the test. Only tests that clear VoI get proposed to your team during the build phase.
Every experiment recalibrates every subsequent recommendation. We learn which methods are reliable for which channels against your spend. Each test makes the next recommendation more precise — which is why a build engagement compounds in value the longer we run it.
Want us to run this on your data?
A diagnostic is a four-week engagement. We run the three-method reconciliation against your spend data, surface the divergences already sitting in your measurement stack, and scope a build engagement if the math says one is worth it. The build is the second phase — we design an AMO for your org and operate it for you. Two SOWs. No software to license. Nothing for your team to stand up or govern.