What the score is and how to calculate it
Create a single 0–100 confidence score that combines three simple inputs: the model's probability, a few business rules, and provenance (where the suggestion came from). Keep the maths simple so a non‑technical person can explain it in a meeting in Fareham or on the South Coast.
Typical formula (easy to implement): take each component as 0–1, apply weights, sum and multiply by 100. Example weights: model_prob * 0.6 + rule_score * 0.3 + provenance_score * 0.1 → scale to 0–100. That gives you a readable score and makes each part auditable.
Useful business rules to convert to rule_score:
- Mandatory fields present (yes=1 / no=0).
- Recent human edit conflict (last 24h edit by staff = 0.2 instead of 1).
- Company domain matches email domain (match=1 / mismatch=0.5).
- Source reliability (form=1, API partner=0.9, scraped=0.4).
Where to store the score, set thresholds and implement gating
Store the fields as CRM properties or sheet columns: confidence_score (0–100), model_probability (0–1), rule_flags (short text), provenance_source, and last_ai_run timestamp. These work in HubSpot, Salesforce, Marketo or Pardot and translate directly into a Google Sheet or Airtable column if you prefer a sheet‑backed middleware flow.
Pick two thresholds and a simple action plan. Example: auto_apply ≥85 (apply change, write audit fields); human_review 50–84 (route to review queue); <50 hold or reject (do not change record). The two thresholds are the auto threshold and the review threshold — tune them after sampling. Implement gating as a small workflow:
- HubSpot: custom properties + workflow that checks confidence_score, applies updates when ≥auto, otherwise adds to a review list and creates a task.
- Salesforce: custom fields + Flow that uses Decision elements to route to queues or update records if ≥auto.
- Sheets/Zapier: have AI write to staging sheet with score; Zapier Filter step routes rows to an update Zap when ≥auto, or to a review spreadsheet/task when below.
Sampling, audit trails, rollback and maintenance effort
Sample 5–10% of auto‑applied changes weekly (focus on scores near thresholds) to check false positives. Log the AI input, model_probability, rule_flags and previous value in a change log column or linked sheet row so reviewers can see context quickly.
For rollback: keep a simple previous_value column or daily snapshot of staged changes. Add a one‑click revert step in your workflow (HubSpot workflows can copy previous_value back to the field; in Salesforce use a rollback flow or a support script). Document a one‑page rollback plan and run a 15–30 minute drill every quarter.
Estimate effort for a small UK team: initial setup 4–16 hours (design weights, add fields, build workflows), weekly maintenance 30–60 minutes (sampling and quick fixes), monthly tuning 1–2 hours (thresholds, weights, problem rules). If you want a practical hand to implement this for HubSpot, Salesforce or a sheet‑backed flow, see our CRM & marketing data optimisation (Fareham) page. If you prefer help building and handing this over, Optira can assist with the first setup and a short coaching handover.