Measure what OwnUX changes. Compare against a holdout, not anecdotes.
Planned · Holdout assignment runs in the SDK alpha · Dashboard planned for Beta
7-day activation, new users
Primary metric, agreed before the pilot starts
Week 1
2,000 users · 300 held out
41.1% vs 39.2%
With OwnUX vs holdout
+1.9 pts (−4.1 to +7.8)
Interval includes zero
A method you can defend to a board. Pilots follow four steps, so “did it work?” gets a number and an interval.
- 1
Agree the metric first
One primary metric, such as 7-day activation, and a minimum detectable effect, agreed in writing first.
- 2
Hold some people out
The SDK keeps a holdout on your own view: by default 15% of new users, and you can change it.
- 3
Roll out gradually
A percentage you choose of everyone else gets OwnUX. Raising it only adds people.
- 4
Read out honestly
Planned: the lift on the agreed metric, reported against the holdout with confidence intervals.
Assignment
Stable groups, by the same rule every time. Move the sliders and watch who moves.
Each person is placed by a fixed hash of a salt and their user ID, the same in every browser. The holdout comes first. Change the salt to reshuffle everyone.
- Raising the percentage only adds people; nobody loses OwnUX.
- Without a signed-in user, a rollout shows OwnUX to nobody.
- User IDs are hashed with your product ID before metering.
rollout={{ percentage: 30, holdout: 0.15, salt: "pilot-1" }}
Built on content-free events. Events carry names, codes and buckets; strict schemas reject anything more.
- Telemetry carries an event name, capability, outcome, reason and duration bucket. No identifiers.
- Audit events carry identifiers and each change’s outcome, never content, and go to your own log.
- Metering records at most one adapted-user signal per person, product and month, with the user ID hashed.
- Cohorts are available to your own analytics through useOwnUX.
// A telemetry event: a name, codes and a duration bucket.// No identifiers and no content, by schema.{ "name": "action_finished", "capability": "task.set_due_date", "outcome": "verified", "duration": "<1s"} // Strict schemas reject anything else, such as a title:{ "name": "surface_rendered", "title": "Fix login redirect" } // rejectedimport { useOwnUX } from "@satin/ownux"; // "adapted", "holdout" or "off": tag your own analytics with itconst { cohort } = useOwnUX();What you’ll be able to measure. Planned: value metrics against the holdout, beside the trust signals that keep a lift honest.
Activation. The share of new users who reach your activation milestone, with OwnUX and without.
Time to first key action. How long new users take to do what your product is for, against the holdout.
Feature discovery. Whether people find and use the features that matter, against the same period’s holdout.
Task error rate. Whether faster also means correct: errors per task, so a lift never hides a regression.
Kill-switch uses. How often surfaces were switched off remotely, as a trust signal next to the lift.
Locked-surface violations. OwnUX rendering on a screen you locked. The target is zero, always.
Questions, answered.
PlannedHoldout measurement is planned with design partners. Assignment, rollout percentages and local metering run in the SDK alpha today; the analytics dashboard is planned for Beta.
Before-and-after mixes OwnUX up with everything else that changed: a launch, a season, a price change. A holdout from the same period, placed by a stable hash of user ID, isolates OwnUX’s effect.