Start Time To Variant

Start Time To Variant (STTV) is a performance metric that measures the elapsed time from when a page begins loading until the A/B testing variant is fully applied and visible to the user.

Meaning & Context

STTV captures the delay caused by A/B testing tools that must first load, execute JavaScript, determine which variant to show, and then modify the page accordingly. This metric is critical for client-side testing tools where variations are applied after the initial page load. Lower STTV values indicate faster variant delivery and reduced flickering or layout shifts.

Why It Matters

High STTV can cause flickering effects where users briefly see the original content before the variant loads, damaging user experience and potentially skewing test results. This delay can also negatively impact conversion rates independent of the actual variant being tested. Optimizing STTV ensures that performance issues don't mask the true impact of your experimental changes.

Example

A SaaS company discovers their A/B testing tool has an STTV of 800ms, causing noticeable flicker on their homepage. After implementing server-side testing, they reduce STTV to under 50ms and see a 3% overall conversion increase across all variants.