Self-hosted split testing, built for engineers
A/B testing that lives on your servers.
No third-party script shipping your visitors’ data to someone else’s cloud, no consent-dialog scramble, no client-side flicker while a snippet decides what to render. Split runs on your own infrastructure — and its UCB bandit stops wasting half your traffic on variants that are already losing.
How it works
A copy test, from YAML to optimized traffic — no deploy required
Split runs as a self-hosted Phoenix API on your own infrastructure. You define tests in config, connect over WebSockets or HTTP, and the bandit algorithm handles traffic allocation from there.
Write a YAML test config
Define a target URL and one or more variants. Each variant lists CSS selectors and the DOM mutation to apply — replace, innerHTML, or remove. Upload it via POST /api/tests. No codebase changes, no release to wait on.
Connect over WebSocket
Clients connect with a user_id (persisted in a 30-day cookie) and the current url. Split auto-assigns and returns variants for every matching test on join — no separate lookup call, no third-party script, no round trip to another company’s cloud.
Apply mutations, track conversions
The client applies the returned DOM mutations directly. Conversions are tracked via dom_event, url, time_on_page, time_on_site, or scroll_depth — whichever fits the test.
UCB shifts traffic automatically
Every impression and conversion updates each variant’s upper-confidence bound. Instead of holding a fixed split for the life of the test, Split keeps moving traffic toward whichever variant is most likely to win — so losing variants stop costing you conversions early.
Features
Built for teams that outgrew marketing suites and aren’t ready for enterprise contracts
No dashboard fluff, no per-visitor pricing tier, no sales call. Split is a focused Elixir/Phoenix service you run yourself, with a small, well-defined API surface.
YAML configuration
Define tests with CSS-selector-based DOM mutations — replace, innerHTML, or remove. Version them like any other config file, and ship a copy test without a code review.
WebSocket API
Real-time variant assignment and conversion tracking over Phoenix Channels, served from your own deployment. Join once, get variants, push conversions — nothing leaves your infrastructure.
Auto-assignment by URL
Connect with the current page URL and Split returns variants for every matching test instantly — exact paths, wildcards, or query params.
UCB bandit algorithm
Upper Confidence Bound selection balances exploration and exploitation automatically, so traffic keeps shifting away from underperforming variants instead of sitting on a fixed split for the whole test.
30-day persistence
Variant assignment stays consistent for a user via a cookie-backed user_id, so returning visitors keep seeing the same variant.
RESTful HTTP API
Bearer-token-authenticated endpoints to upload, list, inspect, and delete tests, plus per-test statistics — self-hosted, so you own the data behind every one of them.
The API surface
A headline test, without a pull request
Upload a YAML test to your own API, then let clients auto-assign over a WebSocket you control. This is the actual shape of both — no vendor snippet in between.
test_id: homepage-hero-v1
url: https://example.com
variants:
variant_a:
"#hero":
mutation: replace
content_type: html
content: |
<h1>Turn data into revenue</h1>
<p>Split testing built for engineers.</p>
"#hero-cta":
mutation: innerHTML
content_type: html
content: |
<button class="btn-primary">Start Free Trial</button>
conversion:
type: dom_event
target: "#hero-cta"
event_name: clickimport { Socket } from "phoenix";
const socket = new Socket("wss://example.com/socket", {
params: { user_id: getUserId(), url: location.href },
});
socket.connect();
const lobby = socket.channel("s", {});
lobby.join().receive("ok", (resp) => {
// resp.tests already contains this visitor's
// variant assignment for every matching test
for (const test of resp.tests ?? []) {
if (!test.is_control) applyMutations(test.mutations);
}
});
// On the CTA click, report the conversion
document.querySelector("#hero-cta").addEventListener("click", () => {
lobby.push("track_conversion", { test_id: "homepage-hero-v1" });
});Algorithm
Stop funding losing variants
A fixed 50/50 split keeps sending traffic to the losing variant for as long as the test runs — that’s the cost of waiting for statistical significance. Split calculates a 95% upper confidence bound for each variant’s conversion rate (a Wilson score interval) and continuously routes traffic to whichever variant currently has the highest bound, so underperformers lose share as soon as the data says so.
Cold start
With zero data, a variant's upper bound starts near 0.87 — maximum uncertainty means new variants get traffic first, without any warm-up period to configure.
Deterministic, not sampled
Unlike Thompson sampling, there's no randomness at decision time — the same inputs always produce the same variant choice, which makes behavior easy to reason about and debug.
| Traditional A/B | Split (UCB) | |
|---|---|---|
| Traffic split | Fixed 50/50 | Shifts to the leading variant automatically |
| New variants | Same traffic share as control | Get priority first, due to higher uncertainty |
| Underperformers | Keep receiving traffic until the test ends | Lose traffic as soon as their upper bound falls behind |
| Decision process | Manual significance check | Continuous — bounds are recalculated on every impression |
FAQ
Frequently asked questions
Is Split actually self-hosted?
Yes. Split is an Elixir/Phoenix API you run on your own infrastructure — `mix phx.server` and it’s up, with your own API key set via `SPLIT_API_KEY`. There’s no third-party script embedded in your pages and no visitor data sent to a vendor’s cloud, so there’s no consent dialog to add just for the test.
What does Split actually run on?
Split is an Elixir/Phoenix API. Tests are uploaded as YAML, variant assignment happens over Phoenix Channels (WebSockets), and management runs through a RESTful HTTP API secured with Bearer token auth.
Do I need a deploy to ship a copy test?
No. A test is a YAML file — a target URL, one or more variants, and the CSS selectors they mutate — uploaded via POST /api/tests. Changing a headline or a CTA is a config upload, not a pull request.
How does a page get a variant?
A client connects to the WebSocket endpoint with a user_id and the current url. On join, Split matches that URL against every active test — exact path, wildcard (*), or recursive wildcard (**) — and returns variant assignments and DOM mutations automatically.
What can a variant actually change?
Each variant is a map of CSS selectors to mutations: replace (swap the outerHTML), innerHTML (swap the contents), or remove. Content can be HTML, Markdown, or plain text.
How is a visitor kept on the same variant?
The client persists user_id in a 30-day cookie and sends it on every connection. Returning visitors within that window get the same variant they saw before — no re-randomization.
How does the traffic-allocation algorithm work?
Split uses an Upper Confidence Bound approach: it computes a 95% Wilson score upper bound on each variant’s conversion rate and always routes to the variant with the highest bound. New variants get priority from uncertainty, and traffic shifts away from underperformers automatically as bounds cross — instead of holding a fixed 50/50 split for the life of the test.
What conversion types are supported?
dom_event (a click or other DOM event on a selector), url (navigating to a target path), time_on_page / time_on_site (a duration threshold), and scroll_depth (a scroll percentage threshold).
Can I handle single-page app navigation?
Yes. Push a url_changed message over the existing socket connection with the new URL, and Split returns fresh variant assignments in the same shape as the initial join response — no reconnect required.
Run split tests on your own servers, without funding losers.
Write a YAML config, connect a socket, and let Split’s UCB algorithm find your winner — no vendor script, no consent dialog, no deploy.
Early access
Get early access to self-hosted split testing
Tell us a bit about what you're building — especially if you're EU-based, privacy-conscious, or migrating off Google Optimize — and we'll reach out when Split is ready for you.