abtestly vs Convert: an honest comparison

I build A/B tests on both of these tools, so the comparison below is from real use, not a spec sheet. Pricing for both tools was checked on their own sites on 29 September 2026; check them for current numbers.

At a glance

dimension	abtestly	convert
Entry paid price (100K MTU)	Pro $249/mo, or $199/mo annually (200K MTU); Starter $99/mo, or $79/mo annually (50K MTU)	Growth $399/mo, or $299/mo annually; Pro higher again
Free option	14 day free trial	15-day trial (no card)
Editor	Code only (JavaScript and CSS), no visual editor	Visual editor plus five code editors
Test types	A/B testing	A/B, split URL, multivariate, full-stack and feature flags (Pro)
Delivery	Cloudflare edge, 300+ cities; config served without an origin round-trip	Akamai CDN; synchronous by default
Snippet and anti-flicker	~0.9KB loader plus ~30KB gzipped runtime; optional full page hide, off by default, capped at two seconds; no hide scoped to one element	Built-in anti-flicker shield, synchronous by default
SPA route changes	Per-site SPA mode; lifecycle re-runs cleanup-first on route change	Re-evaluates on history.pushState and cleans up changes from experiences that no longer apply; your custom JS must guard against running twice
Stats engine	Two-proportion z-test on every plan; sequential and Bayesian on Pro and above; 95% intervals on every lift	Frequentist and Bayesian; sequential testing on every plan
Published methodology	Fully documented, including SRM and verdict thresholds	Feature docs exist; no single published methodology page found
Code in version control	Per variation code history with a diff view in the app, plus a REST API	REST API, a version-control widget, and an MCP server
Integrations	GA4 via the dataLayer, plus Segment, RudderStack, Amplitude and Mixpanel	90+ integrations
Beyond A/B testing	Not yet	Heatmaps and personalization
Compliance posture	GDPR-conscious; no published certifications yet	ISO 27001, SOC 2 Type 2, EU servers
Overage handling	No overage charges; new visitors stop being bucketed	Optional overage billing per block

Where Convert is genuinely better

Convert is the more complete platform, and it is fair to say so plainly. It has a visual editor that a marketer can drive without a developer, where abtestly has nothing comparable because its experiments are code. It does multivariate, split URL, and full-stack testing with feature flags, where abtestly does A/B testing. It carries 90+ one-tag integrations against abtestly's GA4 connection and four product analytics adapters, and bundles heatmaps and personalization. It also brings published ISO 27001 and SOC 2 certifications, EU servers, phone support on Pro, and a decade-plus of operating history. If breadth, a visual editor, or formal compliance decide it for you, Convert wins.

Where abtestly is better

abtestly trades that breadth for speed, transparency, and price. Variants are plain JavaScript and CSS, and every change is kept in a code history for each variation that you can diff, so a reviewer sees exactly what changed since launch. The test config is served from Cloudflare's edge in 300+ cities with no origin round-trip, and the runtime is about 30KB gzipped. Its anti-flicker hide is an opt-in setting, off by default, that hides the whole page for at most two seconds; there is no hide scoped to a single element. The entire statistics methodology, including the confidence-interval math, the sample-ratio-mismatch check, and the verdict thresholds, is published, and nothing is called a winner below 95% confidence. There are no overage charges, and at 100K tested users it costs about a third less than Convert Growth.

Pricing, like for like

At 100K monthly tested users you need abtestly Pro, which covers 200K and costs $249/mo, or $199/mo billed annually, against Convert Growth's $399/mo, or $299/mo annually, so roughly a third less. Convert Growth caps active experiences at 5 per project, so the closest match on capability is Convert Pro, which is higher again. abtestly starts lower too, with a Starter tier at $99/mo, or $79/mo annually, for 50K MTU. Both offer a free trial: 14 days on abtestly, 15 days on Convert. abtestly never charges overage; new visitors simply stop being bucketed once you hit your limit. Convert's overage billing is optional but bills per block when left on.

The developer workflow

This is the difference you feel day to day. On abtestly you write raw JavaScript and CSS per variant in a code editor, and each change lands in a code history you can diff and review. SPA mode re-runs the lifecycle, cleaning up first, on route changes, so a variation does not leak across virtual pages. Convert gives you five code editors, from project scope down to variation scope, plus a visual editor and a REST API, a version-control widget, and an MCP server, so it is genuinely code-friendly too, just with more surface area and a visual path as the default. If everyone building tests writes code, abtestly's code only model fits more naturally. If your team mixes developers and marketers, Convert's visual editor earns its place.

Whichever you choose, the parts that actually break a client-side test are the same. I cover them in why experiments stop firing on SPA route changes and stopping flicker without tanking Core Web Vitals.

Which should you pick

Pick Convert if you need a visual editor for non-developers, multivariate or full-stack testing, a wide integration stack, heatmaps and personalization, or formal compliance certifications, and the budget that comes with a mature platform.

Pick abtestly if your experiments are built by people who write code, you want edge delivery and a code only workflow with a diffable history, you report into GA4, and you would rather read a published statistics methodology than pay for breadth you will not use.

Genuinely unsure? Run both for a week, each on its free trial, and compare visitor counts, conversion rates, and the sample-ratio-mismatch check before you commit.

Common questions

Is abtestly or Convert cheaper?

abtestly is cheaper at the same volume. At 100K monthly tested users you need abtestly Pro, which covers 200K and costs $249/mo, or $199/mo billed annually, against Convert Growth at $399/mo, or $299/mo annually. abtestly also never charges for overage, where Convert bills overage per block unless you switch it off. Both offer a free trial: 14 days on abtestly, 15 days on Convert. Convert Growth caps active experiences at 5 per project, so the closest match on capability is Convert Pro, which is higher again.

Which is better for developers, abtestly or Convert?

Both are code-friendly, but they lean different ways. abtestly is code-first: variants are raw JavaScript and CSS with a built in code history you can diff, delivered from Cloudflare's edge, with a fully published statistics methodology. Convert gives you five code editors plus a visual editor, 90+ integrations, multivariate and full-stack testing, and a longer operating history. Pick abtestly for a lean, edge-native, code only workflow; pick Convert for breadth, a visual editor, and a mature ecosystem.

What does Convert do that abtestly does not?

A visual editor a marketer can drive, multivariate and full-stack testing with feature flags, 90+ integrations, heatmaps and personalization, published ISO 27001 and SOC 2 certifications with EU servers, and a decade-plus of operating history. abtestly is code-first A/B testing; multivariate testing and formal certifications are not in the product today.

What does abtestly do that Convert does not?

No overage charges, a fully published statistics methodology with a 95% interval on every lift and an SRM check, a code-first workflow with a per variation code history and diff view, edge delivery from 300+ Cloudflare cities with a runtime of about 30KB gzipped, and a lower price at the same tested-user volume.

Can I try both before committing?

Yes. Run the same test in both tools at once for a week, each on its free trial, then compare visitor counts, conversion rates, and the sample-ratio-mismatch check before you commit to either.