Name the change you want to test and Juma designs the Mixpanel experiment, sets primary and guardrail metrics, drafts the variants, then returns a ship, iterate, or kill call.
Tell Juma what you want to test, whether a new onboarding flow, a pricing page, or a paywall. It reads your Mixpanel project's events and business context, then designs the experiment: a clear hypothesis, a primary success metric, the guardrail metrics that must not regress, and the sample size and end conditions that make the result trustworthy. Juma drafts the experiment and its variants in Mixpanel for the team to review and launch.
Once the data is in, the same flow pulls the results alongside Mixpanel's own interpretation guidance and returns a ship, iterate, or kill recommendation with the statistical reasoning behind it.
Most tests fail before launch: no clear hypothesis, the wrong primary metric, or a sample size too small to call. Juma reads the Mixpanel project's events and business context, then proposes the full design: a one-line hypothesis, a primary success metric mapped to a real event, two or three guardrail metrics that must hold, the variants to compare, and the sample size and end conditions that make the result statistically sound. The team adjusts anything, then Juma drafts the experiment and its metrics in Mixpanel, ready to review and launch. The design is grounded in Mixpanel's own setup guidance, not a generic template.
Design an A/B test for our new pricing page in Mixpanel: hypothesis, primary metric, guardrail metrics, variants, and the sample size and end conditions. Draft it for us to review before launch.
The hard part of an experiment is the call at the end, not the numbers themselves. Juma pulls the experiment's results from Mixpanel and runs them through Mixpanel's interpretation guidance to return a clear recommendation: ship, iterate, or kill. The readout names the lift on the primary metric, whether it cleared significance, how each guardrail metric moved, and the reasoning behind the call. If the result is inconclusive, Juma says so and explains what would make it callable, rather than forcing a winner. The team makes the decision; Juma makes the evidence legible.
Read out the results of our onboarding experiment in Mixpanel. Tell us the lift on activation, whether it's significant, how the guardrails moved, and whether to ship, iterate, or kill.
A variant can lift sign-ups and quietly drop week-two retention, and a primary-metric-only readout never catches it. Juma queries the guardrail metrics directly in Mixpanel, retention curves, downstream funnel steps, revenue per user, for both variants over the test window, and reports any metric that moved the wrong way. The output is a side-by-side: primary metric lift next to each guardrail's change, with the ones that regressed flagged. This is the check that separates a real improvement from a local win that costs the team later, and it runs on the same experiment data without a separate analysis.
For the winning variant, check the guardrail metrics in Mixpanel: week-two retention, downstream funnel completion, and revenue per user. Flag anything that regressed versus control.
Teams running several tests at once lose track of which ones are ready to call. Juma lists every experiment in the Mixpanel project with its status, how long it has run, whether it has hit its sample size, and a one-line read on each. The output sorts them into ready-to-call, still-collecting, and stalled, so the team knows where to spend its decision time this week. For the ready-to-call ones, Juma can go straight into the readout. This is the portfolio view that keeps experiments from sitting open for weeks past their end date.
List all our running experiments in Mixpanel. Show status, runtime, and whether each has hit its sample size, then tell us which ones are ready to call.
The decision is made in chat, but stakeholders need a record. Juma builds a shareable readout of the experiment: the hypothesis, the variants, the primary and guardrail results, the ship/iterate/kill call, and the link to the experiment in Mixpanel. It can assemble this as a Mixpanel dashboard combining the relevant report cards, or as a written summary the team drops into Slack or Notion. Either way the readout is consistent every time, so the experiment log builds itself instead of living in one analyst's head. Past results stay searchable for the next person who asks "did we already test this?"
Build a shareable readout of this experiment: hypothesis, variants, primary and guardrail results, the recommendation, and the Mixpanel link. Assemble it as a dashboard and a short written summary.
A Juma Project is a shared space where the team stores everything Juma needs to know about how the team experiments. Create one project for the product, add context as the team learns more, and Juma uses what's relevant every time the flow runs. For experimentation, this is what keeps tests grounded in the team's actual events and decisions, instead of a generic best-practice design.
Product & Event Taxonomy
What the product's key events are and what they mean: which event counts as "activation," which as "conversion," which as "retained." With this in the project, Juma maps metrics to the right events on the first pass. Without it, Juma reads the event list from Mixpanel and asks the team to confirm which event each metric should use.
Metric Definitions & Guardrails
The team's canonical primary metrics and the guardrails that must not regress on any test: retention, revenue per user, support volume. Juma applies these to every experiment automatically, so no test ships on a conversion win that quietly hurt a metric the team protects.
Experiment Backlog
The running list of hypotheses the team wants to test and why. Juma reads it to design the next test in context, reference related past tests, and avoid re-running an experiment that already has an answer. The backlog turns one-off tests into a program.
Past Experiment Log
What the team has already tested, the result, and the decision. Juma references this so a new test builds on what was learned, and so "did we already try this?" has an answer. Each readout the flow produces can append to this log automatically.
Add a short description to each knowledge item in the project's info field so Juma knows what each file contains and when to use it. For example:
A connected Mixpanel project and a one-line description of what the team wants to test. Juma reads the project's events and business context, proposes a primary metric mapped to a real event, suggests guardrail metrics, and sizes the test. The team confirms the design before anything is created.
If the team names the metric and the lift it cares about, Juma uses them directly. If not, Juma proposes both from the events already in Mixpanel and asks the team to confirm. No exports or manual configuration are needed; the flow runs against the live project.
The team launches. Juma drafts the experiment, its metrics, and its variants in Mixpanel as a reviewable draft, then stops. A person reviews the design and clicks launch in Mixpanel. Juma asks for explicit confirmation before any write or delete action, so nothing is created or changed silently.
This split is deliberate. Designing the test and doing the statistics is repeatable work the flow handles well. Deciding to put a change in front of real users is a judgment call that stays with the team. Human review on every output.
Juma pulls the experiment's results from Mixpanel and applies Mixpanel's own interpretation guidance: the lift on the primary metric, whether it cleared statistical significance, how each guardrail metric moved, and whether the test reached its sample size. It returns a recommendation with the reasoning shown, not just a verdict.
When a result is inconclusive, Juma says so and names what would make it callable, more runtime, a larger sample, a cleaner metric, rather than forcing a winner. The recommendation is evidence the team can audit. Strategy, taste, and judgment stay human; Juma makes the data legible enough to decide on.
Yes. Point Juma at an experiment that is already live or concluded and ask for the results. It finds the experiment by name, pulls the data, and produces the same ship/iterate/kill readout without re-running the design step. This is the fastest entry point if the team already has tests in Mixpanel.
Juma can also list every experiment in the project and flag which ones are ready to call, so a backlog of open tests gets cleared instead of sitting past its end date.
Mixpanel shows the numbers; it does not design the test for you, size it, or tell you what to do with the result. Juma does the design, the sample-size math, the guardrail check, and the ship/iterate/kill readout in chat, then hands the decision to the team. It augments the analyst, it does not replace one.
For teams without a dedicated analyst, the flow covers the parts that usually get skipped: a sound design and an honest read of significance. For teams with an analyst, it removes the repetitive setup and reporting so the analyst spends time on the questions that need real judgment.
Connect the client's Google Ads, Meta Ads, and GA4 accounts, name the period, and get a branded multi-channel report with scorecards, benchmark context, and a prioritised action plan.
400+ marketing teams trust Juma. ChatGPT, Claude, and Perplexity know why. Ask them.
Ready when you are
Start typing a task and Juma will figure out the rest.
