# G-Test Calculator for AB Tests

## Free online G-Test calculator for your AB test conversion rates

The G-Test is considered to be the best statistical significance test to determine if you have a statistically significant uplift in your conversion rate. Learn more about the G-Test here. You can use this free online tool to generate charts and summary reports (2nd tab) to include into your own AB test reports.

For online AB tests, a general rule of thumb is to achieve 90%+ confidence level. In other industries such as the medical field, a 95%+ threshold is more common. To ensure you account for daily seasonality, it is recommended you run your AB tests for at least a full 7-day cycle. It's also best practice to make a decision on the outcome of your AB test on a 7-day basis i.e. after the AB test has run for 7, 14, 21, 28, ... days.

### Estimated revenue impact calculation

#### Definition

The estimated revenue impact metric is the incremental revenue we can expect from setting the winning variation at 100% of your traffic. It's important to note that this estimate is only accurate once the variation in question is set to 100%.

#### Inputs

You should only add conversion numbers for a KPI that correlates directly with a purchase such as a Visited Checkout Thank You Page KPI. The reason for this is that the revenue is calculated by measuring Average Order Value (AOV) = revenue / conversions. Here's an example to illustrate the reason why.

Suppose we have thousands of conversions for a Click Add to Cart KPI and hundreds of actual purchases. It's obvious that clicks to that button don't always translate into purchases. This means that when we're calculating AOV and using the Click Add to Cart KPI, we'll be counting thousands of purchases instead of hundreds. There are drop-off rates at each step of the funnel, which we could potentially estimate. However, as long as your purchase KPI didn't reach statistical significance, that estimation would not be accurate. That is why it's easiest and best to measure revenue impact using a purchase-related KPI which has reached statistical significance.

#### Calculation

We first estimate how many conversions we expect if the winning variation was running at 100% during the experiment duration:

`Estimated Winner Conversions = Total Visitors * Winning Variation's Conversion Rate`
* Note, Total Visitors is the sum of all visitors across all variations.

We then estimate the number assuming the control variation was running at 100%:

`Estimated Control Conversions = Total Visitors * Control Variation's Conversion Rate`

We're interested in the incremental number of conversions when comparing these 2 scenarios (the winner variation running at 100%) vs (the control variation running at 100%):

`Estimated Incremental Conversions = Estimated Winner Conversions - Estimated Control Conversions`

Then, we want to find out what's the dollar value of these incremental conversions. To do that we estimate the Average Order Value. We calculate the one for the winning variation since we're assuming the results from this variation will carry through when set to 100%

`Test AOV = Winning Variation's Revenue / Winning Variation's Conversions`
* Note, Test Revenue and Test Conversions are the revenue and conversion numbers for the test variation.

So the revenue impact is measured as follows:

`Revenue Impact = Estimated Incremental Conversions * Test AOV`

We then normalize this figure to get daily and annual figures:

`Daily Revenue Impact = Revenue Impact / Experiment Duration`
`Annual Revenue Impact = Daily Revenue Impact * 365`

Here's the calculation for the daily number from the raw initial inputs:

`Daily Revenue Impact = {Total Visitors * (Winner Conversion Rate - Control Conversion Rate) * (Test Revenue / Test Conversions)} / Experiment Duration`

#### Placeholder Example

In the placeholders above, you'll notice that Base Visitors + Test Visitors does not equal to Total Visitors. That's because this example has 3 variations. To avoid having you to input all the numbers for each and every variation, this tool only requires you to input the total visitor number.

Here are the details behind the numbers for the placeholders. Assume Base is control, Test1 is the winning variation and Test2 is another variation.

`Total Visitors = 30,000`
`Base Visitors = Test1 Visitors = Test2 Visitors = 10,000`

`Total Conversions = 1,630`
`Base Conversions = 500; Test1 Conversions = 570; Test2 Conversions = 560`

`Total Revenue = \$40650`
`Base Revenue = \$13,500; Test1 Revenue = \$13,600; Test2 Revenue = \$13,550`

`Estimated Winner Conversions = 30,000 * (570 / 10000) = 1,710`
`Estimated Base Conversions = 30,000 * (500 / 10,000) = 1,500`
`Estimated Incremental Conversions = 210`

`Test AOV = \$13,600 / 570 = \$23.86`
`Revenue Impact = 210 * \$23.86 = \$5,010.53`
`Experiment Duration = 14 days`

`Daily Revenue Impact = \$5,010.53 / 14 = \$358`
`Annual Revenue Impact = \$358 * 365 = \$130,632`