Using A/B Testing in Extensions

Introduction

A/B testing is a method of testing the performance of different variants of a web-page that is shown to users at random. Developers can analyze the resulting test statistics to see which page variant helped users achieve a predetermined conversion goal - for example, a completed purchase.

The process of implementing A/B testing has four steps:

Step 1: Identify your Goal

Identify the goal of your A/B testing. For example, are you interested in whether users click a certain link or button, or do you want to follow a user’s “page-path” through the extension (the sequence of pages a user clicks while navigating through an extension)?

Step 2: Create a Variation

1. Statistically identify the population for each variation. For example, you can split a population into two groups by performing an odd/even check on broadcaster ID.
2. For each of the resulting groups, enable or disable a feature in your front end.
3. Use conditional styling (CSS) to hide or display different elements in each group.

Step 3: Monitor User Behavior in your Extension

Record events as users interact with your extension. For example, record the button clicks or page-path mentioned in Step 1.

Step 4: Analyze Results and Draw Conclusions

Analyze the data you collected. If there is a clear winner, deploy it. If the test is inconclusive, see what insights you can draw from your data and test them in subsequent rounds of A/B tests. 

Sample Code

twitch.ext.onAuthorized(function (auth) {
  //Let’s assume this returns a valid TUID by decoding the token
  tuid = getUserID(auth.token);
  // enable the GREEN button for even number Broadcaster ID
  if (value%2 == 0){
      $('#redButton).hide();
    $('#greenButton).show();
  }
  else{
     $('#redButton).show();
    $('#greenButton).hide();
  }
});