CRM: A/B Testing & Customer Groups in ERP

A/B testing (bucket tests or split-run testing) is a randomized experiment with two variants, A and B, to compare two versions of a single variable,

typically by testing a subject's response to variant A against variant B, and determining which of the two variants is more effective.

“White Listing” 

• Adding an action only for some customers (i.e. adding an “enable delay” SMS)​
• Only customers on the “white list” get the action

“Blacklisting”​

• Removing an action only for some customers (i.e. disable 4-day payment call for late customers)​
• All customers get the action EXCEPT for customers on the blacklist

How to A/B Test

• IT Enables Action for A/B Testing in Actions Engine 

•  Create Customer Group​

•  In Actions Engine, Add Customer Groups to Action

Use in Testing Impact of Interventions (Superfluid)​, Also in use for Bundles (incentives for good payments)​

Disabling action blacklist/whitelist​

At the end of an A/B pilot, you may want to ensure that customers in the test groups no longer have the actions whitelisted or blacklisted​

Step 1: In the Actions Engine module, click on the Action Customer Groups in the navigation panel on the left. ​

• In the Action Customer Groups, you will see a list of groups of customers and the respective actions, and their descriptions that have been blacklisted or whitelisted​
• You should see the action and respective customer group(s) that you want to disable the blacklist/whitelist

Step 2: Click on Action Customer Group that you want to remove the whitelist of blacklist​

• You will see a list of customer groups for which the action has been blacklisted/whitelisted. This will have been earlier created in the CRM module under Customer Groups​
• Click on the Edit button at the top left

Step 3: Disable the blacklist/whitelist by removing/deleting the customer group (s) from the list​

• Click on the delete icon at the end of each customer group​
• Be careful ONLY to remove the customer group(s) that were involved in your pilot. There could be other ongoing pilots with the same action blacklisted/whitelisted and removing the associated groups may impact the pilot outcomes​
• By removing the group of customers, the action will no longer be blacklisted/whitelisted for those customers