Skip to content

Gacha and A/B Testing

In this tutorial we show how to implement a simplified gacha-style spawning system in ChilliConnect that allows the player generate new characters with randomized attributes using in-game currency. We'll use the ChilliConnect Catalog Management feature to store the meta data that drives the system and a Cloud Code Script to perform the spawning. We'll also show how A/B Testing in ChilliConnect can be used to provide different versions of the meta data to different groups of players.

Although this type of system can be implemented directly in client side game code, there are several advantages to using a back-end system like ChilliConnect:

  • The system can be continually updated and refined without the need to distribute new builds of the game.
  • The system can be temporarily modified for specific subsets of players, allowing different configurations to be A/B tested, and time-limited promotions and live events to be run.
  • By implementing the spawning logic using Cloud Code we can also be sure that players cannot cheat by modifying local data on their device.

The full Unity project for this tutorial is available on the ChilliConnect GitHub samples repository under the folder "CharacterGacha". To run the project under your own ChilliConnect game you should change the GAME_TOKEN constant in GachaSceneController.cs.

 Download Sample Code

Creating the Meta Data

For this tutorial, we'll assume that we are creating a squad-based RPG where the player collects characters that are randomly generated using a gacha-style spawning mechanic. Each character can have the following attributes, ranked from 1 to 10: Level, Defence, Attack.

In our game, players can spend in-game currency to generate characters. We'll provide 2 "recipes" for the player that allow them to create a characters with different ranges of attributes for a different amount of currency:

Recipe SmallGenerate a level 1-5 character for 10 coins
Recipe LargeGenerate a level 6-10 character for 100 coins

To start implementing this system in ChilliConnect, we'll first define an Inventory item to represent a character that the player owns. Go to the Catalog section of the dashboard and add a new Inventory Item with the following properties:

AddCharacter

Next, define a new Coins currency, with an initial balance of 100:

AddCoins

We'll then add a new meta data object that will hold the configuration data for Recipe Small. We'll define the cost of the Recipe as well as the attribute ranges using JSON Custom Data. We will later download these definitions to our game to display to the player and also read them from Cloud Code when spawning a new character. Be sure to specify the "Recipe" Tag:

AddRecipe

Repeat the step above for Recipe Large, changing the attributes to the range 6 - 10 and the cost attribute to 100. Once you've created all the Catalog items, Publish the catalog by using the "Publish" button from the Working Copy view.

Downloading Definitions

Now that the meta data for the gacha system has been defined, we can use the GetMetaDataDefinitions method in ChilliConnect to retrieve the available recipes to the game at runtime. In our case, we'll use the "Tag" filter to limit the results to Recipes only:

public void GetRecipeMetaData()
{
    var tags = new List<string>();
    tags.Add ( RECIPE_TAG );

    var getMetadataDefinitionsRequestDesc = new GetMetadataDefinitionsRequestDesc ();
    getMetadataDefinitionsRequestDesc.Tags = tags;

    m_chilliConnect.Catalog.GetMetadataDefinitions (getMetadataDefinitionsRequestDesc,
        (request, response) => RenderRecipeList(response), 
        (request, error) => Debug.LogError (error.ErrorDescription));
}

See Full Code 

Getting Player Characters

Once the player has logged in, we'll request their current character list from ChilliConnect using the GetInventoryForKeys method:

public void GetPlayersCharacterList()
{
    var keys = new List<string>();
    keys.Add ( CHARACTER_KEY );

    m_chilliConnect.Economy.GetInventoryForKeys (keys,
        (request, response) => RenderCharacterList(response), 
        (request, error) => Debug.LogError (error.ErrorDescription));
}

See Full Code 

We also retrieve the players coin balance using GetCurrencyBalance:

public void GetPlayerCoinBalance()
{
    var keys = new List<string>();
    keys.Add ( CURRENCY_KEY );

    var getCurrencyBalanceRequestDesc = new GetCurrencyBalanceRequestDesc ();
    getCurrencyBalanceRequestDesc.Keys = keys;

    m_chilliConnect.Economy.GetCurrencyBalance (getCurrencyBalanceRequestDesc, 
        (request, response) => UpdatePlayerCoinDisplay (response), 
        (request, error) => Debug.LogError (error.ErrorDescription));
}

See Full Code 

Spawning Characters

At this stage, we already have enough information to implement the gacha system - the game code could parse the returned meta data and generate characters itself. While this is a legitimate approach for some games, in this tutorial we are going to show how to do this in a server authoritative manner that prevents players from cheating by manipulating data on their device.

Firstly, add a new Cloud Code script that accepts a single Recipe attribute:

AddScript

Paste in the below code:

try {

    var sdk = ChilliConnect.getSdk("2.0.0")

    /**
     * Load the recipe meta data
     */
    var recipeDefinitions = sdk.Catalog.getMetadataDefinitions(ChilliConnect.Request.Recipe, ["Recipe"]);
    if (recipeDefinitions.Items.length < 1) {
        return { "Error": true, "ErrorMessage": "Invalid Recipe" };
    }

    var recipe = recipeDefinitions.Items[0];

    /**
     * Check that the player has sufficient balance
     */
    var coinsBalance = sdk.Economy.getCurrencyBalance(["COINS"]).Balances[0].Balance;
    if (coinsBalance < recipe.CustomData.Cost) {
        return { "Error": true, "ErrorMessage": "Insufficient Balance" };
    }

    /**
     * Generate the character attributes based on the recipe
     */
    var characterData = {};
    var attributes = ["Level", "Defence", "Attack"];
    _.each(attributes, function (attribute) {
        characterData[attribute] = _.random(recipe.CustomData.AttributesMin, recipe.CustomData.AttributesMax);
    });

    ChilliConnect.Logger.info(characterData);

    /**
     * Add the character to the players inventory and deduct coin balance.
     */
    sdk.Economy.addInventoryItem("CHARACTER", characterData);
    remainingBalance = sdk.Economy.removeCurrencyBalance("COINS", recipe.CustomData.Cost);


    return { "Error": false, "Character": characterData, "CoinBalance": remainingBalance.Balance };

}
catch (e) {
    ChilliConnect.Logger.error("Error:" + e);

    return {
        "Error": true,
        "ErrorMessage": "Unknown Error"
    }
}

We first attempt to load the provided meta data recipe before checking the players coin balance, generating the character and then deducting the cost. On success, the generated character data is returned to the game. You can test the script using the Test tab from the script view. Remember the script must be published before being called from the game. The code to invoke the script from Unity is below:

public void CookRecipe(Recipe recipeKey)
{
    var scriptParams = new Dictionary<string, SdkCore.MultiTypeValue> ();
    scriptParams.Add ("Recipe", recipeKey.RecipeKey);

    var runScriptRequest = new RunScriptRequestDesc(SPAWN_CHARACTER);
    runScriptRequest.Params = scriptParams;

    m_chilliConnect.CloudCode.RunScript( runScriptRequest, 
        (request, response) => PostCharacterCreationActions(response),
        (request, error) => Debug.LogError(error.ErrorDescription) );
}

See Full Code 

A/B Testing

Now we have a basic system up and running we can add additional recipes by simply adding new meta data items through the ChilliConnect dashboard as well as update existing recipes dynamically without needing to rebuild and redistribute our game. We can also use ChilliConnect to perform A/B testing of our system - providing different versions of the system to different players and seeing how these affect the game's performance.

Let's imagine we want to test out a different set combination of recipes to see if they are more effective than the default recipes. Rather than providing 2 recipes, we'll try out a single recipe that costs 50 coins and can generate characters with attributes from 1-10.

To do this, the A/B Testing feature of ChilliConnect can be used to automatically group players into test buckets, optionally segmented by Country, Device Type, Platform or Device Model. We can use the Overrides feature to automatically change the users Catalog based on their group without having to change any of our code.

Creating a Test

First define a new AB test from the "Tests" menu under "Live Ops" in your ChilliConnect dashboard. Name the test as "Single Recipe Test", leaving the default start date and no end date specified.

AddTestDetails

For Segments, we can leave these blank for this tutorial which will simply match all players when assigning test groups.

AddTestSegments

For Test Groups, specify a player limit of 100, and check the box for "New Players Only" - this will ensure that only new player accounts are allocated to our test groups. Keep the existing default Test Group and add a new group with the name "Single Recipe". The default group will act like a control group and continue to use the original recipes.

AddTestSegments

Now we want to create overrides. From the add override dropdown, add "Recipe Large" and "Recipe Small" to the list. We want to override these to be removed from the catalog. To do this, click on "No Override" below "Single Recipe". This will open up the override editor for this item. Simply click "Remove from test group". We will now add a new item override by going to "Add Override", "Metadata", "Add New" and fill in the following properties.

OverrideSingleRecipe

Your Overrides should now look like this

OverrideList

Leave the Custom Data field empty, and save the test. The Tests list on the dashboard should now show our test as Active - this means that the next 100 new players to our game will be allocated to either the "Default" or the "Single Recipe" test group.

Running Demo

Running the demo project with different player accounts, you should now be able to see every second player is assigned to the "Single Recipe" test group and receives the metadata item we defined in our overrides.

Additional Considerations

This was a simplified example, but shows how A/B Testing, Overrides and the Catalog Management features of ChilliConnect can be combined to allow you to update game configuration and meta data at runtime, as well as running A/B tests against different groups of players. When implementing into a full scale game, there are some other points you may want to consider:

  • Rather than loading definitions from the ChilliConnect server each time the game starts, in real games you should check the players Catalog version on login and download and cache catalog definitions only when they change. See the Catalog guide for more information.

  • We loaded Catalog definitions using the GetMetaDataDefinitions method, but for games with a bigger Catalog you might want to consider downloading the Catalog package file instead.

  • We used Cloud Code to implement the spawning logic in the tutorial. This has the benefit of ensuring that the server controls the creation of characters and prevents cheating, but also means that the player must be online to play the game. Depending on your genre and target audience this may not be appropriate for your game.

  • It is also standard practice to include the default data bundled in your game for situations where the player first starts up the game and has no internet connection. Bundling the default data ensures that they will be able to at least start the game, with the default definitions later being updated once their connection is restored. You can do this by downloading and extracting the relevant package for your Catalog in to your game.