Post deployment configuration

Artikel
02/22/2024

Important

The Unreal Cloud Derived Data Cache (DDC) offering has been removed from the Azure Marketplace. As such, we're retiring this documentation on March 31, 2024. Until then, this documentation exists for reference only, with no official support available from Microsoft.

Please contact your Epic Games representative for more information about using Unreal Cloud DDC.

You can either use the following script, or manually complete the configuration in the Azure Portal.

Azure CLI

echo "Subscription Name?           (Type and press enter to continue)" && read -r SUBSCRIPTION_NAME
echo "Resource Group Name?         (Type and press enter to continue)" && read -r RG
echo "Managed Resource Group Name? (Type and press enter to continue)" && read -r RESOURCE_GROUP
echo "Managed App Name?            (Type and press enter to continue)" && read -r APP_NAME
echo "Traffic Manager?             (Type and press enter to continue)" && read -r TM_NAME

az account set -s "$SUBSCRIPTION_NAME"
az account show

Setup IP Addresses

managedResourceGroupId=$(az managedapp show --name $APP_NAME --resource-group $RG --query managedResourceGroupId --output tsv)
resourceGroup=$(echo ${managedResourceGroupId##*/})
nodeResourceGroups=$(az aks list --resource-group $resourceGroup --query [].[nodeResourceGroup] --output tsv)

for nodeResourceGroup in $nodeResourceGroups; do
    ipName=$(az network public-ip list --resource-group $nodeResourceGroup --query "[?contains(name,'kub')].[name]" --output tsv)
    ipID=$(az network public-ip list --resource-group $nodeResourceGroup --query "[?contains(name,'kub')].[id]" --output tsv)
    location=$(az network public-ip list --resource-group $nodeResourceGroup --query "[?contains(name,'kub')].[location]" --output tsv)
    az network public-ip update \
        --name $ipName \
        --dns-name $ipName \
        --resource-group $nodeResourceGroup
    
    az network traffic-manager endpoint create \
        --resource-group $RG \
        --profile-name $TM_NAME \
        -n $location \
        --type azureEndpoints \
        --target-resource-id $ipID \
        --endpoint-status enabled
done

Setup Azure Cosmos DB

account=$(az cosmosdb list --resource-group $resourceGroup --query [].[name] --output tsv)

declare -A tables=(
    [blob_index]="32000"
    [objects]="4000"
    [buckets]="8000"
    [content_id]="4000"
)

keySpace='jupiter'

for table in ${!tables[@]}; do
    az cosmosdb cassandra table throughput update \
        --account-name $account \
        --resource-group $resourceGroup \
        --keyspace-name $keySpace \
        --name $table \
        --throughput ${tables[${table}]} \
    && az cosmosdb cassandra table throughput migrate \
        --account-name $account \
        --resource-group $resourceGroup \
        --keyspace-name $keySpace \
        --name $table \
        --throughput-type "autoscale"
done

declare -A REGION_TABLES=( \
    [replication_log]="4000" \
    [replication_namespace]="4000" \
)

KEY_VAULTS=$(az keyvault list -g "$resourceGroup" --query [].[name] --output tsv)
for table in ${!REGION_TABLES[@]}; do
    for KEY_VAULT in $KEY_VAULTS; do
        LOCATION=$(echo "$KEY_VAULT" | awk -F'-' '{print $1}')
        az cosmosdb cassandra table throughput update \
            --account-name $account \
            --resource-group $resourceGroup \
            --keyspace-name "${keySpace}_local_${LOCATION}" \
            --name $table \
            --throughput ${REGION_TABLES[${table}]} \
        && az cosmosdb cassandra table throughput migrate \
            --account-name $account \
            --resource-group $resourceGroup \
            --keyspace-name "${keySpace}_local_${LOCATION}" \
            --name $table \
            --throughput-type "autoscale"
    done
done

Azure Portal

Configure the Public IP Addresses

Navigate to each Kubernetes service
In the Properties blade, select the Infrastructure resource group
Select the Public IP Address resource which starts with kubernetes-
In the Configuration blade, provide a DNS name label.

Connect to Traffic Manager

Navigate to the Traffic Manager Profile which is registered with your chosen DNS.
Add each Public IP Address Traffic Manager Profile from the Managed Resource Group as a nested profile.
- Go to Settings>Endpoint
- Select Add at the top of the page to start configuring the new endpoint properties
- Under Type, select Azure endpoint
- Under Target resource type, select "Public IP Address" and then select the resource name of the public IP address
- Select the Add button to complete
- Repeat the steps above for each public IP address

Cosmos DB

Adjust the RUs on Cosmos DB Collections. If you encounter 429 errors from Cosmos DB when syncing, the values can be increased further.

blob_index 32k RU
buckets 8k RU
objects 4k RU
content_id 4k RU
replication_log 4k RU
replication_namespace 4k RU

Health test

Test the deployment was successful by navigating to https://<hostname>/health/live/. This page should display "healthy", and be secured with your HTTPS certificate. If Healthy is not displayed and you need help to debug common issues, see Troubleshooting and support.

Healthy deployment screenshot

Advanced Configurations

If you would like to further customzie the deployment you can modify the following helm script. This can also be used if the AKS configurations where not properly set during deployment.

Check Helm Deployment

Check that the Azure Kubernetes Service (AKS) pods have started successfully. If they have, skip to the next section. Otherwise, use this script. Then, restart the AKS pods.

echo "Service Principal Client ID? (Type and press enter to continue)" && read -r APP_ID
echo "Worker Client ID?            (Type and press enter to continue)" && read -r WORKER_ID
echo "Hostname?                    (Type and press enter to continue)" && read -r HOSTNAME
echo "Namespace?                   (Type and press enter to continue)" && read -r Namespace

function setup_ddc(){
    # Disable History Expansion for key vault pointers that contain "!" per https://unix.stackexchange.com/a/33341/38809
    set +H
    HELM_CHART='oci://tchordestoragecontainerregistry.azurecr.io/helm/tc-horde-storage'
    HELM_NAME='myhordetest'
    HELM_NAMESPACE='horde-tests'

    TENANT_ID=$(az account show --query "tenantId" --output tsv)
    SUBSCRIPTION_ID=$(az account show --query "id" --output tsv)

    OAuthClient=$APP_ID
    FEDERATED_ID=$(az identity list -g "$RESOURCE_GROUP" --query "[?contains(name,'""$LOCATION""')].[clientId]" --output tsv)

    AzureConnectionString="akv!$KEY_VAULT|horde-storage-connection-string"
    ScyllaConnectionString="akv!$KEY_VAULT|horde-db-connection-string"
    OAuthClientPointer="akv!$KEY_VAULT|build-app-secret"

    declare -A locations=( [eastus]="East US" [westus]="West US" [southcentralus]="South Central US" [japaneast]="Japan East" [australiaeast]="Australia East")
    LocalDatacenterName=${locations[$LOCATION]}
    LocalKeyspaceSuffix=$LOCATION

    helm upgrade $HELM_NAME $HELM_CHART \
        --namespace $HELM_NAMESPACE \
        --set horde-storage.env[0].name=AZURE_CLIENT_ID \
        --set horde-storage.env[0].value="$FEDERATED_ID" \
        --set horde-storage.env[1].name=AZURE_TENANT_ID \
        --set horde-storage.env[1].value="$TENANT_ID" \
        --set horde-storage.env[2].name=AZURE_FEDERATED_TOKEN_FILE, \
        --set horde-storage.env[2].value='/var/run/secrets/tokens/azure-identity-token' \
        --set horde-storage.service.extraPort[0].name="internal-http" \
        --set horde-storage.service.extraPort[0].port="8080" \
        --set horde-storage.service.extraPort[0].targetPort="internal-http" \
        --set horde-storage.config.Azure.ConnectionString="$AzureConnectionString" \
        --set horde-storage.config.Scylla.ConnectionString="$ScyllaConnectionString" \
        --set horde-storage.config.Scylla.LocalDatacenterName="$LocalDatacenterName" \
        --set horde-storage.config.Scylla.LocalKeyspaceSuffix="$LocalKeyspaceSuffix" \
        --set horde-storage.ingress.hostname=$HOSTNAME \
        --set horde-storage.ingress.tlsCertName='unreal-cloud-ddc-cert' \
        --set horde-storage.secretStore.clientID="$FEDERATED_ID" \
        --set horde-storage.secretStore.keyvaultName="$KEY_VAULT" \
        --set horde-storage.secretStore.resourceGroup="$RESOURCE_GROUP" \
        --set horde-storage.secretStore.subscriptionId="$SUBSCRIPTION_ID" \
        --set horde-storage.secretStore.tenantId="$TENANT_ID" \
        --set horde-storage.serviceAccount.annotations.azure\\.workload\\.identity/client-id="$FEDERATED_ID" \
        --set global.ServiceCredentials.OAuthClientId="$WORKER_ID" \
        --set global.ServiceCredentials.OAuthClientSecret="$OAuthClientPointer" \
        --set global.ServiceCredentials.OAuthLoginUrl="https://login.microsoftonline.com/$TENANT_ID/oauth2/v2.0/token" \
        --set global.ServiceCredentials.OAuthScope="api://$OAuthClient/.default" \
        --set global.auth.schemes.Bearer.jwtAuthority="https://login.microsoftonline.com/$TENANT_ID" \
        --set global.auth.schemes.Bearer.jwtAudience="api://$OAuthClient" \
        --set horde-storage.podForceRestart=true \
        --set horde-storage.worker.env[0].name=AZURE_CLIENT_ID \
        --set horde-storage.worker.env[0].value=$FEDERATED_ID \
        --set horde-storage.worker.env[1].name=AZURE_TENANT_ID \
        --set horde-storage.worker.env[1].value=$TENANT_ID \
        --set horde-storage.worker.env[2].name=AZURE_FEDERATED_TOKEN_FILE \
        --set horde-storage.worker.env[2].value=/var/run/secrets/tokens/azure-identity-token \
        --set horde-storage.worker.Replication.Enabled=true \
        --set horde-storage.worker.Replication[0].ReplicatorName=Replicator \
        --set horde-storage.worker.Replication[0].Namespace=$Namespace \
        --set horde-storage.worker.Replication[0].ConnectionString=$HOSTNAME \
        --create-namespace \
        --install
}

KEY_VAULTS=$(az keyvault list -g "$RESOURCE_GROUP" --query [].[name] --output tsv)

for KEY_VAULT in $KEY_VAULTS; do
    LOCATION=$(echo "$KEY_VAULT" | awk -F'-' '{print $1}')
    CLUSTER_NAME="horde-storage-${LOCATION:0:8}"

    az aks get-credentials \
        --resource-group "$RESOURCE_GROUP" \
        --name "$CLUSTER_NAME" \
        --overwrite-existing

    setup_ddc
done

Dela via