Azure APIM Policy to failover primary backend URL to secondary in case of a status code 429

Question

I have an APIM endpoint that calls a backend HTTP service. If it returns the status code 429, the call needs to be retried to the secondary backend. The backends are working fine without any issues. I tried setting this is retry policy as below but it didn't work.

My requirement is when the primary backend sends a status code of 429, the request should go to the secondary backend without any retry on primary backend. Can someone help me on this?

Below is the policy i used and Please find the trace logs for the same policy here

<policies>
    <inbound>
        <base />
        <set-method>GET</set-method>
        <rewrite-uri id="apim-generated-policy" template="/" />
        <set-header id="apim-generated-policy" name="Ocp-Apim-Subscription-Key" exists-action="delete" />
    </inbound>
    <backend>
        <retry condition="@(context.Response.StatusCode == 429)" count="1" interval="1" first-fast-retry="true">
            <choose>
                <when condition="@(context.Response.StatusCode == 429)">
                    <set-backend-service base-url="https://secondaryserver.eastus.azurecontainerapps.io/" />
                </when>
            </choose>
            <forward-request />
        </retry>
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>