Resolving conflicts between Traffic Director and Formance Operator annotations

When using Google Cloud Service Mesh (Traffic Director) alongside Formance Operator, you may encounter conflicts with service annotations. This occurs because Traffic Director needs to add network endpoint group (NEG) annotations to services, while Formance Operator attempts to maintain its own service configuration.

Symptoms

You may observe:

• Services entering crash loops

• Continuous back-and-forth updates in logs between Traffic Director and Formance Operator

• Network endpoint group annotations being repeatedly added and removed

Solutions

There are two recommended approaches to resolve this conflict:

Option 1: Use an Admission Controller Webhook

The recommended long-term solution is to implement an admission controller webhook (such as Kyverno) in your Kubernetes cluster. This will:

• Modify configurations before they are written to the cluster

• Work transparently with Formance Operator

• Allow dynamic management of annotations

Option 2: Temporary Workaround

As a temporary solution, you can disable Formance Operator's management of specific services by adding the following annotation:

formance.com/skip=true

Required Annotations

The following annotations need to be preserved for Traffic Director to function correctly:

cloud.google.com/neg: {"exposed_ports":{"8080":{}}}
cloud.google.com/neg-status: {
    "network_endpoint_groups":{...},
    "zones":[...]
}