WithRetries Not Waiting Long Enough

[GusPrice]

Checklist

• I have looked into the README and have not found a suitable solution or answer.
• I have looked into the documentation and have not found a suitable solution or answer.
• I have searched the issues and have not found a suitable solution or answer.
• I have upgraded to the latest version of this SDK and the issue still persists.
• I have searched the Auth0 Community forums and have not found a suitable solution or answer.
• I agree to the terms within the Auth0 Code of Conduct.

Description

I am running into an issue where when I receive a 429 response from the management api, it seems the retries are not properly waiting to try again.

It seems that with this retry option, or just using the default retries I frequently run into an issue where the client gets a 429, but retries too soon. It seems like it's using the header value and getting to the final return in backoffDelay() in internal/client/client.go:

return time.Duration(resetAt-time.Now().Unix()) * time.Second

But it seems like maybe this math isn't quite waiting long enough for things to have reset. Here's an example of the debug logs where you can see the reset header value is the current time the code is running at but it continues to fail.

Sample Output

2025/02/06 11:00:31 
HTTP/2.0 429 Too Many Requests
Content-Length: 120
Access-Control-Expose-Headers: WWW-Authenticate,Server-Authorization
Age: 1
Alt-Svc: h3=":443"; ma=86400
Cache-Control: public, max-age=1
Cf-Cache-Status: HIT
Cf-Ray: 90dd62320973645e-SJC
Content-Type: application/json; charset=utf-8
Date: Thu, 06 Feb 2025 19:00:31 GMT
Retry-After: 1
Server: cloudflare
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: origin, Accept-Encoding
X-Auth0-L: 0.025
X-Content-Type-Options: nosniff
X-Ratelimit-Limit: 10
X-Ratelimit-Remaining: 0
X-Ratelimit-Reset: 1738868431

{"statusCode":429,"error":"Too Many Requests","message":"Global limit has been reached","errorCode":"too_many_requests"}

2025/02/06 11:00:31 
GET /api/v2/users-by-email?email=test%gmail.com HTTP/1.1
Host: melkor-dev.us.auth0.com
User-Agent: Go-Auth0/1.15.0
Auth0-Client: eyJuYW1lIjoiZ28tYXV0aDAiLCJ2ZXJzaW9uIjoiMS4xNS4wIiwiZW52Ijp7ImdvIjoiZ28xLjIyLjQifX0=
Content-Type: application/json
Accept-Encoding: gzip


2025/02/06 11:00:31 
HTTP/2.0 429 Too Many Requests
Content-Length: 120
Access-Control-Expose-Headers: WWW-Authenticate,Server-Authorization
Age: 1
Alt-Svc: h3=":443"; ma=86400
Cache-Control: public, max-age=1
Cf-Cache-Status: HIT
Cf-Ray: 90dd623269d7645e-SJC
Content-Type: application/json; charset=utf-8
Date: Thu, 06 Feb 2025 19:00:31 GMT
Retry-After: 1
Server: cloudflare
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: origin, Accept-Encoding
X-Auth0-L: 0.025
X-Content-Type-Options: nosniff
X-Ratelimit-Limit: 10
X-Ratelimit-Remaining: 0
X-Ratelimit-Reset: 1738868431

{"statusCode":429,"error":"Too Many Requests","message":"Global limit has been reached","errorCode":"too_many_requests"}

2025/02/06 11:00:31 
GET /api/v2/users-by-email?email=test%gmail.com HTTP/1.1
Host: melkor-dev.us.auth0.com
User-Agent: Go-Auth0/1.15.0
Auth0-Client: eyJuYW1lIjoiZ28tYXV0aDAiLCJ2ZXJzaW9uIjoiMS4xNS4wIiwiZW52Ijp7ImdvIjoiZ28xLjIyLjQifX0=
Content-Type: application/json
Accept-Encoding: gzip


2025/02/06 11:00:31 
HTTP/2.0 429 Too Many Requests
Content-Length: 120
Access-Control-Expose-Headers: WWW-Authenticate,Server-Authorization
Age: 1
Alt-Svc: h3=":443"; ma=86400
Cache-Control: public, max-age=1
Cf-Cache-Status: HIT
Cf-Ray: 90dd6232ba12645e-SJC
Content-Type: application/json; charset=utf-8
Date: Thu, 06 Feb 2025 19:00:31 GMT
Retry-After: 1
Server: cloudflare
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: origin, Accept-Encoding
X-Auth0-L: 0.025
X-Content-Type-Options: nosniff
X-Ratelimit-Limit: 10
X-Ratelimit-Remaining: 0
X-Ratelimit-Reset: 1738868431

{"statusCode":429,"error":"Too Many Requests","message":"Global limit has been reached","errorCode":"too_many_requests"}

2025/02/06 11:00:31 
GET /api/v2/users-by-email?email=test%gmail.com HTTP/1.1
Host: melkor-dev.us.auth0.com
User-Agent: Go-Auth0/1.15.0
Auth0-Client: eyJuYW1lIjoiZ28tYXV0aDAiLCJ2ZXJzaW9uIjoiMS4xNS4wIiwiZW52Ijp7ImdvIjoiZ28xLjIyLjQifX0=
Content-Type: application/json
Accept-Encoding: gzip


2025/02/06 11:00:31 
HTTP/2.0 429 Too Many Requests
Content-Length: 120
Access-Control-Expose-Headers: WWW-Authenticate,Server-Authorization
Age: 1
Alt-Svc: h3=":443"; ma=86400
Cache-Control: public, max-age=1
Cf-Cache-Status: HIT
Cf-Ray: 90dd62330a69645e-SJC
Content-Type: application/json; charset=utf-8
Date: Thu, 06 Feb 2025 19:00:31 GMT
Retry-After: 1
Server: cloudflare
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: origin, Accept-Encoding
X-Auth0-L: 0.025
X-Content-Type-Options: nosniff
X-Ratelimit-Limit: 10
X-Ratelimit-Remaining: 0
X-Ratelimit-Reset: 1738868431

{"statusCode":429,"error":"Too Many Requests","message":"Global limit has been reached","errorCode":"too_many_requests"}

2025/02/06 11:00:31 
GET /api/v2/users-by-email?email=test%gmail.com HTTP/1.1
Host: melkor-dev.us.auth0.com
User-Agent: Go-Auth0/1.15.0
Auth0-Client: eyJuYW1lIjoiZ28tYXV0aDAiLCJ2ZXJzaW9uIjoiMS4xNS4wIiwiZW52Ijp7ImdvIjoiZ28xLjIyLjQifX0=
Content-Type: application/json
Accept-Encoding: gzip


2025/02/06 11:00:31 
HTTP/2.0 429 Too Many Requests
Content-Length: 120
Access-Control-Expose-Headers: WWW-Authenticate,Server-Authorization
Age: 1
Alt-Svc: h3=":443"; ma=86400
Cache-Control: public, max-age=1
Cf-Cache-Status: HIT
Cf-Ray: 90dd62336aa3645e-SJC
Content-Type: application/json; charset=utf-8
Date: Thu, 06 Feb 2025 19:00:31 GMT
Retry-After: 1
Server: cloudflare
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: origin, Accept-Encoding
X-Auth0-L: 0.025
X-Content-Type-Options: nosniff
X-Ratelimit-Limit: 10
X-Ratelimit-Remaining: 0
X-Ratelimit-Reset: 1738868431

{"statusCode":429,"error":"Too Many Requests","message":"Global limit has been reached","errorCode":"too_many_requests"}

Expectation

I would expect that 1 retry, 2 maximum would wait long enough to avoid this 429 burst limit of 10 that is specific to my ip address. Currently it seems to try again too soon relative the reset value returned by the header.

Reproduction

I am running something similar to the follow example code (simplified):

	auth0Management, err := a0mgmt.New(
		cfg.Auth0.ManagementDomain,
		a0mgmt.WithClientCredentials(ctx, cfg.Auth0.ClientID, cfg.Auth0.ClientSecret),
		a0mgmt.WithDebug(true),
		a0mgmt.WithRetries(5, []int{http.StatusTooManyRequests}),
	)
	if err != nil {
		log.Fatalf("failed to initialize the auth0 management API client: %+v", err)
	}

	emails := []string{"[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]", "[email protected]"}
	for _, email := range emails {
		users, err := a0.User.ListByEmail(ctx, email)
		if err != nil {
			return &a0mgmt.User{}, err
		}
		fmt.Println(users)
	}

Auth0 Go SDK version

1.16.0