What happened?

I'm running the vectorizer on a new large dataset and get this error from openai:

unexpected error: Error code: 400 - {'error': {'message': 'Requested 629204 tokens, max 600000 tokens per request', 'type': 'max_tokens_per_request', 'param': None, 'code': 'max_tokens_per_request'}}

I wonder if that's a configuration error?

pgai extension affected

0.7.0

pgai library affected

No response

PostgreSQL version used

17

What operating system did you use?

latest timescale/timescaledb-ha:pg17

What installation method did you use?

Docker

What platform did you run on?

On prem/Self-hosted

Relevant log output and stack trace

How can we reproduce the bug?

1. Have a large amount of data
2. Create a vectorizer
3. Run the worker

(I can't really pin it down, will try to see if I can reproduce it better)

Are you going to work on the bugfix?

None