1.6 KiB
1.6 KiB
1. Spec and documentation
- 1.1 Create OpenSpec proposal, design, tasks, and capability spec for image concurrency isolation.
- 1.2 Add a local
2uenote for the external image gateway deployment pattern and current non-goals.
2. Config
- 2.1 Add
gateway.image_concurrency.enabledandgateway.image_concurrency.max_concurrent_requestsconfig fields. - 2.2 Register defaults that keep existing behavior unchanged.
- 2.3 Validate max concurrent requests as non-negative.
- 2.4 Update
deploy/config.example.yamlwith safe usage notes. - 2.5 Add image concurrency overflow mode, wait timeout, and max waiting request config.
3. Runtime limiter
- 3.1 Implement a process-level image concurrency limiter with resize-on-config-read behavior.
- 3.2 Acquire/release the limiter around
/v1/images/generationsand/v1/images/editsbefore account scheduling. - 3.3 Acquire/release the limiter around explicit
/v1/responsesimage generation intent before account scheduling. - 3.4 Ensure limiter rejections return
429 rate_limit_errorand do not trigger account failover. - 3.5 Support
rejectandwaitoverflow modes with bounded wait timeout and waiting queue size.
4. Tests and verification
- 4.1 Add config default and validation tests.
- 4.2 Add handler tests for image endpoint limiter rejection.
- 4.3 Add handler tests proving text-only Responses requests are not rejected by the image limiter.
- 4.4 Run focused Go tests for config and OpenAI handler/service paths.
- 4.5 Add limiter tests for wait success, wait timeout, and waiting queue overflow.