Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Kueue service backend #709

Draft
wants to merge 83 commits into
base: main
Choose a base branch
from
Draft

Conversation

cortadocodes
Copy link
Member

@cortadocodes cortadocodes commented Jan 9, 2025

Contents (#709)

IMPORTANT: There is 1 breaking change.

New features

  • Add octue question ask CLI command
  • Add ability to ask remote question from CLI
  • Add octue question events raw command
  • Add octue question events replay CLI command
  • Allow requesting compute resource amounts per question
  • Add cancel method to Service and Child
  • Add octue question cancel CLI command

Enhancements

  • Deserialise octue question ask inputs from JSON
  • Add async option to octue question ask
  • Deprecate octue run command
  • Support parsing already-extracted questions
  • Remove unnecessary log message
  • Improve description of commands
  • Suppress google crc32c warning
  • Improve description of octue question ask local
  • Move get-diagnostics command to octue question diagnostics
  • Allow passing of attributes to octue question ask local
  • 💥 BREAKING CHANGE: Allow setting of Octue services topic name via envvar
  • Log when question event extraction is complete
  • Log question acknowledgement to pub/sub
  • Allow use of service registries with octue question ask remote
  • Remove cloud run runtime warning
  • Add GOOGLE_KUEUE to compute providers and remove old ones
  • Remove option to parse events from Cloud Run
  • Add log message explaining waiting for question acceptance
  • Simplify question cancellation
  • Avoid trying to cancel question if it's already finished

Fixes

  • Fix incorrect type
  • Return question UUID for async octue question ask
  • Fix event and attributes generation
  • Add missing kwarg
  • Fix argument name and backend format in CLI command
  • Fix --asynchronous option in command
  • Fix setting attributes for question event
  • Return data from octue question ask remote properly
  • Await publishing of result event
  • Return early for async question in octue question ask remote
  • Avoid assuming question is a dict
  • Fix typo in OCTUE_SERVICES_TOPIC_NAME envvar
  • Remove non-existent service revision check for now
  • Add missing elif
  • Convert attributes to correct types in make_attributes
  • Extract attributes correctly from passed-in format
  • Explicitly pass credentials to publisher/subscriber clients
  • Refresh credentials before returning
  • Make service config optional for octue question ask remote
  • Only add resource request attributes if non-None
  • Avoid passing invalid ID to child in octue question cancel
  • Add missing key
  • Remove restriction on attributes for PARENT sender type
  • Fix log message
  • Fix string interpolations in log messages
  • Wait for event emission future by default

Operations

  • Increase minor version number
  • Correct version number
  • Remove terraform config

Dependencies

  • Temporarily remove numpy and pandas dev dependencies
  • Drop support for python 3.8 and 3.9
  • Add numpy and pandas back to dev dependencies
  • Update lock file

Refactoring

  • Split octue question ask into ask and ask-local commands
  • Rename answer_question to answer_pub_sub_question
  • Move ask* subcommands under octue question ask group
  • Rename octue question events raw to octue question events get
  • Remove Cloud Run flask app
  • Move dockerfiles into their own directory
  • Move answer_pub_sub_question module and rename
  • Remove push subscription creation logic
  • Remove unused global variable
  • Remove other mentions of cloud run
  • Remove unused dockerfile and workflow
  • Split pub/sub event/attributes extraction into two functions
  • Move attributes extraction into events subpackage
  • Rename function

Testing

  • Update test for python3.13
  • Update mock paths
  • Fix test for different python versions
  • Move cloud run deployment test

Reversions

  • Revert "FIX: Refresh credentials before returning"
  • Revert "FIX: Explicitly pass credentials to publisher/subscriber clients"

Chores

  • Deprecate octue get-diagnostics CLI command

Style

  • Add missing ruff isort pre-commit check

Other

  • Update event schema locally

Upgrade instructions

💥 Allow setting of Octue services topic name via envvar

Update the services topic in production to be named main.octue.services or set the environment variable OCTUE_SERVICES_TOPIC_NAME=octue.services

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Run questions via CLI without starting a service
1 participant