What is the Model Context Management Strategy in AutoGen

[prasenmukh]

Curious to know how AutoGen manages context length for longer conversation. Few questions:

1. Does it send all previous questions and API responses along with every request to LLM.
2. I understand that we can limit the number to last N.
3. For all previous N calls does it use both : prompt and response?
4. Is there a way to apply any Summarization strategy with every AssistantAgent call ?

[ekzhu]

Model context is managed using ChatCompletionContext as part of the AssistantAgent: https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.agents.html#module-autogen_agentchat.agents

You can use BufferedChatCompletionContext to limit the number of messages.

We currently don't have any summarization for context management. We would love to see community contribution on this front. New model context classes can be added to autogen_ext.model_context submodule with external dependencies.