Enhance API Flexibility and Framework Compatibility for BAML

[learnerLj]

Description

After implementing a large language model (LLM) Function Call feature using instructor-ai/instructor as a dependency, I observed some limitations in the flexibility of LLM interpretation. Specifically, when dealing with irregular or exceptional inputs, the robustness of the LLM dropped significantly compared to natural language scenarios. This required me to supplement the LLM with additional mechanisms, such as string similarity checks or correction dictionaries, to filter and preprocess inputs effectively.

This experience highlighted the value of the work done by BoundaryML/BAML. However, I found BAML lacking in the following areas:

1. Limited API Control: The lack of APIs restricts fine-grained control, making it challenging to adapt BAML for specific requirements.
2. Framework Incompatibility: BAML does not support custom frameworks for complete, agent, or dialogue management, limiting its flexibility for tailored use cases.

Suggestions

To enhance BAML’s usability and adoption, I suggest the following improvements:

1. Provide a Comprehensive API: Introduce APIs to enable precise and fine-grained control over the core functionalities. This would make it easier to integrate BAML into custom workflows and frameworks.
2. Improve Framework Compatibility: Enhance BAML to support custom implementations of complete, agent, and dialogue management. This will allow developers to adapt BAML to their unique environments and requirements.

Motivation

With these improvements, BAML could serve as a more robust and flexible solution for managing LLM interactions, especially in scenarios that require handling edge cases or integrating with custom frameworks. I would be eager to try BAML again if these enhancements are implemented.

References

• Instructor-AI/Instructor
• BoundaryML/BAML

Thank you for considering these suggestions!

[rossaai]

We'd like to use specific components of BAML - specifically the Prompt Declaration (.baml files - no clients) and SAP Parsing features - to integrate into our existing pipeline. We don't need the full client implementation since we already have one. Could you help us:

1. Construct agent prompts by passing arguments
2. Access the parsed output

If there's a way to use these features independently with OpenAI's API or other APIs, please let us know. We already have a client implementation and only need these specific components.

[hellovai]

Hi Folks, we are definitely interested in doing this!
sadly this is a much harder than it should be for you all.

We are working on a solution for this in conjunction for exposing just the parser.

However, for now there is 1 workaround you can have:
use a proxy server to capture all the requests.
See: https://github.com/BoundaryML/baml-examples-proxy

Basically you create a in process python proxy server that just handles the openai format prompts and does whatever with it

ideally we should just provide:

prompt = b.prompt.FunctionName(...)

and a:

parser = b.parse.FunctionName(some_string)

But this is likely not going to be able to land until 2025 Q2 since we have a huge backlog!

That said some examples of doing things like dialogue management are here:
https://docs.boundaryml.com/examples/prompt-engineering/chat

Additionally you may find client registry useful:
https://docs.boundaryml.com/guide/baml-advanced/llm-client-registry