Skip to content

Commit

Permalink
Merge pull request #208 from haesleinhuepf/azure
Browse files Browse the repository at this point in the history
Support for Github models and Azure
  • Loading branch information
haesleinhuepf authored Sep 21, 2024
2 parents b7c810a + 6c3356e commit e3b1e64
Show file tree
Hide file tree
Showing 11 changed files with 843 additions and 38 deletions.
34 changes: 32 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
[![License](https://img.shields.io/pypi/l/bia-bob.svg?color=green)](https://github.com/haesleinhuepf/bia-bob/raw/main/LICENSE)

BIA `bob` is a Jupyter-based assistant for interacting with data using large language models which generate Python code for Bio-Image Analysis (BIA).
It can make use of [OpenAI's chatGPT](https://openai.com/blog/openai-api), [Google's Gemini](https://blog.google/technology/ai/google-gemini-ai/), [Anthropic's Claude](https://www.anthropic.com/api), [Helmholtz' blablador](https://helmholtz-blablador.fz-juelich.de/) and [Ollama](https://ollama.com).
It can make use of [OpenAI's chatGPT](https://openai.com/blog/openai-api), [Google's Gemini](https://blog.google/technology/ai/google-gemini-ai/), [Anthropic's Claude](https://www.anthropic.com/api), [Github Models Marketplace](https://github.com/marketplace/models), [Helmholtz' blablador](https://helmholtz-blablador.fz-juelich.de/) and [Ollama](https://ollama.com).
You need an OpenAI API account or a Google Cloud account or a Helmholtz ID account to use it.
Using it with Ollama is free but requires running an Ollama server locally.

Expand All @@ -25,7 +25,7 @@ Users are responsible to verify the generated code according to good scientific
* If you are not sure if an image analysis workflow is valid, consider asking human experts. E.g. reach out via https://image.sc .

> [!CAUTION]
> When using the OpenAI, Google Gemini, Anthropic or any other endpoint via BiA-Bob, you are bound to the terms of service
> When using the OpenAI, Google Gemini, Anthropic, Github Models or any other endpoint via BiA-Bob, you are bound to the terms of service
> of the respective companies or organizations.
> The prompts you enter are transferred to their servers and may be processed and stored there.
> Make sure to not submit any sensitive, confidential or personal data. Also using these services may cost money.
Expand Down Expand Up @@ -159,6 +159,36 @@ Create an OpenAI API Key and add it to your environment variables named `OPENAI_

Create an Anthropic API Key and add it to your environment variables named `ANTHROPIC_API_KEY`.

### Using Github Models Marketplace

If you are using the models from [Github Models Marketplace](https://github.com/marketplace/models), please store your GITHUB API key for accessing the models in an environment variable named `GH_MODELS_API_KEY`.

You can then access the models like this:
```
bob.initialize(
endpoint='github_models',
model='Phi-3.5-mini-instruct')
```

### Using Azure

If you are using the models hosted on [Microsoft Azure](https://azure.microsoft.com/), please store your API key for accessing the models in an environment variable named `AZURE_API_KEY`.

You can then access the models like this:
```
bob.initialize(
endpoint='azure',
model='Phi-3.5-mini-instruct')
```

Alternatively, you can specify the endpoint directly, too:
```
bob.initialize(
endpoint='https://models.inference.ai.azure.com',
model='Phi-3.5-mini-instruct')
```


### Using custom endpoints

Custom endpoints can be used as well if they support the OpenAI API. Examples are [blablador](https://login.helmholtz.de/oauth2-as/oauth2-authz-web-entry) and [ollama](https://ollama.com/).
Expand Down
176 changes: 176 additions & 0 deletions demo/azure.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "b95f8f8a-c13f-4002-a969-12e33bc7797a",
"metadata": {},
"source": [
"# Azure\n",
"You can also access models on [Microsoft Azure](https://azure.microsoft.com/).\n",
"You need to sign up at the platform and store API key to your environment as `AZURE_API_KEY`.\n",
"You can then access models like shown below."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "d5779abe-11fb-4b7e-928e-061a9a6b23cb",
"metadata": {},
"outputs": [],
"source": [
"from bia_bob import bob"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "a2f361fa-ab46-458b-aebe-9be46cc66c76",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" <div style=\"font-size:7pt\">\n",
" This notebook may contain text, code and images generated by artificial intelligence.\n",
" Used model: meta-llama-3.1-405b-instruct, vision model: None, endpoint: https://models.inference.ai.azure.com, bia-bob version: 0.23.0.. Do not enter sensitive or private information and verify generated contents according to good scientific practice. Read more: <a href=\"https://github.com/haesleinhuepf/bia-bob#disclaimer\">https://github.com/haesleinhuepf/bia-bob#disclaimer</a>\n",
" </div>\n",
" "
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"bob.initialize(endpoint=\"azure\", model=\"meta-llama-3.1-405b-instruct\")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "6b3e4eb3-571a-4c1e-b1dc-bb3c2ea32523",
"metadata": {},
"outputs": [],
"source": [
"%bob Please load blobs.tif and print out its dimensions."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e6cb3b4a-33cd-44f3-afb4-5a6d4d667ccf",
"metadata": {},
"outputs": [],
"source": [
"import skimage\n",
"\n",
"# Load the image\n",
"image = skimage.io.imread(\"blobs.tif\")\n",
"\n",
"# Print out the dimensions of the image\n",
"print(image.shape)"
]
},
{
"cell_type": "markdown",
"id": "4707a5d0-8ed0-40b7-b1aa-ef9937929c01",
"metadata": {},
"source": [
"Alternatively, you can also specify the endpoint yourself:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "f76239eb-c3b7-4b69-8eaf-bdc6fb198c16",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" <div style=\"font-size:7pt\">\n",
" This notebook may contain text, code and images generated by artificial intelligence.\n",
" Used model: Phi-3-medium-128k-instruct, vision model: None, endpoint: https://models.inference.ai.azure.com, bia-bob version: 0.23.0.. Do not enter sensitive or private information and verify generated contents according to good scientific practice. Read more: <a href=\"https://github.com/haesleinhuepf/bia-bob#disclaimer\">https://github.com/haesleinhuepf/bia-bob#disclaimer</a>\n",
" </div>\n",
" "
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import os\n",
"\n",
"bob.initialize(endpoint=\"https://models.inference.ai.azure.com\", \n",
" api_key=os.environ.get('AZURE_API_KEY'),\n",
" model=\"Phi-3-medium-128k-instruct\")"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "48c7250a-3b22-4d3e-b0f8-df498db1e8ce",
"metadata": {},
"outputs": [],
"source": [
"%bob Please load blobs.tif and print out its dimensions."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "527de356-1250-42ce-abfc-a1b5385d7f06",
"metadata": {},
"outputs": [],
"source": [
"from PIL import Image\n",
"\n",
"# Load the TIFF image\n",
"with Image.open('blobs.tif') as img:\n",
" # Get the dimensions\n",
" width, height = img.size\n",
" \n",
" # Print the dimensions\n",
" print(f\"Image dimensions: {width} x {height}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "10095188-b106-4fab-91a6-e5c32c0a55ab",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading

0 comments on commit e3b1e64

Please sign in to comment.