Skip to content

Commit

Permalink
chore: formatting
Browse files Browse the repository at this point in the history
  • Loading branch information
@holtskinner
holtskinner committed Feb 20, 2025
1 parent 92c7b22 commit f62736b
Show file tree
Hide file tree
Showing 8 changed files with 3,549 additions and 3,585 deletions.
1,947 changes: 964 additions & 983 deletions gemini/evaluation/evaluate_autorater.ipynb
View file

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion gemini/getting-started/intro_genai_sdk.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1207,7 +1207,7 @@
"source": [
"import pandas as pd\n",
"\n",
"prediction_path = ! gsutil ls {BUCKET_URI}/*/predictions.jsonl \n",
"prediction_path = ! gsutil ls {BUCKET_URI}/*/predictions.jsonl\n",
"print(prediction_path)\n",
"\n",
"# Load the JSONL file into a DataFrame\n",
Expand Down
113 changes: 57 additions & 56 deletions gemini/prompts/prompt_optimizer/vertex_ai_prompt_optimizer_sdk_multimodal_image.ipynb
View file
Original file line number Diff line number Diff line change
@@ -1,19 +1,4 @@
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"toc_visible": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "code",
Expand Down Expand Up @@ -122,6 +107,9 @@
},
{
"cell_type": "markdown",
"metadata": {
"id": "_8BqRNCswb0A"
},
"source": [
"## Objective\n",
"\n",
Expand All @@ -142,10 +130,7 @@
"Run the automatic prompt optimization job.\n",
"Collect the best prompt template and evaluation metric.\n",
"Validate the best prompt template."
],
"metadata": {
"id": "_8BqRNCswb0A"
}
]
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -204,6 +189,9 @@
},
{
"cell_type": "markdown",
"metadata": {
"id": "yXDewg2-zzud"
},
"source": [
"## Costs\n",
"\n",
Expand All @@ -213,10 +201,7 @@
"- Cloud Storage\n",
"\n",
"Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage pricing](https://cloud.google.com/storage/pricing) and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
],
"metadata": {
"id": "yXDewg2-zzud"
}
]
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -250,14 +235,14 @@
},
{
"cell_type": "code",
"source": [
"! mkdir -p ./tutorial/utils && wget https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/prompts/prompt_optimizer/vapo_lib.py -P ./tutorial/utils"
],
"execution_count": null,
"metadata": {
"id": "jKn-zl1Y0NDA"
},
"execution_count": null,
"outputs": []
"outputs": [],
"source": [
"! mkdir -p ./tutorial/utils && wget https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/prompts/prompt_optimizer/vapo_lib.py -P ./tutorial/utils"
]
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -350,7 +335,9 @@
"\n",
"import vertexai\n",
"\n",
"PROJECT_ID = \"\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
"PROJECT_ID = (\n",
" \"\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
")\n",
"if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
" PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
"\n",
Expand Down Expand Up @@ -412,15 +399,15 @@
},
{
"cell_type": "code",
"source": [
"PROJECT_NUMBER = !gcloud projects describe {PROJECT_ID} --format=\"get(projectNumber)\"[0]\n",
"PROJECT_NUMBER = PROJECT_NUMBER[0]"
],
"execution_count": null,
"metadata": {
"id": "4VZ6ADo0vsQe"
},
"execution_count": null,
"outputs": []
"outputs": [],
"source": [
"PROJECT_NUMBER = !gcloud projects describe {PROJECT_ID} --format=\"get(projectNumber)\"[0]\n",
"PROJECT_NUMBER = PROJECT_NUMBER[0]"
]
},
{
"cell_type": "code",
Expand Down Expand Up @@ -525,8 +512,8 @@
"\n",
"# General\n",
"import logging\n",
"import warnings\n",
"from typing import Final\n",
"import warnings\n",
"\n",
"from IPython.display import HTML, display\n",
"from google.cloud import aiplatform\n",
Expand Down Expand Up @@ -577,8 +564,12 @@
},
"outputs": [],
"source": [
"INPUT_DATA_FILE_URI = \"gs://github-repo/prompts/prompt_optimizer/mathvista_dataset/mathvista_input.jsonl\"\n",
"INPUT_DATA_IMAGES_URI = \"gs://github-repo/prompts/prompt_optimizer/mathvista_dataset/images\"\n",
"INPUT_DATA_FILE_URI = (\n",
" \"gs://github-repo/prompts/prompt_optimizer/mathvista_dataset/mathvista_input.jsonl\"\n",
")\n",
"INPUT_DATA_IMAGES_URI = (\n",
" \"gs://github-repo/prompts/prompt_optimizer/mathvista_dataset/images\"\n",
")\n",
"\n",
"EXPERIMENT_NAME = \"mathvista-multimodal-prompt-eval\"\n",
"INPUT_OPTIMIZATION_DATA_URI = epath.Path(WORKSPACE_URI) / \"prompt_optimization_data\"\n",
Expand Down Expand Up @@ -734,6 +725,11 @@
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "zA0wuXZRoPT0"
},
"outputs": [],
"source": [
"SYSTEM_INSTRUCTION_TEMPLATE = \"\"\"\n",
"Solve the problem given the image.\n",
Expand All @@ -744,21 +740,16 @@
"Image: {{image}} @@@image/jpeg\n",
"Answer: {{target}}\n",
"\"\"\""
],
"metadata": {
"id": "zA0wuXZRoPT0"
},
"execution_count": null,
"outputs": []
]
},
{
"cell_type": "markdown",
"source": [
"#### To evaluate the prompt template, follow the method outlined in other notebooks such as [gemini/evaluation/evaluate_multimodal_task_image.ipynb](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/evaluation/evaluate_multimodal_task_image.ipynb)"
],
"metadata": {
"id": "a-FR6GF5lFrh"
}
},
"source": [
"#### To evaluate the prompt template, follow the method outlined in other notebooks such as [gemini/evaluation/evaluate_multimodal_task_image.ipynb](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/evaluation/evaluate_multimodal_task_image.ipynb)"
]
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -1016,8 +1007,6 @@
},
"outputs": [],
"source": [
"from tqdm.asyncio import tqdm_asyncio\n",
"\n",
"gemini_llm = vapo_lib.init_new_model(model_name=\"gemini-1.5-flash-001\")\n",
"\n",
"gemini_predictions = [\n",
Expand Down Expand Up @@ -1061,12 +1050,12 @@
},
{
"cell_type": "markdown",
"source": [
"#### To evaluate the prompt template, follow the method outlined in other notebooks such as [gemini/evaluation/evaluate_multimodal_task_image.ipynb](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/evaluation/evaluate_multimodal_task_image.ipynb)"
],
"metadata": {
"id": "UZY77CxTlgd9"
}
},
"source": [
"#### To evaluate the prompt template, follow the method outlined in other notebooks such as [gemini/evaluation/evaluate_multimodal_task_image.ipynb](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/evaluation/evaluate_multimodal_task_image.ipynb)"
]
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -1106,5 +1095,17 @@
" shutil.rmtree(str(TUTORIAL_PATH))"
]
}
]
}
],
"metadata": {
"colab": {
"name": "vertex_ai_prompt_optimizer_sdk_multimodal_image.ipynb",
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Loading

0 comments on commit f62736b

Please sign in to comment.