Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for model aliases in script function #1164

Merged
merged 1 commit into from
Feb 21, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions docs/src/content/docs/reference/scripts/model-aliases.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
```

Model aliases can be defined as environment varialbles (through the `.env` file),
in a configuration file or through the [cli](/genaiscript/reference/cli/run).
in a configuration file, through the [cli](/genaiscript/reference/cli/run) or in the `script` function.

Check warning on line 27 in docs/src/content/docs/reference/scripts/model-aliases.mdx

View workflow job for this annotation

GitHub Actions / build

Typo in "environment varialbles" should be "environment variables".

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo in "environment varialbles" should be "environment variables".

AI-generated content by pr-docs-review-commit model_alias_typo may be incorrect


This `.env` file defines a `llama32` alias for the `ollama:llama3.2:1b` model.

Expand Down Expand Up @@ -61,6 +61,17 @@
genaiscript run --model-alias llama32=ollama:llama3.2:1b
```

- in the `script`function

```js
script({
model: "llama32",
modelAliases: {
llama32: "ollama:llama3.2:1b",
},
})
```

## Alias of aliases

An model alias can reference another alias as long as cycles are not created.
Expand Down Expand Up @@ -95,4 +106,3 @@
```sh
genaiscript run --provider anthropic
```

7 changes: 5 additions & 2 deletions docs/src/content/docs/reference/scripts/system.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -471,7 +471,9 @@
"web",
"search the web to accomplish tasks.",
`Your are a helpful LLM agent that can use web search.
Answer the question in <QUERY>.`,
Search the web and answer the question in <QUERY>.
- Expand <QUERY> into an optimized search query for better results.
- Answer exclusively with live information from the web.`,

Check failure on line 476 in docs/src/content/docs/reference/scripts/system.mdx

View workflow job for this annotation

GitHub Actions / build

Error message should include a link to the web search documentation for better user guidance.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Error message should include a link to the web search documentation for better user guidance.

AI-generated content by pr-docs-review-commit web_search_error_message may be incorrect

{
system: [
"system.safety_jailbreak",
Expand Down Expand Up @@ -3060,7 +3062,8 @@
count,
ignoreMissingProvider: true,
})
if (!webPages) return "error: no web search provider configured"
if (!webPages)
return "error: no web search provider configured (https://microsoft.github.io/genaiscript/reference/scripts/web-search/)"
return YAML.stringify(
webPages.map((f) => ({
url: f.filename,
Expand Down
8 changes: 8 additions & 0 deletions packages/cli/src/modelalias.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,14 @@
}
}

export function applyScriptModelAliases(script: PromptScript) {
applyModelOptions(script, "script")
if (script.modelAliases)
Object.entries(script.modelAliases).forEach(([name, alias]) => {
runtimeHost.setModelAlias("script", name, alias)
})
}

Check failure on line 46 in packages/cli/src/modelalias.ts

View workflow job for this annotation

GitHub Actions / build

The function `applyScriptModelAliases` is not used anywhere in the codebase and can be removed to clean up the code.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function applyScriptModelAliases is not used anywhere in the codebase and can be removed to clean up the code.

AI-generated content by pr-review-commit unnecessary_function may be incorrect


export function logModelAliases() {
const modelAlias = runtimeHost.modelAliases
if (Object.values(modelAlias).some((m) => m.source !== "default"))
Expand Down
8 changes: 6 additions & 2 deletions packages/cli/src/run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,11 @@
stdout,
} from "../../core/src/logging"
import { ensureDotGenaiscriptPath, setupTraceWriting } from "./trace"
import { applyModelOptions, logModelAliases } from "./modelalias"
import {
applyModelOptions,
applyScriptModelAliases,
logModelAliases,
} from "./modelalias"
import { createCancellationController } from "./cancel"
import { parsePromptScriptMeta } from "../../core/src/template"
import { Fragment } from "../../core/src/generation"
Expand Down Expand Up @@ -351,7 +355,7 @@
const stats = new GenerationStats("")
try {
if (options.label) trace.heading(2, options.label)
applyModelOptions(script, "script")
applyScriptModelAliases(script)

Check failure on line 358 in packages/cli/src/run.ts

View workflow job for this annotation

GitHub Actions / build

The function call `applyScriptModelAliases(script)` is unnecessary as it's already called in the previous line. This can be simplified by removing one of the calls to avoid redundancy.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function call applyScriptModelAliases(script) is unnecessary as it's already called in the previous line. This can be simplified by removing one of the calls to avoid redundancy.

AI-generated content by pr-review-commit unnecessary_function_call may be incorrect

logModelAliases()
const { info } = await resolveModelConnectionInfo(script, {
trace,
Expand Down
5 changes: 5 additions & 0 deletions packages/core/src/types/prompt_template.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -270,10 +270,15 @@
smallModel?: ModelSmallType

/**
* Configure the `vision` model alias.
*/
visionModel?: ModelVisionType

/**
* A list of model aliases to use.
*/
modelAliases?: Record<string, string>
}

Check failure on line 281 in packages/core/src/types/prompt_template.d.ts

View workflow job for this annotation

GitHub Actions / build

The field `modelAliases` in the interface `ModelAliasesOptions` is defined but not used anywhere in the codebase and can be removed to clean up the code.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The field modelAliases in the interface ModelAliasesOptions is defined but not used anywhere in the codebase and can be removed to clean up the code.

AI-generated content by pr-review-commit unused_interface_field may be incorrect


interface ModelOptions extends ModelConnectionOptions, ModelTemplateOptions {
/**
Expand Down
5 changes: 5 additions & 0 deletions packages/sample/genaisrc/model-alias.genai.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
script({
model: "foo",
modelAliases: { foo: "small" },
})
$`Write a poem.`
Loading