Skip to content

Commit

Permalink
✨: Add support for model aliases in script function (#1164)
Browse files Browse the repository at this point in the history
Model aliases can now be defined directly in the script function.
  • Loading branch information
pelikhan authored Feb 21, 2025
1 parent 27bc386 commit 28868bc
Show file tree
Hide file tree
Showing 6 changed files with 41 additions and 6 deletions.
14 changes: 12 additions & 2 deletions docs/src/content/docs/reference/scripts/model-aliases.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ script({
```

Model aliases can be defined as environment varialbles (through the `.env` file),
in a configuration file or through the [cli](/genaiscript/reference/cli/run).
in a configuration file, through the [cli](/genaiscript/reference/cli/run) or in the `script` function.

This `.env` file defines a `llama32` alias for the `ollama:llama3.2:1b` model.

Expand Down Expand Up @@ -61,6 +61,17 @@ The following configuration are support in order importance (last one wins):
genaiscript run --model-alias llama32=ollama:llama3.2:1b
```

- in the `script`function

```js
script({
model: "llama32",
modelAliases: {
llama32: "ollama:llama3.2:1b",
},
})
```

## Alias of aliases

An model alias can reference another alias as long as cycles are not created.
Expand Down Expand Up @@ -95,4 +106,3 @@ The default aliases for a given provider can be loaded using the `provider` opti
```sh
genaiscript run --provider anthropic
```

7 changes: 5 additions & 2 deletions docs/src/content/docs/reference/scripts/system.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -471,7 +471,9 @@ defAgent(
"web",
"search the web to accomplish tasks.",
`Your are a helpful LLM agent that can use web search.
Answer the question in <QUERY>.`,
Search the web and answer the question in <QUERY>.
- Expand <QUERY> into an optimized search query for better results.
- Answer exclusively with live information from the web.`,
{
system: [
"system.safety_jailbreak",
Expand Down Expand Up @@ -3060,7 +3062,8 @@ defTool(
count,
ignoreMissingProvider: true,
})
if (!webPages) return "error: no web search provider configured"
if (!webPages)
return "error: no web search provider configured (https://microsoft.github.io/genaiscript/reference/scripts/web-search/)"
return YAML.stringify(
webPages.map((f) => ({
url: f.filename,
Expand Down
8 changes: 8 additions & 0 deletions packages/cli/src/modelalias.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,14 @@ export function applyModelOptions(
}
}

export function applyScriptModelAliases(script: PromptScript) {
applyModelOptions(script, "script")
if (script.modelAliases)
Object.entries(script.modelAliases).forEach(([name, alias]) => {
runtimeHost.setModelAlias("script", name, alias)
})
}

export function logModelAliases() {
const modelAlias = runtimeHost.modelAliases
if (Object.values(modelAlias).some((m) => m.source !== "default"))
Expand Down
8 changes: 6 additions & 2 deletions packages/cli/src/run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,11 @@ import {
stdout,
} from "../../core/src/logging"
import { ensureDotGenaiscriptPath, setupTraceWriting } from "./trace"
import { applyModelOptions, logModelAliases } from "./modelalias"
import {
applyModelOptions,
applyScriptModelAliases,
logModelAliases,
} from "./modelalias"
import { createCancellationController } from "./cancel"
import { parsePromptScriptMeta } from "../../core/src/template"
import { Fragment } from "../../core/src/generation"
Expand Down Expand Up @@ -351,7 +355,7 @@ export async function runScriptInternal(
const stats = new GenerationStats("")
try {
if (options.label) trace.heading(2, options.label)
applyModelOptions(script, "script")
applyScriptModelAliases(script)
logModelAliases()
const { info } = await resolveModelConnectionInfo(script, {
trace,
Expand Down
5 changes: 5 additions & 0 deletions packages/core/src/types/prompt_template.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,11 @@ interface ModelAliasesOptions {
* Configure the `vision` model alias.
*/
visionModel?: ModelVisionType

/**
* A list of model aliases to use.
*/
modelAliases?: Record<string, string>
}

interface ModelOptions extends ModelConnectionOptions, ModelTemplateOptions {
Expand Down
5 changes: 5 additions & 0 deletions packages/sample/genaisrc/model-alias.genai.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
script({
model: "foo",
modelAliases: { foo: "small" },
})
$`Write a poem.`

0 comments on commit 28868bc

Please sign in to comment.