-
Notifications
You must be signed in to change notification settings - Fork 160
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bedrock support #943
Comments
Hey! Antropic models are interesting and it's definitely an area with a lot of potential use cases. Adding support for them would be great if the team can consider it in future planning. What kind of integration are you envisioning? Would it be as seamless as possible, or do you have specific requirements for how this should be done? Let me know your thoughts!
|
Specifically interested in using models hosted on Bedrock. |
Could you test it? I do not have access to bedrock. |
What's the best way to test it? I created a genaiscript file like this (I'm using the script({
model: "anthropic_bedrock:anthropic.claude-3-haiku-20240307-v1:0",
})
$`Write a short poem.` This is what I'm getting:
|
I think |
Tried using the cjs file in api folder and got the same error:
|
Try
(fix model name) |
Nice! That worked.
|
I'm seeing some errors when I test it with other scripts:
|
I see, we generate system, user messages which become user, user in anthropic. need to collapse them. |
@volkanunsal try the latest in the main branch |
Lgtm! Thanks for the quick turnaround. |
Feel free to raise more issues as they pop up. Mostly relying on the @anthropic/bedrock-sdk package here |
Thank you. One potential area of improvement is allowing builders to
specify a designated credential profile to be used. This way they can use
the default credential profile for their dev work and run the genai scripts
to check their code etc without having to go back and forth between
accounts. I haven’t checked if that exists already…
…On Fri, Dec 13, 2024 at 4:00 PM Peli de Halleux ***@***.***> wrote:
Feel free to raise more issues as they pop up. Mostly relying on the
@anthropic/bedrock-sdk package here
—
Reply to this email directly, view it on GitHub
<#943 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABFAMD25W2WUVNY7NDUNSD2FNDINAVCNFSM6AAAAABTR5Z36GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBSGM3DGOJQGI>
.
You are receiving this because you modified the open/close state.Message
ID: ***@***.***>
|
One thing to try out would be to use the aws SDK in the genai script itself to set things up; then the environment would be ready when the LLM call happens.
Sent from Outlook<http://aka.ms/weboutlook>
________________________________
From: Volkan Unsal ***@***.***>
Sent: Friday, December 13, 2024 1:36 PM
To: microsoft/genaiscript ***@***.***>
Cc: Comment ***@***.***>; Subscribed ***@***.***>
Subject: Re: [microsoft/genaiscript] Bedrock support (Issue #943)
Thank you. One potential area of improvement is allowing builders to
specify a designated credential profile to be used. This way they can use
the default credential profile for their dev work and run the genai scripts
to check their code etc without having to go back and forth between
accounts. I haven’t checked if that exists already…
On Fri, Dec 13, 2024 at 4:00 PM Peli de Halleux ***@***.***> wrote:
Feel free to raise more issues as they pop up. Mostly relying on the
@anthropic/bedrock-sdk package here
—
Reply to this email directly, view it on GitHub
<#943 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABFAMD25W2WUVNY7NDUNSD2FNDINAVCNFSM6AAAAABTR5Z36GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBSGM3DGOJQGI>
.
You are receiving this because you modified the open/close state.Message
ID: ***@***.***>
—
Reply to this email directly, view it on GitHub<#943 (comment)> or unsubscribe<https://github.com/notifications/unsubscribe-auth/AA73QKPXZ6RJ2VWSTDHKGMD2FNHODBFKMF2HI4TJMJ2XIZLTSOBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLLDTOVRGUZLDORPXI6LQMWWES43TOVSUG33NNVSW45FGORXXA2LDOOJIFJDUPFYGLKTSMVYG643JORXXE6NFOZQWY5LFVE3DOOJXHA2DGNRYQKSHI6LQMWSWS43TOVS2K5TBNR2WLKRSG4ZTQMZTHA2TMN5HORZGSZ3HMVZKMY3SMVQXIZI>.
You are receiving this email because you commented on the thread.
Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
I think I see what you mean. Anthropic SDK is using env vars to get the
credential values. Perhaps I can modify them beforehand.
On Fri, Dec 13, 2024 at 4:44 PM Peli de Halleux ***@***.***>
wrote:
… One thing to try out would be to use the aws SDK in the genai script
itself to set things up; then the environment would be ready when the LLM
call happens.
Sent from Outlook<http://aka.ms/weboutlook>
________________________________
From: Volkan Unsal ***@***.***>
Sent: Friday, December 13, 2024 1:36 PM
To: microsoft/genaiscript ***@***.***>
Cc: Comment ***@***.***>; Subscribed ***@***.***>
Subject: Re: [microsoft/genaiscript] Bedrock support (Issue #943)
Thank you. One potential area of improvement is allowing builders to
specify a designated credential profile to be used. This way they can use
the default credential profile for their dev work and run the genai
scripts
to check their code etc without having to go back and forth between
accounts. I haven’t checked if that exists already…
On Fri, Dec 13, 2024 at 4:00 PM Peli de Halleux ***@***.***>
wrote:
> Feel free to raise more issues as they pop up. Mostly relying on the
> @anthropic/bedrock-sdk package here
>
> —
> Reply to this email directly, view it on GitHub
> <
#943 (comment)>,
> or unsubscribe
> <
https://github.com/notifications/unsubscribe-auth/AABFAMD25W2WUVNY7NDUNSD2FNDINAVCNFSM6AAAAABTR5Z36GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBSGM3DGOJQGI>
> .
> You are receiving this because you modified the open/close state.Message
> ID: ***@***.***>
>
—
Reply to this email directly, view it on GitHub<
#943 (comment)>
or unsubscribe<
https://github.com/notifications/unsubscribe-auth/AA73QKPXZ6RJ2VWSTDHKGMD2FNHODBFKMF2HI4TJMJ2XIZLTSOBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLLDTOVRGUZLDORPXI6LQMWWES43TOVSUG33NNVSW45FGORXXA2LDOOJIFJDUPFYGLKTSMVYG643JORXXE6NFOZQWY5LFVE3DOOJXHA2DGNRYQKSHI6LQMWSWS43TOVS2K5TBNR2WLKRSG4ZTQMZTHA2TMN5HORZGSZ3HMVZKMY3SMVQXIZI>.
You are receiving this email because you commented on the thread.
Triage notifications on the go with GitHub Mobile for iOS<
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android<
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
—
Reply to this email directly, view it on GitHub
<#943 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABFAMG7OOCWHB7UID7JH5L2FNIMZAVCNFSM6AAAAABTR5Z36GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBSGQZTANJTGI>
.
You are receiving this because you modified the open/close state.Message
ID: ***@***.***>
|
open to suggestions once you get further in the usage :) |
This worked for me.
To tidy it up, I put the creds in a separate file...
I was also able to make it work by exporting the credentials from |
Are there any plans to add support for OpenAI API compatible models, such as Antopic models, hosted on AWS Bedrock?
The text was updated successfully, but these errors were encountered: