Code

Add simulator.#2703

Code

Merged

nagkumar91 merged 113 commits intomainmicrosoft/promptflow:mainfrom task/addSimulatormicrosoft/promptflow:task/addSimulatorCopy head branch name to clipboardMay 10, 2024

+2,934-620

Commits

Commits on Apr 8, 2024

• Adding simulator templates

  nagkumar91

  committed
  f88b109

  Copy full SHA for f88b109

• Adding synthetic simulator

  nagkumar91

  committed
  432d532

  Copy full SHA for 432d532

• Add tests and dependencies

  nagkumar91

  committed
  1d5697e

  Copy full SHA for 1d5697e

• fix version of aiohttp_retry

  nagkumar91

  committed
  33b6bc8

  Copy full SHA for 33b6bc8

• Install before tests

  nagkumar91

  committed
  1db2ebd

  Copy full SHA for 1db2ebd

• Resolve compliance check

  nagkumar91

  committed
  758de41

  Copy full SHA for 758de41

• Syntax error on the yml

  nagkumar91

  committed
  2ae3b2a

  Copy full SHA for 2ae3b2a

• Perform dev setup on unit test setup stage

  nagkumar91

  committed
  0e9693c

  Copy full SHA for 0e9693c

• Change the setup

  nagkumar91

  committed
  4358014

  Copy full SHA for 4358014

• revert the test file and add install of azure.keyvault

  nagkumar91

  committed
  f386152

  Copy full SHA for f386152

• Changing the install again

  nagkumar91

  committed
  0952488

  Copy full SHA for 0952488

Commits on Apr 10, 2024

• simulator takes on ml_client as an arg

  nagkumar91

  committed
  1452643

  Copy full SHA for 1452643

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  13a0689

  Copy full SHA for 13a0689

• Remove reference to ai client

  nagkumar91

  committed
  9ce7b38

  Copy full SHA for 9ce7b38

• Remove unnecessary method

  nagkumar91

  committed
  d5c6e0b

  Copy full SHA for d5c6e0b

Commits on Apr 11, 2024

• removed unused code

  nagkumar91

  committed
  e39197a

  Copy full SHA for e39197a

• Keyword args for public facing simulaor methods

  nagkumar91

  committed
  8799016

  Copy full SHA for 8799016

• removed dependency

  nagkumar91

  committed
  92cdb2c

  Copy full SHA for 92cdb2c

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  8db2749

  Copy full SHA for 8db2749

• making simulator_templates private

  nagkumar91

  committed
  3160da6

  Copy full SHA for 3160da6

Commits on Apr 12, 2024

• Move tests to right folder

  nagkumar91

  committed
  922accf

  Copy full SHA for 922accf

• updated semaphore limit and load_flow

  nagkumar91

  committed
  9ad30af

  Copy full SHA for 9ad30af

Commits on Apr 13, 2024

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  f7288d5

  Copy full SHA for f7288d5

Commits on Apr 15, 2024

• Remove pdb

  nagkumar91

  committed
  89d1aec

  Copy full SHA for 89d1aec

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  b58e70b

  Copy full SHA for b58e70b

Commits on Apr 16, 2024

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  62ea4c5

  Copy full SHA for 62ea4c5

Commits on Apr 17, 2024

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  02334be

  Copy full SHA for 02334be

Commits on Apr 18, 2024

• Add e2e test

  nagkumar91

  committed
  ab6a3cc

  Copy full SHA for ab6a3cc

• More e2e tests

  nagkumar91

  committed
  ea5d5f3

  Copy full SHA for ea5d5f3

• Integrate tests to evals folder

  nagkumar91

  committed
  5205979

  Copy full SHA for 5205979

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  da941dd

  Copy full SHA for da941dd

• ignore file in cfg and add unknown word

  nagkumar91

  committed
  e22ceeb

  Copy full SHA for e22ceeb

• Add recording for new test

  nagkumar91

  committed
  fa8d1f1

  Copy full SHA for fa8d1f1

• Skip e2e test having DefaultAzureCredential

  nagkumar91

  committed
  d56b607

  Copy full SHA for d56b607

• Fix failing test

  nagkumar91

  committed
  3197fed

  Copy full SHA for 3197fed

• Update the recordings

  nagkumar91

  committed
  58c7a12

  Copy full SHA for 58c7a12

• Update recording

  nagkumar91

  committed
  6447d57

  Copy full SHA for 6447d57

Commits on Apr 19, 2024

• Marking test as skipped to mitigate 429s in recording

  nagkumar91

  committed
  c87e4d0

  Copy full SHA for c87e4d0

• Fix cspell issues

  nagkumar91

  committed
  afb10af

  Copy full SHA for afb10af

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  83c73b7

  Copy full SHA for 83c73b7

• Fix the formatting of docstring

  nagkumar91

  committed
  af5b3f0

  Copy full SHA for af5b3f0

Commits on Apr 22, 2024

• Update the non-adv simulator to use a class to initialize a userbot connection

  nagkumar91

  committed
  3b41ab4

  Copy full SHA for 3b41ab4

Commits on Apr 23, 2024

• Merge accepting remote changes for data file

  nagkumar91

  committed
  10606f8

  Copy full SHA for 10606f8

• Fixed the unittest

  nagkumar91

  committed
  166b546

  Copy full SHA for 166b546

• Fixed e2e tests

  nagkumar91

  committed
  ab451f5

  Copy full SHA for ab451f5

• Skip the failing test

  nagkumar91

  committed
  d99cb39

  Copy full SHA for d99cb39

• Skip test on python 3.9 or lower

  nagkumar91

  committed
  746aa0a

  Copy full SHA for 746aa0a

Commits on Apr 26, 2024

• Initial rewrite begin

  nagkumar91

  committed
  a676092

  Copy full SHA for a676092

• Raise error when incorrect template is being passed

  nagkumar91

  committed
  d2c2ca0

  Copy full SHA for d2c2ca0

• Add e2e test

  nagkumar91

  committed
  897f840

  Copy full SHA for 897f840

• e2e test with and without rai_svc_url

  nagkumar91

  committed
  1f243e4

  Copy full SHA for 1f243e4

Commits on Apr 27, 2024

• State with circular import issue

  nagkumar91

  committed
  51beacb

  Copy full SHA for 51beacb

• e2e test passing for qa

  nagkumar91

  committed
  5f606b7

  Copy full SHA for 5f606b7

• Remove old simulator pieces

  nagkumar91

  committed
  f6ccd8f

  Copy full SHA for f6ccd8f

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  ed24705

  Copy full SHA for ed24705

Commits on Apr 29, 2024

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  8675fbc

  Copy full SHA for 8675fbc

• removed unused test

  nagkumar91

  committed
  1f50ba9

  Copy full SHA for 1f50ba9

• Fix time taken and remove duplicate class

  nagkumar91

  committed
  05fbf29

  Copy full SHA for 05fbf29

• Added unitests for conversation bot

  nagkumar91

  committed
  4cc7c42

  Copy full SHA for 4cc7c42

• Trying a fix for python 3.8 and 3.9

  nagkumar91

  committed
  7de92b6

  Copy full SHA for 7de92b6

• Added more tests for callback conversation bot

  nagkumar91

  committed
  7393ba8

  Copy full SHA for 7393ba8

• Lock in the identity manager

  nagkumar91

  committed
  39d1e76

  Copy full SHA for 39d1e76

• Remove print

  nagkumar91

  committed
  b52c81a

  Copy full SHA for b52c81a

• Making calls to proxy completion models work better

  nagkumar91

  committed
  3770661

  Copy full SHA for 3770661

• remove sending 2 messages with file content

  nagkumar91

  committed
  d5292da

  Copy full SHA for d5292da

• Removed all the old qa things

  nagkumar91

  committed
  bee5c6d

  Copy full SHA for bee5c6d

Commits on Apr 30, 2024

• Add docstring

  nagkumar91

  committed
  710f48c

  Copy full SHA for 710f48c

• Adding e2e test for adv_conversation

  nagkumar91

  committed
  340ac6d

  Copy full SHA for 340ac6d

Commits on May 1, 2024

• More tests and add monitoring to adversarial simulator

  nagkumar91

  committed
  dd58d8d

  Copy full SHA for dd58d8d

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  985a7cc

  Copy full SHA for 985a7cc

• Update test_adv_simulator.py

  nagkumar91

  authored
  a5b8324

  Copy full SHA for a5b8324

Commits on May 2, 2024

• more test changes and remove strict check for template string

  nagkumar91

  committed
  6eb9f72

  Copy full SHA for 6eb9f72

• Fix test and remove unused captioning code

  nagkumar91

  committed
  0472f91

  Copy full SHA for 0472f91

• CHange AzureCLICredential to DefaultAzureCredential

  nagkumar91

  committed
  4b428e3

  Copy full SHA for 4b428e3

Commits on May 3, 2024

• Add readme

  nagkumar91

  committed
  292c76c

  Copy full SHA for 292c76c

• Bug bash instrcutions

  nagkumar91

  committed
  ab40397

  Copy full SHA for ab40397

• Error message for service not available

  nagkumar91

  committed
  736b431

  Copy full SHA for 736b431

• Adding context back to follow chat protocol

  nagkumar91

  committed
  7aca2ce

  Copy full SHA for 7aca2ce

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  52738a3

  Copy full SHA for 52738a3

• fix conftest formatting

  nagkumar91

  committed
  d9854bd

  Copy full SHA for d9854bd

Commits on May 6, 2024

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  e677c56

  Copy full SHA for e677c56

• Remove certain keys from output and add it to tests

  nagkumar91

  committed
  638bb07

  Copy full SHA for 638bb07

• Remove offensive content in readme

  nagkumar91

  committed
  48642f3

  Copy full SHA for 48642f3

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  7e0a614

  Copy full SHA for 7e0a614

Commits on May 7, 2024

• Changed the signature according to discussion

  nagkumar91

  committed
  69a746c

  Copy full SHA for 69a746c

• Better variable name in readme

  nagkumar91

  committed
  71b5b68

  Copy full SHA for 71b5b68

• Merge branch 'main' into task/addSimulator

  nagkumar91

  committed
  ab799e3

  Copy full SHA for ab799e3

• Change project_scope to azure_ai_project

  nagkumar91

  committed
  11f8de7

  Copy full SHA for 11f8de7

• update instructions

  nagkumar91

  committed
  23da5a8

  Copy full SHA for 23da5a8

• update instructions

  nagkumar91

  committed
  c59d0ec

  Copy full SHA for c59d0ec

• update bugbash instructions

  nagkumar91

  committed
  9fb3caa

  Copy full SHA for 9fb3caa

• removed redundant deps

  nagkumar91

  committed
  22b2524

  Copy full SHA for 22b2524

• Remove retry on 424

  nagkumar91

  committed
  18c0977

  Copy full SHA for 18c0977

• Removed the references to mlflow_logger as it was not being used

  nagkumar91

  committed
  7608f72

  Copy full SHA for 7608f72

• Using dataclass

  nagkumar91

  committed
  acb660b

  Copy full SHA for acb660b

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  d2c09bd

  Copy full SHA for d2c09bd

• use project_name instead of workspace_name

  nagkumar91

  committed
  ae8ca58

  Copy full SHA for ae8ca58

• Update the eval part for bug bash

  nagkumar91

  committed
  70c2ba5

  Copy full SHA for 70c2ba5

Commits on May 8, 2024

• Fix the return type docstring and remove tracking for init

  nagkumar91

  committed
  fa96739

  Copy full SHA for fa96739

• uppercase RAI_SVC_URL

  nagkumar91

  committed
  a27b78a

  Copy full SHA for a27b78a

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  5a3cf52

  Copy full SHA for 5a3cf52

• Update import for content safety evaluators

  nagkumar91

  committed
  8e2b5c8

  Copy full SHA for 8e2b5c8

• Fix for jailbreak sim

  nagkumar91

  committed
  b22b47d

  Copy full SHA for b22b47d

• Added test to make sure jailbreak works

  nagkumar91

  committed
  2496b20

  Copy full SHA for 2496b20

• Update bugbash instrctions

  nagkumar91

  committed
  d7c92ed

  Copy full SHA for d7c92ed

Commits on May 9, 2024

• Change to add the enum for adversarial scenarios

  nagkumar91

  committed
  675bea8

  Copy full SHA for 675bea8

• Fixed e2e tests and readme

  nagkumar91

  committed
  ea3b9f4

  Copy full SHA for ea3b9f4

• Singular scenario

  nagkumar91

  committed
  35f2e61

  Copy full SHA for 35f2e61

• Import fix and jupyter notebook comment

  nagkumar91

  committed
  1c9f167

  Copy full SHA for 1c9f167

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  667adb3

  Copy full SHA for 667adb3

Commits on May 10, 2024

• Skipping the e2e tests

  nagkumar91

  committed
  5c25178

  Copy full SHA for 5c25178

• Skipping the e2e tests

  nagkumar91

  committed
  d1e7108

  Copy full SHA for d1e7108

• Merge branch 'main' into task/addSimulator

  nagkumar91

  authored
  0c891c7

  Copy full SHA for 0c891c7