We can run list-jobs (the underlying action, not the cli command) by default whenever the user doesn't pass any job_id to the poll command, and let the user pick the job from the ones listed in the same interaction.

@Changhao-Li This is a good issue to get you started with our development process, and to get you familiarized with the command-line interface of metriq-gym.

The current user workflow is the following:

1. user dispatches a job with the run.py dispatch action
2. user lists all dispatched jobs via run.py list-jobs action
3. from the list of job ids that come from 2., user picks the one they are interested in, and they poll and fetch the results with the run.py poll action, by passing the job id to the --job-id parameter.

There are many things† that can improved in this workflow, but unfortunately the team had to focus on other priorities so far rather than improving this. However, we believe that step 3. is particularly unintuitive and a clear sticking point. In fact, a first-time user may not realize how to get the id for the job. One incremental improvement that we can make immediately is to have the possibility to run the poll action without specify any job id, upon which, the user will be offered a selection of the jobs that are available for polling.

New workflow:

1. python run.py dispatch ...
2. user goes on with their life, make ☕ , etc..
3. user runs python run.py poll
4. metriq-gym displays a table with all the dispatched jobs, and prompt the user a question on which job they want to poll/fetch.
5. user selects the job, and the job is polled/fetched.

We are keeping the CLI very simple, so please don't go with too fancy solutions, as may rewrite it in the future anyway (see #130.)

†As you execute this issue and reflect on the current workflow, don't be shy to create issues on this backlog for whatever improvement you think it's necessary 🙏

Thanks @cosenal for the explanation, looks like a great issue to get started. Feel free to reach out, @Changhao-Li, if you need support.

Thanks @cosenal and @nathanshammah for the information.

I've modified the workflow accordingly by modifying poll_job in run.py, as well as setting the --job-id argument optional for the poll command.

More specifically, the modified poll_job reads:

def poll_job(args: argparse.Namespace, job_manager: JobManager) -> None:
    logger.info("Polling job...")
    if not args.job_id:
        jobs = job_manager.get_jobs()
        if not jobs:
            logger.info("No jobs available for polling.")
            return
        print("Available jobs:")
        for i, job in enumerate(jobs):
            print(f"[{i}] {job.id} - {job.job_type}")
        selected_index = int(input("Select a job index: "))
        args.job_id = jobs[selected_index].id
    
    metriq_job: MetriqGymJob = job_manager.get_job(args.job_id)
    job_type: JobType = JobType(metriq_job.job_type)
    job_data: BenchmarkData = setup_job_data_class(job_type)(**metriq_job.data)
    job_class = setup_job_class(metriq_job.provider_name)
    device = setup_device(metriq_job.provider_name, metriq_job.device_name)
    handler = setup_handler(args, None, job_type)
    quantum_job = [job_class(job_id, device=device) for job_id in job_data.provider_job_ids]
    if all(task.status() == JobStatus.COMPLETED for task in quantum_job):
        result_data: list[ResultData] = [task.result().data for task in quantum_job]
        handler.poll_handler(job_data, result_data)
    else:
        logger.info("Job is not yet completed. Please try again later.")

In cli.py, set

poll_parser.add_argument("--job_id", type=str, required=False, help="Job ID to poll (optional)")

Please let me know if you would like me have a pull request on this to close the issue.

Some other improvements we may consider in future: add a --status flag to poll to allow users to check the status of a job before attempting to poll results; introduce a timeout mechanism when polling jobs to avoid indefinite waits (particularly when there are multi-jobs with large circuits) .

@Changhao-Li Yes, please make a pull request and we discuss there.