@runloop/api-client - v1.4.0
    Preparing search index...

    Class ScenarioRun

    Object-oriented interface for working with Scenario Runs.

    The ScenarioRun class provides a high-level API for managing scenario runs. A scenario run represents a single execution of a scenario on a devbox, including the ability to interact with the devbox, score the run, and retrieve results.

    ScenarioRuns are typically obtained from a Scenario's run() or runAsync() methods:

    import { RunloopSDK } from '@runloop/api-client';

    const runloop = new RunloopSDK();
    const scenario = runloop.scenario.fromId('scn_123');

    // Start a run with agent mounted
    const run = await scenario.run({
      run_name: 'my-run',
      runProfile: {
        mounts: [{
          type: 'agent_mount',
          agent_id: 'agt_123',
          agent_name: null,
          agent_path: '/home/user/agent',
        }],
      },
    });

    // Execute your agent on the devbox
    await run.devbox.cmd.exec('python /home/user/agent/main.py');

    // Score and complete the run
    await run.scoreAndComplete();
    const score = await run.getScore();
    Index

    Accessors

    id devboxId devbox

    Methods

    getInfo awaitEnvReady score awaitScored scoreAndAwait scoreAndComplete complete cancel downloadLogs getScore

    Accessors

    • get devbox(): RunloopSDK.Devbox

      Get the devbox instance for this scenario run.

      This property provides lazy-loaded access to the devbox associated with this scenario run. Use this to interact with the devbox environment during the scenario execution.

      Returns RunloopSDK.Devbox

      The devbox instance

      const run = await scenario.run();
      const devbox = run.devbox;
      await devbox.cmd.exec('npm test');

    Methods

    • Get the complete scenario run data from the API.

      Parameters

      • Optionaloptions: RequestOptions<unknown>

        Request options

      Returns Promise<ScenarioRunView>

      The scenario run data

      const info = await run.getInfo();
      console.log(`Run state: ${info.state}`);
      console.log(`Score: ${info.scoring_contract_result?.score}`);

    • Wait for the scenario environment (devbox) to be ready.

      Blocks until the devbox reaches running state. Call this after using scenario.runAsync() to ensure the devbox is ready for interaction.

      Parameters

      • Optionaloptions: RequestOptions<unknown> & { polling?: Partial<PollingOptions<DevboxView>> }

        Request options with optional polling configuration

      Returns Promise<ScenarioRunView>

      The scenario run data after environment is ready

      const run = await scenario.runAsync();
      await run.awaitEnvReady();
      // Devbox is now ready
      await run.devbox.cmd.exec('ls -la');

    • Submit the scenario run for scoring.

      This triggers the scoring process using the scenario's scoring contract. The scoring runs asynchronously; use awaitScored() or scoreAndAwait() to wait for scoring to complete.

      Parameters

      • Optionaloptions: RequestOptions<unknown>

        Request options

      Returns Promise<ScenarioRunView>

      The updated scenario run data

      await run.score();
      // Scoring is now in progress
      const result = await run.awaitScored();

    • Wait for the scenario run to be scored.

      Blocks until scoring is complete. Call this after score() to wait for the scoring process to finish.

      Parameters

      • Optionaloptions: RequestOptions<unknown> & { polling?: Partial<PollingOptions<ScenarioRunView>> }

        Request options with optional polling configuration

      Returns Promise<ScenarioRunView>

      The scored scenario run data

      await run.score();
      const result = await run.awaitScored();
      console.log(`Final score: ${result.scoring_contract_result?.score}`);

    • Submit for scoring and wait for completion.

      This is a convenience method that combines score() and awaitScored().

      Parameters

      • Optionaloptions: RequestOptions<unknown> & { polling?: Partial<PollingOptions<ScenarioRunView>> }

        Request options with optional polling configuration

      Returns Promise<ScenarioRunView>

      The scored scenario run data

      // Agent has finished working...
      const result = await run.scoreAndAwait();
      console.log(`Final score: ${result.scoring_contract_result?.score}`);

    • Score the run, wait for scoring, then complete and shutdown.

      This is a convenience method that scores the scenario run, waits for scoring to finish, then completes the run and shuts down the devbox. This is the recommended way to finish a scenario run.

      Parameters

      • Optionaloptions: RequestOptions<unknown> & { polling?: Partial<PollingOptions<ScenarioRunView>> }

        Request options with optional polling configuration

      Returns Promise<ScenarioRunView>

      The completed scenario run data with scoring results

      // Agent has finished working...
      const result = await run.scoreAndComplete();
      console.log(`Final score: ${result.scoring_contract_result?.score}`);
      // Devbox has been shut down

    • Complete the scenario run and shutdown the devbox.

      Call this after scoring to finalize the run. The devbox will be shut down and resources released. Note: The run must be in a scored state before calling complete. Use cancel() to end a run without scoring, or scoreAndComplete() to score and complete in one operation.

      Parameters

      • Optionaloptions: RequestOptions<unknown>

        Request options

      Returns Promise<ScenarioRunView>

      The final scenario run data

      // Score first, then complete
      await run.scoreAndAwait();
      await run.complete();

    • Cancel the scenario run and shutdown the devbox.

      Use this to abort a running scenario. The devbox will be shut down and the run marked as canceled.

      Parameters

      • Optionaloptions: RequestOptions<unknown>

        Request options

      Returns Promise<ScenarioRunView>

      The canceled scenario run data

      // Abort the scenario
      await run.cancel();

    • Download all logs for this scenario run to a file.

      Downloads a zip archive containing all logs from the scenario run's associated devbox. This is useful for debugging and analysis.

      Parameters

      • filePath: string

        Path where the zip file will be written

      • Optionaloptions: RequestOptions<unknown>

        Request options

      Returns Promise<void>

      await run.scoreAndComplete();
      await run.downloadLogs('./scenario-logs.zip');

    • Get the scoring result for this run.

      Returns null if the run has not been scored yet. Always makes an API call to retrieve the current scoring result.

      Parameters

      • Optionaloptions: RequestOptions<unknown>

        Request options

      Returns Promise<ScoringContractResultView | null>

      The scoring result or null if not yet scored

      await run.scoreAndAwait();
      const score = await run.getScore();
      if (score) {
        console.log(`Total score: ${score.score}`);
        for (const fn of score.scoring_function_results) {
          console.log(`  ${fn.scoring_function_name}: ${fn.score}`);
        }
      }

    Settings

    Member Visibility

    On This Page

    OverviewQuickstart
    Accessors
    Methods