@runloop/api-client - v1.18.1
    Preparing search index...

    Class ScenarioRun

    Object-oriented interface for working with Scenario Runs.

    The ScenarioRun class provides a high-level API for managing scenario runs. A scenario run represents a single execution of a scenario on a devbox, including the ability to interact with the devbox, score the run, and retrieve results.

    ScenarioRuns are typically obtained from a Scenario's run() or runAsync() methods:

    import { RunloopSDK } from '@runloop/api-client';

    const runloop = new RunloopSDK();
    const scenario = runloop.scenario.fromId('scn_123');

    // Start a run with agent mounted
    const run = await scenario.run({
      run_name: 'my-run',
      runProfile: {
        mounts: [{
          type: 'agent_mount',
          agent_id: 'agt_123',
          agent_name: null,
          agent_path: '/home/user/agent',
        }],
      },
    });

    // Execute your agent on the devbox
    await run.devbox.cmd.exec('python /home/user/agent/main.py');

    // Score and complete the run
    await run.scoreAndComplete();
    const score = await run.getScore();
    Index

    Accessors

    id devboxId devbox

    Methods

    getInfo awaitEnvReady score awaitScored scoreAndAwait scoreAndComplete complete cancel downloadLogs getScore

    Accessors

    • get devbox(): RunloopSDK.Devbox

      Get the devbox instance for this scenario run.

      This property provides lazy-loaded access to the devbox associated with this scenario run. Use this to interact with the devbox environment during the scenario execution.

      Returns RunloopSDK.Devbox

      The devbox instance

      const run = await scenario.run();
      const devbox = run.devbox;
      await devbox.cmd.exec('npm test');

    Methods

    • Get the complete scenario run data from the API.

      Parameters

      • Optionaloptions: RequestOptions<unknown>

        Request options

      Returns Promise<ScenarioRunView>

      The scenario run data

      const info = await run.getInfo();
      console.log(`Run state: ${info.state}`);
      console.log(`Score: ${info.scoring_contract_result?.score}`);

    • Wait for the scenario environment (devbox) to be ready.

      Blocks until the devbox reaches running state. Call this after using scenario.runAsync() to ensure the devbox is ready for interaction.

      Parameters

      Returns Promise<ScenarioRunView>

      The scenario run data after environment is ready

      const run = await scenario.runAsync();
      await run.awaitEnvReady();
      // Devbox is now ready
      await run.devbox.cmd.exec('ls -la');

    • Submit the scenario run for scoring.

      This triggers the scoring process using the scenario's scoring contract. The scoring runs asynchronously; use awaitScored() or scoreAndAwait() to wait for scoring to complete.

      Parameters

      • Optionaloptions: RequestOptions<unknown>

        Request options

      Returns Promise<ScenarioRunView>

      The updated scenario run data

      await run.score();
      // Scoring is now in progress
      const result = await run.awaitScored();

    • Score the run, wait for scoring, then complete and shutdown.

      This is a convenience method that scores the scenario run, waits for scoring to finish, then completes the run and shuts down the devbox. This is the recommended way to finish a scenario run.

      Parameters

      Returns Promise<ScenarioRunView>

      The completed scenario run data with scoring results

      // Agent has finished working...
      const result = await run.scoreAndComplete();
      console.log(`Final score: ${result.scoring_contract_result?.score}`);
      // Devbox has been shut down

    • Complete the scenario run and shutdown the devbox.

      Call this after scoring to finalize the run. The devbox will be shut down and resources released. Note: The run must be in a scored state before calling complete. Use cancel() to end a run without scoring, or scoreAndComplete() to score and complete in one operation.

      Parameters

      • Optionaloptions: RequestOptions<unknown>

        Request options

      Returns Promise<ScenarioRunView>

      The final scenario run data

      // Score first, then complete
      await run.scoreAndAwait();
      await run.complete();

    • Cancel the scenario run and shutdown the devbox.

      Use this to abort a running scenario. The devbox will be shut down and the run marked as canceled.

      Parameters

      • Optionaloptions: RequestOptions<unknown>

        Request options

      Returns Promise<ScenarioRunView>

      The canceled scenario run data

      // Abort the scenario
      await run.cancel();

    • Download all logs for this scenario run to a file.

      Downloads a zip archive containing all logs from the scenario run's associated devbox. This is useful for debugging and analysis.

      Parameters

      • filePath: string

        Path where the zip file will be written

      • Optionaloptions: RequestOptions<unknown>

        Request options

      Returns Promise<void>

      await run.scoreAndComplete();
      await run.downloadLogs('./scenario-logs.zip');

    • Get the scoring result for this run.

      Returns null if the run has not been scored yet. Always makes an API call to retrieve the current scoring result.

      Parameters

      • Optionaloptions: RequestOptions<unknown>

        Request options

      Returns Promise<ScoringContractResultView | null>

      The scoring result or null if not yet scored

      await run.scoreAndAwait();
      const score = await run.getScore();
      if (score) {
        console.log(`Total score: ${score.score}`);
        for (const fn of score.scoring_function_results) {
          console.log(`  ${fn.scoring_function_name}: ${fn.score}`);
        }
      }

    Settings

    Member Visibility

    On This Page

    OverviewQuickstart
    Accessors
    Methods