Class Scorer

Object-oriented interface for working with custom Scorers.

Remarks

Overview

The Scorer class provides a high-level, object-oriented API for managing custom scorers. Scorers define bash scripts that produce a score in the range [0.0, 1.0] for scenario runs.

Quickstart

import { RunloopSDK } from '@runloop/api-client-ts';

const runloop = new RunloopSDK();
const scorer = await runloop.scorer.create({
  type: 'my_scorer',
  bash_script: 'echo "1.0"',
});

const result = await scorer.validate({ scoring_context: { output: 'test' } });
console.log(`Score: ${result.scoring_result.score}`);

Index

Accessors

Methods

getInfo update validate

Accessors

id

get id(): string
Get the scorer ID.

Returns string
The scorer ID
- Defined in src/sdk/scorer.ts:141

Methods

getInfo

getInfo(options?: RequestOptions<unknown>): Promise<ScorerRetrieveResponse>
Fetch current scorer details from the API.
Parameters
- Optionaloptions: RequestOptions<unknown>
  Request options
Returns Promise<ScorerRetrieveResponse>
Current scorer details
Example
```
const info = await scorer.getInfo();
console.log(`Scorer type: ${info.type}`);
```
- Defined in src/sdk/scorer.ts:157

update

update(
params: ScorerUpdateParams,
options?: RequestOptions<unknown>,
): Promise<ScorerUpdateResponse>
Update the scorer's type or bash script.
Parameters
- params: ScorerUpdateParams
  Update parameters
- Optionaloptions: RequestOptions<unknown>
  Request options
Returns Promise<ScorerUpdateResponse>
Updated scorer details
Example
```
const updated = await scorer.update({
  type: 'my_scorer_v2',
  bash_script: 'echo "0.5"'
});
```
- Defined in src/sdk/scorer.ts:176

validate

validate(
params: ScorerValidateParams,
options?: RequestOptions<unknown>,
): Promise<ScorerValidateResponse>
Run the scorer against the provided context and return the result.
Parameters
- params: ScorerValidateParams
  Validation parameters
- Optionaloptions: RequestOptions<unknown>
  Request options
Returns Promise<ScorerValidateResponse>
Validation result with score
Example
```
const result = await scorer.validate({
  scoring_context: { output: 'test output', expected: 'test output' }
});
console.log(`Validation score: ${result.scoring_result.score}`);
console.log(`Output: ${result.scoring_result.output}`);
```
- Defined in src/sdk/scorer.ts:196

Class Scorer

Remarks

Overview

Quickstart

Index

Accessors

Methods

Accessors

id

Returns string

Methods

getInfo

Parameters

Returns Promise<ScorerRetrieveResponse>

Example

update

Parameters

Returns Promise<ScorerUpdateResponse>

Example

validate

Parameters

Returns Promise<ScorerValidateResponse>

Example

Settings

On This Page