Technique

Technique: freeze the evaluation rubric first

Technique: freeze the evaluation rubric first

Define what success means before generating variants, otherwise the best-looking output can hide workflow failure.

Define what success means before generating variants, otherwise the best-looking output can hide workflow failure.

Evaluation drift is one of the easiest ways to mistake taste for evidence.

SOURCE

Prompt QA workflow

ACTION

Write the pass/fail criteria above the prompt before the first run.

RELATED TRANSMISSION

building-a-prompt-qa-pass

Trust over spectacle. Evidence over opinion.

Trust over spectacle. Evidence over opinion.

ACTIVE SIGNAL SYSTEM

ACTIVE SIGNAL SYSTEM