Test Plugin

The test plugin (@functional-examples/test) adds testing capabilities to functional-examples. It reads test definitions from metadata.test on any scanned example and provides the test CLI command.

Installation

npm install @functional-examples/test

Setup

Add the test plugin to your config:

import { createTestPlugin } from '@functional-examples/test';

export default {
  plugins: [
    // ... your extraction plugins
    createTestPlugin(),
  ],
};

Plugin Options

Option	Type	Default	Description
timeout	number	30000	Default timeout per test in milliseconds
reporters	ReporterConfig[]	[]	Additional reporters
defaultReporter	ReporterConfig	Pretty reporter	Reporter for local runs
ciReporter	ReporterConfig	TAP reporter	Reporter for CI environments

Defining Tests

Tests live in the test field of an example's metadata. The test plugin doesn't care which extractor produced the example — it works with any metadata source.

Single Command Tests

The simplest form — run a command and check assertions:

  "functional-examples": {
    "title": "Hello Script",
    "test": [
      {
        "name": "runs hello script",
        "options": { "command": "node hello.js" },
        "assertions": {
          "exitCode": 0,
          "stdout": { "contains": "Hello from example" }
        }
      }
    ]
  },

Multi-Step Tests

For examples that require setup or multiple phases, use steps:

  "functional-examples": {
    "title": "Greeting Function",
    "test": [
      {
        "name": "scan output matches snapshot",
        "steps": [
          {
            "command": "bash ../../scan.sh",
            "assertions": {
              "exitCode": 0
            }
          },
          {
            "command": "true",
            "assertions": {
              "snapshot": {
                "path": "../../output.txt",
                "snapshot": "../../__snapshots__/scan-output.txt"
              }
            }
          }
        ]
      },
      {
        "name": "cleanup",
        "options": {
          "command": "rm -f ../../output.txt"
        },
        "assertions": {
          "exitCode": 0
        }
      }
    ]
  },

Each step runs sequentially. If a step fails, subsequent steps in the same test are skipped.

Assertion Reference

Exit Code

assertions:
  exitCode: 0      # Expected exit code (0 = success)

Standard Output

assertions:
  stdout:
    contains: "expected text"     # Substring match
    matches: "pattern.*regex"     # Regex match

Standard Error

assertions:
  stderr:
    contains: "Error"             # Substring match
    matches: "Error:\\s+.*"       # Regex match

File Assertions

assertions:
  file:
    path: output.txt              # File must exist
    contains: "expected content"  # File content substring
    matches: "pattern"            # File content regex
  files:                          # Multiple file checks
    - path: a.txt
      contains: "content a"
    - path: b.txt

Directory Assertions

assertions:
  dir:
    path: output/                 # Directory must exist
  directories:                    # Multiple directory checks
    - path: dist/
    - path: generated/

Snapshot Assertions

assertions:
  snapshot:
    path: output.txt              # File to compare
    snapshot: __snapshots__/expected.txt  # Reference snapshot
  snapshots:                      # Multiple snapshot checks
    - path: a.txt
      snapshot: __snapshots__/a.txt

See Snapshot Testing for details on the snapshot workflow.

Negation

Wrap any assertion in not to invert it:

assertions:
  not:
    stdout:
      contains: "ERROR"
    file:
      path: should-not-exist.txt

Running Tests

# Run all example tests
npx functional-examples test

# Run tests for a specific directory
npx functional-examples test examples/test-plugin-example

# Update snapshots
npx functional-examples test -u

Reporters

The test plugin supports pluggable reporters:

Reporter	Use Case	Output
Pretty (default)	Local development	Colored, human-readable
TAP	CI pipelines	Machine-parseable TAP format

Use createTapReporter() for CI environments — most CI systems can parse TAP output natively.

See also: Snapshot Testing for snapshot workflows, CI Integration for pipeline setup.