functional-examplesFUNCTIONAL-EXAMPLES
DOCS/Core Concepts

Core Concepts

This guide explains the building blocks of functional-examples: what an example is, how extractors discover them, how plugins compose behavior, and how the scanner ties it all together.

Examples

An Example is the fundamental unit. It represents a single scannable code example with:

  • id — unique identifier (e.g., "basic-usage")
  • title — human-readable name
  • description — what the example demonstrates
  • files — the actual source files belonging to this example
  • metadata — arbitrary key-value data (tags, difficulty, category, etc.)
  • extractorName — which extractor discovered this example

Every plugin ultimately produces Example objects. The scanner collects them all into a uniform ScanResult.

Extractors

An Extractor is a function that receives a list of file candidates and returns examples. It's the core mechanism plugins use to discover examples in your project tree.

The extractor contract:

  1. Receive Dirent[] candidates (files matching the scan patterns)
  2. Inspect files, parse metadata, group related files
  3. Return an ExtractorResult with:
    • examples — the discovered examples
    • errors — any problems encountered
    • claimedFiles — paths this extractor "owns" (prevents double-extraction)

Here's a custom extractor that parses TOML metadata files:

/**
 * Custom TOML-based extractor example.
 *
 * This demonstrates how to create your own extractor that:
 * 1. Scans for a specific file pattern (meta.toml)
 * 2. Parses metadata from that file format
 * 3. Claims files and returns Example objects
 */
import {
  ExampleFile,
  type Example,
  type Extractor,
  type ExtractorResult,
} from 'functional-examples';
import { readdirSync, type Dirent } from 'node:fs';
import { readFile } from 'node:fs/promises';
import path, { join } from 'node:path';

/**
 * Metadata structure for TOML examples.
 * You can define any shape that fits your use case.
 */
export interface TomlMetadata {
  id: string;
  title: string;
  description?: string;
  author?: string;
  [key: string]: unknown;
}

/**
 * Create a custom extractor that reads TOML metadata files.
 *
 * Extractors implement a candidate-based pattern: they're called with
 * pre-filtered candidates (files and directories) and decide which to handle.
 */
export function createTomlExtractor(): Extractor<TomlMetadata> {
  return {
    name: 'toml-extractor',

    async extract(
      candidates: Dirent[]
    ): Promise<ExtractorResult<TomlMetadata>> {
      const examples: Example<TomlMetadata>[] = [];
      const claimedFiles = new Set<string>();
      const errors: { path: string; message: string }[] = [];

      // Find meta.toml files from candidates
      const tomlFiles: string[] = [];

      for (const candidate of candidates) {
        const fullPath = path.join(candidate.parentPath, candidate.name);

        if (candidate.isFile()) {
          // Direct file candidate: check if it's a meta.toml
          if (candidate.name === 'meta.toml') {
            tomlFiles.push(fullPath);
          }
        } else if (candidate.isDirectory()) {
          // Directory candidate: look for meta.toml inside
          const metaPath = path.join(fullPath, 'meta.toml');
          try {
            await readFile(metaPath, 'utf-8');
            tomlFiles.push(metaPath);
          } catch {
            // No meta.toml in this directory, skip
          }
        }
      }

      for (const tomlFile of tomlFiles) {
        try {
          const content = await readFile(tomlFile, 'utf-8');
          const metadata = parseSimpleToml(content);

          const exampleDir = path.dirname(tomlFile);

          // Collect all files in the example directory
          const files = collectExampleFiles(exampleDir);

          // Claim all files
          for (const file of files) {
            claimedFiles.add(file);
          }

          examples.push({
            id: metadata.id,
            title: metadata.title,
            description: metadata.description,
            rootPath: exampleDir,
            files: files.map((f) => new ExampleFile({
              absolutePath: f,
              relativePath: path.relative(exampleDir, f),
            })),
            metadata,
            extractorName: 'toml-extractor',
          });
        } catch (err) {
          errors.push({
            path: tomlFile,
            message: `Failed to parse: ${(err as Error).message}`,
          });
        }
      }

      return { examples, errors, claimedFiles };
    },
  };
}

/**
 * Simplified TOML parser for demonstration.
 * In production, use a proper TOML library like @iarna/toml.
 */
function parseSimpleToml(content: string): TomlMetadata {
  const lines = content.split('\n');
  const result: Record<string, string> = {};

  for (const line of lines) {
    // Match: key = "value"
    const match = line.match(/^(\w+)\s*=\s*"(.*)"/);
    if (match) {
      result[match[1]] = match[2];
    }
  }

  if (!result['id'] || !result['title']) {
    throw new Error('TOML must have id and title fields');
  }

  return {
    id: result['id'],
    title: result['title'],
    description: result['description'],
    author: result['author'],
  };
}

function collectExampleFiles(root: string) {
  if (root.endsWith('node_modules')) {
    return [];
  }

  let files: string[] = [];
  const entries = readdirSync(root, { withFileTypes: true });
  for (const entry of entries) {
    if (entry.isDirectory()) {
      files = files.concat(
        collectExampleFiles(join(entry.parentPath, entry.name))
      );
    } else {
      files.push(join(entry.parentPath, entry.name));
    }
  }
  return files;
}

The claimed files mechanism is important: when multiple plugins are active, each extractor claims the files it processes. This prevents the same file from being extracted twice by different plugins.

Plugins

A Plugin is a container that provides one or more capabilities:

CapabilityPurpose
extractorDiscovers examples from files
validatorsChecks example metadata against rules
schemasJSON Schema definitions for metadata and options
commandsCLI commands the plugin adds
extensionsFile extensions the plugin handles

Plugins are registered in your config:

import { createJavaScriptPlugin } from '@functional-examples/javascript';
import { createTestPlugin } from '@functional-examples/test';
import { createDocumentationPlugin } from '@functional-examples/documentation';

export default {
  plugins: [
    createJavaScriptPlugin(),    // extracts from frontmatter or package.json
    createTestPlugin(),          // reads metadata.test for assertions
    createDocumentationPlugin(), // adds doc generation
  ],
};

Each plugin operates independently. The scanner calls each plugin's extractor in sequence, passing unclaimed files to subsequent extractors.

Regions

Regions (also called hunks) are named sections within a file, marked with #region / #endregion comments:

/**
 * ---
 * id: region-markers
 * title: Region Markers Demo
 * ---
 */

import { scan } from 'functional-examples';

// #region setup
const result = await scan();
// #endregion

// #region execution
console.log(result.examples);
// #endregion

Regions let you reference specific parts of a file in documentation. Instead of showing an entire file, you can pull just the setup region:

<%= example('my-example').region('setup') %\>;

Note

There's a zero width space in the above code to prevent it from being processed by eta, copying it directly will not work.

The JavaScript plugin automatically detects #region / #endregion markers and records their line ranges as hunks on the file.

The Scanner Pipeline

When you call scan() or scanExamples(), here's what happens:

  1. Config resolution — the config file is loaded, plugins are instantiated, and scan patterns are resolved
  2. File discovery — the scanner globs for files matching scan.include patterns (minus scan.exclude)
  3. Path mapping — if pathMappings are configured, files are routed to specific extractors by pattern
  4. Extraction — each plugin's extractor runs against its candidate files, producing examples and claiming files
  5. Validation — if the config includes a metadata schema, each example's metadata is validated against it
  6. Result assembly — examples, errors, and stats are collected into a ScanResult

The result gives you everything you need to generate docs, run tests, or build custom tooling.

Configuration

Configuration lives in a functional-examples.config.ts (or .json) file at your project root. It controls:

  • plugins — which plugins to load
  • scan.root — base directory for scanning
  • scan.include / scan.exclude — glob patterns for file discovery
  • pathMappings — route specific file patterns to specific extractors
  • metadata — JSON Schema for validating example metadata

See the Configuration guide for the full reference.