Duplication explained

Use this page to decide which duplication in your codebase is worth a refactor. It explains what fallow dupes reports, how to read the output, and when to act.

Clone types

Fallow classifies clones with the standard taxonomy from clone detection research.

From strict to semantic, recall increases (fallow finds more clones) and precision decreases (you get more possible false positives).

Detection modes in detail

● Duplicates (1 clone groups)

    196 lines  2 instances  dup:6f87acd9
    src/lib/dutch-holidays.ts:193-388
    src/lib/dutch-holidays.ts:389-584

The two blocks differ only in names such as holidays2024 and holidays2025. Only semantic mode matches them.

Key metrics

Duplication percentage

The share of source lines that overlap at least one clone instance. Fallow computes it over the full analyzed file set, not only the groups that --top shows.

Token count and line count

Each clone group reports both token count and line count.

• Tokens are language-aware units (keywords, identifiers, operators, literals). The detection engine matches on tokens.
• Lines are the source lines that the clone spans. Use lines to estimate the refactoring effort.

A larger clone gives more value when you extract it. A 200-line clone group is worth extracting. A 6-line one probably is not.

Instance count

The number of locations where the same code appears. A clone group with 5 instances means that someone copied the block to 5 places. To fix a bug in that logic, you must update all 5.

Clone groups and families

Clone groups

A clone group is one duplicated code block that fallow found at 2+ locations. Each location is an instance.

● Clone group 1 (8 lines, 2 instances)
  src/validators/userValidator.ts:12-19
  src/validators/orderValidator.ts:8-15

One block of 8 lines appears in both files:

const errors: string[] = [];
for (const [key, rule] of Object.entries(rules)) {
  const value = data[key];
  if (rule.required && (value === undefined || value === null)) {
    errors.push(`${key} is required`);
  }
}
return errors;

One clone group is one duplicated block. The same two files can share more than one block:

● Clone group 2 (6 lines, 2 instances)
  src/validators/userValidator.ts:24-29
  src/validators/orderValidator.ts:20-25

● Clone group 3 (5 lines, 2 instances)
  src/validators/userValidator.ts:35-39
  src/validators/orderValidator.ts:31-35

These are three separate clone groups between the same two files. This pattern is a clone family.

Clone families

A clone family is a set of clone groups that involve the same files. Those files share several distinct clones, so someone probably copy-pasted one from the other.

Family: src/validators/userValidator.ts ↔ src/validators/orderValidator.ts
  3 clone groups, 19 duplicated lines
  → Extract shared validation module

A family needs a different refactor. For a single clone group, you extract a function. For a clone family, you merge or restructure the files.

When duplication is acceptable

Some duplication is fine to keep:

• Test files: Test cases often repeat setup code on purpose. Shared test setup makes tests harder to read and debug. To exclude test, story, and dev files entirely, use --production. For more granular control, use duplicates.ignore patterns.
• Generated code: Codegen output (GraphQL, Prisma, OpenAPI) contains duplication by design. Exclude it with duplicates.ignore.
• Small clones (< 10 lines): Very small clones are often idiomatic patterns, such as error handling or guard clauses.
• Intentional copies in one directory: --skip-local reports only clones that span two or more directories. Use it when you keep side-by-side copies on purpose, such as a .ts and a .js version of one file. For copies in different directories, exclude one side with duplicates.ignore.

Interpreting --threshold results

The --threshold flag sets the maximum duplication percentage. If the project is above the threshold, fallow exits with code 1.

✗ 27,255 lines (19.4%) duplicated across 398 files (0.23s)
Duplication (19.4%) exceeds threshold (15.0%)

Start with a threshold above your current level, then lower it over time. For a more gradual adoption, combine it with --baseline.

How suffix-array detection works

Fallow joins all token streams into one sequence and builds a suffix array with an LCP array. Then it scans for repeated subsequences above the minimum length. This runs in O(n log n) time and needs no pairwise file comparison.

Limitations

• Type-3 clones (gapped clones with inserted or deleted lines): fallow does not detect these by default. To find function-scoped near-miss clones with small structural edits, add --near. Near-miss detection uses semantic normalization and requires at least 80% shingle similarity between every pair in a group.
• Type-4 clones (semantically equivalent but syntactically different): fallow does not detect these. Fallow does not flag two functions that do the same thing but are written differently.
• Cross-file-type detection works only between .ts and .js files (with --cross-language). Fallow does not support other language pairs.
• Minimum size thresholds: fallow does not report clones below --min-tokens or --min-lines. This is intentional, because small clones are usually idiomatic patterns.
• Semantic mode false positives: When fallow normalizes identifiers and literals, it can match code that is structurally similar but semantically unrelated. Review semantic-mode results before you act on them.

JSON _meta object

When you pass --explain, or run through MCP, the JSON output includes a _meta object:

{
  "schema_version": 10,
  "_meta": {
    "docs": "https://docs.fallow.tools/cli/dupes",
    "metrics": {
      "duplication_percentage": {
        "name": "Duplication Percentage",
        "description": "Percentage of source lines that overlap at least one reported clone instance. Computed over the full analyzed file set.",
        "range": "[0, 100]",
        "interpretation": "lower is better"
      },
      "clone_groups": {
        "name": "Clone Groups",
        "description": "A set of code fragments with identical or near-identical normalized token sequences. Each group has 2+ instances across different locations.",
        "interpretation": "each group is a single refactoring opportunity"
      },
      "clone_families": {
        "name": "Clone Families",
        "description": "Groups of clone groups that share the same set of files. Indicates systematic duplication patterns (e.g., mirrored directory structures).",
        "interpretation": "families suggest extract-module refactoring opportunities"
      }
    }
  }
}

This is an excerpt. _meta has an entry for every metric that the report can emit. It also has a field_definitions block that describes shared output fields such as actions[].

With _meta, AI agents and CI systems can interpret the results without external documentation.

See also