How It Works¶

A Sublime Text 4 plugin that automatically detects and sets the correct syntax (language) for views using a cascading strategy pipeline — from user-defined rules through deep learning.

Architecture Overview¶

flowchart TD
    ST["Sublime Text Events"] --> Boot["boot.py<br/>module cleanup"]
    Boot --> Init["plugin/__init__.py<br/>plugin_loaded()"]
    Init --> Custom["Load custom<br/>matches & constraints"]
    Init --> Settings["AioSettings<br/>async setting watcher"]
    Init --> Compile["compile_rules()<br/>per window"]
    Compile --> Collection["SyntaxRuleCollection"]
    Settings -.->|on change| Compile

    ST -->|view events| Listener["EventListener<br/>+ TextChangeListener"]
    Listener --> Pipeline["run_auto_set_syntax_on_view()"]
    Pipeline --> Cascade["Detector Cascade<br/>(9 steps)"]
    Cascade -->|SyntaxDecision| Apply["_apply()"]
    Apply --> Assign["assign_syntax_to_view()"]

    Collection -->|test rules| Pipeline

Entry Point & Lifecycle¶

boot.py¶

Clears previously loaded plugin modules on reload, then imports plugin/__init__.py. This ensures a clean state during development.

plugin/__init__.py¶

On plugin_loaded() (ST lifecycle hook):

1. Adds vendored Python lib path to sys.path
2. Loads custom implementations from AutoSetSyntax-Custom/matches/ and AutoSetSyntax-Custom/constraints/
3. Sets up AioSettings — watches settings files asynchronously; recompiles rules on change
4. Calls set_up_window() for each open window — compiles rules, logs version info, checks Magika availability
5. Optionally runs syntax detection on startup views

On plugin_unloaded():

• Tears down settings watchers and per-window state (log panels, rule collections)

Trigger Events¶

flowchart LR
    subgraph Events["ST Event Listeners"]
        Load["on_load"] -->|"ListenerEvent.LOAD"| Pipeline
        New["on_new"] -->|"ListenerEvent.NEW"| Pipeline
        Save["on_post_save"] -->|"ListenerEvent.SAVE"| Pipeline
        Reload["on_reload"] -->|"ListenerEvent.RELOAD"| Pipeline
        Activate["on_activated"] -->|"ListenerEvent.UNTRANSIENTIZE"| Pipeline
        Change["on_text_changed_async"] -->|"ListenerEvent.MODIFY/PASTE"| Pipeline
        Exec["on_post_window_command"] -->|"ListenerEvent.EXEC"| Pipeline
        Revert["on_revert"] -->|"ListenerEvent.REVERT"| Pipeline
    end

    Pipeline["run_auto_set_syntax_on_view()"]
    Pipeline --> PreChecks

All events converge on run_auto_set_syntax_on_view() in plugin/commands/auto_set_syntax.py.

Per-Event Behavior Detail¶

Prerequisites (pre-flight checks)¶

Before any detector runs, the pipeline verifies:

1. View has a window and is valid
2. View is "syntaxable" — not a widget/panel, not transient, within size limit
3. View is plaintext (if must_plaintext=True)
4. SyntaxRuleCollection is compiled for the window
5. Plugin is ready (G.is_plugin_ready())

The Detector Pipeline (9 Steps)¶

The pipeline tries each detector in order, stopping at the first one to reach a decision:

1. Exec Output — exec_file_syntax for build panels
2. New File — new_file_syntax for untitled files
3. ST Syntax Test — the syntax named on an ST syntax test file's first line
4. Plugin Rules — iterate user-defined SyntaxRule collection
5. First Line — shebang (#!/usr/bin/env) or modeline (-*- mode -*-)
6. Trimmed Filename — strip suffixes and match the base filename
7. Magika (DL) — Google's deep-learning content-type detection
8. Heuristics — content-based guess (currently JSON detection)
9. Give Up — leave as plain text

A detector never touches the view. It inspects the ViewSnapshot and returns either a SyntaxDecision or None:

@dataclass(slots=True, frozen=True)
class SyntaxDecision:
    syntax: sublime.Syntax          # what to assign
    details: dict[str, Any]         # why — logged verbatim
    status_message: str | None      # optional status bar text, shown when applied

Steps 1 and 2 are early-outs handled directly by run_auto_set_syntax_on_view() — exec output runs before the (comparatively expensive) ViewSnapshot is built, since a build panel updates often and needs none of it. Steps 3–8 form the _detect() chain. Whatever the source, the resulting decision reaches the view through a single _apply() call, which emits any status_message and hands off to assign_syntax_to_view().

Rules System¶

The rules system is a tree of SyntaxRule objects, each containing a nested match/constraint tree.

flowchart TD
    SCR["SyntaxRuleCollection<br/>ordered list of SyntaxRules"] -->|test each| SR1["SyntaxRule 1"]
    SCR --> SR2["SyntaxRule 2"]
    SCR --> SR3["... N"]

    SR1 -->|properties| Selector["selector: 'text.plain'"]
    SR1 -->|properties| OnEvents["on_events: [LOAD, SAVE]"]
    SR1 -->|properties| Syntaxes["syntax: 'source.python'"]

    SR1 --> MR["root_rule: MatchRule"]
    MR --> Match["AbstractMatch<br/>(any / all / some / ratio)"]
    Match --> CR1["ConstraintRule<br/>(leaf condition)"]
    Match --> CR2["ConstraintRule"]
    Match --> NestedMatch["MatchRule<br/>(nested)"]

    NestedMatch --> CR3["ConstraintRule"]
    NestedMatch --> CR4["ConstraintRule"]

    CR1 --> Constraint["AbstractConstraint<br/>e.g. is_extension"]

SyntaxRule¶

The top-level rule configured in settings:

{
  "comment": "Python files",
  "syntaxes": ["Python", "scope:source.python"],
  "selector": "text.plain",
  "on_events": ["LOAD", "SAVE"],
  "match": {
    "match": "all",
    "rules": [
      { "constraint": "is_extension", "args": [".py"] }
    ]
  }
}

Fields:

• syntax / syntaxes: The target syntax to assign
• selector: Scope filter — only applies if current scope matches (default: text.plain)
• on_events: Restrict which events trigger this rule (None = all events)
• root_rule: A MatchRule (the match/constraint tree)
• comment: Human-readable label (for logging/debugging)
• src_setting: Reference back to the original setting object

MatchRule + AbstractMatch¶

A MatchRule combines a match strategy (AbstractMatch) with child rules:

Children can be ConstraintRule (leaf) or nested MatchRule (sub-tree).

ConstraintRule + AbstractConstraint¶

A ConstraintRule wraps an AbstractConstraint with optional inversion:

Optimization¶

At compile time, the rule tree is optimized — rules that fold (their result no longer depends on runtime data, so they'd evaluate to the same thing on every view) are pruned so they aren't re-tested on every view.

Every node answers one question, fold(), returning a Fold(value, reason) — the constant it always evaluates to plus why — or UNFOLDED when the result still depends on the view:

SyntaxRule.fold()     → False if no syntax, no events (and not unrestricted), or no root_rule
MatchRule.fold()      → the match's own fold, e.g. some(5) with 3 children is a constant False
ConstraintRule.fold() → the constraint's fold, flipped by `inverted`

The reason travels with the value because only the folding node knows it: by the time the rule reaches the dropped-rules report, "never matches" is all that would be left to say. It is quoted verbatim there, after the verdict — e.g. never matches: no extension was given, or always matches: a "threshold" of 0 is met without finding anything — so phrase it for the user reading Debug Information, not for whoever reads the source.

Knowing which constant a rule folds to is the whole point: an empty all is always True (all([]) == True) while an empty any is always False (any([]) == False), and they are not interchangeable. prunable_child_value() tells a parent match which of those two values is safe to silently remove from its own child list without changing its result:

• Dropping a constant-True child from all(...) is safe (True is AND's identity element: all(True, x) == all(x)), but dropping a constant-False child is not — the whole all(...) must become False instead, since all(False, x) is always False regardless of x.
• The mirror is true for any: dropping constant-False is safe, constant-True is not.
• ratio(n/d)'s prunable_child_value() is None — its goal (ceil(ratio * len(rules))) depends on the count of its children, so no child, constant or not, can ever be safely removed without shifting that goal.

Dropped rules are logged and stored in G.dropped_rules_collection for debugging.

Optimizing is best-effort: it walks the tree mutating it, so if a node's fold() misbehaves (in practice, a Custom Implementation that doesn't return a Fold), a half-finished pass would leave a rule whose root was already detached. compile_rules() catches that, logs the offending class, and rebuilds the collection from the settings to run unoptimized — every rule still tests correctly, it just gets re-evaluated on each view. One broken custom implementation doesn't cost the window the rest of the user's rules.

ViewSnapshot¶

Before the detector chain runs, a ViewSnapshot is created — a frozen snapshot of the view's state at that moment. It is everything a detector is allowed to know about the view:

• view: The sublime.View object
• content: Full text content
• first_line: First line of content
• char_count, line_count: Size metrics
• path_obj: Path() object (or None if unsaved)
• file_extensions: List of suffixes (e.g., .tar.gz → ['.tar', '.gz'])
• file_name, file_name_unhidden: With/without leading dot
• syntax: Current syntax object
• encoding: Encoding (defaults to UTF-8 for unsaved buffers)
• caret_rowcol: Cursor position (row, col) for edit-aware decisions
• content_bytes, encoding_py: Lazy-computed derivatives

Final Assignment¶

_apply() is the only place a decision reaches the view. It is a no-op when no detector decided, emits the decision's status_message if it carries one, then delegates to assign_syntax_to_view():

flowchart TD
    In["assign_syntax_to_view(view, syntax)"] --> Valid{"view.is_valid()?"}
    Valid -->|no| False["return False"]
    Valid -->|yes| Views["Get all views<br/>sharing the buffer"]
    Views --> Loop["For each view"]
    Loop --> Same{"Syntax already<br/>the same?"}
    Same -->|yes| LogKeep["Log: [ALREADY]<br/>skip"]
    Same -->|no| AssignST["view.assign_syntax(syntax)"]
    AssignST --> Flag["Set VIEW_KEY_IS_ASSIGNED"]
    Flag --> LogAssign["Log: old → new<br/>+ reason"]
    LogKeep --> Next["Continue loop"]
    LogAssign --> Next
    Next --> Done["return True"]

1. Validates the view
2. Gets all sibling views sharing the same buffer (via view.buffer().views()), so clones and split views stay in sync
3. For each view: skips if already has the target syntax, otherwise calls view.assign_syntax(syntax)
4. Sets VIEW_KEY_IS_ASSIGNED on view settings
5. Logs the change with full context (old syntax → new syntax + the decision's details)

Extensibility¶

Users can add custom AbstractMatch or AbstractConstraint implementations:

1. Create a Python file in Packages/AutoSetSyntax-Custom/matches/ or Packages/AutoSetSyntax-Custom/constraints/
2. Subclass AbstractMatch or AbstractConstraint
3. Implement test() (and optionally fold())
4. fold() returns Fold(constant, reason) — the constant test() would always return plus a user-facing explanation — or UNFOLDED when the result still depends on the view. Both constants are expressible, so an "always matches" implementation reports True and an "always fails" one reports False — getting this backwards inverts the rule's meaning. It has to be a real Fold: a bare False/None, or a plain (False, "…") tuple, is rejected.
5. For AbstractMatch, also override prunable_child_value() if the match supports safely pruning individual children (see Optimization); the default of "only a constant-False child is prunable" is right for any-like combinators but wrong for all-like ones.

1. The class name convention determines the setting name: FooBarMatch → "foo_bar" and BazConstraint → "baz"

Auto-discovered at plugin load via _load_custom_implementations() using pkgutil.iter_modules().

Magika Integration¶

Magika (Google's deep-learning file type detector) is an optional dependency:

def get_magika_object() -> magika.Magika | None:
    try:
        from magika import Magika
        from magika import PredictionMode
    except ImportError:
        return None
    return Magika(prediction_mode=PredictionMode.HIGH_CONFIDENCE)

• Downloaded separately; not vendored
• Uses HIGH_CONFIDENCE prediction mode (avoids false positives)
• Only invoked for extensionless plaintext files (unless triggered via command)
• Results mapped to ST syntaxes via magika.syntax_map settings

Settings¶

Uses AioSettings — an async settings watcher that:

• Merges per-project settings with plugin defaults
• Automatically recompiles rule collections when settings change
• Tracks per-window settings independently

Key settings groups:

• Strategy control: new_file_syntax, exec_file_syntax, trim_suffixes, magika.*
• Behavior: debounce, run_on_startup_views, enable_log
• Rules: syntax_rules — the array of user-defined SyntaxRule objects

Shared Global State¶

@dataclass
class _GlobalState:
    startup_views: set[sublime.View]         # Views existing at startup
    syntax_rule_collections: WindowKeyedDict  # Per-window compiled rules
    dropped_rules_collection: WindowKeyedDict # Per-window optimized-away rules

G = _GlobalState() — a single shared instance holding per-window compiled state.