How It Works¶

A Sublime Text 4 plugin that automatically detects and sets the correct syntax (language) for views using a cascading strategy pipeline — from user-defined rules through deep learning.

Architecture Overview¶

flowchart TD
    ST["Sublime Text Events"] --> Boot["boot.py<br/>module cleanup"]
    Boot --> Init["plugin/__init__.py<br/>plugin_loaded()"]
    Init --> Custom["Load custom<br/>matches & constraints"]
    Init --> Settings["AioSettings<br/>async setting watcher"]
    Init --> Compile["compile_rules()<br/>per window"]
    Compile --> Collection["SyntaxRuleCollection"]
    Settings -.->|on change| Compile

    ST -->|view events| Listener["EventListener<br/>+ TextChangeListener"]
    Listener --> Pipeline["run_auto_set_syntax_on_view()"]
    Pipeline --> Cascade["Detector Cascade<br/>(9 steps)"]
    Cascade -->|SyntaxDecision| Apply["_apply()"]
    Apply --> Assign["assign_syntax_to_view()"]

    Collection -->|test rules| Pipeline

Entry Point & Lifecycle¶

boot.py¶

Clears previously loaded plugin modules on reload, then imports plugin/__init__.py. This ensures a clean state during development.

plugin/__init__.py¶

On plugin_loaded() (ST lifecycle hook):

1. Adds vendored Python lib path to sys.path
2. Loads custom implementations from AutoSetSyntax-Custom/matches/ and AutoSetSyntax-Custom/constraints/
3. Sets up AioSettings — watches settings files asynchronously; recompiles rules on change
4. Calls set_up_window() for each open window — compiles rules, logs version info, checks Magika availability
5. Runs syntax detection on startup views whose window enables run_on_startup_views

On plugin_unloaded():

• Tears down settings watchers and per-window state (log panels, rule collections)

Trigger Events¶

flowchart LR
    subgraph Events["ST Event Listeners"]
        Load["on_load"] -->|"ListenerEvent.LOAD"| Pipeline
        New["on_new"] -->|"ListenerEvent.NEW"| Pipeline
        Save["on_post_save"] -->|"ListenerEvent.SAVE"| Pipeline
        Reload["on_reload"] -->|"ListenerEvent.RELOAD"| Pipeline
        Activate["on_activated"] -->|"ListenerEvent.UNTRANSIENTIZE"| Pipeline
        Change["on_text_changed_async"] -->|"ListenerEvent.MODIFY/PASTE"| Pipeline
        Exec["on_post_window_command"] -->|"ListenerEvent.EXEC"| Pipeline
        Revert["on_revert"] -->|"ListenerEvent.REVERT"| Pipeline
    end

    Pipeline["run_auto_set_syntax_on_view()"]
    Pipeline --> PreChecks

All events converge on run_auto_set_syntax_on_view() in plugin/commands/auto_set_syntax.py.

Per-Event Behavior Detail¶

Prerequisites (pre-flight checks)¶

Before any detector runs, the pipeline verifies:

1. View has a window and is valid
2. View is "syntaxable" — not a widget/panel, not transient, within size limit
3. View is plaintext (if must_plaintext=True)
4. The window has compiled rules (G.compiled_rules) — i.e. it has been set up (on_new fires before on_new_window for a new window's first tab, so it calls ensure_window_set_up() first)

The Detector Pipeline (9 Steps)¶

The pipeline tries each detector in order, stopping at the first one to reach a decision:

1. Exec Output — exec_file_syntax for build panels
2. New File — new_file_syntax for untitled files
3. ST Syntax Test — the syntax named on an ST syntax test file's first line
4. Plugin Rules — iterate user-defined SyntaxRule collection
5. First Line — shebang (#!/usr/bin/env) or modeline (-*- mode -*-)
6. Trimmed Filename — strip suffixes and match the base filename
7. Magika (DL) — Google's deep-learning content-type detection
8. Heuristics — content-based guess (currently JSON detection)
9. Give Up — leave as plain text

A detector never touches the view. It inspects the ViewSnapshot and returns either a SyntaxDecision or None:

@dataclass(slots=True, frozen=True)
class SyntaxDecision:
    syntax: sublime.Syntax          # what to assign
    reason: str                     # why this detector decided — logged
    details: dict[str, Any]         # further evidence — logged after the reason
    status_message: str | None      # optional status bar text, shown when applied

Not every detector runs on every event: trimmed filename only runs for command, init, load, save and untransientize; Magika additionally runs for modify and paste.

Steps 1 and 2 are early-outs handled directly by run_auto_set_syntax_on_view() — exec output runs before the (comparatively expensive) ViewSnapshot is built, since a build panel updates often and needs none of it. Steps 3–8 form the detect() chain in plugin/detectors.py. Whatever the source, the resulting decision reaches the view through a single _apply() call, which emits any status_message and hands off to assign_syntax_to_view().

Rules System¶

The rules system is a tree of SyntaxRule objects, each containing a nested match/constraint tree.

flowchart TD
    SCR["SyntaxRuleCollection<br/>ordered list of SyntaxRules"] -->|test each| SR1["SyntaxRule 1"]
    SCR --> SR2["SyntaxRule 2"]
    SCR --> SR3["... N"]

    SR1 -->|properties| Selector["selector: 'text.plain'"]
    SR1 -->|properties| OnEvents["on_events: [load, save]"]
    SR1 -->|properties| Syntaxes["syntax: 'source.python'"]

    SR1 --> MR["root_rule: MatchRule"]
    MR --> Match["AbstractMatch<br/>(any / all / some / ratio)"]
    Match --> CR1["ConstraintRule<br/>(leaf condition)"]
    Match --> CR2["ConstraintRule"]
    Match --> NestedMatch["MatchRule<br/>(nested)"]

    NestedMatch --> CR3["ConstraintRule"]
    NestedMatch --> CR4["ConstraintRule"]

    CR1 --> Constraint["AbstractConstraint<br/>e.g. is_extension"]

SyntaxRule¶

The top-level rule configured in settings:

{
  "comment": "Python files",
  "syntaxes": ["Python", "scope:source.python"],
  "selector": "text.plain",
  "on_events": ["load", "save"],
  "match": {
    "match": "all",
    "rules": [
      { "constraint": "is_extension", "args": [".py"] }
    ]
  }
}

Fields:

• syntax / syntaxes: The target syntax to assign
• selector: Scope filter — only applies if current scope matches (default: text.plain)
• on_events: Restrict which events trigger this rule (None = all events)
• root_rule: A MatchRule (the match/constraint tree)
• is_invalid: Set when the setting was understood but a match/constraint in it wasn't; such a rule folds to False
• comment: Human-readable label (for logging/debugging)
• src_setting: Reference back to the original setting object

MatchRule + AbstractMatch¶

A MatchRule combines a match strategy (AbstractMatch) with child rules:

Invalid arguments — some without exactly one non-negative integer, or ratio without a non-negative numerator and a positive denominator — are rejected when the match is built, which drops the whole syntax rule as invalid (see below) rather than letting it match every file.

Children can be ConstraintRule (leaf) or nested MatchRule (sub-tree).

ConstraintRule + AbstractConstraint¶

A ConstraintRule wraps an AbstractConstraint with optional inversion:

Optimization¶

At compile time, the rule tree is optimized — rules that fold (their result no longer depends on runtime data, so they'd evaluate to the same thing on every view) are pruned so they aren't re-tested on every view.

Every node answers one question, fold(), returning a Fold(value, reason) — the constant it always evaluates to plus why — or UNFOLDED when the result still depends on the view:

SyntaxRule.fold()     → False if invalid, no syntax, no events (and not unrestricted), or no root_rule
MatchRule.fold()      → the match's own fold, e.g. some(5) with 3 children is a constant False
ConstraintRule.fold() → the constraint's fold, flipped by `inverted`

The reason travels with the value because only the folding node knows it: by the time the rule reaches the dropped-rules report, "never matches" is all that would be left to say. It is quoted verbatim there, after the verdict — e.g. never matches: no extension was given, or always matches: a "threshold" of 0 is met without finding anything — so phrase it for the user reading Debug Information, not for whoever reads the source.

Knowing which constant a rule folds to is the whole point: an empty all is always True (all([]) == True) while an empty any is always False (any([]) == False), and they are not interchangeable. prunable_child_value() tells a parent match which of those two values is safe to silently remove from its own child list without changing its result:

• Dropping a constant-True child from all(...) is safe (True is AND's identity element: all(True, x) == all(x)), but dropping a constant-False child is not — the whole all(...) folds to False instead, since all(False, x) is always False regardless of x (False is AND's absorbing element). Its reason quotes that child's, so the report still says why: never matches: one of its "all" sub-rules never matches: no extension was given.
• The mirror is true for any: dropping constant-False is safe, while a constant-True child makes the whole any(...) fold to True.
• ratio(n/d)'s prunable_child_value() is None — its goal (ceil(ratio * len(rules))) depends on the count of its children, so no child, constant or not, can ever be safely removed without shifting that goal.

Dropped rules are logged and stored next to the collection, in G.compiled_rules[window].dropped, for debugging.

A rule that can't be compiled at all — an unknown match/constraint name, or arguments its class rejects — fails its enclosing match, and so on up to the syntax rule, which is then dropped with the reason never matches: it contains an unknown or invalid match/constraint. The bad entry is never just removed on its own, since that would change what its parent means: an all whose only child is a typo would be left empty, and an empty all matches everything.

Before any of that, pref_syntax_rules() validates the settings one syntax rule at a time, source by source (core_syntax_rules, project_syntax_rules, user_syntax_rules, default_syntax_rules, in that order). A rule the schema rejects — an unknown key like "constrait", or a wrong type like "args": "py" — is dropped alone and reported with where it is, e.g. invalid setting at user_syntax_rules[3]: rules[0].constrait: Extra inputs are not permitted; every other rule still compiles. (Validating them as one list used to throw away every rule, the bundled ones included, over a single typo.)

Optimizing is best-effort: it walks the tree mutating it, so if a node's fold() misbehaves (in practice, a Custom Implementation that doesn't return a Fold), a half-finished pass would leave a rule whose root was already detached. compile_rules() catches that, logs the offending class, and rebuilds the collection from the settings to run unoptimized — every rule still tests correctly, it just gets re-evaluated on each view. One broken custom implementation doesn't cost the window the rest of the user's rules.

ViewSnapshot¶

Before the detector chain runs, a ViewSnapshot is created — a frozen snapshot of the view's state at that moment. It is everything a detector is allowed to know about the view:

• view: The sublime.View object
• content: Full text content
• first_line: First line of content
• char_count, line_count: Size metrics
• path_obj: Path() object (or None if unsaved)
• file_extensions: List of suffixes (e.g., .tar.gz → ['.tar', '.gz'])
• file_name, file_name_unhidden: With/without leading dot
• syntax: Current syntax object
• file_size: Size on disk in bytes, captured when the snapshot is taken (-1 if not on disk)
• settings: The window's merged plugin settings, so detectors and constraints don't look a window up
• trim_suffixes: The merged trim_suffixes setting, read from settings
• encoding: Encoding (defaults to UTF-8 for unsaved buffers)
• caret_rowcol: Cursor position (row, col) for edit-aware decisions
• content_bytes, encoding_py: Lazy-computed derivatives

Final Assignment¶

_apply() is the only place a decision reaches the view. It is a no-op when no detector decided, emits the decision's status_message if it carries one, then delegates to assign_syntax_to_view():

flowchart TD
    In["assign_syntax_to_view(view, syntax)"] --> Valid{"view.is_valid()?"}
    Valid -->|no| False["return False"]
    Valid -->|yes| Views["Get all views<br/>sharing the buffer"]
    Views --> Loop["For each view"]
    Loop --> Same{"Syntax already<br/>the same?"}
    Same -->|yes| LogKeep["Log: [ALREADY]<br/>skip"]
    Same -->|no| AssignST["view.assign_syntax(syntax)"]
    AssignST --> Flag["Set VIEW_KEY_IS_ASSIGNED"]
    Flag --> LogAssign["Log: old → new<br/>+ reason"]
    LogKeep --> Next["Continue loop"]
    LogAssign --> Next
    Next --> Done["return True"]

1. Validates the view
2. Gets all sibling views sharing the same buffer (via view.buffer().views()), so clones and split views stay in sync
3. For each view: skips if already has the target syntax, otherwise calls view.assign_syntax(syntax)
4. Sets VIEW_KEY_IS_ASSIGNED on view settings
5. Logs the change with full context (old syntax → new syntax + the decision's details)

Extensibility¶

Users can add custom AbstractMatch or AbstractConstraint implementations:

1. Create a Python file in Packages/AutoSetSyntax-Custom/matches/ or Packages/AutoSetSyntax-Custom/constraints/
2. Subclass AbstractMatch or AbstractConstraint
3. Implement test() (and optionally fold())
4. fold() returns Fold(constant, reason) — the constant test() would always return plus a user-facing explanation — or UNFOLDED when the result still depends on the view. Both constants are expressible, so an "always matches" implementation reports True and an "always fails" one reports False — getting this backwards inverts the rule's meaning. It has to be a real Fold: a bare False/None, or a plain (False, "…") tuple, is rejected.
5. To make test() give a verdict that inverted must not flip, raise AlwaysTruthyException or AlwaysFalsyException (from AutoSetSyntax.plugin.rules.constraint) instead of returning.
6. For AbstractMatch, also override prunable_child_value() if the match supports safely pruning individual children (see Optimization). The default is None: nothing is pruned, which is always correct. Name your combinator's identity element (False for any-like ones, True for all-like ones) to opt in; naming the wrong one silently changes the match's result.

1. The class name convention determines the setting name: FooBarMatch → "foo_bar" and BazConstraint → "baz"

Auto-discovered at plugin load via load_custom_implementations() (plugin/custom.py) using pkgutil.iter_modules(). A module that fails to import, for any reason (including a SyntaxError), is logged with its traceback and skipped; the plugin and the other custom modules still load. The custom package is dropped from sys.modules before each load, so every load executes it afresh: on a plugin reload boot.py re-imports only AutoSetSyntax.*, and a cached custom module's classes would still subclass the previous AbstractConstraint/AbstractMatch.

Magika Integration¶

Magika (Google's deep-learning file type detector) is an optional dependency:

def get_magika_object() -> magika.Magika | None:
    try:
        from magika import Magika
        from magika import PredictionMode
    except ImportError:
        return None
    return Magika(prediction_mode=PredictionMode.HIGH_CONFIDENCE)

• Downloaded separately; not vendored
• Uses HIGH_CONFIDENCE prediction mode (avoids false positives)
• Only invoked for extensionless plaintext files (unless triggered via command)
• Results mapped to ST syntaxes via magika.syntax_map settings

Settings¶

Uses AioSettings — an async settings watcher that:

• Merges per-project settings over plugin settings (a null value reads as unset)
• Merges project_, user_ and default_trim_suffixes into one trim_suffixes list (deduplicated, in that order), which ViewSnapshot.trim_suffixes reads
• Recompiles a window's rules when the plugin settings file changes or its project is loaded (on_load_project_async)
• Tracks per-window settings independently; a window is tracked by set_up_window() and forgotten by tear_down_window()

Key settings groups:

• Strategy control: new_file_syntax, exec_file_syntax, trim_suffixes, magika.*
• Behavior: debounce, run_on_startup_views, enable_log
• Rules: core_syntax_rules, project_syntax_rules, user_syntax_rules, default_syntax_rules — arrays of SyntaxRule objects, tested in that order

Shared Global State¶

@dataclass(frozen=True, slots=True)
class CompiledRules:
    collection: SyntaxRuleCollection  # The top-level plugin rules, optimized
    dropped: list[DroppedRule]        # Rules discarded on the way, with why

@dataclass(slots=True)
class _GlobalState:
    startup_views: set[sublime.View]               # Views existing at startup
    compiled_rules: WindowKeyedDict[CompiledRules] # Per window; having an entry == window is ready

G = _GlobalState() — a single shared instance holding per-window compiled state.