cfd-solver-gui — Developer Guide

The runtime sequence for a single solver run:

1. Config editor → namelist write. MainWindow._on_run() resolves the solver path, calls ConfigEditor.get_values(), and passes the flat values dict to LocalRunManager.start(). The run manager creates a durable run directory, rewrites snapshot_freq to max(1, print_freq) when the input value is 0, and writes input.nml there.
2. Namelist → worker spawn. LocalRunManager starts a helper process via QProcess. In bundled apps, entry.py checks for --run-manager-worker before importing any GUI modules and dispatches straight to run_worker.py. The worker writes run_manifest.json and events.jsonl, then launches the solver binary with input.nml as the sole CLI argument, using the run directory as the working directory.
3. Worker poll → structured events. The worker polls the configured snapshot file and checkpoint files. When a new complete snapshot is available it appends progress and snapshot_ready records to events.jsonl, updates run_manifest.json, and mirrors the same events over its stdout pipe.
4. Structured control plane → GUI updates. MainWindow listens to LocalRunManager.event_emitted. progress events drive the status bar and progress bar. snapshot_ready events trigger a fresh read of snapshot.dat for plot updates. Human log lines are forwarded separately via LocalRunManager.log_line and are appended to the log panel without semantic parsing. When CFD_GUI_RUN_TIMING=1 is present in the environment, the GUI and worker also emit diagnostic_timing checkpoints for startup-latency investigation.
5. Terminal event → final result. When the worker emits run_finished, run_failed, or run_stopped, MainWindow loads the configured output file from the run directory (if it exists), calls PlotCanvas.update_numerical() one last time, and restores the UI state.

The current architecture deliberately separates what is computed from how the GUI observes and controls the run.

Data plane:

• input.nml is the authoritative run configuration given to the solver.
• snapshot.dat, result.dat, and checkpoint files are the authoritative solver outputs.
• These files are valid without the GUI and are meant to remain useful for debugging, restart, and future remote/HPC workflows.

Control plane:

• LocalRunManager starts the worker and exposes Qt signals to the GUI.
• The worker emits structured JSON events live over stdout and persists the same events to events.jsonl.
• run_manifest.json stores the latest known run state so other tools can inspect a run without replaying the full event stream.

This split is the core architectural choice behind the current design:

• human-readable logs stay human-only
• machine-readable state is explicit and versionable
• solver artifacts remain inspectable even if the GUI is closed
• future remote or MPI backends can preserve the same control-plane contract

A typical run directory looks like:

<AppDataLocation>/runs/20260319-142530/
├── input.nml
├── run_manifest.json
├── events.jsonl
├── solver_stdout.log
├── snapshot.dat
├── result.dat
├── checkpoint_000100.bin
└── latest_checkpoint        # written by solver if checkpointing is enabled

Responsibilities by file:

• input.nml is written once before launch.
• run_manifest.json is rewritten in place on every major state transition.
• events.jsonl is append-only and is the durable event history.
• solver_stdout.log is the captured merged stdout/stderr stream from the solver process.
• solver-generated files remain exactly where the solver expects them relative to its working directory.

There are four runtime actors:

1. MainWindow owns the user interaction layer: form values, plot updates, status bar text, and log presentation.
2. LocalRunManager is the GUI-side controller: it creates the run directory, writes input.nml, launches the worker process, and translates the worker's JSON Lines stdout into Qt signals.
3. run_worker.py is the local orchestration process: it launches the solver, captures logs, watches snapshot/checkpoint files, and updates run_manifest.json / events.jsonl.
4. The solver remains a pure compute process that knows only about the namelist contract and its normal file outputs.

The GUI never talks to the solver process directly on the primary run path. That indirection is intentional: it gives the project one place to evolve future stop/reconnect/remote submission logic.

Each line in events.jsonl is a JSON object with:

• schema_version
• type
• timestamp
• run_id
• event-specific payload fields

Current event types and expected payloads:

Semantics:

• diagnostic_timing is emitted only when CFD_GUI_RUN_TIMING=1. The process field is either gui or worker. phase is a stable startup checkpoint label, offset_ms is elapsed wall-clock time from the shared startup origin, and details is optional extra context.
• progress and snapshot_ready are emitted only after the worker sees a complete, parseable snapshot file.
• log_line is not part of the machine-control contract; it is preserved only as display/debug text.
• terminal events are mutually exclusive; exactly one of run_finished, run_failed, or run_stopped should close a run.

The manifest state machine is intentionally small:

Rules:

• run_manifest.json is the latest snapshot of truth, not a historical log.
• events.jsonl is the historical log, not the authoritative latest state.
• The GUI should be able to rebuild its user-visible state from either the live event stream or the final manifest plus files in the run directory.

The current implementation is intentionally local only, but the contract is chosen so it can grow without forcing a GUI rewrite.

The intended migration path is:

1. Keep the GUI-side LocalRunManager interface stable.
2. Replace the local worker with a remote runner, scheduler adapter, or MPI launcher when needed.
3. Preserve the same high-level event types and manifest semantics, even if the backing transport changes from local QProcess stdout to a socket, WebSocket, or RPC stream.
4. Keep solver-produced artifacts file-based so post-processing and restart logic do not depend on the GUI process.

The important architectural boundary is therefore GUI ↔ runner, not GUI ↔ solver rank 0. That is what keeps the current implementation useful for future 2-D/3-D and cluster-oriented work.

namelist_io.py owns everything to do with the Fortran namelist contract: the Param dataclass, the PType enum, the SCHEMA list, the parser (parse_namelist, flat_from_namelist), and the writer (write_namelist). It deliberately knows nothing about Qt.

config_editor.py owns the tabbed Qt widget built from SCHEMA. It reads SCHEMA at construction time to generate one tab per group, one widget per parameter. Its only public methods are get_values(), set_values(), and reset_defaults(). It emits value_changed but does not act on it — callers decide what to do with parameter changes.

entry.py owns the first bootstrap decision for python -m cfd_gui and bundled app launches. It decides whether --run-manager-worker should go to the worker path before importing GUI modules.

run_manager.py owns the GUI-side structured run boundary. LocalRunManager creates durable run directories, writes input.nml, launches the helper worker process, and surfaces structured events back to the GUI.

run_worker.py owns worker-side orchestration: run_manifest.json, events.jsonl, stop requests, checkpoint discovery, and progress detection from complete snapshot files.

_run_util.py is a lightweight shared helper module imported by both run_manager.py and run_worker.py. It holds the diagnostic_timing event schema constants (_EVENT_SCHEMA_VERSION, _TIMING_ENV_VAR), JSON utilities, and _timing_enabled() / _timing_payload() so both processes emit identical event shapes. It has no Qt or subprocess imports.

solver_runner.py owns the QProcess lifecycle and nothing else. It does not know about namelists, file paths beyond what it is passed, or the GUI's structured run protocol. It remains as a low-level helper module, but the main window now uses run_manager.py for the primary run path.

snapshot_monitor.py owns the polling loop and the file format contract for snapshot.dat / result.dat. It also exports load_result_file() for one-shot loading (used by MainWindow on solver exit and on File → Load Result).

plot_canvas.py owns the matplotlib figure and its style settings. It knows which physical quantities to plot and how to derive internal energy from p, rho, and gam. It does not know about Qt signals or solver state. The "Exact" legend entry is shown only when update_exact() has been called with data; clear_exact() hides it by setting the line label to _nolegend_ and rebuilding the axes legends.

util/exact_sod.py is a pure-NumPy module with no Qt dependency. It implements the analytic exact Riemann solver via Newton iteration and is called by MainWindow._compute_exact() for Sod runs once t > 0. The public entry point is sod_exact(x, t, rho_L, u_L, p_L, rho_R, u_R, p_R, gam, x0), which returns (rho, u, p) arrays at time t.

main_window.py is the orchestrator. It constructs all other components, wires signals to slots, owns the toolbar/menu actions, and drives the run lifecycle. Business logic that spans multiple components lives here.

settings.py owns QSettings access. All reads/writes of persistent application state go through its helper functions — no other module calls QSettings directly.

This table is the primary update target whenever a signal on the main run path is added, removed, renamed, or its parameter types change.

Notes:

• LocalRunManager.event_emitted is typed as Signal(object) because the payload is a plain Python dict decoded from JSON Lines.
• ConfigEditor.value_changed is defined for subclasses or external callers that want per-keystroke notification. MainWindow reads the editor synchronously via get_values() instead.

SCHEMA in namelist_io.py is a list[Param]. The Param dataclass:

@dataclass
class Param:
    key: str           # Fortran variable name (lowercase)
    group: str         # Namelist group, e.g. 'grid', 'schemes'
    ptype: PType       # INT | FLOAT | BOOL | CHOICE | STRING
    default: ScalarValue
    label: str         # Short GUI label
    tooltip: str = ""
    choices: list[str] = field(default_factory=list)  # CHOICE only
    min_val: float | None = None   # INT/FLOAT only
    max_val: float | None = None   # INT/FLOAT only

PType → widget mapping in ConfigEditor._make_widget():

GROUPS is derived as list(dict.fromkeys(p.group for p in SCHEMA)), which preserves insertion order and deduplicates. Tab order in the editor follows GROUPS order. _GROUP_TITLES in config_editor.py maps group names to human-readable tab titles:

1. Add a Param(...) entry to SCHEMA in namelist_io.py inside the appropriate group block (maintain the # fmt: off / # fmt: on region).
2. If it is PType.CHOICE, ensure the Fortran solver's namelist parser accepts all values in choices. Order the list with the recommended default first if possible, then specify default= to match.
3. If a new namelist group is needed, choose a group name matching the Fortran &group_name and add it to _GROUP_TITLES in config_editor.py. GROUPS updates automatically from SCHEMA — no manual edit needed.
4. If the parameter must always appear in written namelists regardless of its default value, add its key to the sentinel tuple in write_namelist().
5. Run the full test suite: pytest. TestSchemaIntegrity (in test_namelist_io.py) catches structural mistakes automatically (missing choices list for CHOICE type, out-of-range defaults, etc.).
6. Update the parameter reference table in this guide with a new row.
7. If the parameter is user-visible (appears in a config tab), update the relevant tab section under Configuring a Simulation in doc/user-guide.org.

This table must be kept in sync with SCHEMA in namelist_io.py.

Choice lists not shown inline above:

• flux_scheme: lax_friedrichs, steger_warming, van_leer, ausm_plus, hll, hllc, roe
• recon_scheme: upwind1, upwind2, central2, muscl, eno3, weno5, weno5z, weno7, mp5, teno5, linear_weno
• time_scheme: euler, ssprk22, rk3, rk4, ssprk54, beuler, bdf2
• limiter: minmod, superbee, mc, van_leer, koren
• problem_type: sod, shu_osher, smooth_wave, linear_advection, woodward_colella, lax, acoustic_pulse, from_file, udf
• bc_left / bc_right: dirichlet, inflow, outflow, reflecting, periodic, nonreflecting

1. Remove or rename the Param entry in SCHEMA.
2. Check the sentinel tuple in write_namelist() — remove the key if present.
3. Check tests/conftest.py fixtures for hard-coded key names (sample_nml_text uses flux_scheme, recon_scheme, time_scheme, problem_type, output_file, snapshot_freq).
4. Search for the key string across the test files: pytest tests that assert specific values may reference it by name.
5. Update the parameter reference table in this guide.
6. If user-visible, update the relevant tab section under Configuring a Simulation in doc/user-guide.org.

parse_namelist() is regex-based and handles scalar values only (no arrays, no continuation lines).

Processing steps:

1. Strip comments: re.sub(r"!.*", "", text). This is applied to the entire file before group matching.
2. Match namelist groups: re.finditer(r"&(\w+)(.*?)/", text, re.DOTALL). Group name is lowercased.
3. Split group body on , or newline; partition each token on =.
4. Cast the raw value string via _cast_value().

_cast_value() type inference order (first match wins):

1. Boolean: raw (lowercased, stripped of trailing comma) in {".true.", ".t.", "true", "t"} or {".false.", ".f.", "false", "f"}.
2. Fortran string: wrapped in single or double quotes → strip quotes, return str.
3. Float: raw contains ., e, or d (case-insensitive) → replace d/D with e/E, call float().
4. Integer: int() on the raw string.
5. Fallback: return raw string unchanged.

Known limitations:

• Array literals (a = 1, 2, 3) are not supported.
• Continuation lines (& at end of line) are not supported.
• Group name collision: if two &grid groups exist, the second overwrites the first (groups[name] = entries unconditionally).

write_namelist(values, path) takes a flat {key: value} dict and writes a grouped namelist.

Logic:

1. Compute defaults from defaults_dict().
2. For each Param in SCHEMA, include (key, value) in the output group only if value ! default=.
3. Additionally, always include the sentinel keys (regardless of value): n_cell, time_stop, flux_scheme, recon_scheme, time_scheme, problem_type, output_file. This produces self-documenting namelists.
4. Groups with no entries to write are omitted entirely.
5. Group order follows GROUPS (derived from SCHEMA insertion order).

Float formatting in _format_value():

• If abs(val) < 1e-3 or abs(val) > 1e6= (and val ! 0=): scientific notation with Fortran d exponent (e.g. 1.234567d-05).
• Otherwise: %g (shortest representation without trailing zeros).

Safe changes (no risk of breaking existing namelists):

• Widening the comment pattern to also strip !.. comments starting after non-whitespace (currently the whole line after ! is stripped, which is already correct Fortran behaviour).
• Adding tolerance for extra whitespace.

Potentially breaking changes:

• Altering the _cast_value inference order: code that currently produces an int might produce a float, or vice versa.
• Changing string-quote stripping to also handle """ would be safe only if the solver never produces """ in variable values.

The primary run path is now a two-stage launch:

1. LocalRunManager.start() writes input.nml and launches a helper worker via QProcess. The worker is either the bundled executable re-entered with --run-manager-worker or run_worker.py when running from source.
2. run_worker.py launches the solver itself with subprocess.Popen([binary, nml_path], cwd=run_dir, stdout=PIPE, stderr=STDOUT, stdin=DEVNULL, text=True, bufsize=1).

LocalRunManager keeps worker stdout and stderr separate at the QProcess boundary because stdout carries structured JSON events and stderr is reserved for worker-launch failures. The worker then merges the solver's stdout/stderr into a single captured stream and mirrors human log lines back as log_line events.

solver_runner.py remains in the repository as a low-level QProcess wrapper, but it is not the primary integration path for normal GUI runs.

On the primary run path, progress comes from the snapshot file rather than from solver stdout parsing.

run_worker.py polls snapshot.dat every 100 ms and emits progress / snapshot_ready only when all of the following are true:

1. the snapshot path exists
2. stat().st_mtime advanced
3. the file ends with a newline and every data row has exactly four floats
4. the header line matches iter / t metadata
5. the iteration number differs from the last emitted snapshot

Stop behaviour is also worker-mediated:

1. MainWindow calls LocalRunManager.stop().
2. LocalRunManager writes { "type": "stop" } to the worker stdin.
3. The worker marks the manifest stopping, emits stop_requested, and calls proc.terminate() on the solver.
4. If the solver is still alive after 3 seconds, the worker calls proc.kill().

Each run creates an isolated directory:

run_dir = create_run_directory()

The worker launches the solver with this as CWD, so relative paths in the namelist (e.g. output_file = 'result.dat') resolve inside the work directory without collision between concurrent or successive runs.

The run directory is not cleaned up automatically after the run finishes. Its path is printed to the log panel so the user can inspect or copy result files. The same directory also holds run_manifest.json and events.jsonl, so the full run record is preserved in one place.

The primary run path now polls snapshots inside run_worker.py. The polling logic intentionally mirrors the old SnapshotMonitor design because it is portable and robust. SnapshotMonitor remains available as a standalone Qt helper for direct file watching and for tests covering the file-format parser.

Change detection uses two independent gates (both must advance for a new snapshot_ready event):

1. mtime gate: the current stat().st_mtime must advance. This prevents re-parsing an unchanged file.
2. Iteration dedup: the parsed iteration number must differ from the last emitted snapshot. This prevents emitting the same logical snapshot twice if the file is touched but the solver has not yet advanced an iteration.

Why not inotify/FSEvents/kqueue? Portability. Polling works identically on Linux, macOS, and Windows without platform-specific code or optional dependencies.

class SnapshotData:
    __slots__ = ("iteration", "time", "x", "rho", "u", "p")

__slots__ is used for memory efficiency (the object is created on every successful poll). Internal energy is not stored in SnapshotData — it is derived in PlotCanvas.update_numerical() from p, rho, and gam at render time. This keeps the snapshot format faithful to the file and avoids coupling the monitor to the physics value of gam.

LocalRunManager.start() contains:

if values.get("snapshot_freq", 0) == 0:
    values["snapshot_freq"] = max(1, int(values.get("print_freq", 50)))

Rationale: the live plot is a core feature of the GUI. If the user (or a loaded namelist) leaves snapshot_freq at 0, the run manager silently enables snapshot output for that run by matching print_freq. The override only affects the namelist written for the current run; the editor widget is not modified.

PlotCanvas is a QWidget that contains a FigureCanvasQTAgg (the matplotlib figure) and a NavigationToolbar2QT (zoom, pan, home, back/forward).

The _RC dict in plot_canvas.py mirrors the RC settings from the solver's post/_common.py script to ensure GUI plots have the same visual style as offline post-processed figures. Key choices: sans-serif fonts, 8 pt base, no top/right spines, inward ticks, pdf.fonttype=42 (Type 1 embedding).

Two sets of Line2D handles are pre-created at construction time:

• _lines_exact: blue (#0077BB), solid, line only. Exact/reference solution.
• _lines_num: orange (#EE7733), no line, circle markers (ms=2). Numerical solution.

_FIELDS defines the four panels in order: (ρ, Density), (u, Velocity), (p, Pressure), (e, Internal energy).

Internal energy is not in the data file; it is computed in both update_numerical() and update_exact():

e = np.where(rho > 0, p / (rho * (gam - 1.0)), 0.0)

gam is read from ConfigEditor.get_values() at each update (in MainWindow before calling update_numerical()), so the panel stays physically meaningful even if the user changes gam between runs. The np.errstate guard prevents divide-by-zero warnings when rho cells are zero (e.g. initial empty boundaries).

To add a fifth panel (e.g. Mach number):

1. Add a tuple to _FIELDS in plot_canvas.py: (ylabel, title).
2. Change self._fig.subplots(2, 2) to self._fig.subplots(2, 3) (or subplots(3, 2)) and update the layout as needed.
3. Append a new line handle in the loop that creates _lines_num and _lines_exact.
4. Add the derived quantity to update_numerical() and update_exact(), inserting it at the correct index in the zip call over _lines_num.
5. Update matplotlib Embedding in this guide and Live Plot Updates in doc/user-guide.org.

pytest-qt sets QT_QPA_PLATFORM=offscreen automatically when DISPLAY is unset, so Tier 2 tests run headless in CI without xvfb-run.

When to add to each tier:

• Tier 1: any test that does not construct a QWidget or QApplication. Namelist round-trips, schema validation, snapshot file parsing, mtime logic (with time.time() monkeypatching).
• Tier 2: any test that constructs a ConfigEditor, PlotCanvas, or MainWindow, or that asserts on signal emission (use qtbot.waitSignal).

Fixtures defined in tests/conftest.py:

Note: sample_nml_text uses time_scheme = 'rk2' which is not in the current SCHEMA choices list. This is intentional — it tests that the parser handles unknown choice values gracefully (they are stored as strings and trigger no error at parse time; only ConfigEditor.set_values() silently ignores unknown values).

qtbot is provided automatically by pytest-qt; it does not appear in conftest.py.

• Schema integrity: covered automatically by TestSchemaIntegrity in test_namelist_io.py (checks that every CHOICE param has a non-empty choices list, every FLOAT param with min_val has default > min_val=, etc.).
• Round-trip (writer → parser): add a test in TestWriteNamelist that sets the new value, writes a namelist, parses it back, and asserts equality.
• Widget ↔ value contract: add a @pytest.mark.qt test in test_config_editor.py that constructs a ConfigEditor, calls set_values({key: value}), then get_values(), and asserts the round-trip.

mypy is run in strict mode for pure-Python modules (namelist_io, snapshot_monitor except the signal-wired parts, settings). Qt widget modules (config_editor, plot_canvas, main_window) have ignore_errors = true in pyproject.toml due to the limitations of PySide6 stubs.