Image Calibration & Stacking

Workflow overview

The four WStack page buttons filter one observation directory by processing stage. Context-specific controls appear above the preview, so calibration actions are shown on Raw, astrometry controls on Calib, selection and stacking on Astro, and measurement actions on Stack.

Working directory and local state

WStack always opens with a blank session. Click Open directory… and choose the observation directory. A previous directory is used only as the next file-dialog starting point; it is not scanned automatically at application startup.

observation-directory/
├── source FITS and optional physical DSLR RAW files
├── *_TG.fits materialized automatically from DSLR RAW sources
├── pipeline_state.json
├── calibrated/
│   ├── *_CA_DA.fits
│   ├── *_CA_FL.fits
│   └── *_CA_DA_FL.fits
├── astrometry/
│   └── *_WCS.fits
├── ST_...fits
└── CO_...fits

• The root directory may contain ordinary FITS and physical DSLR RAW sources, but the Raw-page working index contains FITS paths only. The page name Raw identifies the first workflow stage; it does not mean that physical RAW files are processed directly.
• calibrated/ contains materialized calibrated FITS products.
• astrometry/ contains newly solved FITS products with WCS and QC metadata.
• pipeline_state.json stores independent Raw→Calib, Calib→Astro and Astro→Stack ON/OFF choices, path relationships and per-group offline astrometry automatic/manual override state. R69 migrates historical physical RAW paths to verified TG FITS paths before session reconciliation and merges duplicate RAW/TG records while preserving the historical ON/OFF cascade.
• Qt QSettings stores the Astrometry.net API key, standalone optical fallbacks and validated stack-dialog preferences. Comet and Stars preferences use separate wstack/comet and wstack/stars namespaces.

If the selected directory is itself named calibrated or astrometry, WStack resolves the parent observation directory. When no output library has been configured, opening a session defaults the calibration library to observation-directory/masters/.

Supported image formats

The materialized TG image is not demosaiced and receives no white balance, gamma, highlight recovery or aesthetic processing. Its FITS metadata uses filter TG, 2×2 binning and, where available, exposure, ISO, timestamp, instrument and camera white-level information.

Physical RAW data are not decoded during discovery and never enter _session_files. Frame-type inference and every downstream operation work on the verified FITS representation.

DSLR RAW → TG FITS materialization

Directory discovery runs before WStack changes the currently open session. When at least one pending or stale DSLR source is an extraction candidate and rawpy/LibRaw is available, WStack immediately starts the materialization batch. No confirmation or bypass dialog is shown.

If rawpy/LibRaw is unavailable, KOPR shows DSLR RAW extraction unavailable. Existing FITS products may still be opened; when the directory has no usable FITS, the previous session remains unchanged. Conflicting, unreadable or unrelated target FITS are not extraction candidates and are reported in a separate warning.

Progress and cancellation

Extraction runs outside the GUI thread. The modal progress dialog shows the current RAW file, item/total count, progress bar and cumulative Extracted, Refreshed, Skipped and Failed counts. Cancel is cooperative between files: the file currently being decoded completes safely, already verified FITS products remain, and unprocessed RAW sources are offered again next time the directory is opened.

FITS-only session reconstruction

After extraction, WStack rescans the directory, verifies provenance and source fingerprints, migrates historical state paths and rebuilds a FITS-only working index. It preselects the first newly created or refreshed TG FITS; when no extraction was required because every representation was already valid, it preselects the first verified green representation. Physical RAW paths are never inserted into preview, calibration input selection, astrometry or stacking.

Direct representation identity

A newly materialized direct green representation carries RAWREP=T. Calibrated and astrometric derivatives carry RAWREP=F. Final stacks remove the single-source identity cards RAWREP, RAWFILE, RAWEXT, RAWMODE, RAWVERS, RAWSIZE and RAWMTIME; their complete multi-frame ancestry remains in HISTORY, REFIMAGE, NCOMBINE and the PROVENANCE extension.

Discovery treats historical RAWFILE/RAWEXT on a calibrated, astrometric or stack product as ancestry, not as a competing direct representation. Existing valid _TG.fits files without RAWREP remain supported. Two genuine direct representations still produce a blocking conflict.

Scientific and Compact output

WStack calculations remain floating point. The setting at Settings › Processing › FITS output format controls only the final primary science HDU written for new masters, calibrated LIGHT frames and final stacks.

Compact is accepted only when its quantization step is no more than one quarter of the robust local pixel-noise estimate. Otherwise that individual output is written as Scientific. No clipping or coarser hidden quantization is used.

The mode is captured at task start, so changing Settings during a running batch does not mix settings within that task. See Scientific and Compact FITS output.

Raw page controls

Raw-page rows are FITS files only, including verified _TG.fits products materialized from DSLR sources. Raw, Calib and Astro file rows have persistent ON/OFF checkboxes. Turning a Raw file OFF excludes it from calibration; turning a Calib file OFF excludes it from astrometry; turning an Astro file OFF excludes it from stacking. The Stack page contains final products and therefore has no processing checkbox.

Files are grouped visually by image type, filter, exposure and dimensions. Selecting a file loads only that image into the preview; the session index otherwise keeps paths and compact header summaries rather than all pixel arrays in memory. Autoscale (1–99 %) controls preview display only.

Calibration master library

Manage masters… opens Dark, Flat and Summary tabs. New files are written to masterdarks/ and masterflats/ below the chosen library root, but the library scan is recursive so compatible older or manually organized masters elsewhere below the root remain visible.

Master dark, dark-flat and bias

1. Select the directory containing the dark-like source set.
2. Use Check only (dry run) first. It performs the complete classification, compatibility grouping and frame QC but writes no master image.
3. Review rejected-frame causes, skipped-group causes, eligible groups and CSV report paths.
4. Use Create master dark… with the same or revised parameters.

Compatibility groups include calibration type, exposure and temperature bins, dimensions, binning, camera, gain, offset and readout mode. Acquisition date is not a compatibility criterion. Small populations of hot or clipped pixels are retained and reported; whole-frame rejection is reserved for gross corruption and robust frame-level outliers.

Master flat

Flat groups additionally require a matching filter. A suitable bias, dark-flat or dark correction may be selected from the same library before each flat is normalized and combined. Isolated clipped pixels are masked; broad clipping or unsuitable ADU level can reject the entire frame. The final master flat is normalized to median 1 and invalid/non-positive response pixels are stored as NaN and reported.

Reports and import

Each build writes frame-level and group-level CSV reports in the respective master subdirectory. The result dialog summarizes the main reasons and exposes full details. Existing FITS masters can be imported after confirming the metadata needed for matching; imported files are copied into the library and marked with MDARK or MFLAT.

Calibrate light frames

Calibrated LIGHT products may be written as Scientific or Compact, but dark subtraction, flat division, invalid-pixel handling and QC are performed in floating point before the final writer is called.

On Raw, switch ON the LIGHT files to process and click Calibrate frames…. The dialog offers automatic master selection or an explicit master dark/flat from the library.

Calibration subtracts the selected additive master and divides by the normalized flat. A small invalid area in an older dark master is repaired with a local median and reported. Invalid or zero flat pixels are masked as NaN; a master flat with an excessive invalid fraction is rejected.

Outputs are placed in calibrated/ with suffixes such as _CA_DA, _CA_FL or _CA_DA_FL. The header records CALSTAT, source file, selected master names, applied corrections, repaired/masked pixel counts and UTC calibration time. A processing_manifest.json records processed, excluded and skipped frames.

Astrometry

On Calib, keep the intended calibrated frames ON. WStack solves enabled frames locally with solve-field, creates a separate astrometry/<stem>_WCS.fits product for each successful frame and applies the configured group-failure policy when a required anchor cannot be solved.

Execution order

1. validate selected CALIB inputs and optional seed fields;
2. resolve a usable solver backend;
3. discover usable 4100/4200 index families and, when required, download the established 4206–4214 set;
4. start the astrometry worker with the already validated backend.

The WStack log reports the selected backend before index handling, for example:

[solve] solver preflight: native solve-field (...)
[solve] solver preflight: WSL solve-field (/usr/bin/solve-field)

Solver backend priority

1. a native solve-field executable on PATH;
2. when KOPR is running natively on Windows, solve-field in the default WSL distribution;
3. the distribution explicitly named by KOPR_WSL_DISTRIBUTION, when set.

The implementation does not automatically search every installed WSL distribution. Use the exact distribution name from wsl --list --verbose when an override is required.

Offline automatic position and scale

Click Astrometry to review the offline group parameters. The dialog displays representative automatic RA/DEC and scale limits. Editing both RA and DEC creates an explicit manual position override; merely opening and accepting the preloaded automatic values does not. Editing both scale limits creates an explicit manual scale override; clearing both scale fields restores the automatic chain.

Entering a comet designation can preview the internal position for the selected Calib frame or the temporal middle of the enabled batch, but automatic mode recomputes the ephemeris independently at every exposure midpoint. The search radius covers the frame half-diagonal plus a conservative margin and never drops below 0.25°.

Blind online astrometry

The online action asks only which group scope to process and uploads each requested FITS as a true blind nova.astrometry.net job. KOPR sends no local centre, radius, scale, parity or downsampling constraint. The offline astrometry-parameter dialog is not opened for this path.

After a blind online job succeeds, the returned field centre and pixel scale are stored as trusted information for the subsequent local solve. While the server job is queued or solving, KOPR prints periodic status heartbeats. It requests the calibration endpoint only after the job reports status=success, avoiding repeated misleading no calibration data available messages.

Three-stage offline solve strategy

Each frame has at most three clearly identified attempts:

1. Attempt 1: the original calibrated FITS with its normal per-frame or adaptive seed.
2. Attempt 2: a temporary robust solver copy, the frame's original constraints and --downsample 2.
3. Attempt 3: the same robust copy with the broad radius/scale fallback.

If an adaptive seed fails, Attempt 2 also returns to the frame's original independent seed. This removes both a possibly incorrect adaptive centre and isolated source-extraction contamination without exceeding three attempts.

Temporary robust solver input

Before Attempts 2–3, KOPR can create a temporary FLOAT32 solver copy inside the isolated solver work directory. A conservative 3×3 local-median impulse classifier replaces only isolated positive or negative extremes; neighbouring multi-pixel stellar profiles are retained. The calibrated source FITS is never modified.

Concise terminal output, structured results and full per-attempt logs report the solver-input mode, effective downsampling and number of replaced impulses.

Anchor failure policy

If a required offline group anchor fails after all three attempts, the default policy stops automatic solving for the remaining frames in that group. They receive SKIPPED_ANCHOR_FAILED; the terminal shows one [FAILED] anchor line, subsequent [SKIPPED] lines and a group summary. Files and group settings remain intact.

The group dialog opened by Astrometry provides a per-group advanced choice:

• Stop and skip remaining frames — recommended default for a homogeneous series.
• Continue with independent seeds — explicit override for genuinely heterogeneous groups.

Index discovery and diagnostics

KOPR discovers application-local astrometry-index/4100 and astrometry-index/4200 families, including sibling families when a direct custom family path is supplied. Terminal diagnostics include:

[INDEX] available=...
[INDEX] recommended_for_wide_fields=4107–4112,4206–4212

The Tycho-2 4100 family is optional. Its absence produces a warning but does not block the established 4200 workflow. Automatic downloading remains unchanged and installs 4206–4214.

Windows/WSL path and cancellation handling

For a WSL backend, input, work-directory, WCS/AXY/config and application-local index paths are translated with wslpath. The generated configuration uses WSL-visible paths and LF line endings; results remain in the Windows working directory. FITS provenance identifies the backend as solve-field/WSL.

When Cancel is pressed, KOPR does not merely terminate the Windows wsl.exe launcher. The solver runs in an isolated Linux process group when setsid is available, publishes its Linux PID through a temporary host-visible file, receives TERM, and after up to three seconds receives KILL if it has not exited. The complete WSL distribution is not terminated.

Validation after solve

• The resulting header must contain a valid two-dimensional celestial WCS.
• Image-center coordinates, scale and inverse pixel/world round-trip must be finite and geometrically stable.
• Where a position seed was required to fall on the image, the solution must remain compatible with that search geometry.
• The AXY source list is measured for star count, FWHM and elongation. A missing AXY list is logged as a diagnostic warning rather than silently inventing quality values.

Pixel storage is preserved

Creating _WCS.fits copies the complete source FITS and updates only WCS/SIP/QC metadata without rescaling raw pixel storage. Scientific remains Scientific and Compact remains Compact with the original BSCALE, BZERO, BLANK and QSTEP; no second int16 quantization is performed.

R76 shared physical scale and light-time

KOPR uses one physical plate-scale priority across the relevant astrometric and measurement paths: a solved WCS or trusted successful online calibration; an explicit manual override; credible FITS scale metadata; and finally active camera/telescope optics with validated observation binning. Binning belongs to the observation image and must not be inferred from a permanently edited camera pixel size.

Internal comet positions use the available DE440s path with the down-leg light-time correction. Comparisons with Guide, Tycho or JPL Horizons are diagnostic checks on selected objects and epochs; the documentation does not impose a universal external-program arcsecond pass/fail limit.

Post-astrometry frame selection

After astrometry, Frame selection… displays one row per solved frame with:

• include/exclude checkbox;
• file name;
• sky median in ADU;
• median stellar FWHM and its robust session z-score;
• median elongation and its robust session z-score;
• number of measured stars.

Green cells indicate better-than-median metrics, orange values are more than 2σ from the session median and red values more than 3σ. FWHM ≥ 6 px and elongation ≥ 1.5 also receive an absolute warning highlight. These colors are decision support; the observer controls inclusion.

Save and close writes the choices into the local pipeline state. Include all and Exclude all are available for batch changes. Stack creation uses only Astro ON files.

Create comet and star stacks

Click Stack comet… on Astro. The default output mode creates a matched comet + stars pair from one shared input group and WCS preflight. A comet-only output can be selected when no separate star-aligned stack is required.

Enabled files are divided into compatible groups by object, filter, camera, binning, gain, offset, readout mode, dimensions and WCS scale. Mean and sigma-clipped mean additionally require matching exposure time. Separate filters therefore produce separate stack groups rather than being mixed or stopping the whole batch.

Comet stacking uses KOPR's internal local-element ephemeris at every exposure midpoint and the saved topocentric observer longitude, latitude and elevation. The active KOPR location is preselected, but another saved location can be chosen for this stack without changing the global KOPR selection. The designation is normalized and must be available in the local orbital-element files.

Each input is reprojected once with bilinear WCS mapping and pixel-area flux scaling. There is no background normalization or quality weighting. The default output keeps the reference-frame dimensions; pixels without valid coverage remain NaN and their actual contribution counts are stored separately.

One-pass Stack Parameters preflight

R75 performs one explicit Stack Parameters preflight before stack execution. Input compatibility and required stack conditions are resolved before processing begins; the preflight is not a second scientific combination stage and does not change the output pixels.

Relative WCS quality control before stacking

A mathematically valid individual WCS can still be inconsistent with the rest of a series. Before stacking, WStack compares solved star positions across the group and selects a consensus reference. Current acceptance thresholds are:

• at least 20 uniquely matched stars;
• match fraction at least 0.20;
• median residual no more than 0.70 px;
• 90th-percentile residual no more than 1.50 px;
• nearest-neighbour search radius 5 px.

Rejected WCS frames are removed before the scientific minimum is checked. If more than 25% of the requested series is rejected, WStack asks for explicit confirmation before continuing. Rejected files are then saved as Astro OFF for the session.

R81 final stacks deliberately omit direct single-RAW identity cards such as RAWREP/RAWFILE/RAWEXT. This prevents a CO/ST stack from being rediscovered as another direct representation while retaining complete ancestry in HISTORY, REFIMAGE, NCOMBINE and PROVENANCE.

Stack headers record NREQUEST, actual NCOMBINE, WCSREJ and accepted-frame diagnostics WCSNACC, WCSFMIN/MED/MAX and WCSMMIN/MED/MAX. The same summary and per-frame rejection details are retained in provenance.

Stack names, headers and extensions

Final products use deterministic names:

ST_<object>_<filter>_<effective-exposure>s_N<used-count>.fits
CO_<object>_<filter>_<effective-exposure>s_N<used-count>.fits

A second compatible group that would otherwise produce the same name receives a _G2, _G3… suffix during the same batch. Existing files with the selected final name are overwritten.

For sigma-clipped mean, NVALID and NSATVAL are post-clipping counts; NCONTRIB and NSATCON are pre-clipping counts. The current implementation performs exactly one clipping pass and leaves partial-coverage output pixels as NaN.

Actual storage mode

KOPRMODE, STORFMT, BITPIX and QSTEP describe the product actually written. A Compact request that fell back to Scientific is recorded through REQMODE, QSAFE and QFALLBK together with the noise-policy diagnostics.

Time contract

Timing is derived from accepted inputs, not copied from the reference frame. DATE-BEG is the first accepted start, DATE-END is the last accepted end, and DATE-OBS/DATE-MID/DATE-AVG are the exposure-weighted midpoint. TELAPSE includes gaps, TIMEALGO='EXP-WMID', and TIMECOMP reports whether all accepted frames had complete timing.

Exposure contract

NCOMBINE counts actual inputs, TOTEXP/XPOSURE sum their exposure times and EXP_NR is the compatibility alias of NCOMBINE. For MEAN and sigma-clipped mean, EXPTIME represents one input exposure; for SUM it equals the summed exposure.

Automatic performance limits

R22 bounds parallel work automatically. The limits are not user settings and should not be added to the WStack settings dialog.

Small workloads, one-CPU systems, insufficient RAM or scheduler initialization failure automatically use a serial path. A serial run is therefore not evidence that parallel processing is broken.

See the complete R22 performance and platform reference.

Stack page and measurement hand-off

The Stack page groups final files by their shared provenance and presents the object, filter, total/effective exposure, frame count, method and composition. A matched measurement group normally contains one comet-aligned file and its matching star-aligned file; a comet-only product can still be opened for manual work.

• Measure opens the ordinary New CCD/DSLR observation dialog with the selected comet stack and optional matched stars stack prefilled.
• The observer can correct the comet designation or paths and choose Automatic Comet Finder, Auto Star Measure and optional JPL position before accepting the observation.
• A comet-only stack remains available to the same manual WCCD workflow.

Pair validation checks shared object, filter, method, frame count, exposure, dimensions, source-frame identity, provenance schema and required extensions. Ambiguous or damaged groups remain visible with a diagnostic tooltip but are not silently treated as valid pairs.

Common problems and responses

Every long-running master, calibration, astrometry and stacking operation uses a progress dialog and cooperative cancellation. Do not close the master-library dialog or force-terminate KOPR while its worker is still shutting down.

Recommended reduction checklist

• Back up the source observation directory before the first reduction.
• When opening DSLR data, confirm that the extraction progress starts automatically for pending/stale sources and that only verified TG FITS appears on the Raw page.
• Verify active KOPR location, telescope and camera.
• Set or confirm the calibration master library.
• Run dry checks before creating new masters.
• Inspect rejected frame and skipped group reasons.
• Calibrate only Raw ON LIGHT files.
• Confirm object designation and independent per-frame astrometry seeds.
• Confirm that solver preflight reports the intended native or WSL backend before any index installation.
• On Windows with a non-default distribution, verify KOPR_WSL_DISTRIBUTION.
• Review FWHM, elongation and star counts before stacking.
• Investigate every relative-WCS rejection.
• After cancelling a Windows/WSL solve, confirm no orphaned solve-field remains if shutdown did not complete normally.
• Before clicking Measure, confirm that the selected Stack group contains the intended comet product and, when available, its matching stars product.
• For a homogeneous series, keep the default Stop and skip remaining frames policy unless there is evidence that independent seeds are appropriate.
• When an anchor fails, compare all three per-attempt lines/logs and note the robust-copy impulse count before changing seed or index settings.
• Treat a missing 4100 family as optional; confirm that the required 4200 indexes are available before diagnosing a solver failure.

Related chapters

Configuration · Astrometry.net installation · CCD photometry · Data formats · Troubleshooting · Scientific / Compact output