Testing and validation

Three validation levels

Recommended commands

Run from the directory containing kopr.py with the KOPR environment activated.

# Syntax of the principal modules
python -m py_compile kopr.py wstack.py wstack_common.py wstack_functions.py   wccd.py autophot.py photom.py koprfunc.py

# Correct KOPR regression target
python -m pytest -q tests

# Targeted Windows/WSL solver contract
python -m pytest -q tests/test_windows_wsl_astrometry_solver.py

# Static Autophot/WCCD build contract
python verify_autophot_build.py

# R22 platform-neutral performance/release contract
python3 validate_performance_release.py
python3 validate_performance_release.py --json

# R22 performance and multiplatform regression contract retained by R27
python -m pytest -q tests/test_performance_release_batch10.py

# R27 photometry release contract
python3 validate_photometry_release.py
python -m pytest -q tests/test_photometry_release_batch4.py

Do not use an unqualified repository-root pytest command as the only release command when unrelated nested projects are present. Target the maintained KOPR tests/ directory.

Current automated test families

R74 adds a focused 58-test gate for automatic DSLR discovery, materialization, session rebuilding, preview selection and WCCD/WStack hand-off. The last complete deterministic baseline remains R73 with 1125 passed and 3 environment-gated skips. A target desktop is still required for real PyQt5 rendering, complete Astropy FITS I/O and rawpy/LibRaw decoding.

R79 cumulative release gate

tests/test_r79_cumulative_release.py contains 13 tests for current release identity, production parsing, automatic RAW materialization, R75 guards, R76 identity/binning/scale/light-time wiring, the strict 0.150-mag global pairwise boundary, lossless AutoStars/Manual Stars parity and the unified Reference stars/Process obs./Af-rho solution.

Scientific and Compact validation

Use the production validator rather than only inspecting header keywords:

python validate_scientific_compact.py \
  --json validation.json \
  --csv validation.csv \
  --markdown validation.md

python validate_scientific_compact.py \
  --compare scientific.fits compact.fits \
  --json pair-validation.json

python validate_scientific_compact.py \
  --source input.fits \
  --output-dir validation-products \
  --keep-files

Optional --center, --aperture-radius, --sky-inner and --sky-outer arguments add a fixed-aperture storage comparison. Required report evidence includes requested/actual mode, fallback reason, QSTEP, local noise, safety ratio, max/RMS/median pixel deviation, optional flux/magnitude/background/centroid differences, primary payload/full FITS size and real-file read/write timings.

Release-blocking cases

• normal coma/star field accepted as Compact with bounded decoded error;
• normalized master flat accepted with sub-response-scale error;
• extreme dynamic range triggers exact Scientific fallback;
• NaN/inf map to BLANK without colliding with valid pixels;
• WCS header update preserves Compact scaling and extensions without requantization;
• manual WCCD results remain consistent across supported integer and float FITS;
• native Scientific and Compact pairs pass the same automatic contract;
• external FITS does not receive invented total exposure or automatic-pair provenance.

Release-blocking scientific invariants

• Manual WCCD and retained shared ACF primitives use production pa-adaptive-upper-v1; the legacy symmetric selector is rollback-only.
• Only positive PA-local outliers are clipped; negative samples are not clipped.
• A Review preview can change only ACF analysis derived from the frozen context, not centroid, background, catalogue stars or zero point.
• Apply is accepted only when context_id and run_id match the current measurement.
• The final signal is the cumulative PA-adaptive ACF value at the selected radius and is not overwritten by ordinary aperture photometry.
• Every accepted measurement retains detailed kopr-acf-provenance-v1 evidence and a concise KACF log/FITS HISTORY line.
• Manual ordinary photometry, ACF, curve-of-growth points and transferred comparison stars use 40 - 2.5 log10(net_flux / EXPTIME); local sky is additive subtraction only.
• Comet and Stars EXPTIME values are validated within 0.001 s during preflight and the accepted value is frozen for the observation; TOTEXP is reporting metadata only.
• Sigma-clipped mean uses one pass and never restores clipped samples.
• Complete source saturation provenance—not the aggregate stack peak—is authoritative in automatic policy.
• An explicit Saturation level edit creates a role-specific manual threshold, intentionally bypasses the map for that slot and cannot leak to a different source path.
• Relative-WCS diagnostics may be expanded, but acceptance thresholds cannot change unintentionally.
• The result plot and ICQ comment use the same physical curve-of-growth model.

A refactor that changes any of these outputs or decision paths is a scientific-method change and requires new reference data, explicit review and release notes—not merely updated unit tests.

• Master-flat correction remains guarded and Stack Parameters uses one preflight pass.
• Physical scale follows the R76 shared source priority and uses validated observation binning.
• DE440s comet positions retain down-leg light-time; external residuals do not define a universal arcsecond gate.
• The canonical reference-star solver applies extinction before global pairwise selection and rejects a pair at exactly 0.150 mag.
• user_enabled and pairwise_accepted remain separate; Process obs. and Af-rho use one final accepted set.

Required GUI and pipeline smoke tests

1. Host tabs. Launch KOPR, open all six tabs in different orders and confirm each widget is created once, menus/status bar remain correct and no duplicate top-level window appears.
2. Settings propagation and persistence. Change active location, telescope, eyepiece and camera; confirm an already-open WStack context refreshes, WCCD uses the host menu state, restart KOPR and verify all four selections are restored rather than reset to index zero.
3. WStack preference persistence. Save different Comet and Stars methods/sigma values and a Comet output mode. Reopen both dialogs and confirm the separate values. Cancel or submit invalid input and verify the prior values remain unchanged.
4. Worker shutdown. Start a cancellable WStack task, close the application and verify bounded cancellation without an orphaned thread or destroyed worker.
5. Windows/WSL preflight. On a native Windows installation with Astrometry.net only in WSL, verify that solver preflight: WSL solve-field (...) appears before any index dialog.
6. Missing Windows/WSL solver. Temporarily select a distribution without Astrometry.net and verify one actionable error appears and no index download begins.
7. Windows/WSL cancellation. Cancel a deliberately long solve and verify pgrep -a solve-field returns no remaining solver process in the selected distribution.
8. Path handling. Solve a frame from a local Windows path containing spaces and verify WCS/AXY/config/index paths translate correctly and output returns to the Windows directory.
9. Calibration. Build dark/flat masters from a known series, inspect QC reports and verify a calibrated LIGHT header references the selected masters.
10. Astrometry/WCS QC. Solve a known series, inspect AXY diagnostics, deliberately include one bad WCS and confirm rejection persists as Astro OFF.
11. R48 blind online path. Inspect the upload payload and confirm all position, radius, scale, parity and downsampling constraints are absent. Verify queued/solving heartbeats and that calibration is requested only after status=success.
12. R48 offline overrides. Open the Astrometry group dialog and accept unchanged automatic RA/DEC and scale values; confirm no override is stored and per-frame ephemerides remain independent. Edit both fields to create an override, then clear both scale limits to restore automatic scale.
13. Stacks. Produce mean, sum and sigma-clipped star/comet pairs; verify names, pair ID, frame counts, NVALID/NCONTRIB, saturation extensions and provenance.
14. Saturation and camera integration. Verify complete-map authority and all scalar base priorities, then use a FLOAT32 SUM fixture with base 65535 and maximum 1036396.5625 to require multiplier 16 and effective limit 1048560. Confirm automatic/manual base parity, non-fatal SATCOMP=NONE, continuation without a source-peak provider and independent flat-top/bleeding rejection. Reconstruct 1370 × 1002 at 3 × 3 as 4110 × 3006 and verify VERIFIED_EXACT; verify a unique 4096 × 3000 active-area configuration returns VERIFIED_COMPATIBLE. Neither verified case may open the camera dialog.
15. COBS source codes. Parameterize all 15 WCCD magnitude-source labels and assert exact two-character case. In particular, APASS Sloan g′/r′/i′ are AG/AR/AI and Gaia mixed-case identifiers remain unchanged.
16. Manual photometry. Reproduce a reference comet measurement with the expected centroid, background, ACF curve and magnitude.
17. Review dialog. Open and resize it; verify zoom/pan, ring selection, active-ring map and raw/upper/filtered profile redraw.
18. Review Close/Reset. Confirm Close without Apply changes nothing and Reset restores exactly the original or last-applied measurement state.
19. Manual Review Apply. Apply a preview while preserving the user-selected fixed aperture radius and then complete the normal save flow.
20. Review lifecycle. Open Review on Comet and Stars images and after reference-star/catalogue/photometric-option changes; confirm these do not invalidate it. Verify new observation/reset/different Comet FITS or a successful non-ACF measurement does invalidate it, while cancelled/failed replacement attempts preserve it.
21. ACF provenance. Confirm detailed diagnostics and the concise KACF line in the log and compatible FITS HISTORY.
22. R45 robust astrometry. Run the supplied problematic frames with local indexes; compare Attempt 1–3 terminal lines and full logs, confirm the calibrated FITS is unchanged, then force an anchor failure and verify the default skip and explicit continuation policies.

WCCD Automatic Helpers manual acceptance

• Open a new observation with all helpers disabled and confirm legacy manual behaviour.
• Run finder with internal KOPR position and with JPL Horizons; disconnect the network and verify fallback.
• Test paired Comet + Stars and Comet-only finder modes.
• Place a valid candidate between 30 and 120 arcsec from the expected position. Verify interactive WCCD review finds it while the unattended automatic-photometry gate remains 30 arcsec.
• Verify displayed automatic rows are consecutive C1..Cn and Primary/Extended classification remains present; click a circle to select its row.
• Click another object and verify M1 uses the same bounded Gaussian comet centroid as manual WCCD and automatic candidates; enable pan/zoom and verify clicks no longer select.
• Force zero numerical candidates and verify the valid cutout still opens for manual visual selection.
• Confirm that accepting Cn or M1 stores only the centre. Verify the same accepted coordinates drive the marker, Slice, aperture measurement and ACF; rejection changes nothing.
• Process a result with automatic and manual saturation policies and verify no automatic Saturation: sentence is added to Results, ICQ or COBS comments while diagnostics retain saturation provenance.
• Run Auto Star Measure with elongations around 2.2 and 2.7, with and without a valid plateau, and with several removable NaNs in the sky annulus.
• Use a crowded field and confirm raw median FWHM can exceed the robust isolated/lower-distribution FWHM; inspect method, source count and 25/50/75% widths.
• Verify the aperture fallback is based on robust FWHM, the final blend gate is no smaller than aperture radius plus 0.75× robust FWHM, and post-aperture neighbour rejection is reported.
• Place NaN in a non-zero-weight aperture pixel and require invalid_aperture; separately test a partially masked/cropped annulus and the guarded balanced_boxes fallback.
• Force each detailed centroid/background rejection family and verify the specific subreason appears.
• Verify <3 final stars fails, 3–7 warns, >40% clipping warns without automatic failure, scatter 0.08–0.15 mag warns and >0.15 mag fails with computed diagnostics retained.
• Verify the result report ordering, expanded FWHM/aperture/blend/background diagnostics, selectable read-only text and persistent window lifetime.
• Run Auto Star Measure on Stars and Comet images, then force saturation and coverage failures and continue manually.
• Edit automatically added stars and verify each displayed aperture and quality flag.
• Enter Measure Stars while finder is active and verify queue behaviour.
• Test Slice and persistent ACF Review lifecycle, then replace/reset the observation and verify stale workers cannot commit and comet_center_meta is empty.
• Open Reference stars with accepted, pairwise-rejected, quality-rejected and user-disabled stars. Verify Use is checked only for the accepted ensemble, the Pairwise status column is absent and Reason explains each exclusion.
• Recheck an incompatible star, verify it remains selected while the solution is pending, then press Recalculate and verify it becomes unchecked again.
• Open a DSLR directory containing direct TG FITS plus CALIB/WCS/CO/ST derivatives with historical RAW cards. Verify no false conflict; then introduce a second genuine direct representation and require a blocking conflict.
• Zoom and pan the active Comet image, choose Measure comet again and verify the viewport is preserved. Repeat for Stars and Tail; verify a wrong-role start still switches and redraws once.

Reported automated status

• R74 focused automatic DSLR gate: 58 passed, 0 failed; the complete pytest run reached 12% without failure before the environment time limit.
• R73 focused saturation/camera integration gate: 75 passed, 0 failed.
• R73 complete deterministic QA: 1125 passed, 3 skipped, 0 failed (157; 324; 327 + 3 skipped; 317).
• R61 focused cumulative R55–R61 gate: 75 passed.
• R61 focused centroid, Comet Finder, hand-off and ACF gate: 78 passed.
• R61 updated historical astrometry/integration gate: 26 passed.
• Retained R50 Automatic Star Measure policy: 13 tests OK.
• R69 complete grouped QA archive: 1062 passed, 3 skipped, 0 failed (236; 421 + 3 skipped; 405).
• The three skips require a complete Astropy FITS runtime.
• Production and QA compilation, release manifest validation, forbidden cache/artifact scan and ZIP integrity: PASS.

Release evidence

The R48 astrometry evidence, R47 complete QA baseline of 870 passed and 3 skipped, R45 robust local solver, R40 role-specific saturation routing, R29 GUI interaction, R27 photometry and R22 performance/multiplatform evidence remain inherited contracts.

For each production build retain:

• source baseline and commit/archive checksum;
• exact test command and complete output;
• targeted Windows/WSL solver test output when the launcher layer changes;
• build verifier output;
• Linux and native-Windows/WSL GUI/pipeline smoke-test checklists;
• reference datasets and expected numerical results;
• intentional algorithm or contract changes;
• checksums of the distributed archive.

The Windows/WSL launcher patch is infrastructure-only: it changes discovery, path translation, launch and cancellation, while preserving FITS pixels, seeds, solve-field parameters, index series, WCS validation, stacking and photometry. Release evidence must nevertheless include a real target-machine WSL solve and cancellation test because WSL cannot be fully validated in a Linux build container.

R81 current release evidence

R81 collected 1231 tests: 1225 passed, 6 environment-gated tests were skipped and none failed. The R80 focused R78–R80 integration gate passed 24 tests. R81 focused RAW/materialization and redraw gates passed 31 and 45 tests. Performance, photometry and Scientific/Compact validators, production/QA compilation/parity, manifests and ZIP integrity passed.

The six skips require complete Astropy FITS/time and, for some GUI paths, PyQt5. Live reopening of the reported DSLR directory, interactive viewport preservation and a real rawpy decode remain target-system checks.

Adding a regression test

1. Reduce the bug to the smallest deterministic input that still fails.
2. Assert the scientific invariant or serialized contract, not incidental GUI coordinates or implementation order.
3. Include a negative/control case so the test cannot pass by returning a constant.
4. When changing FITS output, test both numeric arrays and header/extension metadata.
5. When changing orchestration, add a static contract check plus a live smoke-test step.
6. Record why the previous behavior was wrong in a short report tied to the build baseline.