# Broader reference support and irregular-aperture throughput validation

> Follow-up: [PARTIAL_STATIC_PSF.md](PARTIAL_STATIC_PSF.md) fits five static modes using only retained pixels on the other orbit. It worsens hidden-core/photometric errors and is not adopted.

**2026-10-03 conclusion:** 2,091 additional partial-r3 candidates exist under
the original bright-neighbour catalogue cut, but they are not an accepted
reference ensemble. The existing population PSF with measured temporal WCS
and blur/focus removes most artificial-mask variability on clean controls.
It improves 99.71% of the 2,424 genuinely nonidentity star/mask comparisons,
yet leaves substantial residual errors for heavily masked sources. A source-
specific full-support profile performs better but is unavailable where pixels
are genuinely missing. More reference stars therefore require a validated
throughput/neighbour model, not simply relaxed masks. Full goal remains unmet.

Run: `/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/`.
Code: local branch `codex/temporal-raw-epsf`, worktree
`/home/kshukawa/syndiff_pipeline/dev/.worktrees/temporal-raw-epsf`.
Previous turn was progress: the readout-reference transfer and influence
tests completed. Existing science products, WCS and accepted masks remain intact.

## Available support and the meaning of the catalogue cut

An explicit inventory-only option in `pilot.load` exposes incomplete cores
without changing its default complete-r2 requirement. The original catalogue
selection, fixed bad-column and neighbour masks, and minimum 80 measured
stamp pixels remain. Geometric sky support requires at least 15 outer-ring
pixels and a nonsingular plane design. This inventory is not a scientific
acceptance test and does not establish all-cadence validity.

| Output | Inventory candidates | Complete r3 | Additional partial r3 |
|---|---:|---:|---:|
| A | 521 | 79 | 442 |
| B | 559 | 93 | 466 |
| C | 691 | 119 | 572 |
| D | 751 | 140 | 611 |
| Total | 2,522 | 431 | 2,091 |

All existing 431 sources, their IDs and masks replay exactly. These counts
include the original science sources; they are not counts of new disjoint
references or independent usable measurements.

![Candidate geometry and retained area](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/partial_aperture_benchmark/evidence/qualified_coverage.png)

The original `cont8_13` isolation statistic counts neighbours brighter than
T=13, whereas the pixel-mask recipe includes fainter catalogue sources down
to a relative flux threshold. Consequently, a source may pass that bright-
neighbour cut while most of its core is masked. The earlier inventory figure's
label “catalogue-isolated” is qualified by the figure above; it is not general
isolation from all relevant stars.

For four preselected zero-core examples, tracing the exact catalogue and
WCS/offset recipe finds nearby fainter objects that generated the masks.
For example, the T=12.85 source has a T=15.93 neighbour only 0.61 pixel away;
another T=10.27 source has a T=14.18 neighbour at 0.80 pixel. These catalogue
separations explain why the exclusion circles reach the primary cores. They
do not establish the true fraction of neighbour light in a measured aperture.
Actual contributor IDs, magnitudes, separations and catalogue flux ratios
are in `results/partial_aperture_benchmark/evidence/mask_neighbours.csv`.

![Actual pixels and the catalogue mask contributors](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/partial_aperture_benchmark/evidence/neighbour_mask_trace.png)

## New test: irregular real masks on already clean stars

Before measuring temporal ratios, select four masks per output by their
nearest retained-r3 pixel-area fractions to 0.25, 0.5, 0.75 and 0.95. Apply
these 16 fixed stamp-coordinate patterns, plus an identity control, to all
172 existing clean science stars on the original 36 preselected FFIs. Report
the unchanged 165 eligible subset. The source's original ring sky pixels are
preserved so this isolates the aperture-support question; new-reference
background contamination is a separate, still-required test.

For each star/exposure/mask compare

$$y_{s,t,m}=\log\frac{\sum_p w^{m}_{s,t,p}D_{s,t,p}}
 {\sum_p w^{3}_{s,t,p}D_{s,t,p}},\qquad
\widehat y_{s,t,m}=\log\frac{\sum_p w^{m}_{s,t,p}P_{s,t,p}}
 {\sum_p w^{3}_{s,t,p}P_{s,t,p}}.$$

Here $s$ is a clean control star, $t$ a selected exposure, $m$ a fixed mask
pattern and $p$ a stamp pixel. $D$ is actual calibrated RAW pixel rate in
electrons/second/pixel. The dimensionless weights $w^m$ and $w^3$ implement
the masked and complete r3 apertures with the same response-aware ring-plane
subtraction. $P$ is the existing opposite-source-fold population PSF, rendered
with exact spline pixel integration at the measured per-frame WCS position,
plus the inherited first-order blur/focus fields. Its overall normalization
cancels. Both dimensionless log ratios are mean-centred over the same 36
epochs; the residual is $y-\widehat y$. No stellar trend or mask-specific
temporal coefficient is fitted.

The second prediction uses the source's current empirical temporal profile
learned on full support. It is an **oracle diagnostic only**: genuinely masked
new references do not supply those missing pixels at other epochs. Its lower
residual cannot be counted as a deployable missing-light correction.

The paired pixel-error comparison retains aperture covariance:

$$V_{s,t,m}=\sum_p U_{s,t,p}^2
 \left(\frac{w^m_{s,t,p}}{F^m_{s,t}}-
       \frac{w^3_{s,t,p}}{F^3_{s,t}}\right)^2.$$

Here $U$ is the archived pixel uncertainty in electrons/second/pixel;
$F^m$ and $F^3$ are the measured masked and complete aperture rates, respectively,
in electrons/second; and $V$ is dimensionless log-ratio variance. This is a
fixed-response, independent-recorded-pixel approximation. Learned-profile,
WCS, shared CAL and global-response covariance are not supplied by this
formula, so it is not a certified photon floor.

## Results, including support qualifications

| Mask case | Affected eligible stars | Uncorrected median ratio RMS [ppt] | Population-model residual [ppt] | Full-support oracle residual [ppt] |
|---|---:|---:|---:|---:|
| A, target area 0.50 | 165 | 9.6740 | 0.4517 | 0.2058 |
| C, target area 0.50 | 165 | 18.4699 | 0.6792 | 0.3187 |
| A, target area 0.95 | 165 | 0.2898 | 0.0553 | 0.0357 |
| B, target area 0.95 | 112 | 0.0731 | 0.0256 | 0.0190 |
| D, target area 0.95 | 165 | 1.1655 | 0.0870 | 0.0502 |

The old population model tracks large temporal redistribution well, including
the orbit edges, but accuracy depends strongly on which pixels are missing.
Retained area is not retained flux: the four target-area-0.25 masks retain
median r3 flux fractions about 0.058–0.180 and leave population residuals
0.746–2.232 ppt. The accuracy needed for expanded calibration is not established.

![Actual ratio histories and model predictions](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/partial_aperture_benchmark/evidence/mask_transfer_histories.png)

![Underlying actual temporal pixel redistribution and prediction errors](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/partial_aperture_benchmark/evidence/mask_temporal_pixels.png)

Mask patterns transfer in fixed stamp pixels, so a different subpixel centre
can make a donor's missing aperture pixels fall outside a control's aperture.
This happened for 53/165 controls in case B/0.95 and 163/165 in C/0.95. Such
effective identities are recorded, not counted as successful missing-light
recovery. The C/0.95 case has only two affected stars and cannot establish
population performance. All original scores remain saved; the figure below
and the table above use actual nonidentity support. This qualification is
selected from geometry, never from flux scatter.

![Scores restricted to genuinely affected apertures, with counts](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/partial_aperture_benchmark/active_support/active_mask_scores.png)

The explicit identity mask reproduces complete weights, measured ratios and
both model ratios at numerical precision. Complete-r3 counts replay the
current extraction; sky-basis annihilation is checked for every artificial
aperture. All benchmark source/mask rates are positive. The population PSF
was not retrained here, and no neighbour was synthesized or subtracted.

## Execution and next step

Frozen local jobs `partial_reference_inventory`, `partial_aperture_benchmark`,
`partial_aperture_evidence`, and `partial_mask_active_summary` all exit 0.
The benchmark uses four processes and a 4,000-MB local memory hint (not an
enforced limit); the remaining jobs use two CPUs. Code snapshots, SHA256
manifests and exact commands are under the matching `condor/` directories.
Results, masks, ratios, noise estimates, image evidence and identity-case
records are under `results/partial_reference_inventory/` and
`results/partial_aperture_benchmark/`. All published figures were inspected.

This is aperture-response diagnosis of existing PSFs, not new ePSF training
or an accepted expanded extraction. Next develop and validate a profile/
neighbour model that uses only genuinely available pixels. Lightly masked
sources offer a plausible first expansion, but actual full-cadence support,
neighbour variability, count-loss projection, source/reference injections,
readout response and held-out science-group checks remain required. Heavily
masked/fainter/crowded stars remain part of the original goal, not exclusions
that can be silently dropped to declare success.

**Conclusion:** accurate temporal PSF/WCS modeling removes most artificial
mask-induced changes, but the remaining missing-light error and real faint
neighbours prevent simply adding the 2,091 candidates. The next model must
close that gap without using hidden pixels or flattening real stellar signals.