# Held-out strap sky prediction and support audit — 2026-10-03

> **Repaired-surface follow-up, 2026-10-03:** [REPAIRED_BACKGROUND_RESPONSE.md](REPAIRED_BACKGROUND_RESPONSE.md) completes the 36-epoch repaired-surface validation and full-cadence refit. Selected stars improve further, persistent groups remain.

**Conclusion:** the existing strap model predicts withheld usable background
pixels substantially better than no strap correction, especially in the
high-background image. Removing its floor gives only a small additional gain
on usable columns; the apparently enormous all-column improvement was almost
entirely a known bad-column effect that cannot improve our stellar curves.
Additional spatial flexibility and larger training masks do not give a
consistent improvement in this pilot. No new photometry model is adopted.
The instrumental-systematics/photon-noise goal remains active and unmet.

Run: `/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/`.
Worktree: `/home/kshukawa/syndiff_pipeline/dev/.worktrees/temporal-raw-epsf/`,
local branch `codex/temporal-raw-epsf`. No push or production change.
Preceding result: `STRAP_BACKGROUND.md` in the run directory.

## Conditional prediction, not stellar flatness

The four fixed epochs are FFIs 1, 601, 698 and 1225. At each exposure, fit
the column response using other detector rows and predict withheld pixels.
No stellar light-curve trend or variability threshold trains/selects a model.

$$r_{t,y,x}=R_{t,y,x}/B_{0,t,y,x},\qquad
\widehat R_{t,y,x}=B_{0,t,y,x}\widehat q_{t,x}(y).$$

Here $t$ is exposure, $y$ a zero-based detector row and $x$ a crop-local
detector column (full-FFI column is $x+44$). $R$ is the actual calibrated
RAW-minus-convolved-template pixel in electrons/second, $B_0$ is the saved
pre-strap background surface in the same units, $r$ is their dimensionless
ratio, and $\widehat q$ is a dimensionless column function fitted to training
rows only. $\widehat R$ is the held-out pixel prediction, in electrons/second.
The saved $B_0$ remains fixed. This isolates conditional column prediction;
it is not a holdout of the entire background pipeline or independent truth
for the background underneath stars.

The frozen estimator is from producer `a3d50a0070c504dd9ccc58d0c38236b58759f77f`,
with source hash `54a3ecd53566c64f9b02b2c055eb863f4fb4ee13e80f1df314a104ac98a38571`.
Its complete saved ratio map replays within 1e-12 before alternatives are fit.
The frozen mask constants are used, including FAINT_CAT=32 and STRAP_BIT=4.

Alternate 32-row blocks form two folds. Training uses the opposite fold and
excludes its first/last four rows as a correlation guard. Mask variants grow
the existing bright/source masks by 0, 3 or 6 pixels; all include the saved
extra exclusions. Every model is scored on the same pad-6 evaluation pixels.
Training sigma clipping sees no evaluation value. Columns require at least
20 geometrically valid training pixels in each half at the strictest mask;
all 315 strap columns pass. There are 112,069 initial evaluation pixels per
frame. Larger masks and catalogue flags are not a guarantee of pure sky:
faint, uncatalogued and extended-wing light can remain.

Four predeclared models are compared:

- The native quadratic with response clipped below one.
- The same quadratic without the floor.
- Quadratic B-splines with fixed 256-pixel knots, without the floor.
- Quadratic B-splines with fixed 128-pixel knots, without the floor.

The native smoothing budget is ten times training count times ratio variance.
All 7,560 fitted native curves have only the two endpoint knots: they are
global quadratics, not locally adaptive curves. Spline alternatives solve

$$\widehat c=\arg\min_c\left\{
\sum_{y\in T}(r_y-A_yc)^2+\lambda\|D_2c\|^2\right\},
\qquad \lambda=0.1N_T/n_c.$$

Here $T$ is the retained training-row set after clipping, $N_T$ its size,
$c$ the vector of dimensionless spline coefficients, $n_c$ its length,
$A_y$ the dimensionless quadratic B-spline design row, and $D_2$ the second
coefficient-difference operator. Knot endpoints are fixed at -1 and 2048.
The regularization multiplier is chosen before validation and is not tuned
on withheld pixels. Ill-conditioned/unsupported designs fail rather than
silently filling missing predictions.

## A misleading all-column result caught by inspecting support

The first score suggested a dramatic gain from removing the floor. Attribution
shows that **99.86–99.94%** of that squared-error improvement comes from the
preexisting bad-column set. Column 1967 alone supplies about 99.7–99.8%.
This is crop column 1967, full-FFI column 2011. Its actual calibrated values
are near zero or negative while neighboring background levels are positive;
the free model fits a negative ratio there. That is not a useful stellar
sensitivity correction.

![Actual bad-column data and the competing predictions](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/strap_sky_validation/support_audit/bad_column_evidence.png)

The bad-column list is extracted from the existing `pilot.py` AST, not chosen
from these outcomes. It predates this test:
222, 424, 1023, 1024, 1032, 1033, 1376, 1386, 1505, 1967, 1968 (crop-local).
An assertion confirms that all such pixels have zero support in the saved
science masks and reference aperture weights. Only three intersect this
strap-evaluation sample. Their removal leaves 312 columns and 110,806 common
evaluation pixels per exposure. The original all-column result is retained,
explicitly qualified rather than erased:

![Diagnostic all-column score, including pixels already excluded from stellar measurements](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/strap_sky_validation/heldout_scores.png)

The mask-source hash, per-column contribution and exact support arrays are
in `results/strap_sky_validation/support_audit/`. This is post-diagnostic
attribution to an independently existing mask, not an outcome-based rejection
of difficult science pixels. No source or its measurement support is changed.

## Result on usable columns

Define the dimensionless score

$$\rho=\frac{\sum_{p\in E}(R_p-\widehat R_p)^2}
{\sum_{p\in E}(R_p-B_{0,p})^2}.$$

Here $E$ is the fixed held-out pixel set for one exposure and support cohort;
$p$ indexes its row/column positions. The numerator is prediction squared
error, and the denominator is squared error without a strap correction.
Both sums have units of electrons-squared/second-squared. Lower $\rho$ is
better conditional prediction; it is not a photon-noise statistic.

Native training masks, identical usable evaluation support:

| FFI | Native floor | Free quadratic | Spline256 | Spline128 |
|---|---:|---:|---:|---:|
| 1 | 0.78751 | 0.77865 | 0.78389 | 0.79145 |
| 601 | 0.79923 | 0.79341 | 0.80496 | 0.81580 |
| 698 | 0.04332 | 0.04212 | 0.04097 | 0.04151 |
| 1225 | 0.79832 | 0.79043 | 0.80055 | 0.81416 |

![Matched usable-column scores, including the three training-mask widths](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/strap_sky_validation/support_audit/usable_column_scores.png)

![Actual held-out pixels supporting those scores](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/strap_sky_validation/support_audit/usable_column_images.png)

The high-background frame has visibly large column response before correction;
the native model already removes much of it on withheld pixels. On usable
columns, removing the floor reduces native-model MSE by only about 0.7–2.8%,
descriptively. The splines help slightly in the high-background image but
worsen the low-background cases relative to the free quadratic. Larger
training masks do not improve this pilot's held-out MSE. These repeated
pixels/columns/epochs are correlated: no independent-pixel significance or
universal confidence bound is claimed.

Two additional support checks agree qualitatively. Restricting to columns
touching current science/reference measurement support retains 294 columns
and 104,119 pixels per frame. Restricting to rows inside all models' retained
training ranges retains approximately 100,400–100,600 usable pixels per frame.
All numerical scores and nonpositive-prediction counts are retained in
`support_audit/scores.csv`; usable-column predictions are positive. These
checks prevent a few extrapolated rows from silently defining the conclusion.

## Validation, failures and provenance

`test_strap_column_models.py` passes known quadratic recovery, below-one
response, B-spline partition of unity, held-out perturbation invariance and
array-axis alignment. The physical worker additionally perturbs actual
withheld ratios and requires exactly unchanged predictions. All models have
identical evaluation support; no failed model is scored on an easier subset.

Frozen fit job 2043 on plscience3 (two CPUs/4000 MB) completed the numerical
fits and tables but exited 1 at plotting: `scores.pad` resolves to a pandas
method rather than the `pad` column. Bracket indexing fixes that name collision.
Render job 2048 verifies saved predictions against every summary MSE and
sample count, then completes without refitting. Support audit 2053 completes
with exit 0. Original logs/tables are preserved. All four published figures
above have been inspected; the initial packed-grid figure is not used in this
record because the more relevant usable-column and bad-column images show
the support distinction directly.

Owned scripts: `strap_column_models.py`, `strap_sky_validation.py`,
`strap_sky_support_audit.py`, `test_strap_column_models.py` under
`/home/kshukawa/syndiff_pipeline/dev/.worktrees/temporal-raw-epsf/temporal_raw_epsf/`.
Outputs: `/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/strap_sky_validation/`.
No production calibration, upstream data or accepted light curve is replaced.

## Next discriminating test

Separate a stable additive scene residual from a response proportional to
background illumination, using the existing 36 epochs and withholding whole
time blocks/orbits. A candidate model is

$$R_{t,y,x}=B_{0,t,y,x}q_x(y)+c_{y,x}+\epsilon_{t,y,x}.$$

Here $R$ and $B_0$ have the definitions above, $q_x(y)$ is a dimensionless
response shared across training exposures, $c_{y,x}$ is a static additive
residual in electrons/second (potential template or stellar-wing light), and
$\epsilon$ is the remaining pixel residual in electrons/second. Both $q$ and
$c$ must be learned only from training data. Illumination leverage, parameter
degeneracy and transfer between orbits must be checked rather than assumed.
Do not subtract $c$ as sky or interpret $q$ as a stellar flat without further
physical evidence. This model has not yet been run.

If a background response passes those checks, test a local background plane
multiplied by that response in the RAW pixel fit, then relearn the temporal
profiles with the corrected background. The current profiles were learned
from images containing the strap structure; fixed-profile subtraction alone
does not complete that task. Preserve WCS conventions, variable-source
injections, reference handling, all predefined groups and the full noise
budget. The remaining hp_d scene/removal accounting and broader populations
are still required.

**Conclusion:** the strap lead now has conditional held-out-pixel support,
but simply making its spatial curve more flexible is not a demonstrated
solution. The large apparent floor-removal gain was mostly irrelevant bad
pixels. Temporal separation and the complete pixel/photometry chain remain
the next work; the original goal is unachieved.

## Temporal/profile follow-up completed 2026-10-03

RESPONSE_PROFILE_RELEARN_20261003.md (run copy RESPONSE_PROFILE_RELEARN.md)
records the executed response/static-light separation, full-cadence temporal
profile re-learning, aperture-count reprojection, reference refit and matched
signal/group checks. The independent sky model transfers between orbits and
helps selected large light-curve excursions. Persistent group structure and
the updated physical noise budget remain unresolved; no accepted science
product is replaced.