# Readout-boundary reference mixing and individual-reference influence

> Broader-reference follow-up: [PARTIAL_REFERENCE_SUPPORT.md](PARTIAL_REFERENCE_SUPPORT.md) now inventories additional support and validates irregular-mask throughput on clean controls. It does not yet accept an expanded ensemble.

**2026-10-03 conclusion:** respecting readout boundaries fixes a demonstrated
response failure for injected output-specific gains, but does not solve the
real stellar histories. One persistent group improves modestly while another
worsens; imported reference variability increases in the same-output arm.
No replacement correction is adopted. A separate exact deletion audit finds
that no single reference dominates the two persistent groups, while collective
reference effects remain possible. The full goal is active and unmet.

Run: `/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/`.
Code: local branch `codex/temporal-raw-epsf`, worktree
`/home/kshukawa/syndiff_pipeline/dev/.worktrees/temporal-raw-epsf`.
Previous goal turn: progress, with completed paired-coadd quantization controls
and the fixed-profile noise projection. The shared WCS handoff was re-read;
there is no new peer result beyond previously incorporated centroid evidence.

## Question and fixed comparison

The current empirical calibration combines references across output-channel
boundaries. Because the detector has four gain responses, such mixing can
dilute a true time-dependent output-specific change. The alternative is
tested without estimating gain from the desired flatness of science targets.

Use the current response-aware extraction, all 1,224 original FFIs, all 172
science sources (165 previously eligible), and the same 235 disjoint references.
There are 45/49/57/84 references in outputs A/B/C/D. Three arms were specified
before evaluation: global nearest-80 reference weights, a fixed 50/50 mixture
with same-output weights, and same-output-only weights. The latter uses up
to 80 available references. No source is removed to improve the result.

For each science star the log calibration prediction is

$$p_{s,t}^{(\alpha)}=\sum_r
 \left[(1-\alpha)w^{G}_{r,s,b(t)}+\alpha w^{O}_{r,s,b(t)}\right]Y_{r,t,f(s)}.$$

Here $s$ denotes a science star, $t$ an exposure and $r$ a reference star;
$b(t)$ is its two-day evaluation block; $f(s)$ is its fixed source fold;
$w^G$ is the dimensionless existing spatial/colour/magnitude kernel weight,
and $w^O$ is the corresponding weight restricted to the target's readout
output. Each weight set sums to one; weights are zero for unselected stars.
$Y$ is the dimensionless log reference flux, transported to the reference's
estimated r5 response with the existing opposite-science-fold ratio fit,
then normalized by its fixed full-history median rate. The declared values
of the mixing coefficient $\alpha$ are 0, 0.5 and 1. The corrected science
rate is its unchanged extracted rate multiplied by $\exp(-p)$.

Reference precision is re-estimated using epochs outside the evaluated
two-day block. The exact ratio coefficient for that outer source-fold/block
is evaluated throughout its training set: borrowing other blocks' fitted
ratios there would leak evaluated-block science ratios into precision weights.
The archived full-history normalizations remain fixed constants. WCS, global
response, photon means and profile shapes remain inherited, so this is not
an independent complete-pipeline time holdout.

Reference halves are defined by a fixed catalogue-only random split balanced
within outputs. Each half is predicted entirely from the other half.
Their true stellar variability remains in prediction residuals. A median
combining reciprocal halves can partly cancel errors algebraically; its
flatness is not used as proof of calibration accuracy. Full per-reference
residuals are retained and used for source-level scores.

## Algorithm and signal controls

Weight normalization, exact reproduction of the four readout indicator
vectors, and invariance to reference ordering pass on a synthetic catalogue.
The existing nominal and primary-injection ratio fields are reconstructed
from their recorded coefficients. The original reference prediction and
paired calibrated flux replay within the declared numerical tolerances.

The predeclared primary amplitude tolerance is 0.5%, evaluated on all 172
injected targets. Existing independent analytic profile injections had
already relearned profiles and ratio coefficients; this test then refits
reference precision outside each test block. Reference variability is added
to recorded aperture counts before subtracting the fixed physical count-loss
term. A separate known-output-gain test multiplies calibrated reference and
science rates by the same prescribed function within each readout; it tests
calibration transfer, not detector readout physics.

| Measure | Global k80, time-held weights | Half same-output | Same-output |
|---|---:|---:|---:|
| Worst primary amplitude error [%] | 0.4821 | 0.4016 | 0.3540 |
| Primary signal-error RMS median/p95 [ppm] | 1.698/3.344 | 1.922/3.335 | 2.068/3.572 |
| Imported 1-ppt reference variation RMS median/p95 [ppm] | 83.61/114.49 | 77.56/116.53 | 94.81/140.32 |
| Remaining known output-gain RMS median/p95 [ppm] | 302.69/514.50 | 150.95/257.17 | 1.09/1.80 |

![Actual injected output gains and recovered corrections](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/readout_reference_control/evidence/gain_histories.png)

![Actual primary recovery and reference contamination](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/readout_reference_control/evidence/stellar_signal_histories.png)

The same-output model correctly preserves a gain change at the boundary in
this conditional test. The existing mixed-output correction cannot be treated
as validated removal of arbitrary readout-specific gains. The increased
reference-variable leakage in the same-output arm is a distinct cost of its
smaller, differently weighted reference populations.

## Real-data outcomes

| Six-hour group-median RMS [ppt] | Current recorded | Global k80, time-held weights | Half same-output | Same-output |
|---|---:|---:|---:|---:|
| grid4_shift0_1_0, 10 stars | 0.12332 | 0.12342 | 0.14137 | 0.17116 |
| knn16_104, 16 stars | 0.18360 | 0.18378 | 0.16865 | 0.16119 |

These are the original globally aligned six-hour median statistics on the
unchanged source population. They include astrophysics and do not establish
instrumental variance. The signs and shapes of the persistent histories remain.

![Matched real-data groups, held references and injection summary](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/readout_reference_control/readout_control.png)

![Actual reference histories and their source-disjoint residuals](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/readout_reference_control/reference_histories.png)

All 354 predefined magnitude/colour/spatial/neighbour/readout groups were
recomputed. Their scatter changes in both directions, without a general cure.
Median source six-hour RMS is 0.48690/0.48684/0.48937 ppt in the three new
arms. Held-reference source median RMS is 0.55006/0.54606/0.55860 ppt;
p95 is 2.48309/2.45092/2.42629 ppt. Small aggregate changes are not independent
noise significances. No mixing coefficient is chosen from the science scores.

![All predefined groups and source changes](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/readout_reference_control/evidence/all_groups.png)

## Does one reference drive a persistent group?

The separate deletion audit retains the current recorded estimator and
normalizes the remaining frozen weights after deleting one reference at a
time. Its exact change in a science star's log flux is

$$\delta\ell_{s,t}^{(-r)}=
 \frac{w_{r,s}}{1-w_{r,s}}\left(Y_{r,t,f(s)}-p_{s,t}\right).$$

Here $\ell$ is dimensionless log science flux; $w$ is the current fixed
reference weight; and $Y,p,s,t,r,f$ are as defined above, now for the recorded
current correction. A separate direct matrix recalculation checks the largest
deletion in each group. This is a conditional influence calculation, not a
decision to exclude stars or a full weight refit.

The largest single-reference deletion changes the two group histories by
9.02 and 8.10 ppm RMS. Their current mean-log six-hour RMS is 142.65 and
199.00 ppm. This audit uses means of log histories for an exact additive
decomposition; its values differ from the primary median-group statistics.
The largest individual science/reference weight is 2.84%; the effective
reference count ranges from 58.00 to 71.60, with median 67.57. No single
reference dominates these two groups under this operator. Many-reference
collective signals and refitted-weight effects remain unresolved.

![Exact deletion changes and all constituent science histories](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/reference_influence_audit/reference_influence.png)

![Underlying influential-reference histories and positions](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/reference_influence_audit/influential_references.png)

## Execution and next discriminating work

Scripts: `readout_reference_control.py`, `readout_reference_evidence.py`, and
`reference_influence_audit.py` under the worktree's `temporal_raw_epsf/`.
Matching frozen local jobs and results use the same names under the run's
`condor/` and `results/` directories. All jobs exit 0. The control ran with
four worker processes and a 6,000-MB local resource hint (not enforced), after
checking host capacity. Blank time bins produce expected all-NaN median
warnings; they remain blank and do not enter RMS scores. All seven figures
were inspected. No new ePSF training or production modification occurred.

Next separate fine spatial/colour/background/PSF responses from true output
gain, and expand independent reference coverage with validated partial-support
or neighbour modeling. Repeating the old smaller-aperture reference test
unchanged is not sufficient. Any larger cohort must retain matched original
science support and pass neighbour/source injection tests. A corrected
readout response should not be purchased by importing more stellar variability.
The prior broader-population, physical noise/covariance and absolute-WCS
requirements remain in force.

**Conclusion:** a measurable calibration-transfer weakness is demonstrated,
and the boundary-aware control addresses that injected weakness. It does not
remove the real persistent trends and is not promoted. The individual-reference
audit disfavors a single dominant reference as the explanation, without
clearing collective reference contamination or other instrument effects.