# Column-shaped background response in RAW photometry — 2026-10-03

**Conclusion:** a targeted strap-only subtraction reduces large sampled
orbit-shaped variations in three previously selected stars and avoids the
harmful surface-curvature intervention in a fourth. It leaves the two
predefined group histories essentially unchanged. This is a conditional
development result, not an adopted correction, proof of an upstream SPOC
calibration error or photon-limited photometry. Six specific strap-only
signal controls now show small measured responses; broader independent
background and signal validation remain required.

Run: `/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/`.
Local code: `/home/kshukawa/syndiff_pipeline/dev/.worktrees/temporal-raw-epsf/`,
branch `codex/temporal-raw-epsf`. Code remains local. No production changes.
This follows `BACKGROUND_CURVATURE_PHOTOMETRY.md`; upstream calibration scope
and limits remain in `CALIBRATION_AUDIT.md` and `APPLIED_CALIBRATION.md`.

## Separate two different background effects

The downstream background estimator returns

$$B_{t,p}=B_{0,t,p}q_{t,p}
=B_{0,t,p}+S_{t,p},\qquad S_{t,p}=B_{0,t,p}(q_{t,p}-1).$$

Here $t$ is an exposure and $p$ a detector pixel. $B$ is the total modeled
background in electrons/second/pixel; $B_0$ is its spatially filled, smoothed
and residual-repaired surface before the final strap step, in the same units;
$q$ is the dimensionless downstream column response fitted to ratios of
RAW-minus-template pixels to that surface; $S$ is the resulting additive
strap excess in electrons/second/pixel. The implementation clips $q$ below
one and sets it to one outside designated strap columns. This $q$ is **not**
the archived SPOC flat, a measured stellar sensitivity map, or a justified
factor by which to divide all stellar light.

At FFI 698 the saved candidate surface and strap map replay the earlier
candidate-minus-archive map bit-for-bit at its stored float32 precision.
The exact science-flux operator projects $B_0$ and $S$ separately. Their
direct rate changes add to the full-background change within numerical
precision. These values are before reference calibration, not temporal RMS.

| Gaia source ID | Combined surface effect [ppm] | Combined strap effect [ppm] |
|---|---:|---:|
| 1629810213318424704 | -12417.80 | 0.00 |
| 1625600183294357504 | -63.69 | -9892.34 |
| 1627184373392387072 | -9.06 | -6960.50 |
| 1603178323705979648 | -541.46 | -7456.90 |

The source made worse by full-background subtraction has a surface-curvature
problem. The other three selected interventions are dominated by the strap
term. Their images are column-like; the first source's historical surface has
a broad central feature. These are different pathways and cannot be repaired
by treating every source as one common CCD multiplier.

![Exact component projections into source flux](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/background_curvature_sample/components_f0698/component_response.png)

![Underlying surface and strap component images](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/background_curvature_sample/components_f0698/component_images.png)

The high-background branch at this epoch makes zero SEP anomaly-detector and
positive-RMS calls. Therefore this particular effect must not be attributed
to the separate low-background deblending/RMS failure. The pre-strap surface
still contains multiple processing stages; this split does not yet isolate
which surface operation causes its remaining curvature.

## Strap-only intervention on all 36 selected epochs

Subtract only $S$ derived from the historical saved stages, then retain the
local profile-plus-plane extraction. The historical full-background and
combined-candidate controls are recomputed alongside it. Their saved arrays
are checked against the preceding experiment. The test uses the same 172
science measurements, 165 eligible stars, 235 disjoint references and 354
predefined groups. Both science and reference rates are changed consistently;
physical count restoration, WCS/ePSF, reference transport, weights and
normalizations remain fixed. All rates remain positive. No light-curve trend
is fitted and no additional star is discarded.

Median/p95 temporal RMS of the induced change is **22.19/458.44 ppm**, maximum
2734.86 ppm. This is intervention size, not the RMS of corrected light curves.
The historical-stage strap term is a development choice; the earlier paired
controls for a combined full background do not validate its signal transfer.

![36-epoch induced changes by magnitude, colour and group](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/background_strap_sample/curvature_photometric_effect.png)

![Underlying strap-only curvature, on fixed science support](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/background_strap_sample/curvature_stacks.png)

| Same predefined group | Current sampled RMS [ppm] | Strap-only sampled RMS [ppm] |
|---|---:|---:|
| grid4_shift0_1_0, 10 stars | 228.56 | 231.18 |
| knn16_104, 16 stars | 252.03 | 246.96 |

These changes do not remove the persistent group structure. Sampled RMS
includes astrophysics, shared-reference covariance and nonuniform cadence;
it is not an independent instrumental significance test.

![Actual group samples after the strap-only intervention](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/background_strap_sample/actual_group_samples.png)

![All predefined groups and detector positions](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/background_strap_sample/assessment/multiscale_comparison.png)

![Spatial images behind those comparisons](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/background_strap_sample/assessment/spatial_background_images.png)

Keep the four sources selected in the previous full-background diagnostic,
rather than choosing a new favorable subset. Their actual sampled RMS is:

| Gaia source ID | Current [ppt] | Strap-only [ppt] |
|---|---:|---:|
| 1629810213318424704 | 0.561 | 0.562 |
| 1625600183294357504 | 4.067 | 1.394 |
| 1627184373392387072 | 6.143 | 3.661 |
| 1603178323705979648 | 4.060 | 1.678 |

![Same four source histories, including the new strap-only arm](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/background_strap_sample/assessment/preselected_outlier_histories.png)

The three large histories shrink and the other source avoids the imposed
full-background variation. Substantial variation remains. These selected
examples establish usefulness for further tests, not population acceptance
or proof that the removed component is entirely instrumental.

## Commands, validation and next gates

All jobs use frozen snapshots under this run's `condor/` on the plscience3
scheduler. Component export 2001 uses `background_transfer_probe.py --frame
698 --lane legacy --profile wide --epsilon .001 --rtol 1e-10 --fill-method
harmonic --component-anomalies --positive-rms --save-components --tag
positive_rms_f0698_components`; component score 2005 uses
`background_curvature_components.py`. Both complete with exit 0. The export
has the original same-process paired control as well as baseline components.

Photometry job 2010 uses `background_curvature_photometry.py --sample
--strap-only`; assessment 2011 uses `background_curvature_assessment.py
--input background_strap_sample`. They preserve all sources and pass the
direct remeasurement and source/frame identity checks. All eight figures
in this record were inspected. The general photometry validation numbers
are in the preceding background-curvature record.

Tests address the **strap-only** operator directly. Six paired
probes use the three affected sources at FFIs 1 and 698,
using `background_transfer_probe.py --lane legacy --profile wide --epsilon
.001 --source-id SOURCE --measure-component strap --frame FRAME --tag TAG`.
Tags are `strap_source0_f0001`, `strap_source0_f0698`, and correspondingly
source1/source2; source order is the final three rows of the table. Each
recomputes the complete estimator before/after injection and then measures
only the strap increment. These are conditional RAW cores plus the existing
3–30-pixel empirical wing family, not arbitrary-source truth or full noise
simulation.

Original probes 2014–2019 exited 1 during metadata serialization: the new
source-ID CLI argument duplicated an explicit `source_id` keyword in the
summary constructor. Their arrays and failure records are preserved and are
not counted as completed controls. The redundant keyword was removed and
new jobs 2022–2027 with `_metadata` suffixes rerun the same calculations.
All completed arrays equal the originals exactly. No scientific threshold
or source selection changed.

Assessment 2032 wrote its numerical table but failed on a plotting keyword:
`linthresh` belongs to `set_yscale`, not `Axes.set`. The corrected assessment
2033 completes, retaining the first table and checking identical values.
Its two figures were inspected; this record now includes ten figures.

The six fixed-r3 injected-source errors range from **-0.00401 to -0.03150
ppm** for a +1000-ppm input. The largest remote-reference response is
**0.01181 ppm** among 231–234 remote references per source; remote science
sets contain 162–164 stars. Remote input flux is exactly zero beyond 100
pixels. These are deterministic conditional algorithm responses, not noise
measurements or proof of general signal preservation. The final learned
profile, physical input means and reference calibration are not refitted.

![Specific strap-only paired responses](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/background_strap_sample/paired_controls_checked/responses.png)

![Actual injected and measured strap-response images](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/background_strap_sample/paired_controls_checked/response_images.png)

After signal-transfer checks, validate the column response against withheld
clean sky pixels, split sky rows and widened catalogue exclusions. Use actual
raw/background images, not stellar flatness alone. Compare the historical and
repaired pre-strap surfaces because the fitted ratio depends on that surface.
Treat unknown stellar wings/partial-neighbour light, spectral dependence and
reference covariance explicitly. Full cadence and broader populations follow
only after these gates; no astrophysical flattening objective is substituted.
Include redder matched-brightness sources, off-strap controls, other empirical
wing shapes, negative/zero amplitudes and epochs outside these two conditions.

**Conclusion:** spatially localized detector/background response is now a
concrete, image-supported lead. It does not establish a faulty scalar CCD
calibration or resolve the remaining group trends. The full goal is active
and unachieved; no corrected science product has been promoted.

## Held-out sky follow-up completed 2026-10-03

See STRAP_SKY_VALIDATION_20261003.md (run copy STRAP_SKY_VALIDATION.md).
The existing strap model predicts withheld usable sky pixels better than no
strap term. The large apparent advantage of removing its floor is mostly a
known bad column already excluded from stellar measurements. Extra spatial
flexibility does not improve all tested backgrounds consistently. A temporal
response/static-residual separation is the next test; no new RAW correction
is adopted from this pixel-validation pilot.