.. _validation-plan: Validation run plan ==================== This page documents the planned validation runs for the four stable estimators (**SMF**, **w**\ :sub:`p`\ **(r**\ :sub:`p`\ **)**, **w(θ)**, **ΔΣ(r**\ :sub:`p`\ **)**) and for the in-development LF estimator. Each section lists the dataset, the launch script, the expected output, and the quantitative pass criterion. .. contents:: Contents :local: :depth: 1 ---- Uchuu HOD mocks — SMF, WPRP, WTHETA -------------------------------------- **Purpose**: verify that the estimators recover the truth on a controlled N-body + HOD realisation where the input two-point statistics are known. .. list-table:: :header-rows: 1 :widths: 20 80 * - Item - Details * - Dataset - Uchuu N-body halo catalogue with an HOD applied to match BGS clustering * - Estimators - SMF (1/Vmax), w\ :sub:`p`\ (r\ :sub:`p`), w(θ) * - Launch scripts - ``scripts/measure_joint_sumstat.py --survey mock --stats SMF WP WTHETA`` * - Output - One HDF5 file per stellar-mass threshold; figures from ``scripts/plot_joint_sumstat.py`` * - Pass criterion — SMF - Integrated number density within 20% of the HOD input at each mass bin * - Pass criterion — WPRP - Amplitude and slope consistent with the HOD projected correlation within jackknife errors on scales 0.1–20 Mpc * - Pass criterion — WTHETA - Same, on scales 1–100 arcmin * - Status - Scripts exist; runs not yet formally documented as a validation pass ---- Uchuu HOD mocks — DeltaSigma ------------------------------- **Purpose**: verify the ESD pipeline end-to-end on a mock where the true projected matter density profile is known from the HOD. .. list-table:: :header-rows: 1 :widths: 20 80 * - Item - Details * - Dataset - Uchuu HOD galaxy catalogue (lenses) + synthetic source catalogue with shear computed from the HOD NFW profiles * - Estimators - ΔΣ(r\ :sub:`p`) with jackknife covariance * - Launch scripts - ``scripts/measure_joint_sumstat.py --survey mock --stats ESD_HSC`` (substitute mock survey flag when available) * - Output - ΔΣ(r\ :sub:`p`) vs r\ :sub:`p`; comparison to NFW model expectation * - Pass criterion - Recovered ΔΣ within jackknife errors of the analytic NFW expectation on scales 0.1–10 Mpc; no systematic offset at > 2σ * - Status - **Not yet run**; synthetic source catalogue not yet generated ---- COSMOS — SMF and LF --------------------- **Purpose**: validate the SMF (stable) and LF (development) estimators against the widely-used COSMOS2015 measurements of Ilbert et al. (2013). .. list-table:: :header-rows: 1 :widths: 20 80 * - Item - Details * - Dataset - COSMOS2015 galaxy catalogue (Laigle et al. 2016), photo-z + stellar masses * - Estimators - SMF 1/Vmax, SWML; LF 1/Vmax per band (development) * - Launch scripts - ``scripts/measure_joint_sumstat.py --survey cosmos --stats SMF LF`` * - Reference - `Ilbert et al. (2013) `__ (SMF); `Loveday et al. (2015) `__ (LF) * - Output - φ(M★) per redshift bin; overlay with Ilbert+ 2013 data points * - Pass criterion — SMF - Δφ/φ < 30% in bins with > 20 galaxies (accounting for cosmic variance); Schechter α within 0.2 of Ilbert+ 2013 values at z < 1 * - Pass criterion — LF - *(development)* Δφ/φ < 30% vs Loveday+ 2015 in r-band bins * - Status - Scripts exist; figure script at ``docs/scripts/make_literature_figures.py``; formal pass/fail not yet documented ---- GAMA — SMF and WPRP --------------------- **Purpose**: validate SMF and projected clustering against GAMA, which provides a well-characterised low-redshift galaxy sample. .. list-table:: :header-rows: 1 :widths: 20 80 * - Item - Details * - Dataset - GAMA DR3 galaxy catalogue (Driver et al. 2022); spectroscopic z, stellar masses * - Estimators - SMF 1/Vmax; w\ :sub:`p`\ (r\ :sub:`p`) with jackknife covariance * - Launch scripts - ``scripts/measure_joint_sumstat.py --survey gama --stats SMF LF WP`` * - Reference - `Baldry et al. (2012) `__ (SMF); `Farrow et al. (2015) `__ (WPRP) * - Output - φ(M★) vs Baldry+ 2012; w\ :sub:`p`\ (r\ :sub:`p`) vs Farrow+ 2015 per mass bin * - Pass criterion — SMF - Δφ/φ < 20% in bins with > 50 galaxies; Schechter M★ within 0.15 dex of Baldry+ 2012 * - Pass criterion — WPRP - w\ :sub:`p` within jackknife errors of Farrow+ 2015 on scales 0.1–20 Mpc * - Status - SMF script exists; WPRP script needs ``--survey gama`` support; **not yet run** ---- BGS (DESI Legacy Survey) — full joint suite -------------------------------------------- **Purpose**: the primary science target; validate all four stable estimators jointly on the BGS spectroscopic sample. .. list-table:: :header-rows: 1 :widths: 20 80 * - Item - Details * - Dataset - DESI BGS Bright (EDR or Y1); stellar masses from SED fitting; HSC Y3 / DES Y3 / KiDS sources for lensing * - Estimators - SMF, w\ :sub:`p`\ (r\ :sub:`p`), w(θ), ΔΣ(HSC + DES + KiDS); joint jackknife covariance (100 regions) * - Launch scripts - ``scripts/measure_joint_sumstat.py --survey bgs --stats SMF WP WTHETA ESD_HSC ESD_DES ESD_KIDS`` * - Reference - `Wright et al. (2017) `__ (SMF proxy); GAMA WPRP as low-z reference * - Output - Joint HDF5 file per mass threshold; figures from ``scripts/plot_joint_sumstat.py`` * - Pass criterion - All four estimators consistent with overlapping GAMA/COSMOS measurements at z < 0.3; internal consistency between the three ESD surveys within jackknife errors * - Status - Production runs ongoing (see ``logs/measure_mstar*.log``); formal validation comparison not yet automated