# TESS target-selection rule and shortlist Status: draft pipeline for human verification. It applies a fixed rule and records archive provenance. It does not choose a target, does not claim that any sector is new, and does not claim a research result. The rule comes from `docs/PROBLEMS-DEEP.md` section 1.4. That section asks for a confirmed planet, TESS magnitude brighter than about 11, period shorter than 10 days, at least two TESS sectors, and at least one sector that is not in the published ephemeris. ## Rule as implemented A candidate passes when all of these hold: | Gate | Test | Boundary | | --- | --- | --- | | Confirmed transiting planet | archive `tran_flag == 1` | not applicable | | TESS magnitude | `sy_tmag <= 11.0` | 11.0 passes | | Orbital period | `pl_orbper <= 10.0` days | 10.0 passes | | TESS sectors | at least two unique sector numbers | 2 passes | The boundaries are inclusive. Section 1.4 says "about 11" and "shorter than 10 days", so the exact edge is a choice. It is fixed here, written into every report, and tested at 11.0 and 10.0. A missing magnitude or period is not treated as passing. Shortlist order is deterministic: TESS magnitude ascending, then period ascending, then TIC ascending, then planet name ascending. Archive row order cannot change the output. ### Sector novelty Novelty is not one of the numerical gates. Whether a TESS sector was used in a published ephemeris cannot be established from archive summary fields. The code therefore reports a status instead of guessing: - `POSTDATES_REFERENCE_YEAR`: at least one recorded sector start year is later than the year parsed from the archive's transit-midpoint reference. A sector that started after that reference year cannot have been part of that fit. - `UNVERIFIED`: everything else, including a missing reference year, a sector from the same year as the reference, or no sector timestamps. `POSTDATES_REFERENCE_YEAR` does not prove that the published fit omitted the sector. It is a one-sided inference from recorded dates. Confirming absence requires reading the ephemeris table in the cited paper. ## Layout ```text research/tess-timing/ pyproject.toml README.md src/tess_timing/ model.py data model and MJD conversion selection.py pure rule functions and the novelty flag archive.py read-only Exoplanet Archive and MAST queries report.py demo and fetch runs, JSON and Markdown output provenance.py download records, hashing, and JSONL manifest checks scripts/select_targets.py scripts/provenance.py tests/ data/demo_candidates.json data/raw/ raw response bodies from a fetch run TARGETS.json TARGETS.md ``` Paths are resolved from the script location in `scripts/`, so commands work from the repository root and from any other working directory. ## Demo ```bash python research/tess-timing/scripts/select_targets.py --demo ``` The demo reads `data/demo_candidates.json` and makes no network request. Every row in that fixture is synthetic and labelled `synthetic: true`. Both outputs mark every row synthetic. The fixture covers the 11.0 magnitude boundary, the 10.0 day period boundary, the two-sector minimum, each exclusion gate, the two novelty statuses, and a TIC tie-break. Optional arguments: ```bash python research/tess-timing/scripts/select_targets.py --demo \ --fixture research/tess-timing/data/demo_candidates.json \ --out research/tess-timing/data/demo-run ``` ## Fetch ```bash python research/tess-timing/scripts/select_targets.py --fetch ``` This is a read-only query against public services: - NASA Exoplanet Archive TAP `sync` endpoint, table `pscomppars`, using the ADQL recorded in `TARGETS.json` and `TARGETS.md`. - MAST `Mast.Caom.Filtered` endpoint, TESS timeseries metadata for batches of bare TIC identifiers. The run records the UTC request timestamps, exact queries, HTTP statuses, response digests, and raw response files under `data/raw/`. A failed query writes `TARGETS.json` and `TARGETS.md` with `status: FAILED`, the HTTP status (or `none` for a network error), the stage, and no candidate rows. It never falls back to hand-entered planet rows. MAST can return more than one product for a sector, and some products span multiple sectors. Sectors are counted from TESS timeseries metadata, so an FFI-only observation without a timeseries product is not counted. Rows are deduplicated by sector number. A row whose `t_max - t_min` exceeds 40 days is dropped rather than assigned to one sector, because one TESS sector lasts about 27.4 days. ## Output A completed archive run is kept at `data/fetch-run/`. The two files at the package root come from the demo, so they are synthetic. `TARGETS.json` records the rule, thresholds, query provenance, counts, every evaluated row, and the shortlist with TIC, sectors, ephemeris fields, citations, and sector-novelty status. `TARGETS.md` records the same rule and provenance in readable form, with one section per shortlisted candidate. In the demo, the Markdown also lists every evaluated fixture row and marks it synthetic. Neither output chooses a target, ranks candidates beyond the fixed sort order, or claims novelty. ## Download provenance `scripts/provenance.py` records the download metadata required by `docs/PROBLEMS-DEEP.md` section 1.3. It does not fetch data, contact a service, or use a network client. The `add` command hashes a file that is already on disk. The `verify` command reads the manifest and, optionally, one local file. ### Manifest format The manifest is UTF-8 JSON Lines: one JSON object per line, with a newline after the last record. A record has exactly these twelve fields. Missing fields and unknown fields are rejected. | Field | Type | Meaning | | --- | --- | --- | | `target` | string | Target label as recorded at download | | `tic_id` | integer | TIC identifier, positive | | `sector` | integer | TESS sector number, positive | | `cadence` | integer | Exposure cadence in seconds, positive | | `product_type` | string | Product class label, for example `lightcurve` or `target-pixel-file` | | `file_url` | string | Absolute `http` or `https` URL of the product; embedded credentials are rejected | | `downloaded_utc` | string | ISO 8601 timestamp with an explicit UTC offset; a zero offset is required and writers store `Z` | | `sha256` | string | SHA-256 of the local file, 64 hexadecimal characters, stored lowercase | | `byte_count` | integer | File size in bytes, non-negative | | `aperture` | string | Aperture label or policy used for the reduction | | `quality_mask` | string | Quality-mask label or policy used for the reduction | | `code_version` | string | Version string of the code that wrote the record | `aperture` and `quality_mask` are labels chosen by the person recording the download. The tool does not interpret or verify them against a pipeline. The uniqueness key is `(tic_id, sector, product_type)`, with the product type compared case-insensitively. Cadence is recorded but is not part of the key. Two records with the same key are rejected by both `add` and `verify`. ### Add a record The SHA-256 and byte count are computed from `--file`; they are not accepted as arguments. The download time defaults to the current UTC time. ```bash python research/tess-timing/scripts/provenance.py add \ --manifest "$MANIFEST" \ --file "$LOCAL_FILE" \ --target "$TARGET" \ --tic-id "$TIC_ID" \ --sector "$SECTOR" \ --cadence 120 \ --product-type lightcurve \ --file-url "$PRODUCT_URL" \ --aperture "SPOC SAP" \ --quality-mask "default" ``` Optional arguments are `--downloaded-utc` (ISO 8601 UTC, for a download made earlier) and `--code-version` (default: the package version, `0.1.0`). A successful add prints `status=OK`, the record count, and the digest. A bad timestamp, a bad digest, a duplicate key, or a byte count that does not match the file exits with status 1 and writes nothing. Missing required arguments are rejected by the argument parser before any file is read. Existing line content is preserved; a missing final newline is added before the new record. The write goes through a temporary file in the same directory and an atomic replace, so a failed add does not leave a partial line. ### Verify a manifest ```bash python research/tess-timing/scripts/provenance.py verify --manifest "$MANIFEST" python research/tess-timing/scripts/provenance.py verify \ --manifest "$MANIFEST" --file "$LOCAL_FILE" ``` Without `--file`, verify checks every line for exactly the twelve fields, field types, an ISO 8601 timestamp with a zero UTC offset, a 64-character hexadecimal SHA-256, and duplicate keys. With `--file`, the manifest must hold exactly one record, and its `byte_count` and `sha256` are compared with the local file. The command reports every problem it finds, not only the first. Neither subcommand downloads a product or contacts MAST. Verify cannot prove that a recorded URL, target, TIC, sector, cadence, aperture, quality mask, or code version is correct; it proves that the record is well formed, unique, and, when a file is supplied, consistent with that file. ### Self-test ```bash python research/tess-timing/scripts/provenance.py --self-test ``` The self-test is offline. It writes a small payload and manifest to a temporary directory under `.scratch/`, runs the add and verify paths, and checks ten cases: hashing, the twelve-field record, the append and verify round trip, a missing field, a malformed timestamp, a non-UTC timestamp, a bad SHA-256, a duplicate record on add, a duplicate record on verify, and a byte-count mismatch. Success prints `status=OK`, `self_test=OK`, and the check count, then exits 0. A failure prints `status=FAILED` with the failed label and exits 1. The temporary directory is removed when the run ends. ## Tests ```bash python -m pytest research/tess-timing ``` The tests are offline. They cover the inclusive Tmag and period boundaries, the two-sector minimum, each exclusion gate, deterministic sort order, novelty statuses, MAST product filtering, reference parsing, the failure record, the documented demo command run from the repository root, and the provenance manifest. The provenance tests cover a missing field, an unknown field, a malformed timestamp, a non-UTC timestamp, a bad SHA-256, a duplicate key, and a byte-count mismatch, plus the `add` and `verify` commands run from the repository root and from another directory. ## Dependencies Python 3.12 is the tested version. The package metadata in `pyproject.toml` declares NumPy, SciPy, and Astropy for the analysis scripts, with pytest in the `test` extra. The target-selection library modules use the standard library; lightkurve, batman, emcee, and a network client library are not used. ## Limits - The query selects summary-table rows. It does not inspect light curves, data quality flags, contamination, or transit detectability. - A shortlist entry is not evidence that the planet is suitable for a timing measurement. - The demo fixture is not a dataset and must not be cited as one. - A novelty flag is not a novelty claim. Read the cited ephemeris before saying a sector is unused.