Verification Datasets Meeting 2016-03-16 Minutes

Data Management
#1

Verification Datasets Meeting:

Attendees: Angelo, Zeljko, David, Colin, Hsin-Fang, Tony Tyson, Simon, Frossie, John Swinbank

Hsin-Fang:

  • at UW right now
  • working on obs_decam with Coli, making up for technical debt on obs_decam
  • running stack on raw DECam data
  • obs_decam README updated
  • lots of updates to the stack, dealing with various issues
    • issues with ingest

Colin:

  • getting more ISR implemented in the stack like Subaru

  • pulling over ISR from Subaru, putting it into ip_isr
    can only do intrachip corrections now, not inter-chip

  • Russell is porting HSC non-linearity correction to ip_isr on DM-5462. This will be a task called by IsrTask (right after bias correction). However, HSC uses a polynomial, whereasDECam CP uses a look-up table. We will need a variant task to support lookup tables.

  • maybe put DECam calibration products on obs_decam using git-lsf

    • “compressed” BPM (bounding boxes, similar to subaru), linearity correction and xtalk correction, JIRA tickets filed
    • David will ask Frank/Robert about putting calibration files in obs_decam
    • bias and flats will still need to be downloaded from NOAO

  • jointCal (former simastrom), starting running it on DECam, various states of haphazard working

    • photometric portion probably scheduled in the fall cycle

Simon:

  • at DESC meeting last week
  • twinkles working group
  • twinkles run1 simulated data, one chip, full survey, most nights griz (sometimes u), 1227 total visits
    90% simulated with phosim, 10% took too long to simulate (more than 5 days) when moon was up
  • reduced all simulated images, through stack and forced photometry
  • phot repeatability, 2x worse than Simon’s simple case, phot repeatabiilty ~1.5%
  • coadds looked fine individually, when combined coadd to RGB there are cross-band astrometry issues,
    the offsets are of order of the PSF size,
    not sure what’s happening, investigating
    RMS scatter in WCS fitting solution ~100 milliarcsec
  • example image here https://confluence.lsstcorp.org/display/SQRE/Twinkles+Project
  • there’s rotational dithering in the original images
  • Tony mentioned that DECals has similar astrometic issues between bands
  • airmass? large variety of airmass, but only chose low airmass for images to simulated with phosim
  • working on solving this problem
  • problem likely not DCR, would larger variety of offsets
  • David mentioned this might be the cause of the higher floor on the photometric repeatability, surprised it’s actually not worse, but Simon mentioned that he’s comparing value within each band
  • doing image differencing, seeing lots of SNe
  • no templates yet, but science-to-science image looks very clean

Zeljko:

  • pyastro next week

  • working on revisions of DPDD and LDM-151

  • two intensive days of meetings going over these docs and revising them, next week

  • DPDD good level for requirements, these are the quantities that scientists are interested in

  • Angelo: working on QA database schema

  • Zellko will come out to Tucson soon to chat about QA

Tony:

  • grad student to run deep lens survey through the stack

  • 20 deg^2, 26 mag, similar to LSST depth

  • 145 nights at NOAO, Mayall and Blanco MOSAIC, ~20TB of data

  • flat fielded data are on spinning disk

  • need to set up new obs package

  • Imran Hason, send up to UW for short visit

  • David: should we rename obs_decam to obs_blanco with two imagers, DECam and Blanco.

  • brief discussion on this point and there was a consensus that refactoring obs_decam now would be more
    effort than its worth and people like the idea of separate packages per instrument

  • Zeljko: MOPS update

  • want to do end-to-end test and see if get same results as Lynne/Tim A. got ~4-5 years ago
    using Colin’s detections as well to test false-detections

  • JPL getting better results than us, something wrong with the way we are running our MOPS

  • MOPS convoluted and hard to change, Lynne/Mario thinking about maybe rewriting from scratch

  • should have a better idea in a few months

David:

  • still finishing up the COSMOS rereduction with SDSS as the reference catalog
  • trying to rewrite my LSST IDL workflow in python
  • with science time reducing SMASH data with PHOTRED
  • should have plots for COSMOS next week
#2

I’ve tried to edit the post but got an “500 Internal Server Error” adding more details on my task

Angelo: Making changes on QA database schema. LPM-17 specifies QA samples for single image requirements. We plan to have a single table “QA sources” and Zelko suggested to label the sources for Image Depth, Image Quality, Photometry Quality and Astrometry Quality as it was done in SDSS. I think a good exercise at this point would be to “write the queries” to verify if we actually can select those samples from the outputs specified in LSE-163.