Alert schema changes

Science Alerts & Brokers
#1

The Lasair team has a few questions about changes in the alert schema.

  • We note that standard names have been adopted across all of diaSource, diaForcedSource, and diaNondetectionLimit, namely these six attributes:
    ra, dec, band, midpointMjdTai, psfFlux, psfFluxErr.
    As a result, the Lasair codebase has been changed to use these names consistently throughout. Can we assume that these particular names will not change again?

  • We note that the schema change from 7.0 to 7.1 involves adding two and removing two attributes. Would it be possible to get a summary of changes in some kind of human readable quasi code, for example
    diaForcedSource add column ra double, dec double diaForcedSource drop column x, y

  • We are thinking it is lightcurve features where the majority of schema changes will happen, is that right? (As in DMTN-118). Already there are simple per-band statistics like psfFluxMean and psfFluxChi2, but more will be added. These kind of additions are easy for Lasair to handle. However, our users will build queries against these attributes, therefore it would be great to know that once a feature is added to diaObject, it will not be removed.

#2

Hi @roy ,

  • I think it is likely these will remain unchanged going forward
  • I don’t know that I’d use “quasi-code”, but you’re right to request a changelog. I’ll see that we add something to GitHub - lsst/alert_packet: Alert schemas in Avro format..
  • I think it’s correct that most of the changes are likely to come in the lightcurve features, but there are other fields that we think may require attention as well (for example, the model comparisons between PSF, dipole, and trail fits).

In general, at least until real LSSTCam alert production begins the AP team will continue to consider making breaking schema changes that are backwards incompatible. While we recognize that this is disruptive in the short term it seems preferable to carrying vestigial columns around for a decade. By necessity such changes should slow once we have a production database that must be migrated.

#3

Thank you Eric. Of course at this time we can just delete all data and start again. But once the real data is in the database, we will all tread much more carefully.