Let’s generate a science-driven optimization of the template collection and generation plan for LSST during commissioning.
2 Likes
From the PST-SC chairs conversation on 11/20/2019, being clear that the templates for image differencing will only be available for a fraction of the sky (~10%? ) by the end of commissioning, we should discuss what combination of sky areas and bandpasses should be prioritized and make recommendations for alert-generation in DR1. Different science drivers would lead to a different plan for template collection, e.g. trivially MW science would prefer templates in the galactic plane, EG science and cosmology outside of the plane. Please share this board with all of the SCs. At the appropriate time we will move this discussion to a more formal venue to make official recommendations.
Thanks for the update! I think that for TVS it could be more important to have templates in all bands, rather coverage in one or two bands over a larger area. With coverage in all bands we can use the commission time to test rolling cadence strategies with all possible filter pairs. Also we can test how well we can measure the distance between a transients and its host galaxy for each of the LSST bands. I suppose that for photoz commissioning it will also be important to have good depth in all bands.
It might be good to focus on a decent chunk of sky that has good archival coverage. It will be useful to know before-hand where the AGN are, since these provide a good control sample. Stripe 82 seems like an obvious candidate.
the slides deck of the PST (@leanne)'s presentation to the SCs about alerts in Y1 is here https://drive.google.com/file/d/15kexPAnwuGaJc5xV6bbOBk9QYxFEIWAO/view?usp=sharing
I don’t have strong opinions about the commissioning part. The priority needs to be setting the survey up for success.
Nevertheless…
I guess I would prioritize testing in commissioning how many (2? 4?) visits in each band is sufficient to make a template that is “good enough” to send out alerts without damaging the reputation of the survey and putting out significant “bad” alerts [if that’s possible at all]. Perhaps that is better phrased as ‘at what threshold can we generate good alerts with such a template’ – 10 sigma instead of 5 sigma? Maybe a cut for brighter than 21st mag? Maybe u doesn’t work without a deep template but other filters are okay? [Just made up examples.]
Then I would try to generate such limited alerts in Year 1 as soon as the project has the minimum number of frames in that field to make a good enough template from normal survey operations. That should only be a few months, I think.
I am still trying to come to grips with the impact of Differential Color Refraction with the Difference Image Pipeline. So far as I can tell from Leann’s charts, the Difference Image is generated by a pixel-based subtraction of the current image from the template image. If these are not in the same color, then I would expect every object with a funny color will generate an alert. If these are in the same color but at significantly different zenith distances, the many spurious alerts would be generated. My presumption is that somewhere in the DIA Source process that colors+ZD will be used to suppress these, but all bets are off until the requisite data are available.
Perhaps I am missing something really obvious, but the whole DIA process sounds pretty lame until the survey has enough observations to build the templates and source color tables. It seems that these data will not be generated during Commissioning, no matter what subset of the sky is deemed “important”. Sorry to muddy the water. Clues would be appreciated.
Hi @dgmonet , we are in fact here to discuss what should be prioritized so that it can be scheduled to be generated during commissioning. Of course there is a limited (and not yet finalized) amount of time so not all of the sky can be collected in all of the filters. How much/in how many filters, will get more clear the closer we get to commissioning but we need a plan to make the best of however much (hopefully a lot) time we have in commissioning to get templates. Similarly, template collection will continue thorugh DR1 and we need a plan for prioritizing sky areas/filters. Many considerations will go into that plan, but we, the SCs, have the science knwoledge to present science-driven optimization schemes. All your points are valid, and I think if I had to sum them up I would say that in order to make a careful plan we need to better understand the DIA stack.
Differential Chromatic Refraction (DCR) is something I’ve been working on for the LSST difference image pipeline. We will do a pixel-based subtraction of the science image from the template, but the goal is to build the template for each image based on the observing conditions. Please have a look at https://dmtn-037.lsst.io for a detailed description of how this works.
For commissioning, it is true that we won’t be able to use these DCR-matched templates until we’ve observed a given field at least five times, while more observations will give better results. You can read https://dmtn-121.lsst.io for an analysis of how these templates should perform in the first ~year of taking data.
1 Like
I’m interested in doing follow-up to alerts for young star variability where prompt follow-up on a timescale of weeks is important. So I would like to have the maxim number of possible alerts in the first year. In this case the best policy would be to maximise area (i.e. pick a single colour), since I could sort out the colour variability for a given alert later. As to region, well most of my alerts would be in the Galactic Plane so I would like regions there, and of course in terms of variables per field this is the most productive area.
For WFD extragalactic transients like supernovae, I think we would like a smooth “on-ramp” to the steady state survey. That would favor producing templates in commissioning that would be similar to the templates we would eventually expect to have, i.e. multi-filter, with similar depth. This naturally would mean a smaller area than the full footprint during commissioning, and building up the whole footprint during year 1. Even if this is just 10% or less of the area with good templates from commissioning time, that will still provide a large number of alerts (more than current surveys).
The DDFs are a great place where commissioning can potentially provide full-depth, all-filter templates (for most if not all fields), such that we can start doing extragalactic transient science from day 1 in these fields. There’s a lot we can do with one year of DDF transients.
The statements below come from the DESC SN and SL working groups, and so encompass the needs of those interested in SNIa cosmology and time-domain strong lensing studies (e.g. quasar variability, strongly-lensed SN). The needs of these groups are coordinated through the DESC Commissioning Working Groups and the associated DESC note on commissioning fields. As such these comments are supportive of those baseline needs and fields.
Comments from DESC SN and SL WGs
Q1: Would you want us to focus on building templates for as much of the sky as possible to enable better alerts in LOY1 at the expense of other validation activities?
Response:
The single highest priority is building good DDF templates in commissioning so that SN science operations can begin in LOY1, operating in a mode similar to DES/SNLS/PS1 MDS.
(note DDFs = 4 approved + Akari Deep Field)
SN favors commissioning data templates (i.e., before LOY1) for the DDFs, and LOY1 data templates (i.e., during LOY1) for WFD, with WFD alerts ramping up during LOY1. Having a subset of Stripe 82 (~100-200 sq deg) imaged in gri[uz] during commissioning for validation of the WFD operations prior to LOY1 would be extremely useful for SN and SL, and of use to the DESC Photometric Calibration WG and other groups as well. We do not want building templates for as much of the WFD as possible to detract from other validation activities in commissioning (particularly prompt processing, alert distribution, and DIA).
Q2: Which is the optimal filter if we focus on building wide-area single-filter templates? Or would smaller area with two filters would be better? [r or i band, will avoid the DCR of u and g and the lower signal of z and y.] Are there further optimizations?
Response:
Note that the discussion below only applies to templates for WFD in commissioning (we would like a full complement of filters for the DDFs).
A smaller area with two filters is better than single filter over a large area. Templates in a single filter is not useful for SNWG operations as the lack of color information makes identification and classification impractical. (Three filters for WFD would be ideal.)
Two filters chosen in a moon phase-dependent pattern (g +r in dark - 50% grey, r+i otherwise) is the minimum that is useful to SNWG. This also allows testing of operations in two sets of pairs. Additionally, masking all sources within 2-3 mag of non-linearity will be useful prior to DIA will be useful to help mask spurious sources from DIA with these commissioning templates. Also, investigations of image quality requirements in templates will also be useful for DIA.
Q3: For which locations on the sky do you want templates?
Response:
We want to emphasize that templates for the DDFs during commissioning are crucial for SNWG science operations in LOY1. In addition, for WFD templates, we would prefer Stripe 82 for imaging of any wide continuous strip due to the overlap with existing datasets. An alternate choice would be imaging large contiguous regions surrounding DES S1, S2, S3, or the HSC SN fields.
We note that field have already been identified with the DESC Commissioning WG, which will be released as a DESC note in the coming months.
1 Like
I’d also like to add to @reneehlozek’s post on behalf of DESC SNWG that when visits of the DDF are obtained during commissioning to build co-added templates, spacing those visits by 1-3 nights in each filter (as opposed to obtaining them all on one night) allows:
a) DESC to test DIA with image-image subtractions that we produce internally
b) flagging periodic variables in the DDFs so we can filter them out during LOY1
c) testing the broker teams with real LSST data, and the DESC spectroscopic follow-up system
We’d love to coordinate with the commissioning team here to get contemporaneous observations of the DDFs with other facilities (e.g. DECam)
Hi all : the TVS SC put their thoughts together in 2 research AAS notes
You can find the notes here
https://iopscience.iop.org/article/10.3847/2515-5172/ab812a
And here
https://iopscience.iop.org/article/10.3847/2515-5172/ab8129
For time-critical and non time-critical science respectively (generally speaking science that is and is not based on the alert stream). It was a good exercise to struggle with the tight word limit, which allowed us to get a comprehensive, prioritized look at our needs. I am going to post the synoptic recommendation below, but I encourage you all to read the notes and give us feedback!
1. Recommendations: Time critical
-
- All science cases are enhanced with templates in two filters, most commonly r and g . Some science cases require both filters to be feasible.
-
- Acquiring r , g -band templates of a large region of the extra-galactic sky services a wide range of time-critical science.
-
- Templates in the Galactic Bulge and Magellenic Clouds enable galactic transient science and enhance overall survey yield.
Slowly evolving transients such as Luminous Blue Variables, Superluminous Supernovae and Tidal Disruption Events also benefit from early template observations over large sky regions so that candidates can be identified as early as possible.
We recommend coordination with LIGO/Virgo. The O4 observing run (planned for 2021), overlaps with LSST commissioning and the start of operations. Identification of counterparts to GW sources would be valuable early-science but the lack of templates would delay the contribution of LSST to GW science until LIGO/Virgo-O5 in ~2025. If O4 were to start during year one, incremental template building strategies would be advised.
Observations of Deep Drilling Fields are also of crucial importance to transient and variable science goals that require denser time sampling, and the TVS SC also strongly advocates for early creation of templates for the DDF in multiple filters.
Synthesizing these science cases, we make the following recommendations:
2. Recommendations: Non-Time critical
-
- Coverage of the Galactic Plane and selected nearby galaxies and regions, including crowded regions, is crucial to make strides in all Galactic variable science.
-
- The large range of variability timescales requires that the templates be generated from images collected at a range of time separations. A roughly logarithmic scheme (from ~days to ~6 months) is recommended.
-
- Templates in two colors benefit all science cases through the acquisition of colors that aids classification and extraction of physical parameters.
-
- Template construction in g - and r - or g - and i -bands should be prioritized as they capture the requirements of most science cases.
In addition to being studied in alert-and-follow-up mode, several families of transients will be studies through statistical samples of “standalone” LSST data. Color samples will be necessary for these science cases to decontaminate samples or provide useful priors (see companion paper: Street et al. 2020).
Hey Fed,
I don’t understand the recommendation that templates be built out of logarithmic spaced observations. I understand templates need to be somewhat recent (otherwise proper motions of stars will make difference image dipoles), and templates need to be of comparable hour angle (so DCR doesn’t make dipoles). Let’s say I take 3 images in a month and coadd them to make an image template. Why would taking them on the 1st, 5th, and 30th be better than 1st, 15th, 30th or three observations on the 20th?
Hi @yoachim - The variable stars people can answer this better @aprsa (Kelly is not on Community??) - but the idea is that star variability happens on all time scales: if you build a template that includes images spaced by days and weeks and months (up to 6 months was the original recommendation that got cut because of space constraints) you average over all those time scales, if you do not you will miss some variabilities and there are examples of this happening in previous surveys (Kepler - https://iopscience.iop.org/article/10.1088/0004-6256/141/3/83/meta) .
I get why you want observations spaced at all time scales to get variability, I just don’t understand why that would matter for template generation. You don’t need to use the alert stream to identify variables, you can just look at light curves. I don’t think Kepler even does difference imaging.
The rationale is that we want to avoid bias in templates for the same reason we want to avoid detection issues in variables. If we represented a “static” sky at a certain cadence, and that cadence happened to coincide with the periodicity, then we would be potentially misattributing the brightness to those specific sources. While that is unlikely an all-sky problem, it /can/ be a local problem for DIA, one that seems to be entirely avoidable with an irregular sampling strategy.
Fed & colleagues, Some comments on TVS needs for sky templates.
Time-critical science cases
1.1 “particularly to distinguish supernovae in the Magellanic Clouds.” Aren’t these exceedingly rare (last in 1987)?
1.3 “Fast transients include Fast Blue Optical Transients, Fast Radio Burst counterparts, and fast features of longer duration transients”. Should Gamma-Ray Burst afterglows be mentioned, particularly the more common off-axis orphaned GRBs that don’t show up in Swift/Fermi/Integral.
- Recommendations “3. Templates in the Galactic Bulge and Magellenic Clouds enable galactic transient science and enhance overall survey yield.” May I suggest that templates for “upper Galactic disk” be added? Here is the thought:
Galactic Bulge usually means Baade’s Window or similar regions that emphasizes Pop II populations. Extreme Pop I populations are very interesting (e.g. EXors/FUors in sec 1.2) but the environment is very messy for |b|<1d: patchy emission nebulosity, patchy absorbing cloud, very crowded fields even for LSST. So I suggest that TVS think about not-so-extreme Pop I populations around 3 < |b| < 10 deg and longitudes away from the Galactic Center. Here we can capture large populations of magnetically active dMe flare stars & ZAMS stars, compact accretion binaries, instability strip variables, etc.
Non-time-critical Science Cases
With respect to periodic variables, the document shows concern with cadence structure to reduce aliasing: “Aliasing of variability periods in the collection of the template images could result in systematics … An optimal strategy would collect images in a roughly logarithmic timescale with intervals from days to months.” But for periodic variables, there is also a need for an adequate number of observations in the survey based on these templates. Extracting reliable periods from periodograms (Lomb-Scargle, Stellingwerf, etc) is very troublesome unless there are hundreds of data points, irrespective of the spacing. I have been playing around looking for rotational periods in ZTF DR1 with ~100 observations, and I’ve decided to wait until DR3 due to unreliability of the periodograms peaks.
Now, we don’t plan to extract periods from the templates, but from the later survey. But this does suggest that will want denser cadence coverage of smaller portions of the Galactic Plane. So I suggest that this point be raised, and phrases like “Recommendation 1. Coverage of the Galactic Plane” be changed to something like “portions of the Galactic Plane”.
Eric