Survey Simulations v1.7.1 release (April 2021)

Tags: #<Tag:0x00007f27b1a47cb8> #<Tag:0x00007f27b1a47ba0>

In response to a request for additional simulations from the TVS and SMLV collaborations approved by the SCOC, we have a few additional simulations that you may optionally consider when submitting your Cadence Notes. I know that some collaborations are fairly close to finished with their Cadence Notes – it is absolutely not required to consider these runs for submitting those notes. However, for some science cases particularly relevant to the TVS and SMWLV collaborations, investigating rolling cadence in simulations with denser coverage near the Galactic Plane was a necessity (and not included to any great extent in the previous simulation releases, which demonstrated rolling cadence on a traditional survey footprint with little coverage near the Galactic Plane).

You may also find these simulations useful because they demonstrate additional fine-tuning of the rolling cadence, over a variety of footprints. They demonstrate an improvement in our rolling cadence algorithms which should hopefully do much better at reducing the inter-night gap during rolling seasons.

V1.7.1 Simulations
This is a limited set of 10 simulations, exploring the impact of rolling cadence on three different footprints. The footprints include:

  • a traditional survey footprint (a traditional WFD identical to the baseline)
  • a survey footprint with a simple dec-limited WFD (a WFD ‘stripe’ similar to ‘footprint_gp_smooth’ from the v1.5 footprints family called ‘full_disk*’ here)
  • a survey footprint with a dust-extinction limited WFD coupled with WFD-level visits across the Galactic Bulge and a high-galactic-longitude ‘bridge’ (footprint_6 from the v1.7 footprints_tune family)

image image image

For each of these footprints, we add simulations where there is a 2-dec-band rolling cadence and a 3-dec-band rolling cadence; for the baseline survey footprint, there is an additional 6-dec-band rolling cadence.

All of these simulations are grouped under the new rolling family and make up the v1.7.1 release.

Please note that while several of these simulations use footprints identical to or similar to simulations in previous releases, as always - metric results should be compared to the relevant runs from the same release (i.e. baseline_nexp2_v1.7.1_10yrs is the generic standard here, although to evaluate the impact of rolling on each footprint you may wish to compare to the same footprint without rolling). This is particularly important in this update, as we have also updated the throughput values to current values – this resulted in slightly deeper per-visit magnitude limits in u band, although other bandpasses are generally consistent. We also hope that the impact of rolling cadence in this release is improved over previous releases, due to updates in the rolling cadence algorithm.

As always, you can find the MAF outputs at

and the databases can be downloaded from

The SummaryInfo (alternate viewing location) notebook has been updated, and there is a new summary metric CSV file available with summary stat outputs from these runs as well as previous runs*.

*the updated CSV file (all_summaries_2021_04_07.csv) now does include solar system metric outputs.

Thanks @ljones, this is great. Will these outputs become available on DataLab, too? At the moment I cannot see them in the /sims_maf/ directory. Otherwise one could download them locally and upload them on a DataLab working directory, if the servers allow for a large usage of memory. What do you recommend?

I will ask the Datalab people to add these runs, and can let you know when they would be available (hopefully quickly).
In the meantime, you’re right - you could download them and put them in your datalab working directory (try wget or curl from Datalab directly). There are only 10, which is about 7G — I think this isn’t an unreasonable amount of space to use. Hopefully not everyone has to do this though!
If you find a metric you would like to run on all runs, please let us know.

1 Like

Any update on Datalab?

The installation has been updated and the new runs downloaded!
The new 1.7.1 runs are available in /sims_maf/fbs_1.7.1 (like the previous runs).

Note that you will get some warning messages on import of lsst.sims.maf.db and likely some other pieces of the MAF code. The conda kernel was updated in this version of the LSST software (DM handles this) and the new version of python is stricter about comparisons between strings … the warning messages will go away after the branch I’ve been working on is merged (and then put into the weekly builds and then updated at DataLab) but I don’t expect that to happen until after the cadence note deadline. None of these warnings should cause any errors, however – they just don’t look pretty.

1 Like

Please note: I updated the proposal labelling in all runs, to swap from what I expected/thought the WFD footprint should look like (sometimes inaccurately) to identify the WFD footprint as the part of the sky with at least 750 visits per pointing. This is a better way to do this, because in v1.7 and v1.7.1 we’ve had a widely varying survey footprint design; note for example that footprint_6 includes an extended N/S WFD designed to generally avoid high-dust-extinction areas, with the exception of the galactic bulge and bridge area, and does include the Magellanic Clouds.

Is the CSV without the summary statistics for v1.7 updated as well?

Yes - I have updated the CSV online to add the solar system stats. But, I have not re-run the metrics that use classification as ‘WFD’ (things like “Median Nvisits WFD”, etc.) – although I expect the results to be similar. I also haven’t rerun all of the metadata metrics with the update to fraction of visits in pairs timings that you’d asked for … it’s in progress, but takes a little longer as I will run it on all of the releases. I’ll update separately when that’s done.

Edit… I got an error last night but it has gone away. So never mind

HIya @ljones The link above didn’t work for me. I found the github repo but that has the pre-SSO information. Can you repost the link to the updated CSV file? Thanks

Sorry - it should work now.

Yep that worked. Thanks for the speedy reply.