LSST-related notes from Python in Astronomy 2016
LSST attendees: @mjuric, @timj. @rowen, @yoachim, @parejkoj.
The Python in Astronomy 2016 meeting brings together Python enthusiasts from the community for an annual week-long meeting. This year it was held in Seattle at the University of Washington so that people from the LSST were able to contribute. Presentations from the meeting have been archived on Zenodo and can be cited (and will be indexed on ADS). The talks were all recorded:
The meeting was a mix of formal presentations, tutorial sessions, lightning talks, coding sprints and ad hoc “unconference” sessions where attendees would form into small groups to discuss specific topics.
There were some presentations from LSST members:
- @parejkoj gave an overview of the LSST and its software. (Video)
- @rowen gave a lightning talk on the use of linters. (Video@1h26m)
- @timj gave a short overview of Zenodo.
- @yoachim hosted an unconference session on the LSST simulation analysis.
The intent of the organizers is for there to be a PDF published summarizing the meeting in an “unproceedings”.
Programming Tips
- Conda environments were mentioned many times as a way to sandbox different Python versions and distributions.
- There was a focus on helping people port their code to Python 3 and a number of sessions explaining why Python 3 is the future and explaining how to use tools such as
sixandfutureto retain compatibility with Python 2. - Lots of talk of editors (Atom vs Sublime Text) and plugins,
tmuxandzsh. - There was a presentation on extending Python with compiled code.
- A handy introduction to Python multiprocessing and optimization: https://github.com/mboquien/pyastro16
- A group discussed project management and work tracking systems, e.g. for tracking student work. No one from LSST was there to represent JIRA, but it was discussed in the summary as an option for very large groups.
Other Projects
- CTA are trying to decide on where to put their Python/C++ boundary. Historically the community has been dominated by ROOT and C++.
- SunPy is making (slow) progress trying to move the Solar community away from IDL.
- AstroPy continues to gain momentum with new releases and affiliate packages. Significant changes or additions to AstroPy Core are done via “Astropy Proposals for Enhancement” (APE), which serve the same role as the LSST RFC process.
- The AstroPlan package looks like it has a really nice API.
-
ccdprocversion 1.0 (CCD processing) came out recently and adds some extensions to the AstropyNDDatamodel. - There is a surprising lack of convergence on reducing and analyzing spectra in Python. An attempt at cataloging what has been done can be found at https://github.com/jradavenport/pydis/wiki/Other-Packages
- From the LSST sessions it was clear there was interest in using the LSST software but a strong desire for the software to be installable using
piporcondain the standard manner.
Regions
Region handling is an open issue in Astropy at the moment. There was a hack session in the week to work on an experimental API and incorporate code that can handle DS9 region files. Currently uses an STScI spherical geometry package that is written mainly in Python. There are clear synergies with LSST in this area (which has its own C++ library for that). @timj noted that STC-S regions (usage example) provides a fairly simple exchange format that might be useful.
astropy.modeling
Significant discussion of the limitations of the astropy.modeling fitting system, and whether astropy should provide any “simple” fitters at all, or push all of its fitting to external tools (e.g. scipy, emcee). A few different groups collaborated on a demonstration of how to fit astropy models with particular fitters, including Sherpa and iminuit. This is likely to develop into an APE.
Photometry
There was some discussion of a package for doing aperture photometry on asteroid trails. The package was called Trippy.
NDData
NDData is the container class in Astropy for holding data, variance and masks together. Much of the discussion focused on the “mask” property and on flags. Projects like JWST and LSST need a way to provide information about mask planes and allow the user to select mask planes to apply in a particular situation. In the current framework that requires some interplay between mask (currently a boolean property) and flags (not included in the NDDataBase interface but are present in NDDataArray).
Of particular interest for LSST is that NDData does not have a concept of xy0. However, following the LSST/Astropy summit later in the week, AstroPy is considering adding this (@stsci.perry is working on an “APE” in collaboration with @rowen).
Glueviz
Glueviz is a data visualization system with linked views.
- Interesting usage: 3d view of dark/flat/other stacks to look for time structure (for example looking at Chandra data: x/y/energy, and you could see what was probably the energy loss from charge transfer)
- Click the IPy button below the data window to get an ipython notebook.
dcis the “data component”(s) (e.g. files) you currently have open. - There was some discussion about Bokeh integration. It is possible Bokeh could be used as a frontend visualization tool for glue.
Ginga
Ginga is another visualization tool that can view images and can be embedded in an iPython notebook. It is quite extensible, and one very useful looking extension is a widget for viewing tables that links the data to the displayed image (though the way it is configured to read different data sources may want some work). LSST would benefit from making afw.display talk to Ginga.