Dear all,
I hope you will join us for the May LINCC Tech Talk session that will take place on Thursday, June 13, at 10h PT = 13h ET = 13h CLT = 19h CET on Zoom (https://ls.st/lincc-talks ). We will hear from Neven Caplar and Doug Branton, who will talk about the LINCC Frameworks projects - LSDB & TAPE.
LINCC Frameworks LSDB & TAPE – Scalable analysis across large datasets
This presentation introduces LSDB, a software package that facilitates large-scale analytics across multiple datasets. LSDB’s efficacy lies in its ability to shard expansive datasets and index sources within the Healpix space, leveraging catalog density per index. We will shortly describe the project history and design decisions that we have taken (such as using DASK). We will discuss current and future plans, including supporting science cases, International Virtual Observatory Alliance standardization, and collaboration with NASA. We will then discuss our effort to enable time-domain science, present a use case of searching for rare events and anomalies in existing datasets, and showcase how it can be extended to Rubin-size datasets. We will discuss the challenges of working with LSST-like objects/source tables and then discuss our current plans to refactor the code to implement a “nested” structure that keeps objects and sources tightly coupled. We will demonstrate the current capabilities with a Jupyter notebook demo.
LINCC Tech Talks are held on the second Thursday of every month. Events are also advertised at our web page and also provided in calendar form ; and the #lincc-tech-talks LSSTC Slack channel is always available for discussions before, during, and after the talks.