What to include in DM status presentation to DESC next week?

Data Management
#1

Hi everyone,

I’ve been asked to give next Tuesday a short (15 minute) summary of DM status to DESC (they’re gathering at SLAC for the semi-annual meeting). I’m supposed to focus on new developments since @jbosch gave his DM status talk at Argonne last October.

Here’s a rough outline of what I’m thinking about presenting right now. I’d be super grateful if you looked this over (especially if you’re active in DESC). Are there any things I could safely leave out? What else may be of interest to DESC? What would be of interest to DM, for the DESC members to be aware of?

Current talk outline:

  • DM status

    • Continuing to staff up
      • Even more people shared with the DESC!
    • New management structure
      • Describe short-term goals
      • Working groups

  • Highlights of direct interest to DESC

    • Architectural work DESC cares about
      • Level 1 system design updates
      • [perhaps mention AstroPy efforts? too early? not interesting to this group?]
    • Science pipeline work DESC cares about
      • Functionality ported over from HSC
      • rhl maybe to give a demo at the end of the talk
    • Middleware work DESC will care about
      • butler 2.0
    • Expectations for the next ~6 months
      • [ideas what to specifically mention?]

  • Communications

    • Improved getting started & developer manuals
      • Esp. point out the technical notes
    • Emphasize community.lsst.org
    • Existing work on binary distributions of the stack [too early to mention this?]
    • Continue with close collaboration of related DESC & DM WGs
      • [what examples of this working well do we already have?]
#2

That is a lot to cover. I think what DESC would like to hear about is

  • when will DM work on a particular task (i.e. the roadmap). You don’t need to present it just point to it for others to read when they are interested in a problem
  • how does DM want to interact with DESC. If there is an area where there is an interest in common how do you want the effort to be organized so everyone knows what is happening. You could refer to the PST monthly meeting (where DESC members could raise things they are interested in - through Rachel B et al) as well as community.lsst.org. Also I think Tony is tasked with making sure these tasks are tracked and followed up on so you might want to check with him about how he is planning on doing that.
#3

I see there is some interest in using Qserv (@brianv0 might know more about it) . Let me know what time is your talk, I can try to stop by (can I? I haven’t registered)

#4

I strongly agree that Roadmap and timing are of criticla interest to DESC. Next after that is usability of LSST Project software.

Lead with Communications.
Definitely announce any improved getting started guides.
Remind people about DM Boot Camp slides. These, together with some example notebooks, such as those prepared by Chris Walter, are probably the best entree into DM for DESC members.

The tech notes are not, as of yet, about things that are of particular interest to DESC. However, the idea of the platform may very much be of interest to DESC.

Before you give your presentation, talk to Twinkles folks such as @marshall , @tony_johnson , @danielsf to get a sense for the most recent efforts of DESC members to use LSST DM + Sims software together. (Short story, it wasn’t as easy as it should have been.) Twinkles will be one of the most visible things DESC people will be hearing about that is actively using the LSST Project software.

DESC members would default to wanting to see AstroPy become the default. You could touch upon such efforts briefly, but it’s of course too early to make promises so be very clear about where things stand.

I don’t think most of DESC knows to care about Butler 2.0

HSC stuff will be very much of interest. Simultaneous astrometric and photometric fitting+calibration will also be of interest.

#5

I always like to point out @boutigny’s excellent work on CFHT reprocessing in this category. It may be particularly timely to mention it now, as I believe Dominic have significantly less time to work on this himself going forward, and I believe that could leave a real void between DM and DESC.

#6

I don’t think it should be a problem for you to come. It’s at 9AM in the Kavli Auditorium.

#7

Thanks everyone! I’ll be cooking up the slides this evening, and post them here later tonight.

I’ll share the stage with @RHL, who’ll show some of the results of running the stack code on HSC, so I’ll need to keep my part short (and won’t be able to include all the proposals). But I think it’s a good tradeoff – results are worth a thousand slides :).

#8

Hi everyone; here’s the presentation I plan to show, incorporating a number of your comments (it wasn’t possible to incorporate all of them, but I tried to strike a balance).

PS: I can still make minor fixes in the next ~two hours or so.

#9

Please add it to Zenodo when you are done (or tell me when it’s done). Thanks.

#10

Mario, the links in your presentation seem to be broken. They are not clickable and when you copy-paste them there are spurious characters and you need to rewrite the ‘https’ part for the browser to be happy.

I’m reading your presentation in MacOS X El Capitan with built-in Preview PDF reader.

#11

@FabioHernandez, thanks, I’ll take a look at what’s going on. I think it’s the powerpoint messing things up.

#12

Hi everyone; thanks for the comments and help. This was the final presentation:

.