DM Boot Camp Announcement

Data Management DM Notifications
#1

We will hold a “DM Boot Camp” to introduce new hires and interested users to DM processes and the Science Pipelines stack.

Dates

October 5-7, 2015

Sites

UW, Tucson, and Princeton, plus remote participants

We will try to use BlueJeans for videoconferencing, which can support up to 25 participants. Please try to share connections when possible.

To join the meeting on a computer or mobile phone:
https://bluejeans.com/497859855

Directly from a room system:

  1. Dial: 199.48.152.152 or bjn.vc
  2. Enter Meeting ID: 497859855

Audio-only via phone:

  1. +1.408.740.7256 or +1.888.240.2560 (US Toll Free) or +1.408.317.9253 (Alternate number) or +33.4.84.80.12.56 (France) or other numbers from http://bluejeans.com/numbers
  2. Enter Meeting ID: 497859855

Participants

All available UW and Princeton new hires. DavidN, JonathanS, and AngeloF in Tucson, plus other local users if desired. NateP will travel to Seattle. Hsin-FangC will travel to Princeton.

JohnS and LaurenM will provide local instruction at Princeton. RussellO and TimJ will do the same in Tucson. YusraA and SimonK will instruct at UW.

Remote participants are welcome, up to the limit of our videoconferencing tools. Local participants from other subsystems are also expected in Tucson.

Organization

Each day will involve a few hours of lectures delivered via videoconference by subject experts. Due to time zones, lectures will be from 10-1 Pacific (1-4 Eastern). We plan to record lectures for future reference and usage.

A hands-on tutorial with local instructor(s) will occupy much of the rest of the day (afternoon Pacific, morning Eastern); additional time could be spent on personal projects with assistance from the local tutor(s). We will try to enable inter-site sharing and remote participation in the tutorials, but this may not work well depending on how much preparation we are able to do.

Agenda

Each day’s lectures are scheduled from 10 AM to 1 PM Pacific, 1 PM to 4 PM Eastern.

To access the videos linked below, enter “iwillcommentmycode”.

  • October 5:
  • Presentation (1 hr): Introduction to LSST and DM: science goals, system description, DM’s mission, data products, processing flows for Level 1 (Alert Production) and Level 2 (Data Release Production), available data processing scripts/Tasks. MarioJ, pptx, video
  • Presentation (1 hr): Basic afw Concepts: Image, Exposure, afw.table from the end user’s perspective and Using the Butler: repositories, mappers, registries. KTL, pdf, video
  • Presentation (0.5 hr): Using Tasks: arguments, configuration, retarget/subtasks. JohnS, pdf, video
  • Presentation (0.5 hr): Basic eups Concepts: products, versions, tags, dependencies. JohnS, pdf, video
  • Tutorial: Installing the stack, eups practicalities, using the stack to process data (may incorporate more Day 2 content at UW and Princeton)
  • October 6:
  • Presentation (1 hr): DM Organization: DM org chart and people, communication mechanisms, documents, JIRA. JeffK, pptx, video
  • Presentation (1 hr): DM Code Structure: overall system structure, available third-party packages and middleware, structure of a package, scons and sconsUtils, SWIG (existence, not detailed usage). TimJ, pdf, video
  • Presentation (1 hr): afw: contents, how to use (images, cameraGeom, detections), common pitfalls (XY0). SimonK, pptx, pdf, video
  • Tutorial: Modifying existing code: JIRA, git, GitHub, eups, lsstsw, scons, doxygen, Jenkins, code review
  • October 7:
  • Presentation (1 hr): Detection and Measurement: meas_base, afw.table, meas_*, writing a new measurement plugin. JimB, pdf, figures and iPython notebooks, video
  • Presentation (0.5 hr): Orchestration and Control: HTCondor, ctrl_execute. SteveP, pptx, video
  • Presentation (0.5 hr): Writing a Task: writing a Config, using subtasks, CmdLineTask/argument parser/TaskRunner. RussellO, pptx, video
  • Presentation (1 hr): Creating an obs_* Package: camera description, configuration overrides, task customizations, mapper policy and subclass. RussellO, pptx, video
  • Tutorial: Creating a new package: sconsUtils, .table files, .cfg files, third-party packages, eupspkg, distribution server
1 Like
DM Boot Camp 2 (or alternatives)
DM Boot camp 2019
DM Pipelines Bootcamp 2022
#2

Some of us at NCSA are interested in participating remotely in some of these sessions to become more familiar with LSST Data Management. Is that OK? Are the videoconferencing details available?

#3

Yes, it’s OK, up to the capacity of our videoconferencing. Details will (had better) be available later today.

#4

Hello Boot-campers, I’d like to do a blog posting or E-News article about the DM Boot Camp and need a picture. Perhaps I can combine three individual pictures into a triptych of sorts, not sure. But I’ll need a group photo of each location to begin. I’ve arranged for the Tucson photo to be taken right at 10am today, but Princeton and UW, would you please email me a group shot today of your campers? Thank you, Suzanne
[I sent this to a few by email but wanted to try making a discussion post as well.]

#5

For those like me looking for the support documents for this boot camp, they are available here:

#6

For the final (October 7) lecture session, we have agreed to schedule a 30 minute break at the two hour mark (ie, after “Writing a Task” and before “Creating an obs_* package”).

#7

Does anyone know the location where the videos of the lectures are being posted?

#8

We’re still working on editing, reassembling, and uploading them somewhere. (An initial attempt expanded them to >15GB/day.)

#9

For what it’s worth, I fixed my script for the afw talk so it now will increment the frame like I meant it to.

#10

I just want to say thank you to all the speakers. This was fantastically useful for newbies like me.

I also think the act of describing and talking about the stack, from a high level, was very useful for thinking about what is needed to improve the stack as a cohesive thing. I hope we can regularly do something like this (even if not at such a grand scale) at the all-hands meetings and whenever new functionality/architecture comes into the stack.

1 Like
#11

I have added links to the presentation slides above. We’re still waiting on video.

#12

A news item on the Boot Camp is now available.

#13

Video links have been added above (thanks, Tim and Frossie!). To access the videos, you need to enter “iwillcommentmycode” (and please (continue to) do so :smile:).

DM’s New Developer Guide: developer.lsst.io