When I run the Introduction to the Butler notebook tutorial (4a) in the recommended medium container, it dies at section 4.5, but it works when I select the large container. I’m thinking the output is too much. Also, is this why the TAP service is recommended over the butler for accessing catalogs, since the TAP allows you to only select the data you want?
1 Like
Hi Bob! I can see your message now!
Bob, let me look into this… Maybe I will even have answers… 
Bob, I confirm your result that NB04a will crash within Section 4.5 if one uses the recommended medium container, but will run to completion if one uses a large container on data.lsst.cloud.
I am not sure if preserving memory is a main reason for using the TAP service over the butler for accessing catalogs. I think it may be that the TAP service offers a lot more versatility in accessing and manipulating catalog data. That said, TAP has its own limitations, especially when doing complex manipulations with large fractions of DP0.2 at a given time.
Thank you Douglas! That was very helpful, as always : )
There may be further updates later, Bob. I’ve been asked to investigate whether we can replace the TAP server queries with ones (perhaps using the butler) to reduce the memory load in NB04a, thus permitting use of a “Medium” container. I am at conferences this week and next; so I might not have an immediate result, but, with luck, there will be in the near future.
To follow up, today during Patch Thursday we will release a new version of 04a which will run to completion with a medium container. We have commented-out the cell which causes kernel stop, and added explanatory text for the user to only execute the cell if they are using a large container. The recommended container size in the notebook header has also been updated to ‘large’, but since we know many delegates always choose ‘medium’ by default, we’ve made sure 04a is medium-friendly.
1 Like