Procedure for adding a third-party Python dependency as a pseudo EUPS package (?)

I want to add jsonschema as a third-party python dependency of validate_base (we’ll have an announcement post about that soon) as part of DM-7935. I’m a bit unclear about our handling of python externals, and the Developer Guide page Distributing Third-Party Packages with EUPS doesn’t entirely fit my scenario.

First, it seems that there are two ways we handle python externals in the Stack:

  1. As TaP packages, like those described in Distributing Third-Party Packages with EUPS that contains the python package as a tar file.
  2. As an EUPS ‘pseudo package’ that merely checks that the package is installed. An example is
  3. Are there other alternatives?

I think my preference is to go with #2 so we’re not replicating what PyPI does already. Is this a correct assumption?

Going with #2, the procedure would then be:

  1. Create a repo and implement it like
  2. RFC this jsonschema package. (I’m not 100% sure this is necessary as it will only a dependency of the lsst_qa top-level package; not lsst_apps or lsst_distrib. Maybe @timj can advise.).
  3. Publish the jsonschema package following
  4. Add the jsonschema package to lsstsw’s repos.yaml.
  5. Add jsonschema to the conda package lists in
  6. Document the dependency in pip install jsonschema==2.5.1.
  7. Declare the dependency in validate_base.

I think steps 5, 6 and 7 are what I’m most unsure of. Is anything else needed to ensure that the jsonschema python package is installed for newinstall, lsstsw and conda stack users?

Alternatively, should a jsonschema eups pseudopackage actually run pip install through its ups/

I’ll turn responses to this topic and a corrected procedure into improved Developer Guide documentation.

It partly depends on how many dependencies the python package has. If it has no dependencies itself then I think I’d edge towards a full EUPS package of the tar file as that gives you much more control over versioning.

A two day RFC wouldn’t hurt.

Would also need to add it to lsstsw/bin/deploy so it gets installed in bleeding edge mode (if you are going for option #2).

Looks like jsonschema has only one dependency at setup time, though it’s possibly not necessary for the TaP package. Maybe you’re right that a full TaP package is the way to go to reduce the end-user’s installation burden.

Thanks for checking on the procedure for making the alternative TaP package.