Update of PyYAML

dm-dev
Tags: #<Tag:0x00007f61ab9d5490>

(Tim Jenness) #1

Yesterday I upgraded the stack PyYAML package to version 5.1. This required I make some changes to daf_base and daf_persistence (see DM-18488 for details) since constructors are now associated with specified loaders.

The big change for users though is that PyYAML now complains if you call yaml.load() without specifying a loader class. Previously load() used the unsafe loader by default but this is no longer deemed to be the correct approach. If you call yaml.load() without a loader you will see the following warning:

 YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.

The full details are found at that URL. There are a number of places in our code where we use the default unsafe loader. You will notice them in your pytest output warnings. A quick scan suggests verify, dax_ppdb, validate_base, validate_drp, obs_base, fgcm, and ctrl_iip will need fixing to remove the warnings. The best solution is if SafeLoader (or CSafeLoader) can be used. daf_persistence and daf_base YAML files must be read using FullLoader. Contact me if you want additional help in migrating a package to the new scheme.


(Jmeyers314) #2

Does this update impact DM-10333 ? I see that there are still duplicate keys in the obs_base policy dataset files.


(Tim Jenness) #3

No. I don’t think YAML parsers report warnings from duplicate keys.