Getting started tutorial python problems

Support
#1

Hello,

I’m trying to run Getting started tutorial part 3: displaying exposures and source tables output by single frame processing. While running one of python commands I get error message : AttributeError: module ‘lsst’ has no attribute ‘obs’. What can be done about this and how to update lsst python packges?

Thank you

#2

This implies that you’ve missed a call to setup somewhere since that is what will enable the python lsst.x packages. There should be a setup lsst_distrib somewhere. Getting started tutorial part 1: setting up the Butler data repository — LSST Science Pipelines links to Setting up installed LSST Science Pipelines — LSST Science Pipelines

I’m not sure how you got through step 2 without the python code set up.

#3

Hi @VenIl - I just wanted to check whether running setup lsst_distrib as suggeseted by @timj fixed your issue?

#4

Hi @sgreenstreet , it still doesn’t work with same error message.

#5

Here is the full error message if it helps:

AttributeError Traceback (most recent call last)
File ~/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/site-packages/lsst/utils/doImport.py:52, in doImport..tryImport(module, fromlist, previousError)
51 try:
—> 52 pytype = getattr(pytype, f)
53 except AttributeError as e:

AttributeError: module ‘lsst’ has no attribute ‘obs’

The above exception was the direct cause of the following exception:

ImportError Traceback (most recent call last)
Cell In[3], line 2
1 import os
----> 2 calexp = butler.get(‘calexp’, visit=23718, detector=41)

File ~/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/site-packages/lsst/daf/butler/direct_butler/_direct_butler.py:1177, in DirectButler.get(self, datasetRefOrType, dataId, parameters, collections, storageClass, timespan, **kwargs)
1168 _LOG.debug(“Butler get: %s, dataId=%s, parameters=%s”, datasetRefOrType, dataId, parameters)
1169 ref = self._findDatasetRef(
1170 datasetRefOrType,
1171 dataId,
(…)
1175 **kwargs,
1176 )
→ 1177 return self._datastore.get(ref, parameters=parameters, storageClass=storageClass)

File ~/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/site-packages/lsst/daf/butler/datastores/fileDatastore.py:2288, in FileDatastore.get(self, ref, parameters, storageClass)
2285 if storageClass is not None:
2286 ref = ref.overrideStorageClass(storageClass)
→ 2288 allGetInfo = self._prepare_for_direct_get(ref, parameters)
2289 return get_dataset_as_python_object_from_get_info(
2290 allGetInfo, ref=ref, parameters=parameters, cache_manager=self.cacheManager
2291 )

File ~/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/site-packages/lsst/daf/butler/datastores/fileDatastore.py:826, in FileDatastore._prepare_for_direct_get(self, ref, parameters)
823 if not fileLocations:
824 raise FileNotFoundError(f"None of the component files for dataset {ref} exist.")
→ 826 return generate_datastore_get_information(
827 fileLocations,
828 readStorageClass=refStorageClass,
829 ref=ref,
830 parameters=parameters,
831 )

File ~/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/site-packages/lsst/daf/butler/datastores/file_datastore/get.py:134, in generate_datastore_get_information(fileLocations, ref, parameters, readStorageClass)
131 if disassembled and storedFileInfo.component != refComponent:
132 thisReadStorageClass = writeStorageClass
→ 134 formatter = get_instance_of(
135 storedFileInfo.formatter,
136 FileDescriptor(
137 location,
138 readStorageClass=thisReadStorageClass,
139 storageClass=writeStorageClass,
140 parameters=parameters,
141 component=storedFileInfo.component,
142 ),
143 dataId=ref.dataId,
144 ref=ref,
145 )
147 formatterParams, notFormatterParams = formatter.segregate_parameters()
149 # Of the remaining parameters, extract the ones supported by
150 # this StorageClass (for components not all will be handled)

File ~/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/site-packages/lsst/utils/introspection.py:153, in get_instance_of(typeOrName, *args, **kwargs)
128 def get_instance_of(typeOrName: type | str, *args: Any, **kwargs: Any) → Any:
129 “”“Given the type name or a type, instantiate an object of that type.
130
131 If a type name is given, an attempt will be made to import the type.
(…)
151 Raised if a module is imported rather than a type.
152 “””
→ 153 cls = get_class_of(typeOrName)
154 return cls(*args, **kwargs)

File ~/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/site-packages/lsst/utils/introspection.py:120, in get_class_of(typeOrName)
95 “”“Given the type name or a type, return the python type.
96
97 If a type name is given, an attempt will be made to import the type.
(…)
117 Raised if a module is imported rather than a type.
118 “””
119 if isinstance(typeOrName, str):
→ 120 cls = doImportType(typeOrName)
121 else:
122 if isinstance(typeOrName, types.ModuleType):

File ~/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/site-packages/lsst/utils/doImport.py:114, in doImportType(importable)
90 def doImportType(importable: str) → type:
91 “”“Import a python type given an importable string and return it.
92
93 Parameters
(…)
112 item could not be retrieved from the imported module.
113 “””
→ 114 imported = doImport(importable)
115 if isinstance(imported, types.ModuleType):
116 raise TypeError(f"Import of {importable} returned a module and not a type.")

File ~/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/site-packages/lsst/utils/doImport.py:69, in doImport(importable)
67 while moduleComponents:
68 try:
—> 69 pytype = tryImport(".".join(moduleComponents), infileComponents, previousError)
70 if not infileComponents and hasattr(pytype, moduleComponents[-1]):
71 # This module has an attribute with the same name as the
72 # module itself (like doImport.doImport, actually!).
73 # If that attribute was lifted to the package, we should
74 # return the attribute, not the module.
75 try:

File ~/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/site-packages/lsst/utils/doImport.py:55, in doImport..tryImport(module, fromlist, previousError)
53 except AttributeError as e:
54 extra = f"({previousError})" if previousError is not None else “”
—> 55 raise ImportError(
56 f"Could not get attribute ‘{f}’ from ‘{module}’ when importing ‘{importable}’ {extra}"
57 ) from e
58 return pytype

ImportError: Could not get attribute ‘obs’ from ‘lsst’ when importing ‘lsst.obs.base.formatters.fitsExposure.FitsExposureFormatter’ (No module named ‘lsst.obs’)

#6

How did you install the LSST science pipelines? This looks like a pip install of daf_butler not the eups install.

#7

I Installed it with lsstsw and lsst-build.

#8

I have an Ubuntu laptop.

#9

Except somehow you have also managed to pip install the Rubin middleware packages. That does not happen when you run rebuild lsst_distrib. We expect most tutorial users to follow the installation instructions of installing with lsstinstall since rebuild will give you the cutting edge main version of every package.

What does eups list lsst_distrib report?

#11

output : gc7ba34d93f+8892a0c6f9 b7407 current setup

#12

Ok. So lsst_distrib is set up but the pip installed packages are somehow messing things up.

What about:

$ eups list obs_base
$ echo $OBS_BASE_DIR
$ ls $OBS_BASE_DIR/python
#13

Oh, also please report your $PYTHONPATH value.

#14

g5c8c310e76+15eaa6c88b b7407 current setup
/home/venil360/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/obs_base/g5c8c310e76+15eaa6c88b
lsst

#15

[’/home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/eups/python’, ‘/home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python311.zip’, ‘/home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11’, ‘/home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/lib-dynload’, ‘’, ‘/home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/site-packages’]

#16

That’s the problem. setup lsst_distrib didn’t set any of the PYTHONPATH environment variables. There is meant to be one var per eups package. I still can’t explain how you installed Rubin pip packages. To confirm you installed it as:

$ ./bin/deploy
$ . bin/envconfig
$ rebuild lsst_distrib
$ setup -t bNNNN lsst_distrib

and nothing else?

Can you start a new shell, source the bin/envconfig and try the setup again to see if you get the PYTHONPATH variable set properly?

#17

Did what you told. I get : /home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/eups/python
/home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python311.zip
/home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11
/home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/lib-dynload

/home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/site-packages

#18

Did what you told. I get now : /home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/eups/python
/home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python311.zip
/home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11
/home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/lib-dynload

/home/venil360/lsstsw/miniconda/envs/lsst-scipipe-9.0.0/lib/python3.11/site-packages

#19

That’s very confusing because somehow the setup call is not setting PYTHONPATH at all. This is seemingly impossible. Did you understand how you installed the middleware packages from PyPI? Does rebuild lsst_distrib work for you (since you are using lsstsw)?

Can you please try using lsstinstall as documented here ?

#20

Given

can I assume that you have resolved this topic?

#21

Yes, I reinstalled the whole system.
Thank you.