Oh. You will need a newer build of rubin-env 0.4.3. Not sure how you still got ha770c72_0, but ha770c72_4 is current.
“failed with initial frozen solve” is not usually a problem.
I just ran
RUN conda config --add channels conda-forge && \
conda install rubin-env=0.4.3
like 10 minute ago, so
So going back a step, what version of conda are you using?
RUN wget -nv https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh && \
bash miniconda.sh -b -p /opt/conda && \
conda config --add channels intel && \
conda config --set always_yes yes && \
conda update conda && \
conda create -n myenv python=3.8 pip numpy scipy && \
conda clean -a -y
# conda --version
conda 4.10.3
That last script seems a bit iffy as I don’t think you’re activating the downloaded conda before executing commands. With a fresh miniconda and environment, even with the intel channel and (unnecessary) pre-installation of pip/numpy/scipy, I can’t reproduce your issue, as I get:
(myenv) lsst-devl02:~/stack $ conda list deprecated
# packages in environment at /home/ktlim/stack/conda/envs/myenv:
#
# Name Version Build Channel
deprecated 1.2.10 pyh9f0ad1d_0 conda-forge
(myenv) lsst-devl02:~/stack $ conda list rubin-env
# packages in environment at /home/ktlim/stack/conda/envs/myenv:
#
# Name Version Build Channel
rubin-env 0.4.3 ha770c72_4 conda-forge
Sorry, what do you mean by “that last script”? I’m activating the environment in an /etc/profile.d file.
I’m not sure your profile.d got a chance to run in the middle of this sequence of commands:
I previously added /opt/conda/bin to PATH …
What do you think is going wrong? That ‘conda’ is some different version of conda? (there isn’t)
PS my Dockerfile is
[legacypipe/Dockerfile at lsst · legacysurvey/legacypipe · GitHub]
PPS I’m not a conda expert at all – I don’t really understand how I can have rubin-env of the desired version from the desired channel but not the correct build.
Fwiw, my conda knows about the different builds but for some reason chose the first one
# conda search rubin-env=0.4.3
Loading channels: done
# Name Version Build Channel
rubin-env 0.4.3 ha770c72_0 conda-forge
rubin-env 0.4.3 ha770c72_1 conda-forge
rubin-env 0.4.3 ha770c72_2 conda-forge
rubin-env 0.4.3 ha770c72_3 conda-forge
rubin-env 0.4.3 ha770c72_4 conda-forge
# conda list rubin-env
# packages in environment at /opt/conda/envs/myenv:
#
# Name Version Build Channel
rubin-env 0.4.3 ha770c72_0 conda-forge
If I tell conda to install that exact build:
# conda install rubin-env=0.4.3=ha770c72_4
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /opt/conda/envs/myenv
added / updated specs:
- rubin-env==0.4.3=ha770c72_4
The following packages will be downloaded:
package | build
---------------------------|-----------------
click-7.1.2 | pyh9f0ad1d_0 64 KB conda-forge
deprecated-1.2.10 | pyh9f0ad1d_0 11 KB conda-forge
galsim-2.2.5 | py38h3878127_0 3.0 MB conda-forge
pandas-1.2.5 | py38h1abd341_0 12.1 MB conda-forge
rubin-env-0.4.3 | ha770c72_4 7 KB conda-forge
sqlite-3.34.0 | h74cdb3f_0 1.4 MB conda-forge
------------------------------------------------------------
Total: 16.6 MB
The following packages will be UPDATED:
rubin-env 0.4.3-ha770c72_0 --> 0.4.3-ha770c72_4
The following packages will be SUPERSEDED by a higher-priority channel:
click conda-forge/linux-64::click-8.0.1-py3~ --> conda-forge/noarch::click-7.1.2-pyh9f0ad1d_0
sqlite intel::sqlite-3.35.5-hb9a14ef_1 --> conda-forge::sqlite-3.34.0-h74cdb3f_0
The following packages will be DOWNGRADED:
deprecated 1.2.12-pyh44b312d_0 --> 1.2.10-pyh9f0ad1d_0
galsim 2.3.2-py38h260733e_1 --> 2.2.5-py38h3878127_0
pandas 1.3.1-py38h1abd341_0 --> 1.2.5-py38h1abd341_0
Downloading and Extracting Packages
pandas-1.2.5 | 12.1 MB | ############################################################################################################################################################### | 100%
galsim-2.2.5 | 3.0 MB | ############################################################################################################################################################### | 100%
click-7.1.2 | 64 KB | ############################################################################################################################################################### | 100%
sqlite-3.34.0 | 1.4 MB | ############################################################################################################################################################### | 100%
deprecated-1.2.10 | 11 KB | ############################################################################################################################################################### | 100%
rubin-env-0.4.3 | 7 KB | ############################################################################################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: sed: command not found
bash: grep: command not found
Those final complaints are… weird… as though the PATH has gotten messed up, but once the command completes, my shell path is fine -
# which sed
/bin/sed
# which grep
/bin/grep
I’ve merged DM-31235, so the master version of newinstall.sh should now accept a specified conda environment if it already exists without installing anything.
Unfortunately this caused problems with building our binary tarballs, so I had to back it out. I will reopen the ticket and investigate further how to do this properly.