Since we moved away from Steem, it became essential that the tools needed migration to Hive. One of those tools is Jussi, a reverse proxy that only speaks json-rpc 2.0, which acts as a bridge between the users and the backend servers (steemd, hivemind). First thing in the migration was to ensure that the tools can be built. Unfortunately, as we speak, as far as I'm concerned, we're still using an old (functional) docker image of jussi built by Steemit 6 months ago (steemit/jussi:latest). I tried to build it but it failed. I created an issue but nobody had addressed it yet https://gitlab.syncad.com/hive/jussi/issues/1. So I decided to look into it myself, although I'm not a Python programmer at all. Armed with patience, resolve and lots of googling, I started digging into the problem.
Preliminary trials and errors
To begin, it's pretty straight forward:
git clone https://gitlab.syncad.com/hive/jussi
cd jussi
docker build --tag jussi:test .
Step 24/28 : RUN pipenv install --dev
---> Running in 0f0bed24815f
Creating a virtualenv for this project…
Pipfile: /app/Pipfile
Using /usr/local/bin/python3.6m (3.6.5) to create virtualenv…
⠼ Creating virtual environment...created virtual environment CPython3.6.5.final.0-64 in 253ms
creator CPython3Posix(dest=/app/.venv, clear=False, global=False)
seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1)
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
✔ Successfully created virtual environment!
Virtualenv location: /app/.venv
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
⠴ Locking..✘ Locking Failed!
[pipenv.exceptions.ResolutionFailure]: File "/usr/local/lib/python3.6/site-packages/pipenv/resolver.py", line 69, in resolve
[pipenv.exceptions.ResolutionFailure]: req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]: File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 726, in resolve_deps
[pipenv.exceptions.ResolutionFailure]: req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]: File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]: resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]: File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 395, in resolve
[pipenv.exceptions.ResolutionFailure]: raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]: pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches pytest==3.10.0,>=2.7,>=2.9,>=3.0.0,>=3.0.6,>=3.6.0,>=5.0
[pipenv.exceptions.ResolutionFailure]: Tried: 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.4.0, 2.4.1, 2.4.2, 2.5.0, 2.5.1, 2.5.2, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.7.0, 2.7.1, 2.7.1, 2.7.2, 2.7.2, 2.7.3, 2.7.3, 2.8.0, 2.8.0, 2.8.1, 2.8.1, 2.8.2, 2.8.2, 2.8.3, 2.8.3, 2.8.4, 2.8.4, 2.8.5, 2.8.5, 2.8.6, 2.8.6, 2.8.7, 2.8.7, 2.9.0, 2.9.0, 2.9.1, 2.9.1, 2.9.2, 2.9.2, 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.0.2, 3.0.2, 3.0.3, 3.0.3, 3.0.4, 3.0.4, 3.0.5, 3.0.5, 3.0.6, 3.0.6, 3.0.7, 3.0.7, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.1.2, 3.1.2, 3.1.3, 3.1.3, 3.2.0, 3.2.0, 3.2.1, 3.2.1, 3.2.2, 3.2.2, 3.2.3, 3.2.3, 3.2.4, 3.2.4, 3.2.5, 3.2.5, 3.3.0, 3.3.0, 3.3.1, 3.3.1, 3.3.2, 3.3.2, 3.4.0, 3.4.0, 3.4.1, 3.4.1, 3.4.2, 3.4.2, 3.5.0, 3.5.0, 3.5.1, 3.5.1, 3.6.0, 3.6.0, 3.6.1, 3.6.1, 3.6.2, 3.6.2, 3.6.3, 3.6.3, 3.6.4, 3.6.4, 3.7.0, 3.7.0, 3.7.1, 3.7.1, 3.7.2, 3.7.2, 3.7.3, 3.7.3, 3.7.4, 3.7.4, 3.8.0, 3.8.0, 3.8.1, 3.8.1, 3.8.2, 3.8.2, 3.9.1, 3.9.1, 3.9.2, 3.9.2, 3.9.3, 3.9.3, 3.10.0, 3.10.0, 3.10.1, 3.10.1, 4.0.0, 4.0.0, 4.0.1, 4.0.1, 4.0.2, 4.0.2, 4.1.0, 4.1.0, 4.1.1, 4.1.1, 4.2.0, 4.2.0, 4.2.1, 4.2.1, 4.3.0, 4.3.0, 4.3.1, 4.3.1, 4.4.0, 4.4.0, 4.4.1, 4.4.1, 4.4.2, 4.4.2, 4.5.0, 4.5.0, 4.6.0, 4.6.0, 4.6.1, 4.6.1, 4.6.2, 4.6.2, 4.6.3, 4.6.3, 4.6.4, 4.6.4, 4.6.5, 4.6.5, 4.6.6, 4.6.6, 4.6.7, 4.6.7, 4.6.8, 4.6.8, 4.6.9, 4.6.9, 5.0.0, 5.0.0, 5.0.1, 5.0.1, 5.1.0, 5.1.0, 5.1.1, 5.1.1, 5.1.2, 5.1.2, 5.1.3, 5.1.3, 5.2.0, 5.2.0, 5.2.1, 5.2.1, 5.2.2, 5.2.2, 5.2.3, 5.2.3, 5.2.4, 5.2.4, 5.3.0, 5.3.0, 5.3.1, 5.3.1, 5.3.2, 5.3.2, 5.3.3, 5.3.3, 5.3.4, 5.3.4, 5.3.5, 5.3.5, 5.4.0, 5.4.0, 5.4.1, 5.4.1
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches pytest==3.10.0,>=2.7,>=2.9,>=3.0.0,>=3.0.6,>=3.6.0,>=5.0
Tried: 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.4.0, 2.4.1, 2.4.2, 2.5.0, 2.5.1, 2.5.2, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.7.0, 2.7.1, 2.7.1, 2.7.2, 2.7.2, 2.7.3, 2.7.3, 2.8.0, 2.8.0, 2.8.1, 2.8.1, 2.8.2, 2.8.2, 2.8.3, 2.8.3, 2.8.4, 2.8.4, 2.8.5, 2.8.5, 2.8.6, 2.8.6, 2.8.7, 2.8.7, 2.9.0, 2.9.0, 2.9.1, 2.9.1, 2.9.2, 2.9.2, 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.0.2, 3.0.2, 3.0.3, 3.0.3, 3.0.4, 3.0.4, 3.0.5, 3.0.5, 3.0.6, 3.0.6, 3.0.7, 3.0.7, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.1.2, 3.1.2, 3.1.3, 3.1.3, 3.2.0, 3.2.0, 3.2.1, 3.2.1, 3.2.2, 3.2.2, 3.2.3, 3.2.3, 3.2.4, 3.2.4, 3.2.5, 3.2.5, 3.3.0, 3.3.0, 3.3.1, 3.3.1, 3.3.2, 3.3.2, 3.4.0, 3.4.0, 3.4.1, 3.4.1, 3.4.2, 3.4.2, 3.5.0, 3.5.0, 3.5.1, 3.5.1, 3.6.0, 3.6.0, 3.6.1, 3.6.1, 3.6.2, 3.6.2, 3.6.3, 3.6.3, 3.6.4, 3.6.4, 3.7.0, 3.7.0, 3.7.1, 3.7.1, 3.7.2, 3.7.2, 3.7.3, 3.7.3, 3.7.4, 3.7.4, 3.8.0, 3.8.0, 3.8.1, 3.8.1, 3.8.2, 3.8.2, 3.9.1, 3.9.1, 3.9.2, 3.9.2, 3.9.3, 3.9.3, 3.10.0, 3.10.0, 3.10.1, 3.10.1, 4.0.0, 4.0.0, 4.0.1, 4.0.1, 4.0.2, 4.0.2, 4.1.0, 4.1.0, 4.1.1, 4.1.1, 4.2.0, 4.2.0, 4.2.1, 4.2.1, 4.3.0, 4.3.0, 4.3.1, 4.3.1, 4.4.0, 4.4.0, 4.4.1, 4.4.1, 4.4.2, 4.4.2, 4.5.0, 4.5.0, 4.6.0, 4.6.0, 4.6.1, 4.6.1, 4.6.2, 4.6.2, 4.6.3, 4.6.3, 4.6.4, 4.6.4, 4.6.5, 4.6.5, 4.6.6, 4.6.6, 4.6.7, 4.6.7, 4.6.8, 4.6.8, 4.6.9, 4.6.9, 5.0.0, 5.0.0, 5.0.1, 5.0.1, 5.1.0, 5.1.0, 5.1.1, 5.1.1, 5.1.2, 5.1.2, 5.1.3, 5.1.3, 5.2.0, 5.2.0, 5.2.1, 5.2.1, 5.2.2, 5.2.2, 5.2.3, 5.2.3, 5.2.4, 5.2.4, 5.3.0, 5.3.0, 5.3.1, 5.3.1, 5.3.2, 5.3.2, 5.3.3, 5.3.3, 5.3.4, 5.3.4, 5.3.5, 5.3.5, 5.4.0, 5.4.0, 5.4.1, 5.4.1
There are incompatible versions in the resolved dependencies.
[pipenv.exceptions.ResolutionFailure]: req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]: File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 726, in resolve_deps
[pipenv.exceptions.ResolutionFailure]: req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]: File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]: resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]: File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 395, in resolve
[pipenv.exceptions.ResolutionFailure]: raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]: pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches pytest==3.10.0,>=2.7,>=2.9,>=3.0.0,>=3.0.6,>=3.6.0,>=5.0
[pipenv.exceptions.ResolutionFailure]: Tried: 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.4.0, 2.4.1, 2.4.2, 2.5.0, 2.5.1, 2.5.2, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.7.0, 2.7.1, 2.7.1, 2.7.2, 2.7.2, 2.7.3, 2.7.3, 2.8.0, 2.8.0, 2.8.1, 2.8.1, 2.8.2, 2.8.2, 2.8.3, 2.8.3, 2.8.4, 2.8.4, 2.8.5, 2.8.5, 2.8.6, 2.8.6, 2.8.7, 2.8.7, 2.9.0, 2.9.0, 2.9.1, 2.9.1, 2.9.2, 2.9.2, 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.0.2, 3.0.2, 3.0.3, 3.0.3, 3.0.4, 3.0.4, 3.0.5, 3.0.5, 3.0.6, 3.0.6, 3.0.7, 3.0.7, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.1.2, 3.1.2, 3.1.3, 3.1.3, 3.2.0, 3.2.0, 3.2.1, 3.2.1, 3.2.2, 3.2.2, 3.2.3, 3.2.3, 3.2.4, 3.2.4, 3.2.5, 3.2.5, 3.3.0, 3.3.0, 3.3.1, 3.3.1, 3.3.2, 3.3.2, 3.4.0, 3.4.0, 3.4.1, 3.4.1, 3.4.2, 3.4.2, 3.5.0, 3.5.0, 3.5.1, 3.5.1, 3.6.0, 3.6.0, 3.6.1, 3.6.1, 3.6.2, 3.6.2, 3.6.3, 3.6.3, 3.6.4, 3.6.4, 3.7.0, 3.7.0, 3.7.1, 3.7.1, 3.7.2, 3.7.2, 3.7.3, 3.7.3, 3.7.4, 3.7.4, 3.8.0, 3.8.0, 3.8.1, 3.8.1, 3.8.2, 3.8.2, 3.9.1, 3.9.1, 3.9.2, 3.9.2, 3.9.3, 3.9.3, 3.10.0, 3.10.0, 3.10.1, 3.10.1, 4.0.0, 4.0.0, 4.0.1, 4.0.1, 4.0.2, 4.0.2, 4.1.0, 4.1.0, 4.1.1, 4.1.1, 4.2.0, 4.2.0, 4.2.1, 4.2.1, 4.3.0, 4.3.0, 4.3.1, 4.3.1, 4.4.0, 4.4.0, 4.4.1, 4.4.1, 4.4.2, 4.4.2, 4.5.0, 4.5.0, 4.6.0, 4.6.0, 4.6.1, 4.6.1, 4.6.2, 4.6.2, 4.6.3, 4.6.3, 4.6.4, 4.6.4, 4.6.5, 4.6.5, 4.6.6, 4.6.6, 4.6.7, 4.6.7, 4.6.8, 4.6.8, 4.6.9, 4.6.9, 5.0.0, 5.0.0, 5.0.1, 5.0.1, 5.1.0, 5.1.0, 5.1.1, 5.1.1, 5.1.2, 5.1.2, 5.1.3, 5.1.3, 5.2.0, 5.2.0, 5.2.1, 5.2.1, 5.2.2, 5.2.2, 5.2.3, 5.2.3, 5.2.4, 5.2.4, 5.3.0, 5.3.0, 5.3.1, 5.3.1, 5.3.2, 5.3.2, 5.3.3, 5.3.3, 5.3.4, 5.3.4, 5.3.5, 5.3.5, 5.4.0, 5.4.0, 5.4.1, 5.4.1
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches pytest==3.10.0,>=2.7,>=2.9,>=3.0.0,>=3.0.6,>=3.6.0,>=5.0
Tried: 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.4.0, 2.4.1, 2.4.2, 2.5.0, 2.5.1, 2.5.2, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.7.0, 2.7.1, 2.7.1, 2.7.2, 2.7.2, 2.7.3, 2.7.3, 2.8.0, 2.8.0, 2.8.1, 2.8.1, 2.8.2, 2.8.2, 2.8.3, 2.8.3, 2.8.4, 2.8.4, 2.8.5, 2.8.5, 2.8.6, 2.8.6, 2.8.7, 2.8.7, 2.9.0, 2.9.0, 2.9.1, 2.9.1, 2.9.2, 2.9.2, 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.0.2, 3.0.2, 3.0.3, 3.0.3, 3.0.4, 3.0.4, 3.0.5, 3.0.5, 3.0.6, 3.0.6, 3.0.7, 3.0.7, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.1.2, 3.1.2, 3.1.3, 3.1.3, 3.2.0, 3.2.0, 3.2.1, 3.2.1, 3.2.2, 3.2.2, 3.2.3, 3.2.3, 3.2.4, 3.2.4, 3.2.5, 3.2.5, 3.3.0, 3.3.0, 3.3.1, 3.3.1, 3.3.2, 3.3.2, 3.4.0, 3.4.0, 3.4.1, 3.4.1, 3.4.2, 3.4.2, 3.5.0, 3.5.0, 3.5.1, 3.5.1, 3.6.0, 3.6.0, 3.6.1, 3.6.1, 3.6.2, 3.6.2, 3.6.3, 3.6.3, 3.6.4, 3.6.4, 3.7.0, 3.7.0, 3.7.1, 3.7.1, 3.7.2, 3.7.2, 3.7.3, 3.7.3, 3.7.4, 3.7.4, 3.8.0, 3.8.0, 3.8.1, 3.8.1, 3.8.2, 3.8.2, 3.9.1, 3.9.1, 3.9.2, 3.9.2, 3.9.3, 3.9.3, 3.10.0, 3.10.0, 3.10.1, 3.10.1, 4.0.0, 4.0.0, 4.0.1, 4.0.1, 4.0.2, 4.0.2, 4.1.0, 4.1.0, 4.1.1, 4.1.1, 4.2.0, 4.2.0, 4.2.1, 4.2.1, 4.3.0, 4.3.0, 4.3.1, 4.3.1, 4.4.0, 4.4.0, 4.4.1, 4.4.1, 4.4.2, 4.4.2, 4.5.0, 4.5.0, 4.6.0, 4.6.0, 4.6.1, 4.6.1, 4.6.2, 4.6.2, 4.6.3, 4.6.3, 4.6.4, 4.6.4, 4.6.5, 4.6.5, 4.6.6, 4.6.6, 4.6.7, 4.6.7, 4.6.8, 4.6.8, 4.6.9, 4.6.9, 5.0.0, 5.0.0, 5.0.1, 5.0.1, 5.1.0, 5.1.0, 5.1.1, 5.1.1, 5.1.2, 5.1.2, 5.1.3, 5.1.3, 5.2.0, 5.2.0, 5.2.1, 5.2.1, 5.2.2, 5.2.2, 5.2.3, 5.2.3, 5.2.4, 5.2.4, 5.3.0, 5.3.0, 5.3.1, 5.3.1, 5.3.2, 5.3.2, 5.3.3, 5.3.3, 5.3.4, 5.3.4, 5.3.5, 5.3.5, 5.4.0, 5.4.0, 5.4.1, 5.4.1
There are incompatible versions in the resolved dependencies.
The command '/bin/sh -c pipenv install --dev' returned a non-zero code: 1
As you can see, it was having trouble locking the dependencies versions. I spent hours trying to find a solution for that but nothing worked.
Next, I switched my focus to this pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches pytest==3.10.0,>=2.7,>=2.9,>=3.0.0,>=3.0.6,>=3.6.0,>=5.0. It was weird because pytest 3.10.0 still exists on the python repository, so it's not obsolete or discontinued.
So I looked into the Pipfile, where pytest = "==3.10.0". So I set pytest = ">3.10.0", which would install a higher version of it.
Fire up the docker build again; this time, progress:
Step 24/28 : RUN pipenv install --dev
---> Running in 455f3152c983
Creating a virtualenv for this project…
Pipfile: /app/Pipfile
Using /usr/local/bin/python3.6m (3.6.5) to create virtualenv…
⠼ Creating virtual environment...created virtual environment CPython3.6.5.final.0-64 in 277ms
creator CPython3Posix(dest=/app/.venv, clear=False, global=False)
seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1)
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
✔ Successfully created virtual environment!
Virtualenv location: /app/.venv
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
⠴ Locking..✔ Success!
Locking [packages] dependencies…
⠼ Locking..✔ Success!
Updated Pipfile.lock (606260)!
Installing dependencies from Pipfile.lock (606260)…
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
The docker build was resuming normally, until it ran into another error, where pipenv run pytest was failing:
Step 26/28 : RUN pipenv run pytest
---> Running in 592b4803f184
Traceback (most recent call last):
File "/app/.venv/bin/pytest", line 8, in <module>
sys.exit(main())
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 105, in main
config = _prepareconfig(args, plugins)
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 258, in _prepareconfig
pluginmanager=pluginmanager, args=args
File "/app/.venv/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/app/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/app/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 203, in _multicall
gen.send(outcome)
File "/app/.venv/lib/python3.6/site-packages/_pytest/helpconfig.py", line 90, in pytest_cmdline_parse
config = outcome.get_result()
File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
raise ex[1].with_traceback(ex[2])
File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 836, in pytest_cmdline_parse
self.parse(args)
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 1044, in parse
self._preparse(args, addopts=addopts)
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 992, in _preparse
self.pluginmanager.load_setuptools_entrypoints("pytest11")
File "/app/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 299, in load_setuptools_entrypoints
plugin = ep.load()
File "/app/.venv/lib/python3.6/site-packages/importlib_metadata/__init__.py", line 95, in load
module = import_module(match.group('module'))
File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 665, in _load_unlocked
File "/app/.venv/lib/python3.6/site-packages/_pytest/assertion/rewrite.py", line 152, in exec_module
exec(co, module.__dict__)
File "/app/.venv/lib/python3.6/site-packages/pytest_asyncio/plugin.py", line 8, in <module>
from _pytest.python import transfer_markers
ImportError: cannot import name 'transfer_markers'
The command '/bin/sh -c pipenv run pytest' returned a non-zero code: 1
Looks like we have a nasty ImportError: cannot import name 'transfer_markers' related to pytest_asyncio. Back to Pipfile, the version installed was pytest-asyncio = "==0.9.0".
According to https://stackoverflow.com/questions/54064971/importerror-cannot-import-name-transfer-markers-when-testing-with-pytest
pytest-asyncio is using transfer_markers which is taken out in pytest 4.1.0. Looks like it's fixed and released. Issue#105
EDIT: You can now upgrade pip install pytest-asyncio0.10.0 to fix this issue.
Or update pytest-asyncio0.10.0 in your requirements.txt
This gave me a clue that we're in pytest 4 (vs 3.10.0 where we started). Alright, let's try pytest-asyncio = ">0.9.0" in Pipfile. And.... another error:
Traceback (most recent call last):
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 495, in _importconftest
return self._conftestpath2mod[key]
KeyError: PosixPath('/app/tests/conftest.py')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/.venv/bin/pytest", line 8, in <module>
sys.exit(main())
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 105, in main
config = _prepareconfig(args, plugins)
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 258, in _prepareconfig
pluginmanager=pluginmanager, args=args
File "/app/.venv/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/app/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/app/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 203, in _multicall
gen.send(outcome)
File "/app/.venv/lib/python3.6/site-packages/_pytest/helpconfig.py", line 90, in pytest_cmdline_parse
config = outcome.get_result()
File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
raise ex[1].with_traceback(ex[2])
File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 836, in pytest_cmdline_parse
self.parse(args)
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 1044, in parse
self._preparse(args, addopts=addopts)
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 1002, in _preparse
early_config=self, args=args, parser=self._parser
File "/app/.venv/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/app/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/app/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
return outcome.get_result()
File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
raise ex[1].with_traceback(ex[2])
File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 899, in pytest_load_initial_conftests
self.pluginmanager._set_initial_conftests(early_config.known_args_namespace)
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 444, in _set_initial_conftests
self._try_load_conftest(current)
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 452, in _try_load_conftest
self._getconftestmodules(x)
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 473, in _getconftestmodules
mod = self._importconftest(conftestpath)
File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 501, in _importconftest
mod = conftestpath.pyimport()
File "/app/.venv/lib/python3.6/site-packages/py/_path/local.py", line 701, in pyimport
__import__(modname)
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 665, in _load_unlocked
File "/app/.venv/lib/python3.6/site-packages/_pytest/assertion/rewrite.py", line 152, in exec_module
exec(co, module.__dict__)
File "/app/tests/conftest.py", line 1870, in <module>
@pytest.fixture(params=translatable_steemd_requests_and_responses())
File "/app/.venv/lib/python3.6/site-packages/_pytest/fixtures.py", line 993, in result
fail(message, pytrace=False)
File "/app/.venv/lib/python3.6/site-packages/_pytest/outcomes.py", line 158, in fail
raise Failed(msg=msg, pytrace=pytrace)
Failed: Fixture "translatable_steemd_requests_and_responses" called directly. Fixtures are not meant to be called directly,
but are created automatically when test functions request them as parameters.
See https://docs.pytest.org/en/latest/fixture.html for more information about fixtures, and
https://docs.pytest.org/en/latest/deprecations.html#calling-fixtures-directly about how to update your code.
The command '/bin/sh -c pipenv run pytest' returned a non-zero code: 1
Failed: Fixture "translatable_steemd_requests_and_responses" called directly. Fixtures are not meant to be called directly, Ok so now I need to learn Python and try to figure wth is happening in the code itself. Fuck no, already wasted too much time on this, and I'm not going to learn python. I need a break. Screw this chain of unhappy packages.
Let's try something else
How about checking which packages/versions are being installed? There's a nifty command pipenv graph that outputs all installed packages with their version, requirements and dependencies.
Since pytest = ">3.10.0" initially solved the Pipfile locking problem, I retried it with this added in Dockerfile
RUN pipenv install --dev && pipenv graph. The graph command gave a very long output (I won't paste it) that was extremely helpful! By looking at which packages were using pytest, I found:
pytest-asyncio==0.9.0
- pytest [required: >=3.0.6, installed: 5.4.1]
...
pytest-console-scripts==0.1.7
- mock [required: >=2.0.0, installed: 4.0.2]
- pytest [required: >=3.0.0, installed: 5.4.1]
...
pytest-cov==2.6.0
- coverage [required: >=4.4, installed: 5.0.4]
- pytest [required: >=2.9, installed: 5.4.1]
...
pytest-mock==1.10.0
- pytest [required: >=2.7, installed: 5.4.1]
...
pytest-profiling==1.7.0
- gprof2dot [required: Any, installed: 2019.11.30]
- pytest [required: Any, installed: 5.4.1]
...
pytest-pylint==0.15.1
- pylint [required: >=2.0.0, installed: 2.2.3]
- astroid [required: >=2.0,<2.2.0, installed: 2.1.0]
- lazy-object-proxy [required: Any, installed: 1.4.3]
- six [required: Any, installed: 1.14.0]
- typed-ast [required: Any, installed: 1.4.1]
- wrapt [required: Any, installed: 1.12.1]
- isort [required: >=4.2.5, installed: 4.3.21]
- mccabe [required: Any, installed: 0.6.1]
- pytest [required: >=5.0, installed: 5.4.1]
...
pytest-sanic==0.1.13
- aiohttp [required: Any, installed: 3.6.2]
- async-timeout [required: >=3.0,<4.0, installed: 3.0.1]
- attrs [required: >=17.3.0, installed: 19.1.0]
- chardet [required: >=2.0,<4.0, installed: 3.0.4]
- idna-ssl [required: >=1.0, installed: 1.1.0]
- idna [required: >=2.0, installed: 2.9]
- multidict [required: >=4.5,<5.0, installed: 4.7.5]
- typing-extensions [required: >=3.6.5, installed: 3.7.4.2]
- yarl [required: >=1.0,<2.0, installed: 1.4.2]
- idna [required: >=2.0, installed: 2.9]
- multidict [required: >=4.0, installed: 4.7.5]
- async-generator [required: Any, installed: 1.10]
- pytest [required: Any, installed: 5.4.1]
...
pytest-timeout==1.3.4
- pytest [required: >=3.6.0, installed: 5.4.1]
Notice that all those packages work fine with up to pytest 3.x or any version of it, except pytest-pylint==0.15.1 that needs pytest [required: >=5.0, installed: 5.4.1]!!! Could it be that pytest-pylint was causing all this mayhem?
Sure enough, it does https://pypi.org/project/pytest-pylint/#history
Rememeber I mentioned that the last jussi docker image built by Steemit was 6 months ago? That puts pytest-pylint 0.14.1 right about that period.
Ok, one last try with Pipfile modified only with pytest-pylint = "==0.14.1".
SUCCESS. Jussi builds at last, although with many test warning like this
tests/test_jussi_request_response_middleware.py::test_urn_parts_not_in_options_response_headers
/app/.venv/lib/python3.6/site-packages/sanic/testing.py:26: DeprecationWarning: verify_ssl is deprecated, use ssl=False instead
conn = aiohttp.TCPConnector(verify_ssl=False)
But that's beyond the scope of this fix. It appears to be a benign problem, I'll let others look into it if it's critical.
Updating PyPi URL
And while we're at it, the Python Package Index (PyPI) has moved to a new domain. https://packaging.python.org/guides/migrating-to-pypi-org/
Uploads through pypi.python.org were switched off on July 3, 2017. As of April 13th, 2018, pypi.org is the URL for PyPI.
So fixing Pipfile
url = "https://pypi.python.org/simple"
to
url = "https://pypi.org/simple"
The final fix is at:
https://gitlab.syncad.com/hive/jussi/-/tree/fix-build
EDIT: Merged it into the autoclave branch https://gitlab.syncad.com/hive/jussi/-/tree/autoclave
https://gitlab.syncad.com/hive/jussi/-/commit/420af54d5d0d725045843e91c2ab027b06739221
Available & Reliable. I am your Witness. I want to represent You.
🗳 If you like what I do, consider voting for me 🗳
