Makefile¶
All relevant build tasks are implemented in Origin: manage.sh and for CI or
IDE integration a small Makefile
wrapper is available. If you are not
familiar with Makefiles, we recommend to read gnu-make introduction.
The usage is simple, just type make {target-name}
to build a target.
Calling the help
target gives a first overview (make help
):
buildenv
rebuild ./utils/brand.env
babel.compile
pybabel compile ./searx/translations
data.*
all : update searx/languages.py and ./data/*
languages : update searx/data/engines_languages.json & searx/languages.py
useragents: update searx/data/useragents.json with the most recent versions of Firefox.
docs.*
html : build HTML documentation
live : autobuild HTML documentation while editing
gh-pages : deploy on gh-pages branch
prebuild : build reST include files (./build/docs/includes)
clean : clean documentation build
docker
build : build docker image
push : build and push docker image
gecko.driver
download & install geckodriver if not already installed (required for
robot_tests)
node.*
env : download & install npm dependencies locally
clean : drop npm installations
py.*
build : Build python packages at ./dist
clean : delete virtualenv and intermediate py files
pyenv.* :
install : developer install of searx into virtualenv
uninstall : uninstall developer installation
cmd ... : run command ... in virtualenv
OK : test if virtualenv is OK
pypi.upload:
Upload python packages to PyPi (to test use pypi.upload.test)
test.* :
pylint : lint PYLINT_FILES, searx/engines, searx & tests
pep8 : pycodestyle (pep8) for all files except PYLINT_FILES
unit : run unit tests
coverage : run unit tests with coverage
robot : run robot test
clean : clean intermediate test stuff
themes.* :
all : build all themes
oscar : build oscar theme
simple : build simple theme
----
run - run developer instance
install - developer install of searx into virtualenv
uninstall - uninstall developer installation
clean - clean up working tree
search.checker - check search engines
test - run shell & CI tests
test.sh - test shell scripts
ci.test - run CI tests
Python environment¶
We do no longer need to build up the virtualenv manually. Jump into your git
working tree and release a make install
to get a virtualenv with a
developer install of searx (Origin: setup.py).
$ cd ~/searx-clone
$ make install
PYENV [virtualenv] installing ./requirements*.txt into local/py3
...
PYENV OK
PYENV [install] pip install -e 'searx[test]'
...
Successfully installed argparse-1.4.0 searx
BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
BUILDENV INFO:searx:Initialisation done
BUILDENV build utils/brand.env
If you release make install
multiple times the installation will only
rebuild if the sha256 sum of the requirement files fails. With other words:
the check fails if you edit the requirements listed in
Origin: requirements-dev.txt and Origin: requirements.txt).
$ make install
PYENV OK
PYENV [virtualenv] requirements.sha256 failed
[virtualenv] - 6cea6eb6def9e14a18bf32f8a3e... ./requirements-dev.txt
[virtualenv] - 471efef6c73558e391c3adb35f4... ./requirements.txt
...
PYENV [virtualenv] installing ./requirements*.txt into local/py3
...
PYENV OK
PYENV [install] pip install -e 'searx[test]'
...
Successfully installed argparse-1.4.0 searx
BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
BUILDENV INFO:searx:Initialisation done
BUILDENV build utils/brand.env
If you think, something goes wrong with your ./local environment or you change the Origin: setup.py file, you have to call make clean.
make run
¶
To get up a running a developer instance simply call make run
. This enables
debug option in Origin: searx/settings.yml, starts a ./searx/webapp.py
instance, disables debug option again and opens the URL in your favorite WEB
browser (man: xdg-open):
$ make run
PYENV OK
SEARX_DEBUG=1 ./manage.sh pyenv.cmd python ./searx/webapp.py
...
INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
make clean
¶
Drop all intermediate files, all builds, but keep sources untouched. Before
calling make clean
stop all processes using Python environment.
$ make clean
CLEAN pyenv
PYENV [virtualenv] drop ./local/py3
CLEAN docs -- ./build/docs ./dist/docs
CLEAN locally installed npm dependencies
CLEAN test stuff
CLEAN common files
make docs docs.autobuild docs.clean
¶
We describe the usage of the doc.*
targets in the How to contribute /
Documentation section. If you want to edit the documentation
read our live build section. If you are working in your own brand,
adjust your Global Settings.
make docs.gh-pages
¶
To deploy on github.io first adjust your Global Settings. For any further read deploy on github.io.
make test
¶
Runs a series of tests: make test.pylint, test.pep8
, test.unit
and test.robot
. You can run tests selective, e.g.:
$ make test.pep8 test.unit test.sh
TEST test.pep8 OK
...
TEST test.unit OK
...
TEST test.sh OK
make test.sh
¶
Lint shell scripts / if you have changed some bash scripting run this test before commit.
make test.pylint
¶
Pylint is known as one of the best source-code, bug and quality checker for the Python programming language. The pylint profile we use at searx project is found in project’s root folder Origin: .pylintrc.
search.checker.{engine name}
¶
To check all engines:
make search.checker
To check a engine with whitespace in the name like google news replace space by underline:
make search.checker.google_news
To see HTTP requests and more use SEARX_DEBUG:
make SEARX_DEBUG=1 search.checker.google_news
To filter out HTTP redirects (3xx):
make SEARX_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]"
...
Engine google news Checking
https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
--
https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
--
make pybuild
¶
Build Python packages in ./dist/py
:
$ make pybuild
...
BUILD pybuild
running sdist
running egg_info
...
running bdist_wheel
$ ls ./dist
searx-0.18.0-py3-none-any.whl searx-0.18.0.tar.gz
To upload packages to PyPi, there is also a pypi.upload
target (to test use
pypi.upload.test
). Since you are not the owner of PyPi: searx you will
never need to upload.