# Searxng > =============================== --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/answer-captcha.rst # Path: docs/admin/answer-captcha.rst =============================== Answer CAPTCHA from server's IP =============================== With a SSH tunnel we can send requests from server's IP and solve a CAPTCHA that blocks requests from this IP. If your SearXNG instance is hosted at ``example.org`` and your login is ``user`` you can setup a proxy simply by :man:`ssh`: .. code:: bash # SOCKS server: socks://127.0.0.1:8080 $ ssh -q -N -D 8080 user@example.org The ``socks://localhost:8080`` from above can be tested by: .. tabs:: .. group-tab:: server's IP .. code:: bash $ curl -x socks://127.0.0.1:8080 http://ipecho.net/plain n.n.n.n .. group-tab:: desktop's IP .. code:: bash $ curl http://ipecho.net/plain x.x.x.x In the settings of the WEB browser open the *"Network Settings"* and setup a proxy on ``SOCKS5 127.0.0.1:8080`` (see screenshot below). In the WEB browser check the IP from the server is used: - http://ipecho.net/plain Now open the search engine that blocks requests from your server's IP. If you have `issues with the qwant engine `__, solve the CAPTCHA from `qwant.com `__. ----- .. tabs:: .. group-tab:: Firefox .. kernel-figure:: /assets/answer-captcha/ffox-setting-proxy-socks.png :alt: FFox proxy on SOCKS5, 127.0.0.1:8080 Firefox's network settings .. admonition:: :man:`ssh` manual: -D [bind_address:]port Specifies a local “dynamic” application-level port forwarding. This works by allocating a socket to listen to port on the local side .. Whenever a connection is made to this port, the connection is forwarded over the secure channel, and the application protocol is then used to determine where to connect to from the remote machine .. ssh will act as a SOCKS server .. -N Do not execute a remote command. This is useful for just forwarding ports. --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/api.rst # Path: docs/admin/api.rst .. _adminapi: ================== Administration API ================== Get configuration data ====================== .. code:: http GET /config HTTP/1.1 Sample response --------------- .. code:: json { "autocomplete": "", "categories": [ "map", "it", "images", ], "default_locale": "", "default_theme": "simple", "engines": [ { "categories": [ "map" ], "enabled": true, "name": "openstreetmap", "shortcut": "osm" }, { "categories": [ "it" ], "enabled": true, "name": "arch linux wiki", "shortcut": "al" }, { "categories": [ "images" ], "enabled": true, "name": "google images", "shortcut": "goi" }, { "categories": [ "it" ], "enabled": false, "name": "bitbucket", "shortcut": "bb" }, ], "instance_name": "SearXNG", "locales": { "de": "Deutsch (German)", "en": "English", "eo": "Esperanto (Esperanto)", }, "plugins": [ { "enabled": true, "name": "HTTPS rewrite" } ], "safe_search": 0 } Embed search bar ================ The search bar can be embedded into websites. Just paste the example into the HTML of the site. URL of the SearXNG instance and values are customizable. .. code:: html
--- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/architecture.rst # Path: docs/admin/architecture.rst .. _architecture: ============ Architecture ============ .. sidebar:: Further reading - Reverse Proxy: :ref:`Apache ` & :ref:`nginx ` - uWSGI: :ref:`searxng uwsgi` - SearXNG: :ref:`installation basic` Herein you will find some hints and suggestions about typical architectures of SearXNG infrastructures. .. _architecture uWSGI: uWSGI Setup =========== We start with a *reference* setup for public SearXNG instances which can be build up and maintained by the scripts from our :ref:`toolboxing`. .. _arch public: .. kernel-figure:: arch_public.dot :alt: arch_public.dot Reference architecture of a public SearXNG setup. The reference installation activates ``server.limiter`` and ``server.image_proxy`` (:origin:`/etc/searxng/settings.yml `) .. literalinclude:: ../../utils/templates/etc/searxng/settings.yml :language: yaml :end-before: # preferences: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/buildhosts.rst # Path: docs/admin/buildhosts.rst .. _buildhosts: ========== Buildhosts ========== .. contents:: :depth: 2 :local: :backlinks: entry To get best results from build, it's recommend to install additional packages on build hosts (see :ref:`searxng.sh`). .. _searxng.sh install buildhost: Build and Development tools =========================== To Install tools used by build and development tasks in once: .. tabs:: .. group-tab:: SearXNG's development tools .. code:: sh $ sudo -H ./utils/searxng.sh install buildhost This will install packages needed by SearXNG: .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START distro-packages :end-before: END distro-packages and packages needed to build documentation and run tests: .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START build-packages :end-before: END build-packages .. _docs build: Build docs ========== .. _Graphviz: https://graphviz.gitlab.io .. _ImageMagick: https://www.imagemagick.org .. _XeTeX: https://tug.org/xetex/ .. _dvisvgm: https://dvisvgm.de/ .. sidebar:: Sphinx build needs - ImageMagick_ - Graphviz_ - XeTeX_ - dvisvgm_ Most of the sphinx requirements are installed from :origin:`setup.py` and the docs can be build from scratch with ``make docs.html``. For better math and image processing additional packages are needed. The XeTeX_ needed not only for PDF creation, it's also needed for :ref:`math` when HTML output is build. To be able to do :ref:`sphinx:math-support` without CDNs, the math are rendered as images (``sphinx.ext.imgmath`` extension). Here is the extract from the :origin:`docs/conf.py` file, setting math renderer to ``imgmath``: .. literalinclude:: ../conf.py :language: python :start-after: # sphinx.ext.imgmath setup :end-before: # sphinx.ext.imgmath setup END If your docs build (``make docs.html``) shows warnings like this:: WARNING: dot(1) not found, for better output quality install \ graphviz from https://www.graphviz.org .. WARNING: LaTeX command 'latex' cannot be run (needed for math \ display), check the imgmath_latex setting you need to install additional packages on your build host, to get better HTML output (:ref:`install buildhost `). .. tabs:: .. group-tab:: Ubuntu / debian .. code-block:: sh $ sudo apt install graphviz imagemagick texlive-xetex librsvg2-bin .. group-tab:: Arch Linux .. code-block:: sh $ sudo pacman -S graphviz imagemagick texlive-bin extra/librsvg .. group-tab:: Fedora / RHEL .. code-block:: sh $ sudo dnf install graphviz graphviz-gd ImageMagick texlive-xetex-bin librsvg2-tools For PDF output you also need: .. tabs:: .. group-tab:: Ubuntu / debian .. code:: sh $ sudo apt texlive-latex-recommended texlive-extra-utils ttf-dejavu .. group-tab:: Arch Linux .. code:: sh $ sudo pacman -S texlive-core texlive-latexextra ttf-dejavu .. group-tab:: Fedora / RHEL .. code:: sh $ sudo dnf install \ texlive-collection-fontsrecommended texlive-collection-latex \ dejavu-sans-fonts dejavu-serif-fonts dejavu-sans-mono-fonts .. _sh lint: Lint shell scripts ================== .. _ShellCheck: https://github.com/koalaman/shellcheck To lint shell scripts we use ShellCheck_ - a shell script static analysis tool (:ref:`install buildhost `). .. SNIP sh lint requirements .. tabs:: .. group-tab:: Ubuntu / debian .. code-block:: sh $ sudo apt install shellcheck .. group-tab:: Arch Linux .. code-block:: sh $ sudo pacman -S shellcheck .. group-tab:: Fedora / RHEL .. code-block:: sh $ sudo dnf install ShellCheck .. SNAP sh lint requirements --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/index.rst # Path: docs/admin/index.rst =========================== Administrator documentation =========================== .. toctree:: :maxdepth: 2 settings/index installation installation-docker installation-scripts installation-searxng installation-granian installation-uwsgi installation-nginx installation-apache update-searxng answer-captcha searx.favicons searx.limiter api architecture plugins buildhosts --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/installation-apache.rst # Path: docs/admin/installation-apache.rst .. _installation apache: ====== Apache ====== .. _Apache: https://httpd.apache.org/ .. _Apache Debian: https://cwiki.apache.org/confluence/display/HTTPD/DistrosDefaultLayout#DistrosDefaultLayout-Debian,Ubuntu(Apachehttpd2.x): .. _apache2.README.Debian: https://salsa.debian.org/apache-team/apache2/raw/master/debian/apache2.README.Debian .. _Apache Arch Linux: https://wiki.archlinux.org/index.php/Apache_HTTP_Server .. _Apache Fedora: https://docs.fedoraproject.org/en-US/quick-docs/getting-started-with-apache-http-server/index.html .. _Apache directives: https://httpd.apache.org/docs/trunk/mod/directives.html .. _Getting Started: https://httpd.apache.org/docs/current/en/getting-started.html .. _Terms Used to Describe Directives: https://httpd.apache.org/docs/current/en/mod/directive-dict.html .. _Configuration Files: https://httpd.apache.org/docs/current/en/configuring.html .. _ProxyPreserveHost: https://httpd.apache.org/docs/trunk/mod/mod_proxy.html#proxypreservehost .. _LoadModule: https://httpd.apache.org/docs/mod/mod_so.html#loadmodule .. _IncludeOptional: https://httpd.apache.org/docs/mod/core.html#includeoptional .. _DocumentRoot: https://httpd.apache.org/docs/trunk/mod/core.html#documentroot .. _Location: https://httpd.apache.org/docs/trunk/mod/core.html#location .. _uWSGI Apache support: https://uwsgi-docs.readthedocs.io/en/latest/Apache.html .. _mod_proxy_uwsgi: https://uwsgi-docs.readthedocs.io/en/latest/Apache.html#mod-proxy-uwsgi .. _mod_proxy_http: https://httpd.apache.org/docs/current/mod/mod_proxy_http.html .. _mod_proxy: https://httpd.apache.org/docs/current/mod/mod_proxy.html This section explains how to set up a SearXNG instance using the HTTP server Apache_. If you did use the :ref:`installation scripts` and do not have any special preferences you can install the :ref:`SearXNG site ` using :ref:`searxng.sh `: .. code:: bash $ sudo -H ./utils/searxng.sh install apache If you have special interests or problems with setting up Apache, the following section might give you some guidance. .. sidebar:: further read - `Apache Arch Linux`_ - `Apache Debian`_ - `apache2.README.Debian`_ - `Apache Fedora`_ - `Apache directives`_ .. contents:: :depth: 2 :local: :backlinks: entry The Apache HTTP server ====================== If Apache_ is not installed, install it now. If apache_ is new to you, the `Getting Started`_, `Configuration Files`_ and `Terms Used to Describe Directives`_ documentation gives first orientation. There is also a list of `Apache directives`_ *to keep in the pocket*. .. tabs:: .. group-tab:: Ubuntu / debian .. code:: bash sudo -H apt-get install apache2 .. group-tab:: Arch Linux .. code:: bash sudo -H pacman -S apache sudo -H systemctl enable httpd sudo -H systemctl start http .. group-tab:: Fedora / RHEL .. code:: bash sudo -H dnf install httpd sudo -H systemctl enable httpd sudo -H systemctl start httpd Now at http://localhost you should see some kind of *Welcome* or *Test* page. How this default site is configured, depends on the linux distribution (compare `Apache directives`_). .. tabs:: .. group-tab:: Ubuntu / debian .. code:: bash less /etc/apache2/sites-enabled/000-default.conf In this file, there is a line setting the `DocumentRoot`_ directive: .. code:: apache DocumentRoot /var/www/html And the *welcome* page is the HTML file at ``/var/www/html/index.html``. .. group-tab:: Arch Linux .. code:: bash less /etc/httpd/conf/httpd.conf In this file, there is a line setting the `DocumentRoot`_ directive: .. code:: apache DocumentRoot "/srv/http" Options Indexes FollowSymLinks AllowOverride None Require all granted The *welcome* page of Arch Linux is a page showing the directory located at ``DocumentRoot``. This *directory* page is generated by the Module `mod_autoindex `_: .. code:: apache LoadModule autoindex_module modules/mod_autoindex.so ... Include conf/extra/httpd-autoindex.conf .. group-tab:: Fedora / RHEL .. code:: bash less /etc/httpd/conf/httpd.conf In this file, there is a line setting the ``DocumentRoot`` directive: .. code:: apache DocumentRoot "/var/www/html" ... AllowOverride None # Allow open access: Require all granted On fresh installations, the ``/var/www`` is empty and the *default welcome page* is shown, the configuration is located at:: less /etc/httpd/conf.d/welcome.conf .. _Debian's Apache layout: Debian's Apache layout ---------------------- Be aware, Debian's Apache layout is quite different from the standard Apache configuration. For details look at the apache2.README.Debian_ (``/usr/share/doc/apache2/README.Debian.gz``). Some commands you should know on Debian: * :man:`apache2ctl`: Apache HTTP server control interface * :man:`a2enmod`, :man:`a2dismod`: switch on/off modules * :man:`a2enconf`, :man:`a2disconf`: switch on/off configurations * :man:`a2ensite`, :man:`a2dissite`: switch on/off sites .. _apache modules: Apache modules -------------- To load additional modules, in most distributions you have to uncomment the lines with the corresponding LoadModule_ directive, except in :ref:`Debian's Apache layout`. .. tabs:: .. group-tab:: Ubuntu / debian :ref:`Debian's Apache layout` uses :man:`a2enmod` and :man:`a2dismod` to activate or disable modules: .. code:: bash sudo -H a2enmod ssl sudo -H a2enmod headers sudo -H a2enmod proxy sudo -H a2enmod proxy_http sudo -H a2enmod proxy_uwsgi .. group-tab:: Arch Linux In the ``/etc/httpd/conf/httpd.conf`` file, activate LoadModule_ directives: .. code:: apache LoadModule ssl_module modules/mod_ssl.so LoadModule headers_module modules/mod_headers.so LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_http_module modules/mod_proxy_http.so LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so .. group-tab:: Fedora / RHEL In the ``/etc/httpd/conf/httpd.conf`` file, activate LoadModule_ directives: .. code:: apache LoadModule ssl_module modules/mod_ssl.so LoadModule headers_module modules/mod_headers.so LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_http_module modules/mod_proxy_http.so LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so .. _apache sites: Apache sites ------------ .. tabs:: .. group-tab:: Ubuntu / debian In :ref:`Debian's Apache layout` you create a ``searxng.conf`` with the ```` directive and save this file in the *sites available* folder at ``/etc/apache2/sites-available``. To enable the ``searxng.conf`` use :man:`a2ensite`: .. code:: bash sudo -H a2ensite searxng.conf .. group-tab:: Arch Linux In the ``/etc/httpd/conf/httpd.conf`` file add a IncludeOptional_ directive: .. code:: apache IncludeOptional sites-enabled/*.conf Create two folders, one for the *available sites* and one for the *enabled sites*: .. code:: bash mkdir -p /etc/httpd/sites-available mkdir -p /etc/httpd/sites-enabled Create configuration at ``/etc/httpd/sites-available`` and place a symlink to ``sites-enabled``: .. code:: bash sudo -H ln -s /etc/httpd/sites-available/searxng.conf \ /etc/httpd/sites-enabled/searxng.conf .. group-tab:: Fedora / RHEL In the ``/etc/httpd/conf/httpd.conf`` file add a IncludeOptional_ directive: .. code:: apache IncludeOptional sites-enabled/*.conf Create two folders, one for the *available sites* and one for the *enabled sites*: .. code:: bash mkdir -p /etc/httpd/sites-available mkdir -p /etc/httpd/sites-enabled Create configuration at ``/etc/httpd/sites-available`` and place a symlink to ``sites-enabled``: .. code:: bash sudo -H ln -s /etc/httpd/sites-available/searxng.conf \ /etc/httpd/sites-enabled/searxng.conf .. _apache searxng site: Apache's SearXNG site ===================== .. _mod_uwsgi: https://uwsgi-docs.readthedocs.io/en/latest/Apache.html#mod-uwsgi .. sidebar:: uWSGI Use mod_proxy_uwsgi_ / don't use the old mod_uwsgi_ anymore. To proxy the incoming requests to the SearXNG instance Apache needs the mod_proxy_ module (:ref:`apache modules`). .. sidebar:: HTTP headers With ProxyPreserveHost_ the incoming ``Host`` header is passed to the proxied host. Depending on what your SearXNG installation is listening on, you need a http mod_proxy_http_) or socket (mod_proxy_uwsgi_) communication to upstream. The :ref:`installation scripts` installs the :ref:`reference setup ` and a :ref:`uwsgi setup` that listens on a socket by default. You can install and activate your own ``searxng.conf`` like shown in :ref:`apache sites`. .. tabs:: .. group-tab:: socket .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START apache socket :end-before: END apache socket .. group-tab:: http .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START apache http :end-before: END apache http .. _restart apache: Restart service: .. tabs:: .. group-tab:: Ubuntu / debian .. code:: bash sudo -H systemctl restart apache2 sudo -H service uwsgi restart searxng .. group-tab:: Arch Linux .. code:: bash sudo -H systemctl restart httpd sudo -H systemctl restart uwsgi@searxng .. group-tab:: Fedora / RHEL .. code:: bash sudo -H systemctl restart httpd sudo -H touch /etc/uwsgi.d/searxng.ini disable logs ============ For better privacy you can disable Apache logs. In the examples above activate one of the lines and `restart apache`_: .. code:: apache SetEnvIf Request_URI "/searxng" dontlog # CustomLog /dev/null combined env=dontlog The ``CustomLog`` directive disables logs for the entire (virtual) server, use it when the URL of the service does not have a path component (``/searxng``), so when SearXNG is located at root (``/``). --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/installation-docker.rst # Path: docs/admin/installation-docker.rst .. _installation container: ====================== Installation container ====================== .. _Docker 101: https://docs.docker.com/get-started/docker-overview .. _Docker cheat sheet (PDF doc): https://docs.docker.com/get-started/docker_cheatsheet.pdf .. _Podman rootless containers: https://github.com/containers/podman/blob/main/docs/tutorials/rootless_tutorial.md .. _DockerHub mirror: https://hub.docker.com/r/searxng/searxng .. _GHCR mirror: https://ghcr.io/searxng/searxng .. _Docker compose: https://github.com/searxng/searxng-docker .. sidebar:: info - `Docker 101`_ - `Docker cheat sheet (PDF doc)`_ - `Podman rootless containers`_ .. important:: Understanding container architecture basics is essential for properly maintaining your SearXNG instance. This guide assumes familiarity with container concepts and provides deployment steps at a high level. If you're new to containers, we recommend learning the fundamentals at `Docker 101`_ before proceeding. Container images are the basis for deployments in containerized environments, `Docker compose`_, Kubernetes and more. .. _Container installation: Installation ============ .. _Container prerequisites: Prerequisites ------------- You need a working Docker or Podman installation on your system. Choose the option that works best for your environment: - `Docker `_ (recommended for most users) - `Podman `_ In the case of Docker, you need to add the user running the container to the ``docker`` group and restart the session: .. code:: sh $ sudo usermod -aG docker $USER In the case of Podman, no additional steps are generally required, but there are some considerations when running `Podman rootless containers`_. .. _Container pulling images: Pulling images -------------- .. note:: DockerHub now applies rate limits to unauthenticated image pulls. If you are affected by this, you can use the `GHCR mirror`_ instead. The official images are mirrored at: - `DockerHub mirror`_ - `GHCR mirror`_ (GitHub Container Registry) Pull the latest image: .. code:: sh $ docker pull docker.io/searxng/searxng:latest \.\. or if you want to lock in to a specific version: .. code:: sh $ docker pull docker.io/searxng/searxng:2025.8.1-3d96414 .. _Container instancing: Instancing ========== This section is intended for advanced users who need custom deployments. We recommend using `Docker compose`_, which provides a preconfigured environment with sensible defaults. Basic container instancing example: .. code:: sh # Create directories for configuration and persistent data $ mkdir -p ./searxng/config/ ./searxng/data/ $ cd ./searxng/ # Run the container $ docker run --name searxng -d \ -p 8888:8080 \ -v "./config/:/etc/searxng/" \ -v "./data/:/var/cache/searxng/" \ docker.io/searxng/searxng:latest This will start SearXNG in the background, accessible at http://localhost:8888 .. _Container management: Management ---------- List running containers: .. code:: sh $ docker container list CONTAINER ID IMAGE ... CREATED PORTS NAMES 1af574997e63 ... ... 3 minutes ago 0.0.0.0:8888->8080/tcp searxng Access the container shell (troubleshooting): .. code:: sh $ docker container exec -it --user root searxng /bin/sh -l 1af574997e63:/usr/local/searxng# Stop and remove the container: .. code:: sh $ docker container stop searxng $ docker container rm searxng .. _Container volumes: Volumes ======= Two volumes are exposed that should be mounted to preserve its contents: - ``/etc/searxng``: Configuration files (settings.yml, etc.) - ``/var/cache/searxng``: Persistent data (faviconcache.db, etc.) .. _Container environment variables: Environment variables ===================== The following environment variables can be configured: - ``$SEARXNG_*``: Controls the SearXNG configuration options, look out for environment ``$SEARXNG_*`` in :ref:`settings server` and :ref:`settings general`. - ``$GRANIAN_*``: Controls the :ref:`Granian server options `. - ``$FORCE_OWNERSHIP``: Ensures mounted volumes/files are owned by the ``searxng:searxng`` user (default: ``true``) Container internal paths (don't modify unless you know what you're doing): - ``$CONFIG_PATH``: Path to the SearXNG configuration directory (default: ``/etc/searxng``) - ``$SEARXNG_SETTINGS_PATH``: Path to the SearXNG settings file (default: ``$CONFIG_PATH/settings.yml``) - ``$DATA_PATH``: Path to the SearXNG data directory (default: ``/var/cache/searxng``) .. _Container custom certificates: Custom certificates =================== You can mount ``/usr/local/share/ca-certificates/`` folder to add/remove additional certificates as needed. They will be available on container (re)start or when running ``update-ca-certificates`` in the container shell. .. _Container custom images: Custom images ============= To build your own SearXNG container image from source (please note, custom container images are not officially supported): .. code:: sh $ git clone https://github.com/searxng/searxng.git $ cd ./searxng/ # Run the container build script $ make container $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE localhost/searxng/searxng 2025.8.1-3d96414 ... About a minute ago 183 MB localhost/searxng/searxng latest ... About a minute ago 183 MB localhost/searxng/searxng builder ... About a minute ago 524 MB ghcr.io/searxng/base searxng-builder ... 2 days ago 378 MB ghcr.io/searxng/base searxng ... 2 days ago 42.2 MB --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/installation-granian.rst # Path: docs/admin/installation-granian.rst .. _searxng granian: ======= Granian ======= .. _Options: https://github.com/emmett-framework/granian/blob/master/README.md#options .. _Workers and threads: https://github.com/emmett-framework/granian/blob/master/README.md#workers-and-threads .. _Backpressure: https://github.com/emmett-framework/granian/blob/master/README.md#backpressure .. _Runtime mode: https://github.com/emmett-framework/granian/blob/master/README.md#runtime-mode .. sidebar:: further reading - `Options`_ - `Workers and threads`_ - `Backpressure`_ - `Runtime mode`_ .. note:: Granian will be the future replacement for :ref:`searxng uwsgi` in SearXNG. At the moment, it's only officially supported in the :ref:`installation container`. .. _Granian installation: Installation ============ We only recommend installing Granian with pip, as officially documented. Run the following command in the Python environment of the SearXNG installation: .. code:: sh $ pip install granian .. _Granian configuration: Configuration ============= .. note:: It's not advised to modify the amount of workers, expect increased resource usage and potential issues with :ref:`botdetection`. Granian can be configured via option parameters and environment variables (``$GRANIAN_*``). We provide sane defaults that should fit most use cases, however if you feel you should change something, Granian documents all available parameters in the `Options`_ section. --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/installation-nginx.rst # Path: docs/admin/installation-nginx.rst .. _installation nginx: ===== NGINX ===== .. _nginx: https://docs.nginx.com/nginx/admin-guide/ .. _nginx server configuration: https://docs.nginx.com/nginx/admin-guide/web-server/web-server/#setting-up-virtual-servers .. _nginx beginners guide: https://nginx.org/en/docs/beginners_guide.html .. _Getting Started wiki: https://www.nginx.com/resources/wiki/start/ .. _uWSGI support from nginx: https://uwsgi-docs.readthedocs.io/en/latest/Nginx.html .. _uwsgi_params: https://uwsgi-docs.readthedocs.io/en/latest/Nginx.html#configuring-nginx .. _SCRIPT_NAME: https://werkzeug.palletsprojects.com/en/1.0.x/wsgi/#werkzeug.wsgi.get_script_name This section explains how to set up a SearXNG instance using the HTTP server nginx_. If you have used the :ref:`installation scripts` and do not have any special preferences you can install the :ref:`SearXNG site ` using :ref:`searxng.sh `: .. code:: bash $ sudo -H ./utils/searxng.sh install nginx If you have special interests or problems with setting up nginx, the following section might give you some guidance. .. sidebar:: further reading - nginx_ - `nginx beginners guide`_ - `nginx server configuration`_ - `Getting Started wiki`_ - `uWSGI support from nginx`_ .. contents:: :depth: 2 :local: :backlinks: entry The nginx HTTP server ===================== If nginx_ is not installed, install it now. .. tabs:: .. group-tab:: Ubuntu / debian .. code:: bash sudo -H apt-get install nginx .. group-tab:: Arch Linux .. code-block:: sh sudo -H pacman -S nginx-mainline sudo -H systemctl enable nginx sudo -H systemctl start nginx .. group-tab:: Fedora / RHEL .. code-block:: sh sudo -H dnf install nginx sudo -H systemctl enable nginx sudo -H systemctl start nginx Now at http://localhost you should see a *Welcome to nginx!* page, on Fedora you see a *Fedora Webserver - Test Page*. The test page comes from the default `nginx server configuration`_. How this default site is configured, depends on the linux distribution: .. tabs:: .. group-tab:: Ubuntu / debian .. code:: bash less /etc/nginx/nginx.conf There is one line that includes site configurations from: .. code:: nginx include /etc/nginx/sites-enabled/*; .. group-tab:: Arch Linux .. code-block:: sh less /etc/nginx/nginx.conf There is a configuration section named ``server``: .. code-block:: nginx server { listen 80; server_name localhost; # ... } .. group-tab:: Fedora / RHEL .. code-block:: sh less /etc/nginx/nginx.conf There is one line that includes site configurations from: .. code:: nginx include /etc/nginx/conf.d/*.conf; .. _nginx searxng site: NGINX's SearXNG site ==================== Now you have to create a configuration file (``searxng.conf``) for the SearXNG site. If nginx_ is new to you, the `nginx beginners guide`_ is a good starting point and the `Getting Started wiki`_ is always a good resource *to keep in the pocket*. Depending on what your SearXNG installation is listening on, you need a http or socket communication to upstream. .. tabs:: .. group-tab:: socket .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START nginx socket :end-before: END nginx socket .. group-tab:: http .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START nginx http :end-before: END nginx http The :ref:`installation scripts` installs the :ref:`reference setup ` and a :ref:`uwsgi setup` that listens on a socket by default. .. tabs:: .. group-tab:: Ubuntu / debian Create configuration at ``/etc/nginx/sites-available/`` and place a symlink to ``sites-enabled``: .. code:: bash sudo -H ln -s /etc/nginx/sites-available/searxng.conf \ /etc/nginx/sites-enabled/searxng.conf .. group-tab:: Arch Linux In the ``/etc/nginx/nginx.conf`` file, in the ``server`` section add a `include `_ directive: .. code:: nginx server { # ... include /etc/nginx/default.d/*.conf; # ... } Create two folders, one for the *available sites* and one for the *enabled sites*: .. code:: bash mkdir -p /etc/nginx/default.d mkdir -p /etc/nginx/default.apps-available Create configuration at ``/etc/nginx/default.apps-available`` and place a symlink to ``default.d``: .. code:: bash sudo -H ln -s /etc/nginx/default.apps-available/searxng.conf \ /etc/nginx/default.d/searxng.conf .. group-tab:: Fedora / RHEL Create a folder for the *available sites*: .. code:: bash mkdir -p /etc/nginx/default.apps-available Create configuration at ``/etc/nginx/default.apps-available`` and place a symlink to ``conf.d``: .. code:: bash sudo -H ln -s /etc/nginx/default.apps-available/searxng.conf \ /etc/nginx/conf.d/searxng.conf Restart services: .. tabs:: .. group-tab:: Ubuntu / debian .. code:: bash sudo -H systemctl restart nginx sudo -H service uwsgi restart searxng .. group-tab:: Arch Linux .. code:: bash sudo -H systemctl restart nginx sudo -H systemctl restart uwsgi@searxng .. group-tab:: Fedora / RHEL .. code:: bash sudo -H systemctl restart nginx sudo -H touch /etc/uwsgi.d/searxng.ini Disable logs ============ For better privacy you can disable nginx logs in ``/etc/nginx/nginx.conf``. .. code:: nginx http { # ... access_log /dev/null; error_log /dev/null; # ... } --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/installation-scripts.rst # Path: docs/admin/installation-scripts.rst .. _installation scripts: =================== Installation Script =================== .. sidebar:: Update the OS first! To avoid unwanted side effects, update your OS before installing SearXNG. The following will install a setup as shown in :ref:`the reference architecture `. First you need to get a clone of the repository. The clone is only needed for the installation procedure and some maintenance tasks. .. sidebar:: further read - :ref:`toolboxing` Jump to a folder that is readable by *others* and start to clone SearXNG, alternatively you can create your own fork and clone from there. .. code:: bash $ cd ~/Downloads $ git clone https://github.com/searxng/searxng.git searxng $ cd searxng .. sidebar:: further read - :ref:`inspect searxng` To install a SearXNG :ref:`reference setup ` including a :ref:`uWSGI setup ` as described in the :ref:`installation basic` and in the :ref:`searxng uwsgi` section type: .. code:: bash $ sudo -H ./utils/searxng.sh install all .. attention:: For the installation procedure, use a *sudoer* login to run the scripts. If you install from ``root``, take into account that the scripts are creating a ``searxng`` user. In the installation procedure this new created user does need to have read access to the cloned SearXNG repository, which is not the case if you clone it into a folder below ``/root``! .. sidebar:: further read - :ref:`update searxng` .. _caddy: https://hub.docker.com/_/caddy When all services are installed and running fine, you can add SearXNG to your HTTP server. We do not have any preferences regarding the HTTP server, you can use whatever you prefer. We implemented installation procedures for: - :ref:`installation nginx` - :ref:`installation apache` --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/installation-searxng.rst # Path: docs/admin/installation-searxng.rst .. _installation basic: ========================= Step by step installation ========================= .. contents:: :depth: 2 :local: :backlinks: entry In this section we show the setup of a SearXNG instance that will be installed by the :ref:`installation scripts`. .. _install packages: Install packages ================ .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START distro-packages :end-before: END distro-packages .. hint:: This installs also the packages needed by :ref:`searxng uwsgi` .. _create searxng user: Create user =========== .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START create user :end-before: END create user .. _searxng-src: Install SearXNG & dependencies ============================== Start a interactive shell from new created user and clone SearXNG: .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START clone searxng :end-before: END clone searxng In the same shell create *virtualenv*: .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START create virtualenv :end-before: END create virtualenv To install SearXNG's dependencies, exit the SearXNG *bash* session you opened above and start a new one. Before installing, check if your *virtualenv* was sourced from the login (*~/.profile*): .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START manage.sh update_packages :end-before: END manage.sh update_packages .. tip:: Open a second terminal for the configuration tasks and leave the ``(searx)$`` terminal open for the tasks below. .. _use_default_settings.yml: Configuration ============= .. sidebar:: ``use_default_settings: True`` - :ref:`settings.yml` - :ref:`settings location` - :ref:`settings use_default_settings` - :origin:`/etc/searxng/settings.yml ` To create a initial ``/etc/searxng/settings.yml`` we recommend to start with a copy of the file :origin:`utils/templates/etc/searxng/settings.yml`. This setup :ref:`use default settings ` from :origin:`searx/settings.yml` and is shown in the tab *"Use default settings"* below. This setup: - enables :ref:`limiter ` to protect against bots - enables :ref:`image proxy ` for better privacy Modify the ``/etc/searxng/settings.yml`` to your needs: .. tabs:: .. group-tab:: Use default settings .. literalinclude:: ../../utils/templates/etc/searxng/settings.yml :language: yaml :end-before: # preferences: To see the entire file jump to :origin:`utils/templates/etc/searxng/settings.yml` .. group-tab:: searx/settings.yml .. literalinclude:: ../../searx/settings.yml :language: yaml :end-before: # hostnames: To see the entire file jump to :origin:`searx/settings.yml` For a *minimal setup* you need to set ``server:secret_key``. .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START searxng config :end-before: END searxng config Check ===== To check your SearXNG setup, optional enable debugging and start the *webapp*. SearXNG looks at the exported environment ``$SEARXNG_SETTINGS_PATH`` for a configuration file. .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START check searxng installation :end-before: END check searxng installation If everything works fine, hit ``[CTRL-C]`` to stop the *webapp* and disable the debug option in ``settings.yml``. You can now exit SearXNG user bash session (enter exit command twice). At this point SearXNG is not demonized; uwsgi allows this. --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/installation-uwsgi.rst # Path: docs/admin/installation-uwsgi.rst .. _searxng uwsgi: ===== uWSGI ===== .. sidebar:: further reading - `systemd.unit`_ - `uWSGI Emperor`_ .. contents:: :depth: 2 :local: :backlinks: entry .. _systemd.unit: https://www.freedesktop.org/software/systemd/man/systemd.unit.html .. _One service per app in systemd: https://uwsgi-docs.readthedocs.io/en/latest/Systemd.html#one-service-per-app-in-systemd .. _uWSGI Emperor: https://uwsgi-docs.readthedocs.io/en/latest/Emperor.html .. _uwsgi ini file: https://uwsgi-docs.readthedocs.io/en/latest/Configuration.html#ini-files .. _systemd unit template: http://0pointer.de/blog/projects/instances.html Origin uWSGI ============ .. _Tyrant mode: https://uwsgi-docs.readthedocs.io/en/latest/Emperor.html#tyrant-mode-secure-multi-user-hosting How uWSGI is implemented by distributors varies. The uWSGI project itself recommends two methods: 1. `systemd.unit`_ template file as described here `One service per app in systemd`_: There is one `systemd unit template`_ on the system installed and one `uwsgi ini file`_ per uWSGI-app placed at dedicated locations. Take archlinux and a ``searxng.ini`` as example:: systemd template unit: /usr/lib/systemd/system/uwsgi@.service contains: [Service] ExecStart=/usr/bin/uwsgi --ini /etc/uwsgi/%I.ini SearXNG application: /etc/uwsgi/searxng.ini links to: /etc/uwsgi/apps-available/searxng.ini The SearXNG app (template ``/etc/uwsgi/%I.ini``) can be maintained as known from common systemd units: .. code:: sh $ systemctl enable uwsgi@searxng $ systemctl start uwsgi@searxng $ systemctl restart uwsgi@searxng $ systemctl stop uwsgi@searxng 2. The `uWSGI Emperor`_ which fits for maintaining a large range of uwsgi apps and there is a `Tyrant mode`_ to secure multi-user hosting. The Emperor mode is a special uWSGI instance that will monitor specific events. The Emperor mode (the service) is started by a (common, not template) systemd unit. The Emperor service will scan specific directories for `uwsgi ini file`_\s (also know as *vassals*). If a *vassal* is added, removed or the timestamp is modified, a corresponding action takes place: a new uWSGI instance is started, reload or stopped. Take Fedora and a ``searxng.ini`` as example:: to install & start SearXNG instance create --> /etc/uwsgi.d/searxng.ini to reload the instance edit timestamp --> touch /etc/uwsgi.d/searxng.ini to stop instance remove ini --> rm /etc/uwsgi.d/searxng.ini Distributors ============ The `uWSGI Emperor`_ mode and `systemd unit template`_ is what the distributors mostly offer their users, even if they differ in the way they implement both modes and their defaults. Another point they might differ in is the packaging of plugins (if so, compare :ref:`install packages`) and what the default python interpreter is (python2 vs. python3). While archlinux does not start a uWSGI service by default, Fedora (RHEL) starts a Emperor in `Tyrant mode`_ by default (you should have read :ref:`uWSGI Tyrant mode pitfalls`). Worth to know; debian (ubuntu) follow a complete different approach, read see :ref:`Debian's uWSGI layout`. .. _Debian's uWSGI layout: Debian's uWSGI layout --------------------- .. _uwsgi.README.Debian: https://salsa.debian.org/uwsgi-team/uwsgi/-/raw/debian/latest/debian/uwsgi.README.Debian Be aware, Debian's uWSGI layout is quite different from the standard uWSGI configuration. Your are familiar with :ref:`Debian's Apache layout`? .. they do a similar thing for the uWSGI infrastructure. The folders are:: /etc/uwsgi/apps-available/ /etc/uwsgi/apps-enabled/ The `uwsgi ini file`_ is enabled by a symbolic link:: ln -s /etc/uwsgi/apps-available/searxng.ini /etc/uwsgi/apps-enabled/ More details can be found in the uwsgi.README.Debian_ (``/usr/share/doc/uwsgi/README.Debian.gz``). Some commands you should know on Debian: .. code:: none Commands recognized by init.d script ==================================== You can issue to init.d script following commands: * start | starts daemon * stop | stops daemon * reload | sends to daemon SIGHUP signal * force-reload | sends to daemon SIGTERM signal * restart | issues 'stop', then 'start' commands * status | shows status of daemon instance (running/not running) 'status' command must be issued with exactly one argument: ''. Controlling specific instances of uWSGI ======================================= You could control specific instance(s) by issuing: SYSTEMCTL_SKIP_REDIRECT=1 service uwsgi ... where: * is one of 'start', 'stop' etc. * is the name of configuration file (without extension) For example, this is how instance for /etc/uwsgi/apps-enabled/hello.xml is started: SYSTEMCTL_SKIP_REDIRECT=1 service uwsgi start hello .. _uWSGI maintenance: uWSGI maintenance ================= .. tabs:: .. group-tab:: Ubuntu / debian .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START searxng uwsgi-description ubuntu-20.04 :end-before: END searxng uwsgi-description ubuntu-20.04 .. hotfix: a bug group-tab need this comment .. group-tab:: Arch Linux .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START searxng uwsgi-description arch :end-before: END searxng uwsgi-description arch .. hotfix: a bug group-tab need this comment .. group-tab:: Fedora / RHEL .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START searxng uwsgi-description fedora :end-before: END searxng uwsgi-description fedora .. _uwsgi setup: uWSGI setup =========== Create the configuration ini-file according to your distribution and restart the uwsgi application. As shown below, the :ref:`installation scripts` installs by default a uWSGI setup that listens on a socket. .. tabs:: .. group-tab:: Ubuntu / debian .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START searxng uwsgi-appini ubuntu-20.04 :end-before: END searxng uwsgi-appini ubuntu-20.04 .. hotfix: a bug group-tab need this comment .. group-tab:: Arch Linux .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START searxng uwsgi-appini arch :end-before: END searxng uwsgi-appini arch .. hotfix: a bug group-tab need this comment .. group-tab:: Fedora / RHEL .. kernel-include:: $DOCS_BUILD/includes/searxng.rst :start-after: START searxng uwsgi-appini fedora :end-before: END searxng uwsgi-appini fedora .. _uWSGI Tyrant mode pitfalls: Pitfalls of the Tyrant mode =========================== The implementation of the process owners and groups in the `Tyrant mode`_ is somewhat unusual and requires special consideration. In `Tyrant mode`_ mode the Emperor will run the vassal using the UID/GID of the vassal configuration file (user and group of the app ``.ini`` file). .. _#2099@uWSGI: https://github.com/unbit/uwsgi/issues/2099 .. _#752@uWSGI: https://github.com/unbit/uwsgi/pull/752 .. _#2425uWSGI: https://github.com/unbit/uwsgi/issues/2425 Without option ``emperor-tyrant-initgroups=true`` in ``/etc/uwsgi.ini`` the process won't get the additional groups, but this option is not available in 2.0.x branch (see `#2099@uWSGI`_) the feature `#752@uWSGI`_ has been merged (on Oct. 2014) to the master branch of uWSGI but had never been released; the last major release is from Dec. 2013, since the there had been only bugfix releases (see `#2425uWSGI`_). To shorten up: **In Tyrant mode, there is no way to get additional groups, and the uWSGI process misses additional permissions that may be needed.** For example on Fedora (RHEL): If you try to install a valkey DB with socket communication and you want to connect to it from the SearXNG uWSGI, you will see a *Permission denied* in the log of your instance:: ERROR:searx.valkeydb: [searxng (993)] can't connect valkey DB ... ERROR:searx.valkeydb: Error 13 connecting to unix socket: /usr/local/searxng-valkey/run/valkey.sock. Permission denied. ERROR:searx.plugins.limiter: init limiter DB failed!!! Even if your *searxng* user of the uWSGI process is added to additional groups to give access to the socket from the valkey DB:: $ groups searxng searxng : searxng searxng-valkey To see the effective groups of the uwsgi process, you have to look at the status of the process, by example:: $ ps -aef | grep '/usr/sbin/uwsgi --ini searxng.ini' searxng 93 92 0 12:43 ? 00:00:00 /usr/sbin/uwsgi --ini searxng.ini searxng 186 93 0 12:44 ? 00:00:01 /usr/sbin/uwsgi --ini searxng.ini Here you can see that the additional "Groups" of PID 186 are unset (missing gid of ``searxng-valkey``):: $ cat /proc/186/task/186/status ... Uid: 993 993 993 993 Gid: 993 993 993 993 FDSize: 128 Groups: ... --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/installation.rst # Path: docs/admin/installation.rst .. _installation: ============ Installation ============ *You're spoilt for choice*, choose your preferred method of installation. - :ref:`installation container` - :ref:`installation scripts` - :ref:`installation basic` The :ref:`installation basic` is an excellent illustration of *how a SearXNG instance is build up* (see :ref:`architecture uWSGI`). If you do not have any special preferences, it's recommended to use the :ref:`installation container` or the :ref:`installation scripts`. .. attention:: SearXNG is growing rapidly, you should regularly read our :ref:`migrate and stay tuned` section. If you want to upgrade an existing instance, you should read this section first! --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/plugins.rst # Path: docs/admin/plugins.rst .. _plugins admin: =============== List of plugins =============== Further reading .. - :ref:`SearXNG settings ` - :ref:`dev plugin` .. _configured plugins: .. jinja:: searx .. flat-table:: Plugins configured at built time (defaults) :header-rows: 1 :stub-columns: 1 :widths: 3 1 9 * - Name - Active - Description {% for plg in plugins %} * - {{plg.info.name}} - {{(plg.active and "yes") or "no"}} - {{plg.info.description}} {% endfor %} --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/searx.favicons.rst # Path: docs/admin/searx.favicons.rst .. _favicons: ======== Favicons ======== .. sidebar:: warning Don't activate the favicons before reading the documentation. .. contents:: :depth: 2 :local: :backlinks: entry Activating the favicons in SearXNG is very easy, but this **generates a significantly higher load** in the client/server communication and increases resources needed on the server. To mitigate these disadvantages, various methods have been implemented, including a *cache*. The cache must be parameterized according to your own requirements and maintained regularly. To activate favicons in SearXNG's result list, set a default ``favicon_resolver`` in the :ref:`search ` settings: .. code:: yaml search: favicon_resolver: "duckduckgo" By default and without any extensions, SearXNG serves these resolvers: - ``duckduckgo`` - ``allesedv`` - ``google`` - ``yandex`` With the above setting favicons are displayed, the user has the option to deactivate this feature in his settings. If the user is to have the option of selecting from several *resolvers*, a further setting is required / but this setting will be discussed :ref:`later ` in this article, first we have to setup the favicons cache. Infrastructure ============== The infrastructure for providing the favicons essentially consists of three parts: - :py:obj:`Favicons-Proxy <.favicons.proxy>` (aka *proxy*) - :py:obj:`Favicons-Resolvers <.favicons.resolvers>` (aka *resolver*) - :py:obj:`Favicons-Cache <.favicons.cache>` (aka *cache*) To protect the privacy of users, the favicons are provided via a *proxy*. This *proxy* is automatically activated with the above activation of a *resolver*. Additional requests are required to provide the favicons: firstly, the *proxy* must process the incoming requests and secondly, the *resolver* must make outgoing requests to obtain the favicons from external sources. A *cache* has been developed to massively reduce both, incoming and outgoing requests. This *cache* is also activated automatically with the above activation of a *resolver*. In its defaults, however, the *cache* is minimal and not well suitable for a production environment! .. _favicon cache setup: Setting up the cache ==================== To parameterize the *cache* and more settings of the favicons infrastructure, a TOML_ configuration is created in the file ``/etc/searxng/favicons.toml``. .. code:: toml [favicons] cfg_schema = 1 # config's schema version no. [favicons.cache] db_url = "/var/cache/searxng/faviconcache.db" # default: "/tmp/faviconcache.db" LIMIT_TOTAL_BYTES = 2147483648 # 2 GB / default: 50 MB # HOLD_TIME = 5184000 # 60 days / default: 30 days # BLOB_MAX_BYTES = 40960 # 40 KB / default 20 KB # MAINTENANCE_MODE = "off" # default: "auto" # MAINTENANCE_PERIOD = 600 # 10min / default: 1h :py:obj:`cfg_schema <.FaviconConfig.cfg_schema>`: Is required to trigger any processes required for future upgrades / don't change it. :py:obj:`cache.db_url <.FaviconCacheConfig.db_url>`: The path to the (SQLite_) database file. The default path is in the `/tmp`_ folder, which is deleted on every reboot and is therefore unsuitable for a production environment. The FHS_ provides the folder `/var/cache`_ for the cache of applications, so a suitable storage location of SearXNG's caches is folder ``/var/cache/searxng``. In a standard installation (compare :ref:`create searxng user`), the folder must be created and the user under which the SearXNG process is running must be given write permission to this folder. .. code:: bash $ sudo mkdir /var/cache/searxng $ sudo chown root:searxng /var/cache/searxng/ $ sudo chmod g+w /var/cache/searxng/ In container systems, a volume should be mounted for this folder. Check whether the process in the container has read/write access to the mounted folder. :py:obj:`cache.LIMIT_TOTAL_BYTES <.FaviconCacheConfig.LIMIT_TOTAL_BYTES>`: Maximum of bytes stored in the cache of all blobs. The limit is only reached at each maintenance interval after which the oldest BLOBs are deleted; the limit is exceeded during the maintenance period. .. attention:: If the maintenance period is too long or maintenance is switched off completely, the cache grows uncontrollably. SearXNG hosters can change other parameters of the cache as required: - :py:obj:`cache.HOLD_TIME <.FaviconCacheConfig.HOLD_TIME>` - :py:obj:`cache.BLOB_MAX_BYTES <.FaviconCacheConfig.BLOB_MAX_BYTES>` Maintenance of the cache ------------------------ Regular maintenance of the cache is required! By default, regular maintenance is triggered automatically as part of the client requests: - :py:obj:`cache.MAINTENANCE_MODE <.FaviconCacheConfig.MAINTENANCE_MODE>` (default ``auto``) - :py:obj:`cache.MAINTENANCE_PERIOD <.FaviconCacheConfig.MAINTENANCE_PERIOD>` (default ``6000`` / 1h) As an alternative to maintenance as part of the client request process, it is also possible to carry out maintenance using an external process. For example, by creating a :man:`crontab` entry for maintenance: .. code:: bash $ python -m searx.favicons cache maintenance The following command can be used to display the state of the cache: .. code:: bash $ python -m searx.favicons cache state .. _favicon proxy setup: Proxy configuration =================== Most of the options of the :py:obj:`Favicons-Proxy <.favicons.proxy>` are already set sensibly with settings from the :ref:`settings.yml ` and should not normally be adjusted. .. code:: toml [favicons.proxy] max_age = 5184000 # 60 days / default: 7 days (604800 sec) :py:obj:`max_age <.FaviconProxyConfig.max_age>`: The `HTTP Cache-Control max-age`_ response directive indicates that the response remains fresh until N seconds after the response is generated. This setting therefore determines how long a favicon remains in the client's cache. As a rule, in the favicons infrastructure of SearXNG's this setting only affects favicons whose byte size exceeds :ref:`BLOB_MAX_BYTES ` (the other favicons that are already in the cache are embedded as `data URL`_ in the :py:obj:`generated HTML <.favicons.proxy.favicon_url>`, which can greatly reduce the number of additional requests). .. _register resolvers: Register resolvers ------------------ A :py:obj:`resolver <.favicon.resolvers>` is a function that obtains the favicon from an external source. The resolver functions available to the user are registered with their fully qualified name (FQN_) in a ``resolver_map``. If no ``resolver_map`` is defined in the ``favicon.toml``, the favicon infrastructure of SearXNG generates this ``resolver_map`` automatically depending on the ``settings.yml``. SearXNG would automatically generate the following TOML configuration from the following YAML configuration: .. code:: yaml search: favicon_resolver: "duckduckgo" .. code:: toml [favicons.proxy.resolver_map] "duckduckgo" = "searx.favicons.resolvers.duckduckgo" If this automatism is not desired, then (and only then) a separate ``resolver_map`` must be created. For example, to give the user two resolvers to choose from, the following configuration could be used: .. code:: toml [favicons.proxy.resolver_map] "duckduckgo" = "searx.favicons.resolvers.duckduckgo" "allesedv" = "searx.favicons.resolvers.allesedv" # "google" = "searx.favicons.resolvers.google" # "yandex" = "searx.favicons.resolvers.yandex" .. note:: With each resolver, the resource requirement increases significantly. The number of resolvers increases: - the number of incoming/outgoing requests and - the number of favicons to be stored in the cache. In the following we list the resolvers available in the core of SearXNG, but via the FQN_ it is also possible to implement your own resolvers and integrate them into the *proxy*: - :py:obj:`searx.favicons.resolvers.duckduckgo` - :py:obj:`searx.favicons.resolvers.allesedv` - :py:obj:`searx.favicons.resolvers.google` - :py:obj:`searx.favicons.resolvers.yandex` .. _SQLite: https://www.sqlite.org/ .. _FHS: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html .. _`/var/cache`: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s05.html .. _`/tmp`: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s18.html .. _TOML: https://toml.io/en/ .. _HTTP Cache-Control max-age: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#response_directives .. _data URL: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URLs .. _FQN: https://en.wikipedia.org/wiki/Fully_qualified_name --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/searx.limiter.rst # Path: docs/admin/searx.limiter.rst .. _limiter: ======= Limiter ======= .. sidebar:: info The limiter requires a :ref:`Valkey ` database. .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.limiter :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/settings/index.rst # Path: docs/admin/settings/index.rst .. _searxng settings.yml: ======== Settings ======== .. sidebar:: Further reading .. - :ref:`engine settings` - :ref:`engine file` .. toctree:: :maxdepth: 2 settings settings_engines settings_brand settings_general settings_search settings_server settings_ui settings_redis settings_valkey settings_outgoing settings_categories_as_tabs settings_plugins --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/settings/settings.rst # Path: docs/admin/settings/settings.rst .. _settings.yml: ================ ``settings.yml`` ================ This page describe the options possibilities of the :origin:`searx/settings.yml` file. .. sidebar:: Further reading .. - :ref:`use_default_settings.yml` - :ref:`search API` .. contents:: :depth: 2 :local: :backlinks: entry .. _settings location: settings.yml location ===================== The initial ``settings.yml`` we be load from these locations: 1. the full path specified in the ``SEARXNG_SETTINGS_PATH`` environment variable. 2. ``/etc/searxng/settings.yml`` If these files don't exist (or are empty or can't be read), SearXNG uses the :origin:`searx/settings.yml` file. Read :ref:`settings use_default_settings` to see how you can simplify your *user defined* ``settings.yml``. .. _settings use_default_settings: use_default_settings ==================== .. sidebar:: ``use_default_settings: true`` - :ref:`settings location` - :ref:`use_default_settings.yml` - :origin:`/etc/searxng/settings.yml ` The user defined ``settings.yml`` is loaded from the :ref:`settings location` and can relied on the default configuration :origin:`searx/settings.yml` using: ``use_default_settings: true`` ``server:`` In the following example, the actual settings are the default settings defined in :origin:`searx/settings.yml` with the exception of the ``secret_key`` and the ``bind_address``: .. code:: yaml use_default_settings: true server: secret_key: "ultrasecretkey" # change this! bind_address: "[::]" ``engines:`` With ``use_default_settings: true``, each settings can be override in a similar way, the ``engines`` section is merged according to the engine ``name``. In this example, SearXNG will load all the default engines, will enable the ``bing`` engine and define a :ref:`token ` for the arch linux engine: .. code:: yaml use_default_settings: true server: secret_key: "ultrasecretkey" # change this! engines: - name: arch linux wiki tokens: ['$ecretValue'] - name: bing disabled: false ``engines:`` / ``remove:`` It is possible to remove some engines from the default settings. The following example is similar to the above one, but SearXNG doesn't load the the google engine: .. code:: yaml use_default_settings: engines: remove: - google server: secret_key: "ultrasecretkey" # change this! engines: - name: arch linux wiki tokens: ['$ecretValue'] ``engines:`` / ``keep_only:`` As an alternative, it is possible to specify the engines to keep. In the following example, SearXNG has only two engines: .. code:: yaml use_default_settings: engines: keep_only: - google - duckduckgo server: secret_key: "ultrasecretkey" # change this! engines: - name: google tokens: ['$ecretValue'] - name: duckduckgo tokens: ['$ecretValue'] --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/settings/settings_brand.rst # Path: docs/admin/settings/settings_brand.rst .. _settings brand: ========== ``brand:`` ========== .. autoclass:: searx.brand.SettingsBrand :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/settings/settings_categories_as_tabs.rst # Path: docs/admin/settings/settings_categories_as_tabs.rst .. _settings categories_as_tabs: ======================= ``categories_as_tabs:`` ======================= A list of the categories that are displayed as tabs in the user interface. Categories not listed here can still be searched with the :ref:`search-syntax`. .. code:: yaml categories_as_tabs: general: images: videos: news: map: music: it: science: files: social media: Engines are added to ``categories:`` (compare :ref:`engine categories`), the categories listed in ``categories_as_tabs`` are shown as tabs in the UI. If there are no active engines in a category, the tab is not displayed (e.g. if a user disables all engines in a category). On the preferences page (``/preferences``) -- under *engines* -- there is an additional tab, called *other*. In this tab are all engines listed that are not in one of the UI tabs (not included in ``categories_as_tabs``). --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/settings/settings_engines.rst # Path: docs/admin/settings/settings_engines.rst .. _settings engines: ============ ``engines:`` ============ .. sidebar:: Further reading .. - :ref:`configured engines` - :ref:`engines-dev` In the section ``engines:`` is a list of the engines that are to be made available in the instance. Each list entry is in turn a key/value mapping. .. code:: yaml engines: - name: dummy.online engine: dummy .. - name: dummy.offline engine: dummy-offline .. .. In the code example below a *full fledged* example of a YAML setup from a dummy engine is shown. Most of the options have a default value or even are optional. .. hint:: A few more options are possible, but they are pretty specific to some engines (:ref:`engine implementations`). .. code:: yaml - name: example engine: example shortcut: demo base_url: 'https://{language}.example.com/' send_accept_language_header: false categories: general timeout: 3.0 api_key: 'apikey' disabled: false language: en_US tokens: [ 'my-secret-token' ] weight: 1 display_error_messages: true about: website: https://example.com wikidata_id: Q306656 official_api_documentation: https://example.com/api-doc use_official_api: true require_api_key: true results: HTML # overwrite values from section 'outgoing:' enable_http2: false retries: 1 max_connections: 100 max_keepalive_connections: 10 keepalive_expiry: 5.0 using_tor_proxy: false proxies: http: - http://proxy1:8080 - http://proxy2:8080 https: - http://proxy1:8080 - http://proxy2:8080 - socks5://user:password@proxy3:1080 - socks5h://user:password@proxy4:1080 # other network settings enable_http: false retry_on_http_error: true # or 403 or [404, 429] ``name`` : Name that will be used across SearXNG to define this engine. In settings, on the result page... ``engine`` : Name of the python file used to handle requests and responses to and from this search engine. ``shortcut`` : Code used to execute bang requests (in this case using ``!bi``) ``base_url`` : optional Part of the URL that should be stable across every request. Can be useful to use multiple sites using only one engine, or updating the site URL without touching at the code. ``send_accept_language_header`` : Several engines that support languages (or regions) deal with the HTTP header ``Accept-Language`` to build a response that fits to the locale. When this option is activated, the language (locale) that is selected by the user is used to build and send a ``Accept-Language`` header in the request to the origin search engine. .. _engine categories: ``categories`` : optional Specifies to which categories the engine should be added. Engines can be assigned to multiple categories. Categories can be shown as tabs (:ref:`settings categories_as_tabs`) in the UI. A search in a tab (in the UI) will query all engines that are active in this tab. In the preferences page (``/preferences``) -- under *engines* -- users can select what engine should be active when querying in this tab. Alternatively, :ref:`\!bang ` can be used to search all engines in a category, regardless of whether they are active or not, or whether they are in a tab of the UI or not. For example, ``!dictionaries`` can be used to query all search engines in that category (group). ``timeout`` : optional Timeout of the search with the current search engine. Overwrites ``request_timeout`` from :ref:`settings outgoing`. **Be careful, it will modify the global timeout of SearXNG.** ``api_key`` : optional In a few cases, using an API needs the use of a secret key. How to obtain them is described in the file. Engines that require an API key are set to ``inactive: true`` by default. To enable such an engine, provide the API key and set ``inactive: false``. ``disabled`` : optional To disable by default the engine, but not deleting it. It will allow the user to manually activate it in the settings. ``inactive``: optional Remove the engine from the settings (*disabled & removed*). This defaults to ``true`` for engines that require an API key, please see the ``api_key`` section if you want to enable such an engine. ``language`` : optional If you want to use another language for a specific engine, you can define it by using the ISO code of language (and region), like ``fr``, ``en-US``, ``de-DE``. ``tokens`` : optional A list of secret tokens to make this engine *private*, more details see :ref:`private engines`. ``weight`` : default ``1`` Weighting of the results of this engine. ``display_error_messages`` : default ``true`` When an engine returns an error, the message is displayed on the user interface. .. _engine network: ``network`` : optional Use the network configuration from another engine. In addition, there are two default networks: - ``ipv4`` set ``local_addresses`` to ``0.0.0.0`` (use only IPv4 local addresses) - ``ipv6`` set ``local_addresses`` to ``::`` (use only IPv6 local addresses) ``enable_http`` : optional Enable HTTP for this engine (by default only HTTPS is enabled). ``retry_on_http_error`` : optional Retry request on some HTTP status code. Example: * ``true`` : on HTTP status code between 400 and 599. * ``403`` : on HTTP status code 403. * ``[403, 429]``: on HTTP status code 403 and 429. ``proxies`` : Overwrites proxy settings from :ref:`settings outgoing`. ``using_tor_proxy`` : Using tor proxy (``true``) or not (``false``) for this engine. The default is taken from ``using_tor_proxy`` of the :ref:`settings outgoing`. .. _Pool limit configuration: https://www.python-httpx.org/advanced/#pool-limit-configuration ``max_keepalive_connection#s`` : `Pool limit configuration`_, overwrites value ``pool_maxsize`` from :ref:`settings outgoing` for this engine. ``max_connections`` : `Pool limit configuration`_, overwrites value ``pool_connections`` from :ref:`settings outgoing` for this engine. ``keepalive_expiry`` : `Pool limit configuration`_, overwrites value ``keepalive_expiry`` from :ref:`settings outgoing` for this engine. .. _private engines: Private Engines (``tokens``) ============================ Administrators might find themselves wanting to limit access to some of the enabled engines on their instances. It might be because they do not want to expose some private information through :ref:`offline engines`. Or they would rather share engines only with their trusted friends or colleagues. .. sidebar:: info Initial sponsored by `Search and Discovery Fund `_ of `NLnet Foundation `_. To solve this issue the concept of *private engines* exists. A new option was added to engines named `tokens`. It expects a list of strings. If the user making a request presents one of the tokens of an engine, they can access information about the engine and make search requests. Example configuration to restrict access to the Arch Linux Wiki engine: .. code:: yaml - name: arch linux wiki engine: archlinux shortcut: al tokens: [ 'my-secret-token' ] Unless a user has configured the right token, the engine is going to be hidden from them. It is not going to be included in the list of engines on the Preferences page and in the output of `/config` REST API call. Tokens can be added to one's configuration on the Preferences page under "Engine tokens". The input expects a comma separated list of strings. The distribution of the tokens from the administrator to the users is not carved in stone. As providing access to such engines implies that the admin knows and trusts the user, we do not see necessary to come up with a strict process. Instead, we would like to add guidelines to the documentation of the feature. Example: Multilingual Search ============================ SearXNG does not support true multilingual search. You have to use the language prefix in your search query when searching in a different language. But there is a workaround: By adding a new search engine with a different language, SearXNG will search in your default and other language. Example configuration in settings.yml for a German and English speaker: .. code-block:: yaml search: default_lang : "de" ... engines: - name : google english engine : google language : en ... When searching, the default google engine will return German results and "google english" will return English results. --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/settings/settings_general.rst # Path: docs/admin/settings/settings_general.rst .. _settings general: ============ ``general:`` ============ .. code:: yaml general: debug: false instance_name: "SearXNG" privacypolicy_url: false donation_url: false contact_url: false enable_metrics: true open_metrics: '' ``debug`` : ``$SEARXNG_DEBUG`` In debug mode, the server provides an interactive debugger, will reload when code is changed and activates a verbose logging. .. attention:: The debug setting is intended for local development server. Don't activate debug (don't use a development server) when deploying to production. ``donation_url`` : Set value to ``true`` to use your own donation page written in the :ref:`searx/info/en/donate.md ` and use ``false`` to disable the donation link altogether. ``privacypolicy_url``: Link to privacy policy. ``contact_url``: Contact ``mailto:`` address or WEB form. ``enable_metrics``: Enabled by default. Record various anonymous metrics available at ``/stats``, ``/stats/errors`` and ``/preferences``. ``open_metrics``: Disabled by default. Set to a secret password to expose an `OpenMetrics API `_ at ``/metrics``, e.g. for usage with Prometheus. The ``/metrics`` endpoint is using HTTP Basic Auth, where the password is the value of ``open_metrics`` set above. The username used for Basic Auth can be randomly chosen as only the password is being validated. --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/settings/settings_outgoing.rst # Path: docs/admin/settings/settings_outgoing.rst .. _settings outgoing: ============= ``outgoing:`` ============= Communication with search engines. .. code:: yaml outgoing: request_timeout: 2.0 # default timeout in seconds, can be override by engine max_request_timeout: 10.0 # the maximum timeout in seconds useragent_suffix: "" # information like an email address to the administrator pool_connections: 100 # Maximum number of allowable connections, or null # for no limits. The default is 100. pool_maxsize: 10 # Number of allowable keep-alive connections, or null # to always allow. The default is 10. enable_http2: true # See https://www.python-httpx.org/http2/ # uncomment below section if you want to use a custom server certificate # see https://www.python-httpx.org/advanced/#changing-the-verification-defaults # and https://www.python-httpx.org/compatibility/#ssl-configuration # verify: ~/.mitmproxy/mitmproxy-ca-cert.cer # # uncomment below section if you want to use a proxyq see: SOCKS proxies # https://2.python-requests.org/en/latest/user/advanced/#proxies # are also supported: see # https://2.python-requests.org/en/latest/user/advanced/#socks # # proxies: # all://: # - http://proxy1:8080 # - http://proxy2:8080 # # using_tor_proxy: true # # Extra seconds to add in order to account for the time taken by the proxy # # extra_proxy_timeout: 10.0 # ``request_timeout`` : Global timeout of the requests made to others engines in seconds. A bigger timeout will allow to wait for answers from slow engines, but in consequence will slow SearXNG reactivity (the result page may take the time specified in the timeout to load). Can be override by ``timeout`` in the :ref:`settings engines`. ``useragent_suffix`` : Suffix to the user-agent SearXNG uses to send requests to others engines. If an engine wish to block you, a contact info here may be useful to avoid that. .. _Pool limit configuration: https://www.python-httpx.org/advanced/#pool-limit-configuration ``pool_maxsize``: Number of allowable keep-alive connections, or ``null`` to always allow. The default is 10. See ``max_keepalive_connections`` `Pool limit configuration`_. ``pool_connections`` : Maximum number of allowable connections, or ``null`` # for no limits. The default is 100. See ``max_connections`` `Pool limit configuration`_. ``keepalive_expiry`` : Number of seconds to keep a connection in the pool. By default 5.0 seconds. See ``keepalive_expiry`` `Pool limit configuration`_. .. _httpx proxies: https://www.python-httpx.org/advanced/#http-proxying ``proxies`` : Define one or more proxies you wish to use, see `httpx proxies`_. If there are more than one proxy for one protocol (http, https), requests to the engines are distributed in a round-robin fashion. ``source_ips`` : If you use multiple network interfaces, define from which IP the requests must be made. Example: * ``0.0.0.0`` any local IPv4 address. * ``::`` any local IPv6 address. * ``192.168.0.1`` * ``[ 192.168.0.1, 192.168.0.2 ]`` these two specific IP addresses * ``fe80::60a2:1691:e5a2:ee1f`` * ``fe80::60a2:1691:e5a2:ee1f/126`` all IP addresses in this network. * ``[ 192.168.0.1, fe80::/126 ]`` ``retries`` : Number of retry in case of an HTTP error. On each retry, SearXNG uses an different proxy and source ip. ``enable_http2`` : Enable by default. Set to ``false`` to disable HTTP/2. .. _httpx verification defaults: https://www.python-httpx.org/advanced/#changing-the-verification-defaults .. _httpx ssl configuration: https://www.python-httpx.org/compatibility/#ssl-configuration ``verify``: : ``$SSL_CERT_FILE``, ``$SSL_CERT_DIR`` Allow to specify a path to certificate. see `httpx verification defaults`_. In addition to ``verify``, SearXNG supports the ``$SSL_CERT_FILE`` (for a file) and ``$SSL_CERT_DIR`` (for a directory) OpenSSL variables. see `httpx ssl configuration`_. ``max_redirects`` : 30 by default. Maximum redirect before it is an error. ``using_tor_proxy`` : Using tor proxy (``true``) or not (``false``) for all engines. The default is ``false`` and can be overwritten in the :ref:`settings engines` --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/settings/settings_plugins.rst # Path: docs/admin/settings/settings_plugins.rst .. _settings plugins: ============ ``plugins:`` ============ .. attention:: The ``enabled_plugins:`` section in SearXNG's settings no longer exists. There is no longer a distinction between built-in and external plugin, all plugins are registered via the settings in the ``plugins:`` section. .. sidebar:: Further reading .. - :ref:`plugins admin` - :ref:`dev plugin` In SearXNG, plugins can be registered in the :py:obj:`PluginStore ` via a fully qualified class name. A configuration (:py:obj:`PluginCfg `) can be transferred to the plugin, e.g. to activate it by default / *opt-in* or *opt-out* from user's point of view. Please note that some plugins, such as the :ref:`hostnames plugin` plugin, require further configuration before they can be made available for selection. By default the :ref:`settings built in plugins` are loaded. To change the list of plugins to be loaded, the value for ``plugins:`` in ``/etc/searxng/settings.yml`` must be overwritten. Following is an example that uses :ref:`settings use_default_settings` and only two plugins are registered: the calculator can be activated by the user and the unit converter is active by default. .. code:: yaml use_default_settings: true plugins: searx.plugins.calculator.SXNGPlugin: active: false searx.plugins.unit_converter.SXNGPlugin: active: true To prevent any plugins from loading, the following setting can be used: .. code:: yaml use_default_settings: true plugins: {} .. _settings built in plugins: built-in plugins ================ The built-in plugins are all located in the namespace `searx.plugins`. .. code:: yaml plugins: searx.plugins.calculator.SXNGPlugin: active: true searx.plugins.infinite_scroll.SXNGPlugin: active: false searx.plugins.hash_plugin.SXNGPlugin: active: true searx.plugins.self_info.SXNGPlugin: active: true searx.plugins.tracker_url_remover.SXNGPlugin: active: true searx.plugins.unit_converter.SXNGPlugin: active: true searx.plugins.ahmia_filter.SXNGPlugin: active: true searx.plugins.hostnames.SXNGPlugin: active: true searx.plugins.oa_doi_rewrite.SXNGPlugin: active: false searx.plugins.tor_check.SXNGPlugin: active: false .. _settings external_plugins: external plugins ================ .. _Only show green hosted results: https://github.com/return42/tgwf-searx-plugins/ SearXNG supports *external plugins* / there is no need to install one, SearXNG runs out of the box. - `Only show green hosted results`_ - .. --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/settings/settings_redis.rst # Path: docs/admin/settings/settings_redis.rst .. _settings redis: ========== ``redis:`` ========== .. _Valkey: https://valkey.io .. attention:: SearXNG is switching from the Redis DB to Valkey_. The configuration description of Valkey_ in SearXNG can be found here: :ref:`settings `. If you have built and installed a local Redis DB for SearXNG, it is recommended to uninstall it now and replace it with the installation of a Valkey_ DB. .. _Redis Developer Notes: Redis Developer Notes ===================== To uninstall SearXNG's local Redis DB you can use: .. code:: sh # stop your SearXNG instance $ ./utils/searxng.sh remove.redis Remove the Redis DB in your YAML setting: .. code:: yaml redis: url: unix:///usr/local/searxng-redis/run/redis.sock?db=0 To install Valkey_ read: :ref:`Valkey Developer Notes` --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/settings/settings_search.rst # Path: docs/admin/settings/settings_search.rst .. _settings search: =========== ``search:`` =========== .. code:: yaml search: safe_search: 0 autocomplete: "" favicon_resolver: "" default_lang: "" ban_time_on_fail: 5 max_page: 0 max_ban_time_on_fail: 120 suspended_times: SearxEngineAccessDenied: 86400 SearxEngineCaptcha: 86400 SearxEngineTooManyRequests: 3600 cf_SearxEngineCaptcha: 1296000 cf_SearxEngineAccessDenied: 86400 recaptcha_SearxEngineCaptcha: 604800 formats: - html ``safe_search``: Filter results. - ``0``: None - ``1``: Moderate - ``2``: Strict ``autocomplete``: Existing autocomplete backends, leave blank to turn it off. - ``360search`` - ``baidu`` - ``brave`` - ``dbpedia`` - ``duckduckgo`` - ``google`` - ``mwmbl`` - ``naver`` - ``quark`` - ``qwant`` - ``seznam`` - ``sogou`` - ``startpage`` - ``stract`` - ``swisscows`` - ``wikipedia`` - ``yandex`` ``favicon_resolver``: To activate favicons in SearXNG's result list select a default favicon-resolver, leave blank to turn off the feature. Don't activate the favicons before reading the :ref:`Favicons documentation `. ``default_lang``: Default search language - leave blank to detect from browser information or use codes from :origin:`searx/sxng_locales.py`. ``languages``: List of available languages - leave unset to use all codes from :origin:`searx/sxng_locales.py`. Otherwise list codes of available languages. The ``all`` value is shown as the ``Default language`` in the user interface (in most cases, it is meant to send the query without a language parameter ; in some cases, it means the English language) Example: .. code:: yaml languages: - all - en - en-US - de - it-IT - fr - fr-BE ``max_page``: If engine supports paging, 0 means unlimited numbers of pages. The value is only applied if the engine itself does not have a max value that is lower than this one. ``ban_time_on_fail``: Ban time in seconds after engine errors. ``max_ban_time_on_fail``: Max ban time in seconds after engine errors. ``suspended_times``: Engine suspension time after error (in seconds; set to 0 to disable) ``SearxEngineAccessDenied``: 86400 For error "Access denied" and "HTTP error [402, 403]" ``SearxEngineCaptcha``: 86400 For error "CAPTCHA" ``SearxEngineTooManyRequests``: 3600 For error "Too many request" and "HTTP error 429" Cloudflare CAPTCHA: - ``cf_SearxEngineCaptcha``: 1296000 - ``cf_SearxEngineAccessDenied``: 86400 Google CAPTCHA: - ``recaptcha_SearxEngineCaptcha``: 604800 ``formats``: Result formats available from web, remove format to deny access (use lower case). - ``html`` - ``csv`` - ``json`` - ``rss`` --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/settings/settings_server.rst # Path: docs/admin/settings/settings_server.rst .. _settings server: =========== ``server:`` =========== .. code:: yaml server: base_url: http://example.org/location # change this! port: 8888 bind_address: "127.0.0.1" secret_key: "ultrasecretkey" # change this! limiter: false public_instance: false image_proxy: false method: "POST" default_http_headers: X-Content-Type-Options : nosniff X-Download-Options : noopen X-Robots-Tag : noindex, nofollow Referrer-Policy : no-referrer ``base_url`` : ``$SEARXNG_BASE_URL`` The base URL where SearXNG is deployed. Used to create correct inbound links. ``port`` & ``bind_address``: ``$SEARXNG_PORT`` & ``$SEARXNG_BIND_ADDRESS`` Port number and *bind address* of the SearXNG web application if you run it directly using ``python searx/webapp.py``. Doesn't apply to a SearXNG services running behind a proxy and using socket communications. .. _server.secret_key: ``secret_key`` : ``$SEARXNG_SECRET`` Used for cryptography purpose. ``limiter`` : ``$SEARXNG_LIMITER`` Rate limit the number of request on the instance, block some bots. The :ref:`limiter` requires a :ref:`settings valkey` database. .. _public_instance: ``public_instance`` : ``$SEARXNG_PUBLIC_INSTANCE`` Setting that allows to enable features specifically for public instances (not needed for local usage). By set to ``true`` the following features are activated: - :py:obj:`searx.botdetection.link_token` in the :ref:`limiter` .. _image_proxy: ``image_proxy`` : ``$SEARXNG_IMAGE_PROXY`` Allow your instance of SearXNG of being able to proxy images. Uses memory space. .. _method: ``method`` : ``GET`` | ``POST`` HTTP method. By defaults ``POST`` is used / The ``POST`` method has the advantage with some WEB browsers that the history is not easy to read, but there are also various disadvantages that sometimes **severely restrict the ease of use for the end user** (e.g. back button to jump back to the previous search page and drag & drop of search term to new tabs do not work as expected .. and several more). We had some discussions about the *pros versus cons*: - `[doc] adds the missing documentation of the server.method settings `__ - look out for `label:"http methods GET & POST" `__ .. _HTTP headers: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers ``default_http_headers`` : Set additional `HTTP headers`_, see `#755 `__ --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/settings/settings_ui.rst # Path: docs/admin/settings/settings_ui.rst .. _settings ui: ======= ``ui:`` ======= .. _cache busting: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#caching_static_assets_with_cache_busting .. code:: yaml ui: default_locale: "" query_in_title: false center_alignment: false cache_url: https://web.archive.org/web/ default_theme: simple theme_args: simple_style: auto search_on_category_select: true hotkeys: default url_formatting: pretty ``default_locale`` : SearXNG interface language. If blank, the locale is detected by using the browser language. If it doesn't work, or you are deploying a language specific instance of SearXNG, a locale can be defined using an ISO language code, like ``fr``, ``en``, ``de``. ``query_in_title`` : When true, the result page's titles contains the query it decreases the privacy, since the browser can records the page titles. ``center_alignment`` : default ``false`` When enabled, the results are centered instead of being in the left (or RTL) side of the screen. This setting only affects the *desktop layout* (:origin:`min-width: @tablet `) .. cache_url: ``cache_url`` : ``https://web.archive.org/web/`` URL prefix of the internet archive or cache, don't forget trailing slash (if needed). The default is https://web.archive.org/web/ alternatives are: - https://webcache.googleusercontent.com/search?q=cache: - https://archive.today/ ``default_theme`` : Name of the theme you want to use by default on your SearXNG instance. ``theme_args.simple_style``: Style of simple theme: ``auto``, ``light``, ``dark``, ``black`` ``results_on_new_tab``: Open result links in a new tab by default. ``search_on_category_select``: Perform search immediately if a category selected. Disable to select multiple categories. ``hotkeys``: Hotkeys to use in the search interface: ``default``, ``vim`` (Vim-like). ``url_formatting``: Formatting type to use for result URLs: ``pretty``, ``full`` or ``host``. --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/settings/settings_valkey.rst # Path: docs/admin/settings/settings_valkey.rst .. _settings valkey: =========== ``valkey:`` =========== .. _Valkey: https://valkey.io .. _Valkey-Installation: https://valkey.io/topics/installation/ .. _There are several ways to specify a database number: https://valkey-py.readthedocs.io/en/stable/connections.html#valkey.Valkey.from_url A Valkey_ DB can be connected by an URL, in section :ref:`valkey db` you will find a description to test your valkey connection in SearXNG. ``url`` : ``$SEARXNG_VALKEY_URL`` URL to connect valkey database. `There are several ways to specify a database number`_:: valkey://[[username]:[password]]@localhost:6379/0 valkeys://[[username]:[password]]@localhost:6379/0 unix://[[username]:[password]]@/path/to/socket.sock?db=0 When using sockets, don't forget to check the access rights on the socket:: ls -la /usr/local/searxng-valkey/run/valkey.sock srwxrwx--- 1 searxng-valkey searxng-valkey ... /usr/local/searxng-valkey/run/valkey.sock In this example read/write access is given to the *searxng-valkey* group. To get access rights to valkey instance (the socket), your SearXNG (or even your developer) account needs to be added to the *searxng-valkey* group. .. _Valkey Developer Notes: Valkey Developer Notes ====================== To set up a local Valkey_ DB, set the URL connector in your YAML setting: .. code:: yaml valkey: url: valkey://localhost:6379/0 To install a local Valkey_ DB from package manager read `Valkey-Installation`_ or use: .. code:: sh $ ./utils/searxng.sh install valkey # restart your SearXNG instance --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/admin/update-searxng.rst # Path: docs/admin/update-searxng.rst .. _searxng maintenance: =================== SearXNG maintenance =================== .. sidebar:: further read - :ref:`toolboxing` - :ref:`uWSGI maintenance` .. contents:: :depth: 2 :local: :backlinks: entry .. _update searxng: How to update ============= How to update depends on the :ref:`installation` method. If you have used the :ref:`installation scripts`, use the ``update`` command from the :ref:`searxng.sh` script. .. code:: sh sudo -H ./utils/searxng.sh instance update .. _inspect searxng: How to inspect & debug ====================== How to debug depends on the :ref:`installation` method. If you have used the :ref:`installation scripts`, use the ``inspect`` command from the :ref:`searxng.sh` script. .. code:: sh sudo -H ./utils/searxng.sh instance inspect .. _migrate and stay tuned: Migrate and stay tuned! ======================= .. sidebar:: info - :pull:`1332` - :pull:`456` - :pull:`A comment about rolling release <446#issuecomment-954730358>` SearXNG is a *rolling release*; each commit to the master branch is a release. SearXNG is growing rapidly, the services and opportunities are change every now and then, to name just a few: - Bot protection has been switched from filtron to SearXNG's :ref:`limiter `, this requires a :ref:`Valkey ` database. To stay tuned and get in use of the new features, instance maintainers have to update the SearXNG code regularly (see :ref:`update searxng`). As the above examples show, this is not always enough, sometimes services have to be set up or reconfigured and sometimes services that are no longer needed should be uninstalled. Here you will find a list of changes that affect the infrastructure. Please check to what extent it is necessary to update your installations: :pull:`1595`: ``[fix] uWSGI: increase buffer-size`` Re-install uWSGI (:ref:`searxng.sh`) or fix your uWSGI ``searxng.ini`` file manually. Check after Installation ------------------------ Once you have done your installation, you can run a SearXNG *check* procedure, to see if there are some left overs. In this example there exists a *old* ``/etc/searx/settings.yml``:: $ sudo -H ./utils/searxng.sh instance check SearXNG checks -------------- ERROR: settings.yml in /etc/searx/ is deprecated, move file to folder /etc/searxng/ ... INFO searx.valkeydb : connecting to Valkey db=0 path='/usr/local/searxng-valkey/run/valkey.sock' INFO searx.valkeydb : connected to Valkey --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/build-templates/searxng.rst # Path: docs/build-templates/searxng.rst .. template evaluated by: ./utils/searxng.sh searxng.doc.rst .. hint: all dollar-names are variables, dollar sign itself is quoted by: \\$ .. START distro-packages .. tabs:: .. group-tab:: Ubuntu / debian .. code-block:: sh $ sudo -H apt-get install -y \\ ${debian} .. group-tab:: Arch Linux .. code-block:: sh $ sudo -H pacman -S --noconfirm \\ ${arch} .. group-tab:: Fedora / RHEL .. code-block:: sh $ sudo -H dnf install -y \\ ${fedora} .. END distro-packages .. START build-packages .. tabs:: .. group-tab:: Ubuntu / debian .. code-block:: sh $ sudo -H apt-get install -y \\ ${debian_build} .. group-tab:: Arch Linux .. code-block:: sh $ sudo -H pacman -S --noconfirm \\ ${arch_build} .. group-tab:: Fedora / RHEL .. code-block:: sh $ sudo -H dnf install -y \\ ${fedora_build} .. END build-packages .. START create user .. tabs:: .. group-tab:: bash .. code-block:: sh $ sudo -H useradd --shell /bin/bash --system \\ --home-dir \"$SERVICE_HOME\" \\ --comment 'Privacy-respecting metasearch engine' \\ $SERVICE_USER $ sudo -H mkdir \"$SERVICE_HOME\" $ sudo -H chown -R \"$SERVICE_GROUP:$SERVICE_GROUP\" \"$SERVICE_HOME\" .. END create user .. START clone searxng .. tabs:: .. group-tab:: bash .. code-block:: sh $ sudo -H -u ${SERVICE_USER} -i (${SERVICE_USER})$ git clone \"$GIT_URL\" \\ \"$SEARXNG_SRC\" .. END clone searxng .. START create virtualenv .. tabs:: .. group-tab:: bash .. code-block:: sh (${SERVICE_USER})$ python3 -m venv \"${SEARXNG_PYENV}\" (${SERVICE_USER})$ echo \". ${SEARXNG_PYENV}/bin/activate\" \\ >> \"$SERVICE_HOME/.profile\" .. END create virtualenv .. START manage.sh update_packages .. tabs:: .. group-tab:: bash .. code-block:: sh $ sudo -H -u ${SERVICE_USER} -i (${SERVICE_USER})$ command -v python && python --version $SEARXNG_PYENV/bin/python Python 3.11.10 # update pip's boilerplate .. pip install -U pip pip install -U setuptools pip install -U wheel pip install -U pyyaml pip install -U msgspec # jump to SearXNG's working tree and install SearXNG into virtualenv (${SERVICE_USER})$ cd \"$SEARXNG_SRC\" (${SERVICE_USER})$ pip install --use-pep517 --no-build-isolation -e . .. END manage.sh update_packages .. START searxng config .. tabs:: .. group-tab:: Use default settings .. code-block:: sh $ sudo -H mkdir -p \"$(dirname ${SEARXNG_SETTINGS_PATH})\" $ sudo -H cp \"$SEARXNG_SRC/utils/templates/etc/searxng/settings.yml\" \\ \"${SEARXNG_SETTINGS_PATH}\" .. group-tab:: minimal setup .. code-block:: sh $ sudo -H sed -i -e \"s/ultrasecretkey/\$(openssl rand -hex 16)/g\" \\ \"$SEARXNG_SETTINGS_PATH\" .. END searxng config .. START check searxng installation .. tabs:: .. group-tab:: bash .. code-block:: sh # enable debug .. $ sudo -H sed -i -e \"s/debug : False/debug : True/g\" \"$SEARXNG_SETTINGS_PATH\" # start webapp $ sudo -H -u ${SERVICE_USER} -i (${SERVICE_USER})$ cd ${SEARXNG_SRC} (${SERVICE_USER})$ export SEARXNG_SETTINGS_PATH=\"${SEARXNG_SETTINGS_PATH}\" (${SERVICE_USER})$ python searx/webapp.py # disable debug $ sudo -H sed -i -e \"s/debug : True/debug : False/g\" \"$SEARXNG_SETTINGS_PATH\" Open WEB browser and visit http://$SEARXNG_INTERNAL_HTTP . If you are inside a container or in a script, test with curl: .. tabs:: .. group-tab:: WEB browser .. code-block:: sh $ xdg-open http://$SEARXNG_INTERNAL_HTTP .. group-tab:: curl .. code-block:: none $ curl --location --verbose --head --insecure $SEARXNG_INTERNAL_HTTP * Trying 127.0.0.1:8888... * TCP_NODELAY set * Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0) > HEAD / HTTP/1.1 > Host: 127.0.0.1:8888 > User-Agent: curl/7.68.0 > Accept: */* > * Mark bundle as not supporting multiuse * HTTP 1.0, assume close after body < HTTP/1.0 200 OK HTTP/1.0 200 OK ... .. END check searxng installation --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/answerers/builtins.rst # Path: docs/dev/answerers/builtins.rst .. _builtin answerers: ================== Built-in Answerers ================== .. toctree:: :maxdepth: 1 random statistics --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/answerers/development.rst # Path: docs/dev/answerers/development.rst .. _dev answerers: ==================== Answerer Development ==================== .. automodule:: searx.answerers --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/answerers/index.rst # Path: docs/dev/answerers/index.rst ========= Answerers ========= .. toctree:: :maxdepth: 2 development builtins --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/answerers/random.rst # Path: docs/dev/answerers/random.rst .. _answerer.random: ====== Random ====== .. autoclass:: searx.answerers.random.SXNGAnswerer :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/answerers/statistics.rst # Path: docs/dev/answerers/statistics.rst .. _answerer.statistics: ========== Statistics ========== .. autoclass:: searx.answerers.statistics.SXNGAnswerer :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/commits.rst # Path: docs/dev/commits.rst .. _create commit: =============================== Git Commits & Change Management =============================== .. sidebar:: Create good commits! - `Conventional Commits`_ - `Structural split of changes`_ - `Git Commit Good Practice`_ A commit and its commit message are among the most important information available to a developer for bug fixing and further development. A commit is a change and changes have a context (a change request). In a SCM system (git), the change history is derived from the commit history. A commit message is therefore part of the documentation for change management and thus elementary for the traceability of changes. **What a commit is not**: *A commit to an SCM system is not used to save files!* A commit should always have a context and the commit message describes what is to be changed in that context, just as a function description should describe what the intention and the goal of the function is, a commit message should describe what the intention and the goal of that commit is. The commit messages form the history and are the first and therefore most important information a developer has when he has to research when and why a change had to be made and how it was made (what the goal was). Like any text, a commit message should be written for the reader and not from the perspective of the author. When scrolling through the history, the first thing one see is the title of the commit message. Therefore the title should describe the change as briefly and precisely as possible ... followed by a blank line and then a somewhat detailed description of the change. ---- The follwing rules should be in mind, when creating a commit: - **Commit history should be read like a history book.** - **Commit messages are for the reader not for the author of the commit.** - **A commit is the atomic code-modification of a change in change management.** - **Think about which descriptions from your PR might belong in the commit message.** - **The maximum line length in a commit message is 80 characters.** ---- Choose meaningful commit messages: .. code:: [type] optional scope: description [body] [optional trailers] ``[type]``: Commits MUST be prefixed with a type .. ``feat``, ``fix``, ``refactor``, ``mod``, ``upd``, ``doc``, ``l10n``, ``build`` .. ``[body]`` `Information in commit messages`_ ``[optional trailers]``: - `Signed-off-by`_: certify that the committer has the rights to submit the work under the project’s license. That the developer has this right is a prerequisite for a merge. If the `Signed-off-by`_ is not set in the commit, the contributor enters his `Developer's Certificate of Origin` at the latest when creating a PR! - Closes: Link to the bug report or the bug number (e.g. ``Closes: #10``) - `Co-authored-by`_: email address of the co-author - Reported-by: email address (if there is no bug report) - Suggested-by: email address (if there is no bug report) ---- To give examples at hand, here are a few commits. Follow the links to see the full commit messages: :patch:`44d941c93` ``[fix] mojeek web engine: don't add empty fmt argument for web searches`` :patch:`feb15e387` ``[fix] brave.news engine: response is HTML and no longer JSON`` :patch:`bdfe1c2a1` ``[mod] engines: migration of the individual cache solutions to EngineCache`` .. _Conventional Commits: https://www.conventionalcommits.org/ .. _Structural split of changes: https://wiki.openstack.org/wiki/GitCommitMessages#Structural_split_of_changes .. _Git Commit Good Practice: https://wiki.openstack.org/wiki/GitCommitMessages .. _Information in commit messages: https://wiki.openstack.org/wiki/GitCommitMessages#Information_in_commit_messages .. _`Developer's Certificate of Origin`: https://developercertificate.org/ .. _Signed-off-by: https://git-scm.com/docs/git-commit#Documentation/git-commit.txt-code--signoffcode .. _Co-authored-by: https://docs.github.com/en/pull-requests/committing-changes-to-your-project/creating-and-editing-commits/creating-a-commit-with-multiple-authors --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/contribution_guide.rst # Path: docs/dev/contribution_guide.rst .. _how to contribute: ================= How to contribute ================= .. contents:: :depth: 2 :local: :backlinks: entry Prime directives: Privacy, Hackability ====================================== SearXNG has two prime directives, **privacy-by-design and hackability** . The hackability comes in three levels: - support of search engines - plugins to alter search behaviour - hacking SearXNG itself Note the lack of "world domination" among the directives. SearXNG has no intention of wide mass-adoption, rounded corners, etc. The prime directive "privacy" deserves a separate chapter, as it's quite uncommon unfortunately. Privacy-by-design ----------------- SearXNG was born out of the need for a **privacy-respecting** search tool which can be extended easily to maximize both its search and its privacy protecting capabilities. Some widely used search engine features may work differently, may be turned off by default, or may not be implemented at all in SearXNG **as a consequence of a privacy-by-design approach**. Following this approach, features reducing the privacy preserving aspects of SearXNG should be switched off by default or should not be implemented at all. There are plenty of search engines already providing such features. If a feature reduces SearXNG's efficacy in protecting a user's privacy, the user must be informed about the effect of choosing to enable it. Features that protect privacy but differ from the expectations of the user should also be carefully explained to them. Also, if you think that something works weird with SearXNG, it might be because the tool you are using is designed in a way that interferes with SearXNG's privacy aspects. Submitting a bug report to the vendor of the tool that misbehaves might be a good feedback for them to reconsider the disrespect to their customers (e.g., ``GET`` vs ``POST`` requests in various browsers). Remember the other prime directive of SearXNG is to be hackable, so if the above privacy concerns do not fancy you, simply fork it. *Happy hacking.* Code ==== .. _PEP8: https://www.python.org/dev/peps/pep-0008/ .. _Structural split of changes: https://wiki.openstack.org/wiki/GitCommitMessages#Structural_split_of_changes .. sidebar:: Create good commits! - :ref:`create commit` In order to submit a patch, please follow the steps below: - Follow coding conventions. - PEP8_ standards apply, except the convention of line length - Maximum line length is 120 characters - The cardinal rule for creating good commits is to ensure there is only one *logical change* per commit / read `Structural split of changes`_ - Check if your code breaks existing tests. If so, update the tests or fix your code. - If your code can be unit-tested, add unit tests. - Add yourself to the :origin:`AUTHORS.rst` file. - Choose meaningful commit messages, see :ref:`create commit` - Create a pull request. For more help on getting started with SearXNG development, see :ref:`devquickstart`. Translation =========== Translation currently takes place on :ref:`weblate `. .. _contrib docs: Documentation ============= .. _Sphinx: https://www.sphinx-doc.org .. _reST: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html .. sidebar:: The reST sources has been moved from ``gh-branch`` into ``master`` (:origin:`docs`). The documentation is built using Sphinx_. So in order to be able to generate the required files, you have to install it on your system. Much easier, use our :ref:`makefile`. Here is an example which makes a complete rebuild: .. code:: sh $ make docs.clean docs.html ... The HTML pages are in dist/docs. .. _make docs.live: Live build ---------- .. _sphinx-autobuild: https://github.com/executablebooks/sphinx-autobuild/blob/master/README.md .. sidebar:: docs.clean It is recommended to assert a complete rebuild before deploying (use ``docs.clean``). Live build is like WYSIWYG. It's the recommended way to go if you want to edit the documentation. The Makefile target ``docs.live`` builds the docs, opens URL in your favorite browser and rebuilds every time a reST file has been changed (:ref:`make docs.clean`). .. code:: sh $ make docs.live ... The HTML pages are in dist/docs. ... Serving on http://0.0.0.0:8000 ... Start watching changes Live builds are implemented by sphinx-autobuild_. Use environment ``$(SPHINXOPTS)`` to pass arguments to the sphinx-autobuild_ command. You can pass any argument except for the ``--host`` option (which is always set to ``0.0.0.0``). E.g., to find and use a free port, use: .. code:: sh $ SPHINXOPTS="--port 0" make docs.live ... ... Serving on http://0.0.0.0:50593 ... .. _deploy on github.io: deploy on github.io ------------------- To deploy documentation at :docs:`github.io <.>` use Makefile target :ref:`make docs.gh-pages`, which builds the documentation and runs all the needed git add, commit and push: .. code:: sh $ make docs.clean docs.gh-pages .. attention:: If you are working in your own brand, don't forget to adjust your :ref:`settings brand`. --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/demo/demo_offline.rst # Path: docs/dev/engines/demo/demo_offline.rst .. _demo offline engine: =================== Demo Offline Engine =================== .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.demo_offline :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/demo/demo_online.rst # Path: docs/dev/engines/demo/demo_online.rst .. _demo online engine: ================== Demo Online Engine ================== .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.demo_online :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/engine_overview.rst # Path: docs/dev/engines/engine_overview.rst .. _engines-dev: =============== Engine Overview =============== .. contents:: :depth: 3 :local: :backlinks: entry .. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine .. sidebar:: Further reading .. - :ref:`configured engines` - :ref:`settings engines` SearXNG is a metasearch-engine_, so it uses different search engines to provide better results. Because there is no general search API which could be used for every search engine, an adapter has to be built between SearXNG and the external search engines. Adapters are stored under the folder :origin:`searx/engines`. .. _general engine configuration: General Engine Configuration ============================ It is required to tell SearXNG the type of results the engine provides. The arguments can be set in the engine file or in the settings file (normally ``settings.yml``). The arguments in the settings file override the ones in the engine file. It does not matter if an option is stored in the engine file or in the settings. However, the standard way is the following: .. _engine file: Engine File ----------- .. table:: Common options in the engine module :width: 100% ======================= =========== ======================================================== argument type information ======================= =========== ======================================================== categories list categories, in which the engine is working paging boolean support multiple pages time_range_support boolean support search time range engine_type str - ``online`` :ref:`[ref] ` by default, other possibles values are: - ``offline`` :ref:`[ref] ` - ``online_dictionary`` :ref:`[ref] ` - ``online_currency`` :ref:`[ref] ` - ``online_url_search`` :ref:`[ref] ` ======================= =========== ======================================================== .. _engine settings: Engine ``settings.yml`` ----------------------- For a more detailed description, see :ref:`settings engines` in the :ref:`settings.yml`. .. table:: Common options in the engine setup (``settings.yml``) :width: 100% ======================= =========== ================================================== argument type information ======================= =========== ================================================== name string name of search-engine engine string name of searxng-engine (file name without ``.py``) enable_http bool enable HTTP (by default only HTTPS is enabled). shortcut string shortcut of search-engine timeout string specific timeout for search-engine display_error_messages boolean display error messages on the web UI proxies dict set proxies for a specific engine (e.g. ``proxies : {http: socks5://proxy:port, https: socks5://proxy:port}``) ======================= =========== ================================================== .. _engine overrides: Overrides --------- A few of the options have default values in the namespace of the engine's python module, but are often overwritten by the settings. If ``None`` is assigned to an option in the engine file, it has to be redefined in the settings, otherwise SearXNG will not start with that engine (global names with a leading underline can be ``None``). Here is an very simple example of the global names in the namespace of engine's module: .. code:: python # engine dependent config categories = ['general'] paging = True _non_overwritten_global = 'foo' .. table:: The naming of overrides is arbitrary / recommended overrides are: :width: 100% ======================= =========== =========================================== argument type information ======================= =========== =========================================== base_url string base-url, can be overwritten to use same engine on other URL number_of_results int maximum number of results per request language string ISO code of language and country like en_US api_key string api-key if required by engine ======================= =========== =========================================== .. _engine request: Making a Request ================ To perform a search an URL have to be specified. In addition to specifying an URL, arguments can be passed to the query. .. _engine request arguments: Passed Arguments (request) -------------------------- These arguments can be used to construct the search query. Furthermore, parameters with default value can be redefined for special purposes. .. _engine request online: .. table:: If the ``engine_type`` is :py:obj:`online ` :width: 100% ====================== ============== ======================================================================== argument type default-value, information ====================== ============== ======================================================================== url str ``''`` method str ``'GET'`` headers dict ``{}`` data dict ``{}`` cookies dict ``{}`` verify bool ``True`` headers.User-Agent str a random User-Agent category str current category, like ``'general'`` safesearch int ``0``, between ``0`` and ``2`` (normal, moderate, strict) time_range Optional[str] ``None``, can be ``day``, ``week``, ``month``, ``year`` pageno int current pagenumber searxng_locale str SearXNG's locale selected by user. Specific language code like ``'en'``, ``'en-US'``, or ``'all'`` if unspecified. ====================== ============== ======================================================================== .. _engine request online_dictionary: .. table:: If the ``engine_type`` is :py:obj:`online_dictionary `, in addition to the :ref:`online ` arguments: :width: 100% ====================== ============== ======================================================================== argument type default-value, information ====================== ============== ======================================================================== from_lang str specific language code like ``'en_US'`` to_lang str specific language code like ``'en_US'`` query str the text query without the languages ====================== ============== ======================================================================== .. _engine request online_currency: .. table:: If the ``engine_type`` is :py:obj:`online_currency `, in addition to the :ref:`online ` arguments: :width: 100% ====================== ============== ======================================================================== argument type default-value, information ====================== ============== ======================================================================== amount float the amount to convert from str ISO 4217 code to str ISO 4217 code from_name str currency name to_name str currency name ====================== ============== ======================================================================== .. _engine request online_url_search: .. table:: If the ``engine_type`` is :py:obj:`online_url_search `, in addition to the :ref:`online ` arguments: :width: 100% ====================== ============== ======================================================================== argument type default-value, information ====================== ============== ======================================================================== search_url dict URLs from the search query: .. code:: python { 'http': str, 'ftp': str, 'data:image': str } ====================== ============== ======================================================================== Specify Request --------------- The function :py:func:`def request(query, params): ` always returns the ``params`` variable, the following parameters can be used to specify a search request: .. table:: :width: 100% =================== =========== ========================================================================== argument type information =================== =========== ========================================================================== url str requested url method str HTTP request method headers dict HTTP header information data dict HTTP data information cookies dict HTTP cookies verify bool Performing SSL-Validity check allow_redirects bool Follow redirects max_redirects int maximum redirects, hard limit soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300 =================== =========== ========================================================================== Making a Response ================= In the ``response`` function of the engine, the HTTP response (``resp``) is parsed and a list of results is returned. A engine can append result-items of different media-types and different result-types to the result list. The list of the result items is render to HTML by templates. For more details read section: - :ref:`simple theme templates` - :ref:`result types` --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/enginelib.rst # Path: docs/dev/engines/enginelib.rst .. _searx.enginelib: ============== Engine Library ============== .. automodule:: searx.enginelib :members: .. _searx.enginelib.traits: Engine traits ============= .. automodule:: searx.enginelib.traits :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/engines.rst # Path: docs/dev/engines/engines.rst .. _searx.engines loader: ======================== SearXNG's engines loader ======================== .. automodule:: searx.engines :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/index.rst # Path: docs/dev/engines/index.rst .. _engine implementations: ====================== Engine Implementations ====================== .. contents:: :depth: 2 :local: :backlinks: entry .. toctree:: :caption: Framework Components :maxdepth: 2 enginelib engines engine_overview ResultList and engines ====================== .. autoclass:: searx.result_types.ResultList .. autoclass:: searx.result_types.EngineResults Engine Types ============ The :py:obj:`engine_type ` of an engine determines which :ref:`search processor ` is used by the engine. In this section a list of the engines that are documented is given, a complete list of the engines can be found in the source under: :origin:`searx/engines`. .. _online engines: Online Engines -------------- .. sidebar:: info - :py:obj:`processors.online ` .. toctree:: :maxdepth: 1 :glob: demo/demo_online xpath mediawiki json_engine .. toctree:: :maxdepth: 1 :glob: online/* .. _offline engines: Offline Engines --------------- .. sidebar:: info - :py:obj:`processors.offline ` .. toctree:: :maxdepth: 1 :glob: offline_concept demo/demo_offline offline/* .. _online url search: Online URL Search ----------------- .. sidebar:: info - :py:obj:`processors.online_url_search ` .. toctree:: :maxdepth: 1 :glob: online_url_search/* .. _online currency: Online Currency --------------- .. sidebar:: info - :py:obj:`processors.online_currency ` *no engine of this type is documented yet / coming soon* .. _online dictionary: Online Dictionary ----------------- .. sidebar:: info - :py:obj:`processors.online_dictionary ` *no engine of this type is documented yet / coming soon* --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/json_engine.rst # Path: docs/dev/engines/json_engine.rst .. _json_engine engine: ============ JSON Engine ============ .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.json_engine :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/mediawiki.rst # Path: docs/dev/engines/mediawiki.rst .. _mediawiki engine: ================ MediaWiki Engine ================ .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.mediawiki :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/offline/command-line-engines.rst # Path: docs/dev/engines/offline/command-line-engines.rst .. _engine command: ==================== Command Line Engines ==================== .. sidebar:: info - :origin:`command.py ` - :ref:`offline engines` .. contents:: :depth: 2 :local: :backlinks: entry .. sidebar:: info Initial sponsored by `Search and Discovery Fund `_ of `NLnet Foundation `_. .. automodule:: searx.engines.command :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/offline/nosql-engines.rst # Path: docs/dev/engines/offline/nosql-engines.rst .. _nosql engines: =============== NoSQL databases =============== .. sidebar:: further read - `NoSQL databases `_ - `valkey.io `_ - `MongoDB `_ .. contents:: :depth: 2 :local: :backlinks: entry .. sidebar:: info Initial sponsored by `Search and Discovery Fund `_ of `NLnet Foundation `_. The following `NoSQL databases`_ are supported: - :ref:`engine valkey_server` - :ref:`engine mongodb` All of the engines above are just commented out in the :origin:`settings.yml `, as you have to set various options and install dependencies before using them. By default, the engines use the ``key-value`` template for displaying results / see :origin:`simple ` theme. If you are not satisfied with the original result layout, you can use your own template, set ``result_template`` attribute to ``{template_name}`` and place the templates at:: searx/templates/{theme_name}/result_templates/{template_name} Furthermore, if you do not wish to expose these engines on a public instance, you can still add them and limit the access by setting ``tokens`` as described in section :ref:`private engines`. Extra Dependencies ================== For using :ref:`engine valkey_server` or :ref:`engine mongodb` you need to install additional packages in Python's Virtual Environment of your SearXNG instance. To switch into the environment (:ref:`searxng-src`) you can use :ref:`searxng.sh`:: $ sudo utils/searxng.sh instance cmd bash (searxng-pyenv)$ pip install ... Configure the engines ===================== `NoSQL databases`_ are used for storing arbitrary data without first defining their structure. .. _engine valkey_server: Valkey Server ------------- .. _valkey: https://github.com/andymccurdy/valkey-py#installation .. sidebar:: info - ``pip install`` valkey_ - valkey.io_ - :origin:`valkey_server.py ` .. automodule:: searx.engines.valkey_server :members: .. _engine mongodb: MongoDB ------- .. _pymongo: https://github.com/mongodb/mongo-python-driver#installation .. sidebar:: info - ``pip install`` pymongo_ - MongoDB_ - :origin:`mongodb.py ` .. automodule:: searx.engines.mongodb :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/offline/search-indexer-engines.rst # Path: docs/dev/engines/offline/search-indexer-engines.rst ================= Local Search APIs ================= .. sidebar:: further read - `Comparison to alternatives `_ .. contents:: :depth: 1 :local: :backlinks: entry .. sidebar:: info Initial sponsored by `Search and Discovery Fund `_ of `NLnet Foundation `_. Administrators might find themselves wanting to integrate locally running search engines. The following ones are supported for now: * `Elasticsearch`_ * `Meilisearch`_ * `Solr`_ Each search engine is powerful, capable of full-text search. All of the engines above are added to ``settings.yml`` just commented out, as you have to ``base_url`` for all them. Please note that if you are not using HTTPS to access these engines, you have to enable HTTP requests by setting ``enable_http`` to ``True``. Furthermore, if you do not want to expose these engines on a public instance, you can still add them and limit the access by setting ``tokens`` as described in section :ref:`private engines`. .. _engine meilisearch: MeiliSearch =========== .. automodule:: searx.engines.meilisearch :members: .. _engine elasticsearch: Elasticsearch ============= .. automodule:: searx.engines.elasticsearch :members: .. _engine solr: Solr ==== .. automodule:: searx.engines.solr :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/offline/sql-engines.rst # Path: docs/dev/engines/offline/sql-engines.rst .. _sql engines: =========== SQL Engines =========== .. sidebar:: further read - `SQLite `_ - `PostgreSQL `_ - `MySQL `_ .. contents:: :depth: 2 :local: :backlinks: entry .. sidebar:: info Initial sponsored by `Search and Discovery Fund `_ of `NLnet Foundation `_. With the *SQL engines* you can bind SQL databases into SearXNG. The following Relational Database Management System (RDBMS) are supported: - :ref:`engine sqlite` - :ref:`engine postgresql` - :ref:`engine mysql_server` & :ref:`engine mariadb_server` All of the engines above are just commented out in the :origin:`settings.yml `, as you have to set the required attributes for the engines, e.g. ``database:`` ... .. code:: yaml - name: ... engine: {sqlite|postgresql|mysql_server} database: ... result_template: {template_name} query_str: ... By default, the engines use the ``key-value`` template for displaying results / see :origin:`simple ` theme. If you are not satisfied with the original result layout, you can use your own template, set ``result_template`` attribute to ``{template_name}`` and place the templates at:: searx/templates/{theme_name}/result_templates/{template_name} If you do not wish to expose these engines on a public instance, you can still add them and limit the access by setting ``tokens`` as described in section :ref:`private engines`. Extra Dependencies ================== For using :ref:`engine postgresql` or :ref:`engine mysql_server` you need to install additional packages in Python's Virtual Environment of your SearXNG instance. To switch into the environment (:ref:`searxng-src`) you can use :ref:`searxng.sh`:: $ sudo utils/searxng.sh instance cmd bash (searxng-pyenv)$ pip install ... Configure the engines ===================== The configuration of the new database engines are similar. You must put a valid SQL-SELECT query in ``query_str``. At the moment you can only bind at most one parameter in your query. By setting the attribute ``limit`` you can define how many results you want from the SQL server. Basically, it is the same as the ``LIMIT`` keyword in SQL. Please, do not include ``LIMIT`` or ``OFFSET`` in your SQL query as the engines rely on these keywords during paging. If you want to configure the number of returned results use the option ``limit``. .. _engine sqlite: SQLite ------ .. sidebar:: info - :origin:`sqlite.py ` .. automodule:: searx.engines.sqlite :members: .. _engine postgresql: PostgreSQL ---------- .. _psycopg2: https://www.psycopg.org/install .. sidebar:: info - :origin:`postgresql.py ` - ``pip install`` `psycopg2-binary `_ .. automodule:: searx.engines.postgresql :members: .. _engine mysql_server: MySQL ----- .. sidebar:: info - :origin:`mysql_server.py ` - ``pip install`` :pypi:`mysql-connector-python ` .. automodule:: searx.engines.mysql_server :members: .. _engine mariadb_server: MariaDB -------- .. sidebar:: info - :origin:`mariadb_server.py ` - ``pip install`` :pypi:`mariadb ` .. automodule:: searx.engines.mariadb_server :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/offline_concept.rst # Path: docs/dev/engines/offline_concept.rst =============== Offline Concept =============== .. sidebar:: offline engines - :ref:`demo offline engine` - :ref:`engine command` - :ref:`sql engines` - :ref:`nosql engines` - :py:obj:`searx.search.processors.offline` To extend the functionality of SearXNG, offline engines are going to be introduced. An offline engine is an engine which does not need Internet connection to perform a search and does not use HTTP to communicate. Offline engines can be configured, by adding those to the `engines` list of :origin:`settings.yml `. An example skeleton for offline engines can be found in :ref:`demo offline engine` (:origin:`demo_offline.py `). Programming Interface ===================== :py:func:`init(engine_settings=None) ` All offline engines can have their own init function to setup the engine before accepting requests. The function gets the settings from settings.yml as a parameter. This function can be omitted, if there is no need to setup anything in advance. :py:func:`search(query, params) ` Each offline engine has a function named ``search``. This function is responsible to perform a search and return the results in a presentable format. (Where *presentable* means presentable by the selected result template.) The return value is a list of results retrieved by the engine. Engine representation in ``/config`` If an engine is offline, the attribute ``offline`` is set to ``True``. .. _offline requirements: Extra Dependencies ================== If an offline engine depends on an external tool, SearXNG does not install it by default. When an administrator configures such engine and starts the instance, the process returns an error with the list of missing dependencies. Also, required dependencies will be added to the comment/description of the engine, so admins can install packages in advance. If there is a need to install additional packages in *Python's Virtual Environment* of your SearXNG instance you need to switch into the environment (:ref:`searxng-src`) first, for this you can use :ref:`searxng.sh`:: $ sudo utils/searxng.sh instance cmd bash (searxng-pyenv)$ pip install ... Private engines (Security) ========================== To limit the access to offline engines, if an instance is available publicly, administrators can set token(s) for each of the :ref:`private engines`. If a query contains a valid token, then SearXNG performs the requested private search. If not, requests from an offline engines return errors. --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/adobe_stock.rst # Path: docs/dev/engines/online/adobe_stock.rst .. _adobe stock engine: =========== Adobe Stock =========== .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.adobe_stock :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/alpinelinux.rst # Path: docs/dev/engines/online/alpinelinux.rst .. _alpinelinux engine: ===================== Alpine Linux Packages ===================== .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.alpinelinux :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/annas_archive.rst # Path: docs/dev/engines/online/annas_archive.rst .. _annas_archive engine: ============== Anna's Archive ============== .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.annas_archive :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/archlinux.rst # Path: docs/dev/engines/online/archlinux.rst .. _archlinux engine: ========== Arch Linux ========== .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.archlinux :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/arxiv.rst # Path: docs/dev/engines/online/arxiv.rst .. _arxiv engine: ===== arXiv ===== .. automodule:: searx.engines.arxiv :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/astrophysics_data_system.rst # Path: docs/dev/engines/online/astrophysics_data_system.rst .. _astrophysics_data_system engine: ============================== Astrophysics Data System (ADS) ============================== .. automodule:: searx.engines.astrophysics_data_system :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/azure.rst # Path: docs/dev/engines/online/azure.rst .. _azure engine: =============== Azure Resources =============== .. automodule:: searx.engines.azure :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/bing.rst # Path: docs/dev/engines/online/bing.rst .. _bing engines: ============ Bing Engines ============ .. contents:: :depth: 2 :local: :backlinks: entry .. _bing web engine: Bing WEB ======== .. automodule:: searx.engines.bing :members: .. _bing images engine: Bing Images =========== .. automodule:: searx.engines.bing_images :members: .. _bing videos engine: Bing Videos =========== .. automodule:: searx.engines.bing_videos :members: .. _bing news engine: Bing News ========= .. automodule:: searx.engines.bing_news :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/bpb.rst # Path: docs/dev/engines/online/bpb.rst .. _bpb engine: === Bpb === .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.bpb :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/brave.rst # Path: docs/dev/engines/online/brave.rst .. _brave engine: ============= Brave Engines ============= .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.brave :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/bt4g.rst # Path: docs/dev/engines/online/bt4g.rst .. _bt4g engine: ==== BT4G ==== .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.bt4g :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/chinaso.rst # Path: docs/dev/engines/online/chinaso.rst .. _chinaso engine: ======= ChinaSo ======= .. automodule:: searx.engines.chinaso :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/core.rst # Path: docs/dev/engines/online/core.rst .. _core engine: ==== CORE ==== .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.core :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/crossref.rst # Path: docs/dev/engines/online/crossref.rst .. _crossref engine: ======== Crossref ======== .. automodule:: searx.engines.crossref :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/dailymotion.rst # Path: docs/dev/engines/online/dailymotion.rst .. _dailymotion engine: =========== Dailymotion =========== .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.dailymotion :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/discourse.rst # Path: docs/dev/engines/online/discourse.rst .. _discourse engine: ================ Discourse Forums ================ .. automodule:: searx.engines.discourse :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/duckduckgo.rst # Path: docs/dev/engines/online/duckduckgo.rst .. _duckduckgo engines: ================== DuckDuckGo Engines ================== .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.duckduckgo :members: .. automodule:: searx.engines.duckduckgo_extra :members: .. automodule:: searx.engines.duckduckgo_definitions :members: .. automodule:: searx.engines.duckduckgo_weather :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/geizhals.rst # Path: docs/dev/engines/online/geizhals.rst .. _gitea geizhals: ======== Geizhals ======== .. automodule:: searx.engines.geizhals :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/gitea.rst # Path: docs/dev/engines/online/gitea.rst .. _gitea engine: ===== Gitea ===== .. automodule:: searx.engines.gitea :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/github_code.rst # Path: docs/dev/engines/online/github_code.rst .. _github code engine: =========== Github Code =========== .. automodule:: searx.engines.github_code :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/gitlab.rst # Path: docs/dev/engines/online/gitlab.rst .. _gitlab engine: ====== GitLab ====== .. automodule:: searx.engines.gitlab :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/google.rst # Path: docs/dev/engines/online/google.rst .. _google engines: ============== Google Engines ============== .. contents:: :depth: 2 :local: :backlinks: entry .. _google API: Google API ========== .. _Query Parameter Definitions: https://developers.google.com/custom-search/docs/xml_results#WebSearch_Query_Parameter_Definitions SearXNG's implementation of the Google API is mainly done in :py:obj:`get_google_info `. For detailed description of the *REST-full* API see: `Query Parameter Definitions`_. The linked API documentation can sometimes be helpful during reverse engineering. However, we cannot use it in the freely accessible WEB services; not all parameters can be applied and some engines are more *special* than other (e.g. :ref:`google news engine`). .. _google web engine: Google WEB ========== .. automodule:: searx.engines.google :members: .. _google autocomplete: Google Autocomplete ==================== .. autofunction:: searx.autocomplete.google_complete .. _google images engine: Google Images ============= .. automodule:: searx.engines.google_images :members: .. _google videos engine: Google Videos ============= .. automodule:: searx.engines.google_videos :members: .. _google news engine: Google News =========== .. automodule:: searx.engines.google_news :members: .. _google scholar engine: Google Scholar ============== .. automodule:: searx.engines.google_scholar :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/huggingface.rst # Path: docs/dev/engines/online/huggingface.rst .. _huggingface engine: ============ Hugging Face ============ .. automodule:: searx.engines.huggingface :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/lemmy.rst # Path: docs/dev/engines/online/lemmy.rst .. _lemmy engine: ===== Lemmy ===== .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.lemmy :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/loc.rst # Path: docs/dev/engines/online/loc.rst .. _loc engine: =================== Library of Congress =================== .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.loc :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/marginalia.rst # Path: docs/dev/engines/online/marginalia.rst .. _marginalia engine: ================= Marginalia Search ================= .. automodule:: searx.engines.marginalia :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/mastodon.rst # Path: docs/dev/engines/online/mastodon.rst .. _mastodon engine: ======== Mastodon ======== .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.mastodon :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/moviepilot.rst # Path: docs/dev/engines/online/moviepilot.rst .. _moviepilot engine: ========== Moviepilot ========== .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.moviepilot :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/mrs.rst # Path: docs/dev/engines/online/mrs.rst .. _mrs engine: ========================= Matrix Rooms Search (MRS) ========================= .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.mrs :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/mwmbl.rst # Path: docs/dev/engines/online/mwmbl.rst .. _Mwmbl engine: ============ Mwmbl Engine ============ .. contents:: :depth: 2 :local: :backlinks: entry .. _mwmbl web engine: Mwmbl WEB ========= .. automodule:: searx.engines.mwmbl :members: .. _mwmbl autocomplete: Mwmbl Autocomplete ================== .. autofunction:: searx.autocomplete.mwmbl --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/odysee.rst # Path: docs/dev/engines/online/odysee.rst .. _odysee engine: ====== Odysee ====== .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.odysee :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/openalex.rst # Path: docs/dev/engines/online/openalex.rst .. _openalex engine: ======== OpenAlex ======== .. automodule:: searx.engines.openalex :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/openlibrary.rst # Path: docs/dev/engines/online/openlibrary.rst .. _openlibrary engine: ============ Open Library ============ .. automodule:: searx.engines.openlibrary :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/peertube.rst # Path: docs/dev/engines/online/peertube.rst .. _peertube engines: ================ Peertube Engines ================ .. contents:: :depth: 2 :local: :backlinks: entry .. _peertube video engine: Peertube Video ============== .. automodule:: searx.engines.peertube :members: .. _sepiasearch engine: SepiaSearch =========== .. automodule:: searx.engines.sepiasearch :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/piped.rst # Path: docs/dev/engines/online/piped.rst .. _piped engine: ===== Piped ===== .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.piped :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/presearch.rst # Path: docs/dev/engines/online/presearch.rst .. _engine presearch: ================ Presearch Engine ================ .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.presearch :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/pubmed.rst # Path: docs/dev/engines/online/pubmed.rst .. _pubmed engine: ====== PubMed ====== .. automodule:: searx.engines.pubmed :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/qwant.rst # Path: docs/dev/engines/online/qwant.rst .. _qwant engine: ===== Qwant ===== .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.qwant :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/radio_browser.rst # Path: docs/dev/engines/online/radio_browser.rst .. _RadioBrowser engine: ============ RadioBrowser ============ .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.radio_browser :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/recoll.rst # Path: docs/dev/engines/online/recoll.rst .. _engine recoll: ============= Recoll Engine ============= .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.recoll :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/repology.rst # Path: docs/dev/engines/online/repology.rst .. _repology engine: ======== Repology ======== .. automodule:: searx.engines.repology :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/reuters.rst # Path: docs/dev/engines/online/reuters.rst .. _reuters engine: ======= Reuters ======= .. automodule:: searx.engines.reuters :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/seekr.rst # Path: docs/dev/engines/online/seekr.rst .. _seekr engine: ============= Seekr Engines ============= .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.seekr :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/semantic_scholar.rst # Path: docs/dev/engines/online/semantic_scholar.rst .. _semantic_scholar engine: ================ Semantic Scholar ================ .. automodule:: searx.engines.semantic_scholar :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/soundcloud.rst # Path: docs/dev/engines/online/soundcloud.rst .. _soundcloud engine: ========== Soundcloud ========== .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.soundcloud :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/sourcehut.rst # Path: docs/dev/engines/online/sourcehut.rst .. _sourcehut engine: ========= Sourcehut ========= .. automodule:: searx.engines.sourcehut :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/springer.rst # Path: docs/dev/engines/online/springer.rst .. _springer engine: =============== Springer Nature =============== .. automodule:: searx.engines.springer :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/startpage.rst # Path: docs/dev/engines/online/startpage.rst .. _startpage engines: ================= Startpage Engines ================= .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.startpage :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/tagesschau.rst # Path: docs/dev/engines/online/tagesschau.rst .. _tagesschau engine: ============== Tagesschau API ============== .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.tagesschau :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/torznab.rst # Path: docs/dev/engines/online/torznab.rst .. _torznab engine: ============== Torznab WebAPI ============== .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.torznab :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/tubearchivist.rst # Path: docs/dev/engines/online/tubearchivist.rst .. _tubearchivist engine: ============== Tube Archivist ============== .. automodule:: searx.engines.tubearchivist :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/void.rst # Path: docs/dev/engines/online/void.rst .. _voidlinux engine: ========================== Void Linux binary packages ========================== .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.voidlinux :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/wallhaven.rst # Path: docs/dev/engines/online/wallhaven.rst .. _wallhaven engine: ========= Wallhaven ========= .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.wallhaven :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/wikipedia.rst # Path: docs/dev/engines/online/wikipedia.rst .. _wikimedia engines: ========= Wikimedia ========= .. contents:: :depth: 2 :local: :backlinks: entry .. _wikipedia engine: Wikipedia ========= .. automodule:: searx.engines.wikipedia :members: .. _wikidata engine: Wikidata ========= .. automodule:: searx.engines.wikidata :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/yacy.rst # Path: docs/dev/engines/online/yacy.rst .. _yacy engine: ==== Yacy ==== .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.yacy :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/yahoo.rst # Path: docs/dev/engines/online/yahoo.rst .. _yahoo engine: ============ Yahoo Engine ============ .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.yahoo :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online/zlibrary.rst # Path: docs/dev/engines/online/zlibrary.rst .. _zlibrary engine: ========= Z-Library ========= .. contents:: Contents :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.zlibrary :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/online_url_search/tineye.rst # Path: docs/dev/engines/online_url_search/tineye.rst .. _tineye engine: ====== Tineye ====== .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.tineye :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/engines/xpath.rst # Path: docs/dev/engines/xpath.rst .. _xpath engine: ============ XPath Engine ============ .. contents:: :depth: 2 :local: :backlinks: entry .. automodule:: searx.engines.xpath :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/extended_types.rst # Path: docs/dev/extended_types.rst .. _extended_types.: ============== Extended Types ============== .. automodule:: searx.extended_types --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/index.rst # Path: docs/dev/index.rst ======================= Developer documentation ======================= .. toctree:: :maxdepth: 2 quickstart commits contribution_guide extended_types engines/index result_types/index templates search_api plugins/index answerers/index translation makefile reST searxng_extra/index --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/makefile.rst # Path: docs/dev/makefile.rst .. _makefile: ======================= Makefile & ``./manage`` ======================= .. _gnu-make: https://www.gnu.org/software/make/manual/make.html#Introduction All relevant build and development tasks are implemented in the :origin:`./manage ` script and for CI or IDE integration a small :origin:`Makefile` wrapper is available. If you are not familiar with Makefiles, we recommend to read gnu-make_ introduction. .. sidebar:: build environment Before looking deeper at the targets, first read about :ref:`make install`. To install developer requirements follow :ref:`buildhosts`. .. contents:: :depth: 2 :local: :backlinks: entry The usage is simple, just type ``make {target-name}`` to *build* a target. Calling the ``help`` target gives a first overview (``make help``): .. tabs:: .. group-tab:: ``make`` .. program-output:: bash -c "cd ..; make --no-print-directory help" .. group-tab:: ``./manage`` The Makefile targets are implemented for comfort, if you can do without tab-completion and need to have a more granular control, use :origin:`manage` without the Makefile wrappers. .. code:: sh $ ./manage help .. _make install: Python environment (``make install``) ===================================== .. sidebar:: activate environment ``source ./local/py3/bin/activate`` We do no longer need to build up the virtualenv manually. Jump into your git working tree and release a ``make install`` to get a virtualenv with a *developer install* of SearXNG (:origin:`setup.py`). :: $ cd ~/searxng-clone $ make install PYENV [virtualenv] installing ./requirements*.txt into local/py3 ... PYENV [install] pip install --use-pep517 --no-build-isolation -e 'searx[test]' ... Successfully installed searxng-2023.7.19+a446dea1b If you release ``make install`` multiple times the installation will only rebuild if the sha256 sum of the *requirement files* fails. With other words: the check fails if you edit the requirements listed in :origin:`requirements-dev.txt` and :origin:`requirements.txt`). :: $ make install PYENV OK PYENV [virtualenv] requirements.sha256 failed [virtualenv] - 6cea6eb6def9e14a18bf32f8a3e... ./requirements-dev.txt [virtualenv] - 471efef6c73558e391c3adb35f4... ./requirements.txt ... PYENV [virtualenv] installing ./requirements*.txt into local/py3 ... PYENV [install] pip install --use-pep517 --no-build-isolation -e 'searx[test]' ... Successfully installed searxng-2023.7.19+a446dea1b .. sidebar:: drop environment To get rid of the existing environment before re-build use :ref:`clean target ` first. If you think, something goes wrong with your ./local environment or you change the :origin:`setup.py` file, you have to call :ref:`make clean`. .. _make node.env: Node.js environment (``make node.env``) ======================================= .. _Node.js: https://nodejs.org/ .. _nvm: https://github.com/nvm-sh .. _npm: https://www.npmjs.com/ .. jinja:: searx Node.js_ version {{version.node}} or higher is required to build the themes. If the requirement is not met, the build chain uses nvm_ (Node Version Manager) to install Node.js_ locally: there is no need to install nvm_ or npm_ on your system. To install NVM_ and Node.js_ in once you can use :ref:`make nvm.nodejs`. .. _make nvm: NVM ``make nvm.install nvm.status`` ----------------------------------- Use ``make nvm.status`` to get the current status of your Node.js_ and nvm_ setup. .. tabs:: .. group-tab:: nvm.install .. code:: sh $ LANG=C make nvm.install INFO: install (update) NVM at ./searxng/.nvm INFO: clone: https://github.com/nvm-sh/nvm.git || Cloning into './searxng/.nvm'... INFO: checkout v0.39.4 || HEAD is now at 8fbf8ab v0.39.4 .. group-tab:: nvm.status (ubu2004) Here is the output you will typically get on a Ubuntu 20.04 system which serves only a `no longer active `_ Release `Node.js v10.19.0 `_. .. code:: sh $ make nvm.status INFO: Node.js is installed at /usr/bin/node INFO: Node.js is version v10.19.0 WARN: minimal Node.js version is 16.13.0 INFO: npm is installed at /usr/bin/npm INFO: npm is version 6.14.4 WARN: NVM is not installed .. _make nvm.nodejs: ``make nvm.nodejs`` ------------------- Install latest Node.js_ locally (uses nvm_):: $ make nvm.nodejs INFO: install (update) NVM at /share/searxng/.nvm INFO: clone: https://github.com/nvm-sh/nvm.git ... Downloading and installing node v16.13.0... ... INFO: Node.js is installed at searxng/.nvm/versions/node/v16.13.0/bin/node INFO: Node.js is version v16.13.0 INFO: npm is installed at searxng/.nvm/versions/node/v16.13.0/bin/npm INFO: npm is version 8.1.0 INFO: NVM is installed at searxng/.nvm .. _make run: ``make run`` ============ To get up a running a developer instance simply call ``make run``. This enables *debug* option in :origin:`searx/settings.yml`, starts a ``./searx/webapp.py`` instance and opens the URL in your favorite WEB browser (:man:`xdg-open`):: $ make run Changes to theme's HTML templates (jinja2) are instant. Changes to the CSS & JS sources of the theme need to be rebuild. You can do that by running:: $ make themes.all .. ToDo: vite server is not implemented yet / will be done in a follow up PR Alternatively to ``themes.all`` you can run *live builds* of the theme you are modify (:ref:`make themes`):: $ LIVE_THEME=simple make run .. _make format: ``make format`` ====================== .. _Black code style: https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html .. _shfmt: https://github.com/mvdan/sh?tab=readme-ov-file#shfmt .. _EditorConfig: https://github.com/patrickvane/shfmt?tab=readme-ov-file#description - Format Python source code using `Black code style`_. See ``$BLACK_OPTIONS`` and ``$BLACK_TARGETS`` in :origin:`Makefile`. - Format Shell scripts using shfmt_. The formatter ``shfmt`` reads the rules from the EditorConfig_ files. .. _make clean: ``make clean`` ============== Drops all intermediate files, all builds, but keep sources untouched. Before calling ``make clean`` stop all processes using the :ref:`make install` or :ref:`make node.env`. :: $ make clean CLEAN pyenv PYENV [virtualenv] drop local/py3 CLEAN docs -- build/docs dist/docs CLEAN themes -- locally installed npm dependencies ... CLEAN test stuff CLEAN common files .. _make docs: ``make docs`` ============= Target ``docs`` builds the documentation: .. code:: bash $ make docs HTML ./docs --> file:// DOCS build build/docs/includes ... The HTML pages are in dist/docs. .. _make docs.clean: ``make docs.clean docs.live`` ---------------------------------- We describe the usage of the ``doc.*`` targets in the :ref:`How to contribute / Documentation ` section. If you want to edit the documentation read our :ref:`make docs.live` section. If you are working in your own brand, adjust your :ref:`settings brand`. .. _make docs.gh-pages: ``make docs.gh-pages`` ---------------------- To deploy on github.io first adjust your :ref:`settings brand`. For any further read :ref:`deploy on github.io`. .. _make test: ``make test`` ============= Runs a series of tests: :ref:`make test.pylint`, ``test.pep8``, ``test.unit`` and ``test.robot``. You can run tests selective, e.g.:: $ make test.pep8 test.unit test.shell TEST test.pep8 OK ... TEST test.unit OK ... TEST test.shell OK .. _make test.shell: ``make test.shell`` ------------------- :ref:`sh lint` / if you have changed some bash scripting run this test before commit. .. _make test.pylint: ``make test.pylint`` -------------------- .. _Pylint: https://www.pylint.org/ Pylint_ is known as one of the best source-code, bug and quality checker for the Python programming language. The pylint profile used in the SearXNG project is found in project's root folder :origin:`.pylintrc`. .. _make search.checker: ``make search.checker.{engine name}`` ===================================== To check all engines:: make search.checker To check a engine with whitespace in the name like *google news* replace space by underline:: make search.checker.google_news To see HTTP requests and more use SEARXNG_DEBUG:: make SEARXNG_DEBUG=1 search.checker.google_news .. _3xx: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_redirection To filter out HTTP redirects (3xx_):: make SEARXNG_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]" ... Engine google news Checking https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0 https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None -- https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0 https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None -- .. _make themes: ``make themes.*`` ================= .. sidebar:: further read - :ref:`devquickstart` The :origin:`Makefile` targets ``make theme.*`` cover common tasks to build the theme(s). The ``./manage themes.*`` command line can be used to convenient run common theme build tasks. .. program-output:: bash -c "cd ..; ./manage themes.help" To get live builds while modifying CSS & JS use (:ref:`make run`): .. code:: sh $ LIVE_THEME=simple make run .. _make static.build: ``make static.build.*`` ======================= .. sidebar:: further read - :ref:`devquickstart` The :origin:`Makefile` targets ``static.build.*`` cover common tasks to build (a commit of) the static files. The ``./manage static.build..*`` command line can be used to convenient run common build tasks of the static files. .. program-output:: bash -c "cd ..; ./manage static.help" .. _manage go.help: ``./manage go.help`` ==================== The ``./manage go.*`` command line can be used to convenient run common `go (wiki)`_ tasks. .. _go (wiki): https://en.wikipedia.org/wiki/Go_(programming_language) .. program-output:: bash -c "cd ..; ./manage go.help" --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/plugins/builtins.rst # Path: docs/dev/plugins/builtins.rst .. _builtin plugins: ================ Built-in Plugins ================ .. toctree:: :maxdepth: 1 calculator hash_plugin hostnames infinite_scroll self_info tor_check unit_converter time_zone --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/plugins/calculator.rst # Path: docs/dev/plugins/calculator.rst .. _plugins.calculator: ========== Calculator ========== .. automodule:: searx.plugins.calculator :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/plugins/development.rst # Path: docs/dev/plugins/development.rst .. _dev plugin: ================== Plugin Development ================== .. automodule:: searx.plugins --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/plugins/hash_plugin.rst # Path: docs/dev/plugins/hash_plugin.rst .. _hash_plugin plugin: =========== Hash Values =========== .. autoclass:: searx.plugins.hash_plugin.SXNGPlugin :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/plugins/hostnames.rst # Path: docs/dev/plugins/hostnames.rst .. _hostnames plugin: ========= Hostnames ========= .. automodule:: searx.plugins.hostnames :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/plugins/index.rst # Path: docs/dev/plugins/index.rst ======= Plugins ======= .. toctree:: :maxdepth: 2 development builtins --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/plugins/infinite_scroll.rst # Path: docs/dev/plugins/infinite_scroll.rst .. _plugins.infinite_scroll: =============== Infinite scroll =============== .. automodule:: searx.plugins.infinite_scroll :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/plugins/self_info.rst # Path: docs/dev/plugins/self_info.rst .. _self_info plugin: ========= Self-Info ========= .. autoclass:: searx.plugins.self_info.SXNGPlugin :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/plugins/time_zone.rst # Path: docs/dev/plugins/time_zone.rst .. _time zone plugin: ========= Time Zone ========= .. automodule:: searx.plugins.time_zone :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/plugins/tor_check.rst # Path: docs/dev/plugins/tor_check.rst .. _tor check plugin: ========= Tor check ========= .. automodule:: searx.plugins.tor_check :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/plugins/unit_converter.rst # Path: docs/dev/plugins/unit_converter.rst .. _unit converter plugin: ============== Unit Converter ============== .. automodule:: searx.plugins.unit_converter :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/quickstart.rst # Path: docs/dev/quickstart.rst .. _devquickstart: ====================== Development Quickstart ====================== .. _npm: https://www.npmjs.com/ .. _Node.js: https://nodejs.org/ .. _Biome: https://biomejs.dev/ .. _Stylelint: https://stylelint.io/ .. sidebar:: further read - :ref:`makefile` - :ref:`buildhosts` SearXNG loves developers; Developers do not need to worry about tool chains, the usual developer tasks can be comfortably executed via :ref:`make `. Don't hesitate, just clone SearXNG's sources and start hacking right now .. .. code:: bash git clone https://github.com/searxng/searxng.git searxng For the developer environment, `mise en place`_ is recommended: .. code:: sh $ curl https://mise.run | sh The tools required for a developer environment are provided via `mise.toml`_, trust the project: .. code:: sh $ mise trust .. _mise.toml: https://github.com/searxng/searxng/blob/master/mise.toml .. _mise en place: https://mise.jdx.dev/getting-started.html Here is how a minimal workflow looks like: 1. *start* hacking 2. *run* your code: :ref:`make run` 3. *format & test* your code: :ref:`make format` and :ref:`make test` If you think at some point something fails, go back to *start*. Otherwise, choose a meaningful commit message and we are happy to receive your pull request. To not end in *wild west* we have some directives, please pay attention to our ":ref:`how to contribute`" guideline. .. _pdb: https://docs.python.org/3/library/pdb.html#module-pdb If you want to debug with the *good old Python Debugger* pdb_: Alternatively to ``make run`` (2.) which starts a :ref:`searxng granian` server you can jump into the developer environment and start a python based HTTP server by:: $ ./manage dev.env ... (dev.env)$ SEARXNG_DEBUG=1 python -m searx.webapp Since this is a pure Python solution, you can set breakpoints in your code with ``pdb.set_trace()`` and the debugger will wait for you in the terminal prompt. .. sidebar:: further read - :ref:`make nvm` - :ref:`make themes` If you implement themes, you will need to setup a :ref:`Node.js environment `. Before you call *make run* (2.), you need to compile the modified styles and JavaScript: ``make node.clean themes.all``. If Biome_ or Stylelint_ reports issues, try ``make themes.fix``. Alternatively you can also compile selective the theme you have modified, e.g. the *simple* theme. .. code:: bash make themes.simple .. tip:: To get live builds while modifying CSS & JS use: ``LIVE_THEME=simple make run`` .. sidebar:: further read - :ref:`make static.build` If you finished your *tests* you can start to commit your changes. To separate the modified source code from the build products first run: .. code:: bash make static.build.restore This will restore the old build products and only your changes of the code remain in the working tree which can now be added & committed. When all sources are committed, you can commit the build products simply by: .. code:: bash make static.build.commit Committing the build products should be the last step, just before you send us your PR. There is also a make target to rewind this last build commit: .. code:: bash make static.build.drop --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/reST.rst # Path: docs/dev/reST.rst .. _reST primer: =========== reST primer =========== .. sidebar:: KISS_ and readability_ Instead of defining more and more roles, we at SearXNG encourage our contributors to follow principles like KISS_ and readability_. We at SearXNG are using reStructuredText (aka reST_) markup for all kind of documentation. With the builders from the Sphinx_ project a HTML output is generated and deployed at docs.searxng.org_. For build prerequisites read :ref:`docs build`. .. _docs.searxng.org: https://docs.searxng.org/ The source files of SearXNG's documentation are located at :origin:`docs`. Sphinx assumes source files to be encoded in UTF-8 by default. Run :ref:`make docs.live ` to build HTML while editing. .. sidebar:: Further reading - Sphinx-Primer_ - `Sphinx markup constructs`_ - reST_, docutils_, `docutils FAQ`_ - Sphinx_, `sphinx-doc FAQ`_ - `sphinx config`_, doctree_ - `sphinx cross references`_ - linuxdoc_ - intersphinx_ - sphinx-jinja_ - `Sphinx's autodoc`_ - `Sphinx's Python domain`_, `Sphinx's C domain`_ - SVG_, ImageMagick_ - DOT_, `Graphviz's dot`_, Graphviz_ .. contents:: :depth: 3 :local: :backlinks: entry Sphinx_ and reST_ have their place in the python ecosystem. Over that reST is used in popular projects, e.g the Linux kernel documentation `[kernel doc]`_. .. _[kernel doc]: https://www.kernel.org/doc/html/latest/doc-guide/sphinx.html .. sidebar:: Content matters The readability_ of the reST sources has its value, therefore we recommend to make sparse usage of reST markup / .. content matters! **reST** is a plaintext markup language, its markup is *mostly* intuitive and you will not need to learn much to produce well formed articles with. I use the word *mostly*: like everything in live, reST has its advantages and disadvantages, some markups feel a bit grumpy (especially if you are used to other plaintext markups). Soft skills =========== Before going any deeper into the markup let's face on some **soft skills** a trained author brings with, to reach a well feedback from readers: - Documentation is dedicated to an audience and answers questions from the audience point of view. - Don't detail things which are general knowledge from the audience point of view. - Limit the subject, use cross links for any further reading. To be more concrete what a *point of view* means. In the (:origin:`docs`) folder we have three sections (and the *blog* folder), each dedicate to a different group of audience. User's POV: :origin:`docs/user` A typical user knows about search engines and might have heard about meta crawlers and privacy. Admin's POV: :origin:`docs/admin` A typical Admin knows about setting up services on a linux system, but he does not know all the pros and cons of a SearXNG setup. Developer's POV: :origin:`docs/dev` Depending on the readability_ of code, a typical developer is able to read and understand source code. Describe what a item aims to do (e.g. a function). If the chronological order matters, describe it. Name the *out-of-limits conditions* and all the side effects a external developer will not know. .. _reST inline markup: Basic inline markup =================== .. sidebar:: Inline markup - :ref:`reST roles` - :ref:`reST smart ref` Basic inline markup is done with asterisks and backquotes. If asterisks or backquotes appear in running text and could be confused with inline markup delimiters, they have to be escaped with a backslash (``\*pointer``). .. table:: basic inline markup :widths: 4 3 7 ================================================ ==================== ======================== description rendered markup ================================================ ==================== ======================== one asterisk for emphasis *italics* ``*italics*`` two asterisks for strong emphasis **boldface** ``**boldface**`` backquotes for code samples and literals ``foo()`` ````foo()```` quote asterisks or backquotes \*foo is a pointer ``\*foo is a pointer`` ================================================ ==================== ======================== .. _reST basic structure: Basic article structure ======================= The basic structure of an article makes use of heading adornments to markup chapter, sections and subsections. .. _reST template: reST template ------------- reST template for an simple article: .. code:: reST .. _doc refname: ============== Document title ============== Lorem ipsum dolor sit amet, consectetur adipisici elit .. Further read :ref:`chapter refname`. .. _chapter refname: Chapter ======= Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquid ex ea commodi consequat ... .. _section refname: Section ------- lorem .. .. _subsection refname: Subsection ~~~~~~~~~~ lorem .. Headings -------- #. title - with overline for document title: .. code:: reST ============== Document title ============== #. chapter - with anchor named ``anchor name``: .. code:: reST .. _anchor name: Chapter ======= #. section .. code:: reST Section ------- #. subsection .. code:: reST Subsection ~~~~~~~~~~ Anchors & Links =============== .. _reST anchor: Anchors ------- .. _ref role: https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-ref To refer a point in the documentation a anchor is needed. The :ref:`reST template ` shows an example where a chapter titled *"Chapters"* gets an anchor named ``chapter title``. Another example from *this* document, where the anchor named ``reST anchor``: .. code:: reST .. _reST anchor: Anchors ------- To refer a point in the documentation a anchor is needed ... To refer anchors use the `ref role`_ markup: .. code:: reST Visit chapter :ref:`reST anchor`. Or set hyperlink text manually :ref:`foo bar `. .. admonition:: ``:ref:`` role :class: rst-example Visit chapter :ref:`reST anchor`. Or set hyperlink text manually :ref:`foo bar `. .. _reST ordinary ref: Link ordinary URL ----------------- If you need to reference external URLs use *named* hyperlinks to maintain readability of reST sources. Here is a example taken from *this* article: .. code:: reST .. _Sphinx Field Lists: https://www.sphinx-doc.org/en/master/usage/restructuredtext/field-lists.html With the *named* hyperlink `Sphinx Field Lists`_, the raw text is much more readable. And this shows the alternative (less readable) hyperlink markup `Sphinx Field Lists `__. .. admonition:: Named hyperlink :class: rst-example With the *named* hyperlink `Sphinx Field Lists`_, the raw text is much more readable. And this shows the alternative (less readable) hyperlink markup `Sphinx Field Lists `__. .. _reST smart ref: Smart refs ---------- With the power of sphinx.ext.extlinks_ and intersphinx_ referencing external content becomes smart. .. table:: smart refs with sphinx.ext.extlinks_ and intersphinx_ :widths: 4 3 7 ========================== ================================== ==================================== refer ... rendered example markup ========================== ================================== ==================================== :rst:role:`rfc` :rfc:`822` ``:rfc:`822``` :rst:role:`pep` :pep:`8` ``:pep:`8``` sphinx.ext.extlinks_ -------------------------------------------------------------------------------------------------- project's wiki article :wiki:`Offline-engines` ``:wiki:`Offline-engines``` to docs public URL :docs:`dev/reST.html` ``:docs:`dev/reST.html``` files & folders origin :origin:`docs/dev/reST.rst` ``:origin:`docs/dev/reST.rst``` pull request :pull:`4` ``:pull:`4``` patch :patch:`af2cae6` ``:patch:`af2cae6``` PyPi package :pypi:`httpx` ``:pypi:`httpx``` manual page man :man:`bash` ``:man:`bash``` intersphinx_ -------------------------------------------------------------------------------------------------- external anchor :ref:`python:and` ``:ref:`python:and``` external doc anchor :doc:`jinja:templates` ``:doc:`jinja:templates``` python code object :py:obj:`datetime.datetime` ``:py:obj:`datetime.datetime``` flask code object :py:obj:`flask.Flask` ``:py:obj:`flask.Flask``` ========================== ================================== ==================================== Intersphinx is configured in :origin:`docs/conf.py`: .. code:: python intersphinx_mapping = { "python": ("https://docs.python.org/3/", None), "flask": ("https://flask.palletsprojects.com/", None), "jinja": ("https://jinja.palletsprojects.com/", None), "linuxdoc" : ("https://return42.github.io/linuxdoc/", None), "sphinx" : ("https://www.sphinx-doc.org/en/master/", None), } To list all anchors of the inventory (e.g. ``python``) use: .. code:: sh $ python -m sphinx.ext.intersphinx https://docs.python.org/3/objects.inv ... $ python -m sphinx.ext.intersphinx https://docs.searxng.org/objects.inv ... Literal blocks ============== The simplest form of :duref:`literal-blocks` is a indented block introduced by two colons (``::``). For highlighting use :dudir:`highlight` or :ref:`reST code` directive. To include literals from external files use :rst:dir:`literalinclude` or :ref:`kernel-include ` directive (latter one expands environment variables in the path name). .. _reST literal: ``::`` ------ .. code:: reST :: Literal block Lorem ipsum dolor:: Literal block Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore :: Literal block .. admonition:: Literal block :class: rst-example :: Literal block Lorem ipsum dolor:: Literal block Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore :: Literal block .. _reST code: ``code-block`` -------------- .. _pygments: https://pygments.org/languages/ .. sidebar:: Syntax highlighting is handled by pygments_. The :rst:dir:`code-block` directive is a variant of the :dudir:`code` directive with additional options. To learn more about code literals visit :ref:`sphinx:code-examples`. .. code-block:: reST The URL ``/stats`` handle is shown in :ref:`stats-handle` .. code-block:: Python :caption: python code block :name: stats-handle @app.route('/stats', methods=['GET']) def stats(): """Render engine statistics page.""" stats = get_engines_stats() return render( 'stats.html' , stats = stats ) .. code-block:: reST .. admonition:: Code block :class: rst-example The URL ``/stats`` handle is shown in :ref:`stats-handle` .. code-block:: Python :caption: python code block :name: stats-handle @app.route('/stats', methods=['GET']) def stats(): """Render engine statistics page.""" stats = get_engines_stats() return render( 'stats.html' , stats = stats ) Unicode substitution ==================== The :dudir:`unicode directive ` converts Unicode character codes (numerical values) to characters. This directive can only be used within a substitution definition. .. code-block:: reST .. |copy| unicode:: 0xA9 .. copyright sign .. |(TM)| unicode:: U+2122 Trademark |(TM)| and copyright |copy| glyphs. .. admonition:: Unicode :class: rst-example .. |copy| unicode:: 0xA9 .. copyright sign .. |(TM)| unicode:: U+2122 Trademark |(TM)| and copyright |copy| glyphs. .. _reST roles: Roles ===== .. sidebar:: Further reading - `Sphinx Roles`_ - :doc:`sphinx:usage/restructuredtext/domains` A *custom interpreted text role* (:duref:`ref `) is an inline piece of explicit markup. It signifies that that the enclosed text should be interpreted in a specific way. The general markup is one of: .. code:: reST :rolename:`ref-name` :rolename:`ref text ` .. table:: smart refs with sphinx.ext.extlinks_ and intersphinx_ :widths: 4 3 7 ========================== ================================== ==================================== role rendered example markup ========================== ================================== ==================================== :rst:role:`guilabel` :guilabel:`&Cancel` ``:guilabel:`&Cancel``` :rst:role:`kbd` :kbd:`C-x C-f` ``:kbd:`C-x C-f``` :rst:role:`menuselection` :menuselection:`Open --> File` ``:menuselection:`Open --> File``` :rst:role:`download` :download:`this file ` ``:download:`this file ``` math_ :math:`a^2 + b^2 = c^2` ``:math:`a^2 + b^2 = c^2``` :rst:role:`ref` :ref:`svg image example` ``:ref:`svg image example``` :rst:role:`command` :command:`ls -la` ``:command:`ls -la``` :durole:`emphasis` :emphasis:`italic` ``:emphasis:`italic``` :durole:`strong` :strong:`bold` ``:strong:`bold``` :durole:`literal` :literal:`foo()` ``:literal:`foo()``` :durole:`subscript` H\ :sub:`2`\ O ``H\ :sub:`2`\ O`` :durole:`superscript` E = mc\ :sup:`2` ``E = mc\ :sup:`2``` :durole:`title-reference` :title:`Time` ``:title:`Time``` ========================== ================================== ==================================== Figures & Images ================ .. sidebar:: Image processing With the directives from :ref:`linuxdoc ` the build process is flexible. To get best results in the generated output format, install ImageMagick_ and Graphviz_. SearXNG's sphinx setup includes: :ref:`linuxdoc:kfigure`. Scalable here means; scalable in sense of the build process. Normally in absence of a converter tool, the build process will break. From the authors POV it’s annoying to care about the build process when handling with images, especially since he has no access to the build process. With :ref:`linuxdoc:kfigure` the build process continues and scales output quality in dependence of installed image processors. If you want to add an image, you should use the ``kernel-figure`` (inheritance of :dudir:`figure`) and ``kernel-image`` (inheritance of :dudir:`image`) directives. E.g. to insert a figure with a scalable image format use SVG (:ref:`svg image example`): .. code:: reST .. _svg image example: .. kernel-figure:: svg_image.svg :alt: SVG image example Simple SVG image To refer the figure, a caption block is needed: :ref:`svg image example`. .. _svg image example: .. kernel-figure:: svg_image.svg :alt: SVG image example Simple SVG image. To refer the figure, a caption block is needed: :ref:`svg image example`. DOT files (aka Graphviz) ------------------------ With :ref:`linuxdoc:kernel-figure` reST support for **DOT** formatted files is given. - `Graphviz's dot`_ - DOT_ - Graphviz_ A simple example is shown in :ref:`dot file example`: .. code:: reST .. _dot file example: .. kernel-figure:: hello.dot :alt: hello world DOT's hello world example .. admonition:: hello.dot :class: rst-example .. _dot file example: .. kernel-figure:: hello.dot :alt: hello world DOT's hello world example ``kernel-render`` DOT --------------------- Embed *render* markups (or languages) like Graphviz's **DOT** is provided by the :ref:`linuxdoc:kernel-render` directive. A simple example of embedded DOT_ is shown in figure :ref:`dot render example`: .. code:: reST .. _dot render example: .. kernel-render:: DOT :alt: digraph :caption: Embedded DOT (Graphviz) code digraph foo { "bar" -> "baz"; } Attribute ``caption`` is needed, if you want to refer the figure: :ref:`dot render example`. Please note :ref:`build tools `. If Graphviz_ is installed, you will see an vector image. If not, the raw markup is inserted as *literal-block*. .. admonition:: kernel-render DOT :class: rst-example .. _dot render example: .. kernel-render:: DOT :alt: digraph :caption: Embedded DOT (Graphviz) code digraph foo { "bar" -> "baz"; } Attribute ``caption`` is needed, if you want to refer the figure: :ref:`dot render example`. ``kernel-render`` SVG --------------------- A simple example of embedded SVG_ is shown in figure :ref:`svg render example`: .. code:: reST .. _svg render example: .. kernel-render:: SVG :caption: Embedded **SVG** markup :alt: so-nw-arrow .. .. code:: xml .. admonition:: kernel-render SVG :class: rst-example .. _svg render example: .. kernel-render:: SVG :caption: Embedded **SVG** markup :alt: so-nw-arrow .. _reST lists: List markups ============ Bullet list ----------- List markup (:duref:`ref `) is simple: .. code:: reST - This is a bulleted list. 1. Nested lists are possible, but be aware that they must be separated from the parent list items by blank line 2. Second item of nested list - It has two items, the second item uses two lines. #. This is a numbered list. #. It has two items too. .. admonition:: bullet list :class: rst-example - This is a bulleted list. 1. Nested lists are possible, but be aware that they must be separated from the parent list items by blank line 2. Second item of nested list - It has two items, the second item uses two lines. #. This is a numbered list. #. It has two items too. Horizontal list --------------- The :rst:dir:`.. hlist:: ` transforms a bullet list into a more compact list. .. code:: reST .. hlist:: - first list item - second list item - third list item ... .. admonition:: hlist :class: rst-example .. hlist:: - first list item - second list item - third list item - next list item - next list item xxxx - next list item yyyy - next list item zzzz Definition list --------------- .. sidebar:: Note .. - the term cannot have more than one line of text - there is **no blank line between term and definition block** // this distinguishes definition lists (:duref:`ref `) from block quotes (:duref:`ref `). Each definition list (:duref:`ref `) item contains a term, optional classifiers and a definition. A term is a simple one-line word or phrase. Optional classifiers may follow the term on the same line, each after an inline ' : ' (**space, colon, space**). A definition is a block indented relative to the term, and may contain multiple paragraphs and other body elements. There may be no blank line between a term line and a definition block (*this distinguishes definition lists from block quotes*). Blank lines are required before the first and after the last definition list item, but are optional in-between. Definition lists are created as follows: .. code:: reST term 1 (up to a line of text) Definition 1. See the typo : this line is not a term! And this is not term's definition. **There is a blank line** in between the line above and this paragraph. That's why this paragraph is taken as **block quote** (:duref:`ref `) and not as term's definition! term 2 Definition 2, paragraph 1. Definition 2, paragraph 2. term 3 : classifier Definition 3. term 4 : classifier one : classifier two Definition 4. .. admonition:: definition list :class: rst-example term 1 (up to a line of text) Definition 1. See the typo : this line is not a term! And this is not term's definition. **There is a blank line** in between the line above and this paragraph. That's why this paragraph is taken as **block quote** (:duref:`ref `) and not as term's definition! term 2 Definition 2, paragraph 1. Definition 2, paragraph 2. term 3 : classifier Definition 3. term 4 : classifier one : classifier two Quoted paragraphs ----------------- Quoted paragraphs (:duref:`ref `) are created by just indenting them more than the surrounding paragraphs. Line blocks (:duref:`ref `) are a way of preserving line breaks: .. code:: reST normal paragraph ... lorem ipsum. Quoted paragraph ... lorem ipsum. | These lines are | broken exactly like in | the source file. .. admonition:: Quoted paragraph and line block :class: rst-example normal paragraph ... lorem ipsum. Quoted paragraph ... lorem ipsum. | These lines are | broken exactly like in | the source file. .. _reST field list: Field Lists ----------- .. _Sphinx Field Lists: https://www.sphinx-doc.org/en/master/usage/restructuredtext/field-lists.html .. sidebar:: bibliographic fields First lines fields are bibliographic fields, see `Sphinx Field Lists`_. Field lists are used as part of an extension syntax, such as options for directives, or database-like records meant for further processing. Field lists are mappings from field names to field bodies. They marked up like this: .. code:: reST :fieldname: Field content :foo: first paragraph in field foo second paragraph in field foo :bar: Field content .. admonition:: Field List :class: rst-example :fieldname: Field content :foo: first paragraph in field foo second paragraph in field foo :bar: Field content They are commonly used in Python documentation: .. code:: python def my_function(my_arg, my_other_arg): """A function just for me. :param my_arg: The first of my arguments. :param my_other_arg: The second of my arguments. :returns: A message (just for me, of course). """ Further list blocks ------------------- - field lists (:duref:`ref `, with caveats noted in :ref:`reST field list`) - option lists (:duref:`ref `) - quoted literal blocks (:duref:`ref `) - doctest blocks (:duref:`ref `) Admonitions =========== Sidebar ------- Sidebar is an eye catcher, often used for admonitions pointing further stuff or site effects. Here is the source of the sidebar :ref:`on top of this page `. .. code:: reST .. sidebar:: KISS_ and readability_ Instead of defining more and more roles, we at SearXNG encourage our contributors to follow principles like KISS_ and readability_. Generic admonition ------------------ The generic :dudir:`admonition ` needs a title: .. code:: reST .. admonition:: generic admonition title lorem ipsum .. .. admonition:: generic admonition title lorem ipsum .. Specific admonitions -------------------- Specific admonitions: :dudir:`hint`, :dudir:`note`, :dudir:`tip` :dudir:`attention`, :dudir:`caution`, :dudir:`danger`, :dudir:`error`, , :dudir:`important`, and :dudir:`warning` . .. code:: reST .. hint:: lorem ipsum .. .. note:: lorem ipsum .. .. warning:: lorem ipsum .. .. hint:: lorem ipsum .. .. note:: lorem ipsum .. .. tip:: lorem ipsum .. .. attention:: lorem ipsum .. .. caution:: lorem ipsum .. .. danger:: lorem ipsum .. .. important:: lorem ipsum .. .. error:: lorem ipsum .. .. warning:: lorem ipsum .. Tables ====== .. sidebar:: Nested tables Nested tables are ugly! Not all builder support nested tables, don't use them! ASCII-art tables like :ref:`reST simple table` and :ref:`reST grid table` might be comfortable for readers of the text-files, but they have huge disadvantages in the creation and modifying. First, they are hard to edit. Think about adding a row or a column to a ASCII-art table or adding a paragraph in a cell, it is a nightmare on big tables. .. sidebar:: List tables For meaningful patch and diff use :ref:`reST flat table`. Second the diff of modifying ASCII-art tables is not meaningful, e.g. widening a cell generates a diff in which also changes are included, which are only ascribable to the ASCII-art. Anyway, if you prefer ASCII-art for any reason, here are some helpers: * `Emacs Table Mode`_ * `Online Tables Generator`_ .. _reST simple table: Simple tables ------------- :duref:`Simple tables ` allow *colspan* but not *rowspan*. If your table need some metadata (e.g. a title) you need to add the ``.. table:: directive`` :dudir:`(ref) ` in front and place the table in its body: .. code:: reST .. table:: foo gate truth table :widths: grid :align: left ====== ====== ====== Inputs Output ------------- ------ A B A or B ====== ====== ====== False -------------------- True -------------------- True False True (foo) ------ ------ ------ False True (foo) ====== ============= .. admonition:: Simple ASCII table :class: rst-example .. table:: foo gate truth table :widths: grid :align: left ====== ====== ====== Inputs Output ------------- ------ A B A or B ====== ====== ====== False -------------------- True -------------------- True False True (foo) ------ ------ ------ False True (foo) ====== ============= .. _reST grid table: Grid tables ----------- :duref:`Grid tables ` allow colspan *colspan* and *rowspan*: .. code:: reST .. table:: grid table example :widths: 1 1 5 +------------+------------+-----------+ | Header 1 | Header 2 | Header 3 | +============+============+===========+ | body row 1 | column 2 | column 3 | +------------+------------+-----------+ | body row 2 | Cells may span columns.| +------------+------------+-----------+ | body row 3 | Cells may | - Cells | +------------+ span rows. | - contain | | body row 4 | | - blocks. | +------------+------------+-----------+ .. admonition:: ASCII grid table :class: rst-example .. table:: grid table example :widths: 1 1 5 +------------+------------+-----------+ | Header 1 | Header 2 | Header 3 | +============+============+===========+ | body row 1 | column 2 | column 3 | +------------+------------+-----------+ | body row 2 | Cells may span columns.| +------------+------------+-----------+ | body row 3 | Cells may | - Cells | +------------+ span rows. | - contain | | body row 4 | | - blocks. | +------------+------------+-----------+ .. _reST flat table: flat-table ---------- The ``flat-table`` is a further developed variant of the :ref:`list tables `. It is a double-stage list similar to the :dudir:`list-table` with some additional features: column-span: ``cspan`` with the role ``cspan`` a cell can be extended through additional columns row-span: ``rspan`` with the role ``rspan`` a cell can be extended through additional rows auto-span: spans rightmost cell of a table row over the missing cells on the right side of that table-row. With Option ``:fill-cells:`` this behavior can changed from *auto span* to *auto fill*, which automatically inserts (empty) cells instead of spanning the last cell. options: :header-rows: [int] count of header rows :stub-columns: [int] count of stub columns :widths: [[int] [int] ... ] widths of columns :fill-cells: instead of auto-span missing cells, insert missing cells roles: :cspan: [int] additional columns (*morecols*) :rspan: [int] additional rows (*morerows*) The example below shows how to use this markup. The first level of the staged list is the *table-row*. In the *table-row* there is only one markup allowed, the list of the cells in this *table-row*. Exception are *comments* ( ``..`` ) and *targets* (e.g. a ref to :ref:`row 2 of table's body `). .. code:: reST .. flat-table:: ``flat-table`` example :header-rows: 2 :stub-columns: 1 :widths: 1 1 1 1 2 * - :rspan:`1` head / stub - :cspan:`3` head 1.1-4 * - head 2.1 - head 2.2 - head 2.3 - head 2.4 * .. row body 1 / this is a comment - row 1 - :rspan:`2` cell 1-3.1 - cell 1.2 - cell 1.3 - cell 1.4 * .. Comments and targets are allowed on *table-row* stage. .. _`row body 2`: - row 2 - cell 2.2 - :rspan:`1` :cspan:`1` cell 2.3 with a span over * col 3-4 & * row 2-3 * - row 3 - cell 3.2 * - row 4 - cell 4.1 - cell 4.2 - cell 4.3 - cell 4.4 * - row 5 - cell 5.1 with automatic span to right end * - row 6 - cell 6.1 - .. .. admonition:: List table :class: rst-example .. flat-table:: ``flat-table`` example :header-rows: 2 :stub-columns: 1 :widths: 1 1 1 1 2 * - :rspan:`1` head / stub - :cspan:`3` head 1.1-4 * - head 2.1 - head 2.2 - head 2.3 - head 2.4 * .. row body 1 / this is a comment - row 1 - :rspan:`2` cell 1-3.1 - cell 1.2 - cell 1.3 - cell 1.4 * .. Comments and targets are allowed on *table-row* stage. .. _`row body 2`: - row 2 - cell 2.2 - :rspan:`1` :cspan:`1` cell 2.3 with a span over * col 3-4 & * row 2-3 * - row 3 - cell 3.2 * - row 4 - cell 4.1 - cell 4.2 - cell 4.3 - cell 4.4 * - row 5 - cell 5.1 with automatic span to right end * - row 6 - cell 6.1 - .. CSV table --------- CSV table might be the choice if you want to include CSV-data from a outstanding (build) process into your documentation. .. code:: reST .. csv-table:: CSV table example :header: .. , Column 1, Column 2 :widths: 2 5 5 :stub-columns: 1 :file: csv_table.txt Content of file ``csv_table.txt``: .. literalinclude:: csv_table.txt .. admonition:: CSV table :class: rst-example .. csv-table:: CSV table example :header: .. , Column 1, Column 2 :widths: 3 5 5 :stub-columns: 1 :file: csv_table.txt Templating ========== .. sidebar:: Build environment All *generic-doc* tasks are running in the :ref:`make install`. Templating is suitable for documentation which is created generic at the build time. The sphinx-jinja_ extension evaluates jinja_ templates in the :ref:`make install` (with SearXNG modules installed). We use this e.g. to build chapter: :ref:`configured engines`. Below the jinja directive from the :origin:`docs/admin/engines.rst` is shown: .. literalinclude:: ../user/configured_engines.rst :language: reST :start-after: .. _configured engines: The context for the template is selected in the line ``.. jinja:: searx``. In sphinx's build configuration (:origin:`docs/conf.py`) the ``searx`` context contains the ``engines`` and ``plugins``. .. code:: py import searx.search import searx.engines import searx.plugins searx.search.initialize() jinja_contexts = { 'searx': { 'engines': searx.engines.engines, 'plugins': searx.plugins.plugins }, } Tabbed views ============ .. _sphinx-tabs: https://github.com/djungelorm/sphinx-tabs .. _basic-tabs: https://github.com/djungelorm/sphinx-tabs#basic-tabs .. _group-tabs: https://github.com/djungelorm/sphinx-tabs#group-tabs .. _code-tabs: https://github.com/djungelorm/sphinx-tabs#code-tabs With `sphinx-tabs`_ extension we have *tabbed views*. To provide installation instructions with one tab per distribution we use the `group-tabs`_ directive, others are basic-tabs_ and code-tabs_. Below a *group-tab* example from :ref:`docs build` is shown: .. literalinclude:: ../admin/buildhosts.rst :language: reST :start-after: .. SNIP sh lint requirements :end-before: .. SNAP sh lint requirements .. _math: Math equations ============== .. _Mathematics: https://en.wikibooks.org/wiki/LaTeX/Mathematics .. _amsmath user guide: http://vesta.informatik.rwth-aachen.de/ftp/pub/mirror/ctan/macros/latex/required/amsmath/amsldoc.pdf .. sidebar:: About LaTeX - `amsmath user guide`_ - Mathematics_ - :ref:`docs build` The input language for mathematics is LaTeX markup using the :ctan:`amsmath` package. To embed LaTeX markup in reST documents, use role :rst:role:`:math: ` for inline and directive :rst:dir:`.. math:: ` for block markup. .. code:: reST In :math:numref:`schroedinger general` the time-dependent Schrödinger equation is shown. .. math:: :label: schroedinger general \mathrm{i}\hbar\dfrac{\partial}{\partial t} |\,\psi (t) \rangle = \hat{H} |\,\psi (t) \rangle. .. admonition:: LaTeX math equation :class: rst-example In :math:numref:`schroedinger general` the time-dependent Schrödinger equation is shown. .. math:: :label: schroedinger general \mathrm{i}\hbar\dfrac{\partial}{\partial t} |\,\psi (t) \rangle = \hat{H} |\,\psi (t) \rangle. The next example shows the difference of ``\tfrac`` (*textstyle*) and ``\dfrac`` (*displaystyle*) used in a inline markup or another fraction. .. code:: reST ``\tfrac`` **inline example** :math:`\tfrac{\tfrac{1}{x}+\tfrac{1}{y}}{y-z}` ``\dfrac`` **inline example** :math:`\dfrac{\dfrac{1}{x}+\dfrac{1}{y}}{y-z}` .. admonition:: Line spacing :class: rst-example Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. ... ``\tfrac`` **inline example** :math:`\tfrac{\tfrac{1}{x}+\tfrac{1}{y}}{y-z}` At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. ... ``\tfrac`` **inline example** :math:`\dfrac{\dfrac{1}{x}+\dfrac{1}{y}}{y-z}` At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. .. _KISS: https://en.wikipedia.org/wiki/KISS_principle .. _readability: https://docs.python-guide.org/writing/style/ .. _Sphinx-Primer: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html .. _reST: https://docutils.sourceforge.io/rst.html .. _Sphinx Roles: https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html .. _Sphinx: https://www.sphinx-doc.org .. _`sphinx-doc FAQ`: https://www.sphinx-doc.org/en/stable/faq.html .. _Sphinx markup constructs: https://www.sphinx-doc.org/en/stable/markup/index.html .. _`sphinx cross references`: https://www.sphinx-doc.org/en/stable/markup/inline.html#cross-referencing-arbitrary-locations .. _sphinx.ext.extlinks: https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html .. _intersphinx: https://www.sphinx-doc.org/en/stable/ext/intersphinx.html .. _sphinx config: https://www.sphinx-doc.org/en/stable/config.html .. _Sphinx's autodoc: https://www.sphinx-doc.org/en/stable/ext/autodoc.html .. _Sphinx's Python domain: https://www.sphinx-doc.org/en/stable/domains.html#the-python-domain .. _Sphinx's C domain: https://www.sphinx-doc.org/en/stable/domains.html#cross-referencing-c-constructs .. _doctree: https://www.sphinx-doc.org/en/master/extdev/tutorial.html?highlight=doctree#build-phases .. _docutils: http://docutils.sourceforge.net/docs/index.html .. _docutils FAQ: http://docutils.sourceforge.net/FAQ.html .. _linuxdoc: https://return42.github.io/linuxdoc .. _jinja: https://jinja.palletsprojects.com/ .. _sphinx-jinja: https://github.com/tardyp/sphinx-jinja .. _SVG: https://www.w3.org/TR/SVG11/expanded-toc.html .. _DOT: https://graphviz.gitlab.io/_pages/doc/info/lang.html .. _`Graphviz's dot`: https://graphviz.gitlab.io/_pages/pdf/dotguide.pdf .. _Graphviz: https://graphviz.gitlab.io .. _ImageMagick: https://www.imagemagick.org .. _`Emacs Table Mode`: https://www.emacswiki.org/emacs/TableMode .. _`Online Tables Generator`: https://www.tablesgenerator.com/text_tables .. _`OASIS XML Exchange Table Model`: https://www.oasis-open.org/specs/tm9901.html --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/result_types/answer.rst # Path: docs/dev/result_types/answer.rst .. _result_types.answer: ============== Answer Results ============== The :ref:`area answer results` is an area in which additional information can be displayed. .. automodule:: searx.result_types.answer --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/result_types/base_result.rst # Path: docs/dev/result_types/base_result.rst ====== Result ====== .. automodule:: searx.result_types._base --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/result_types/correction.rst # Path: docs/dev/result_types/correction.rst .. _result_types.corrections: ================== Correction Results ================== .. hint:: There is still no typing for these result items. The templates can be used as orientation until the final typing is complete. The :ref:`area corrections results` shows the user alternative search terms. A result of this type is a very simple dictionary with only one key/value pair .. code:: python {"correction" : "lorem ipsum .."} From this simple dict another dict is build up: .. code:: python # use RawTextQuery to get the corrections URLs with the same bang {"url" : "!bang lorem ipsum ..", "title": "lorem ipsum .." } and used in the template :origin:`corrections.html `: title : :py:class:`str` Corrected search term. url : :py:class:`str` Not really an URL, its the value to insert in a HTML form for a SearXNG query. --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/result_types/index.rst # Path: docs/dev/result_types/index.rst .. _result types: ============ Result Types ============ To understand the typification of the results, let's take a brief look at the structure of SearXNG .. At its core, SearXNG is nothing more than an aggregator that aggregates the results from various sources, renders them via templates and displays them to the user. The **sources** can be: 1. :ref:`engines ` 2. :ref:`plugins ` 3. :ref:`answerers ` The sources provide the results, which are displayed in different **areas** depending on the type of result. The areas are: .. _area main results: :ref:`area main results
` It is the main area in which -- as is typical for search engines -- the results that a search engine has found for the search term are displayed. .. _area answer results: :ref:`area answers ` This area displays short answers that could be found for the search term. .. _area info box: :ref:`area info box ` An area in which additional information can be displayed, e.g. excerpts from wikipedia or other sources such as maps. .. _area suggestions results: :ref:`area suggestions ` Suggestions for alternative search terms can be found in this area. These can be clicked on and a search is carried out with these search terms. .. _area corrections results: :ref:`area corrections ` Results in this area are like the suggestion of alternative search terms, which usually result from spelling corrections At this point it is important to note that all **sources** can contribute results to all of the areas mentioned above. In most cases, however, the :ref:`engines ` will fill the *main results* and the :ref:`answerers ` will generally provide the contributions for the *answer* area. Not necessary to mention here but for a better understanding: the plugins can also filter out or change results from the main results area (e.g. the URL of the link). The result items are organized in the :py:obj:`results.ResultContainer` and after all sources have delivered their results, this container is passed to the templating to build a HTML output. The output is usually HTML, but it is also possible to output the result lists as JSON or RSS feed. Thats quite all we need to know before we dive into typification of result items. .. hint:: Typification of result items: we are at the very first beginng! The first thing we have to realize is that there is no typification of the result items so far, we have to build it up first .. and that is quite a big task, which we will only be able to accomplish gradually. The foundation for the typeless results was laid back in 2013 in the very first commit :commit:`ae9fb1d7d`, and the principle has not changed since then. At the time, the approach was perfectly adequate, but we have since evolved and the demands on SearXNG increase with every feature request. **Motivation:** in the meantime, it has become very difficult to develop new features that require structural changes and it is especially hard for newcomers to find their way in this typeless world. As long as the results are only simple key/value dictionaries, it is not even possible for the IDEs to support the application developer in his work. **Planning:** The procedure for subsequent typing will have to be based on the circumstances .. .. attention:: As long as there is no type defined for a kind of result the HTML template specify what the properties of a type are. In this sense, you will either find a type definition here in the documentation or, if this does not yet exist, a description of the HTML template. .. toctree:: :maxdepth: 2 base_result main_result answer correction suggestion infobox --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/result_types/infobox.rst # Path: docs/dev/result_types/infobox.rst .. _result_types.infobox: =============== Infobox Results =============== .. hint:: There is still no typing for these result items. The templates can be used as orientation until the final typing is complete. The :ref:`area info box` is an area where addtional infos shown to the user. Fields used in the :origin:`infobox.html `: img_src: :py:class:`str` URL of a image or thumbnail that is displayed in the infobox. infobox: :py:class:`str` Title of the info box. content: :py:class:`str` Text of the info box. The infobox has additional subsections for *attributes*, *urls* and *relatedTopics*: attributes: :py:class:`List `\ [\ :py:class:`dict`\ ] A list of attributes. An *attribute* is a dictionary with keys: - label :py:class:`str`: (mandatory) - value :py:class:`str`: (mandatory) - image :py:class:`List `\ [\ :py:class:`dict`\ ] (optional) A list of images. An *image* is a dictionary with keys: - src :py:class:`str`: URL of an image/thumbnail (mandatory) - alt :py:class:`str`: alternative text for the image (mandatory) urls: :py:class:`List `\ [\ :py:class:`dict`\ ] A list of links. An *link* is a dictionary with keys: - url :py:class:`str`: URL of the link (mandatory) - title :py:class:`str`: Title of the link (mandatory) relatedTopics: :py:class:`List `\ [\ :py:class:`dict`\ ] A list of topics. An *topic* is a dictionary with keys: - name: :py:class:`str`: (mandatory) - suggestions: :py:class:`List `\ [\ :py:class:`dict`\ ] (optional) A list of suggestions. A *suggestion* is simple dictionary with just one key/value pair: - suggestion: :py:class:`str`: suggested search term (mandatory) --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/result_types/main/code.rst # Path: docs/dev/result_types/main/code.rst .. _result_types.code: ============ Code Results ============ .. automodule:: searx.result_types.code --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/result_types/main/file.rst # Path: docs/dev/result_types/main/file.rst .. _result_types.file: ============ File Results ============ .. automodule:: searx.result_types.file --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/result_types/main/keyvalue.rst # Path: docs/dev/result_types/main/keyvalue.rst .. _result_types.keyvalue: ================= Key-Value Results ================= .. automodule:: searx.result_types.keyvalue --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/result_types/main/mainresult.rst # Path: docs/dev/result_types/main/mainresult.rst .. _result_types.mainresult: .. autoclass:: searx.result_types._base.MainResult :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/result_types/main/paper.rst # Path: docs/dev/result_types/main/paper.rst .. _result_types.paper: ============= Paper Results ============= .. automodule:: searx.result_types.paper --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/result_types/main_result.rst # Path: docs/dev/result_types/main_result.rst .. _main search results: =================== Main Search Results =================== In the :ref:`area main results` the results that a search engine has found for the search term are displayed. There is still no typing for all items in the :ref:`main result list`. The following types have been implemented so far .. .. toctree:: :maxdepth: 2 main/mainresult main/keyvalue main/code main/paper main/file The :ref:`LegacyResult ` is used internally for the results that have not yet been typed. The templates can be used as orientation until the final typing is complete. - :ref:`template default` / :py:obj:`Result` - :ref:`template images` - :ref:`template videos` - :ref:`template torrent` - :ref:`template map` - :ref:`template packages` - :ref:`template products` --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/result_types/suggestion.rst # Path: docs/dev/result_types/suggestion.rst .. _result_types.suggestion: ================== Suggestion Results ================== .. hint:: There is still no typing for these result items. The templates can be used as orientation until the final typing is complete. The :ref:`area suggestions results` shows the user alternative search terms. A result of this type is a very simple dictionary with only one key/value pair .. code:: python {"suggestion" : "lorem ipsum .."} From this simple dict another dict is build up: .. code:: python {"url" : "!bang lorem ipsum ..", "title": "lorem ipsum" } and used in the template :origin:`suggestions.html `: .. code:: python # use RawTextQuery to get the suggestion URLs with the same bang {"url" : "!bang lorem ipsum ..", "title": "lorem ipsum" } title : :py:class:`str` Suggested search term url : :py:class:`str` Not really an URL, its the value to insert in a HTML form for a SearXNG query. --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/search_api.rst # Path: docs/dev/search_api.rst .. _search API: ========== Search API ========== SearXNG supports querying via a simple HTTP API. Two endpoints, ``/`` and ``/search``, are supported for both GET and POST methods. The GET method expects parameters as URL query parameters, while the POST method expects parameters as form data. If you want to consume the results as JSON, CSV, or RSS, you need to set the ``format`` parameter accordingly. Supported formats are defined in ``settings.yml``, under the ``search`` section. Requesting an unset format will return a 403 Forbidden error. Be aware that many public instances have these formats disabled. Endpoints: ``GET /`` ``GET /search`` ``POST /`` ``POST /search`` example cURL calls: .. code-block:: bash curl 'https://searx.example.org/search?q=searxng&format=json' curl -X POST 'https://searx.example.org/search' -d 'q=searxng&format=csv' curl -L -X POST -d 'q=searxng&format=json' 'https://searx.example.org/' Parameters ========== .. sidebar:: Further reading .. - :ref:`engines-dev` - :ref:`settings.yml` - :ref:`configured engines` ``q`` : required The search query. This string is passed to external search services. Thus, SearXNG supports syntax of each search service. For example, ``site:github.com SearXNG`` is a valid query for Google. However, if simply the query above is passed to any search engine which does not filter its results based on this syntax, you might not get the results you wanted. See more at :ref:`search-syntax` ``categories`` : optional Comma separated list, specifies the active search categories (see :ref:`configured engines`) ``engines`` : optional Comma separated list, specifies the active search engines (see :ref:`configured engines`). ``language`` : default from :ref:`settings search` Code of the language. ``pageno`` : default ``1`` Search page number. ``time_range`` : optional [ ``day``, ``month``, ``year`` ] Time range of search for engines which support it. See if an engine supports time range search in the preferences page of an instance. ``format`` : optional [ ``json``, ``csv``, ``rss`` ] Output format of results. Format needs to be activated in :ref:`settings search`. ``results_on_new_tab`` : default ``0`` [ ``0``, ``1`` ] Open search results on new tab. ``image_proxy`` : default from :ref:`settings server` [ ``True``, ``False`` ] Proxy image results through SearXNG. ``autocomplete`` : default from :ref:`settings search` [ ``google``, ``dbpedia``, ``duckduckgo``, ``mwmbl``, ``startpage``, ``wikipedia``, ``stract``, ``swisscows``, ``qwant`` ] Service which completes words as you type. ``safesearch`` : default from :ref:`settings search` [ ``0``, ``1``, ``2`` ] Filter search results of engines which support safe search. See if an engine supports safe search in the preferences page of an instance. ``theme`` : default ``simple`` [ ``simple`` ] Theme of instance. Please note, available themes depend on an instance. It is possible that an instance administrator deleted, created or renamed themes on their instance. See the available options in the preferences page of the instance. ``enabled_plugins`` : optional List of enabled plugins. :default: ``Hash_plugin``, ``Self_Information``, ``Tracker_URL_remover``, ``Ahmia_blacklist`` :values: .. enabled by default ``Hash_plugin``, ``Self_Information``, ``Tracker_URL_remover``, ``Ahmia_blacklist``, .. disabled by default ``Hostnames_plugin``, ``Open_Access_DOI_rewrite``, ``Vim-like_hotkeys``, ``Tor_check_plugin`` ``disabled_plugins``: optional List of disabled plugins. :default: ``Hostnames_plugin``, ``Open_Access_DOI_rewrite``, ``Vim-like_hotkeys``, ``Tor_check_plugin`` :values: see values from ``enabled_plugins`` ``enabled_engines`` : optional : *all* :origin:`engines ` List of enabled engines. ``disabled_engines`` : optional : *all* :origin:`engines ` List of disabled engines. --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/searxng_extra/index.rst # Path: docs/dev/searxng_extra/index.rst .. _searxng_extra: ============================= Tooling box ``searxng_extra`` ============================= In the folder :origin:`searxng_extra/` we maintain some tools useful for CI and developers. .. toctree:: :maxdepth: 2 update --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/searxng_extra/update.rst # Path: docs/dev/searxng_extra/update.rst ========================= ``searxng_extra/update/`` ========================= :origin:`[source] ` Scripts to update static data in :origin:`searx/data/` .. _update_ahmia_blacklist.py: ``update_ahmia_blacklist.py`` ============================= :origin:`[source] ` .. automodule:: searxng_extra.update.update_ahmia_blacklist :members: ``update_currencies.py`` ======================== :origin:`[source] ` .. automodule:: searxng_extra.update.update_currencies :members: ``update_engine_descriptions.py`` ================================= :origin:`[source] ` .. automodule:: searxng_extra.update.update_engine_descriptions :members: ``update_external_bangs.py`` ============================ :origin:`[source] ` .. automodule:: searxng_extra.update.update_external_bangs :members: ``update_firefox_version.py`` ============================= :origin:`[source] ` .. automodule:: searxng_extra.update.update_firefox_version :members: ``update_engine_traits.py`` =========================== :origin:`[source] ` .. automodule:: searxng_extra.update.update_engine_traits :members: .. _update_osm_keys_tags.py: ``update_osm_keys_tags.py`` =========================== :origin:`[source] ` .. automodule:: searxng_extra.update.update_osm_keys_tags :members: ``update_pygments.py`` ====================== :origin:`[source] ` .. automodule:: searxng_extra.update.update_pygments :members: .. _update_locales.py: ``update_locales.py`` ===================== :origin:`[source] ` .. automodule:: searxng_extra.update.update_locales :members: ``update_wikidata_units.py`` ============================ :origin:`[source] ` .. automodule:: searxng_extra.update.update_wikidata_units :members: --- # SearXNG Documentation # Source: https://raw.githubusercontent.com/searxng/searxng/master/docs/dev/templates.rst # Path: docs/dev/templates.rst .. _simple theme templates: ====================== Simple Theme Templates ====================== The simple template is complex, it consists of many different elements and also uses macros and include statements. The following is a rough overview that we would like to give the developer at hand, details must still be taken from the :origin:`sources `. A :ref:`result item ` can be of different media types. The media type of a result is defined by the :py:obj:`result_type.Result.template`. To set another media-type as :ref:`template default`, the field ``template`` in the result item must be set to the desired type. .. contents:: Contents :depth: 2 :local: :backlinks: entry .. _result template macros: Result template macros ====================== .. _macro result_header: ``result_header`` ----------------- Execpt ``image.html`` and some others this macro is used in nearly all result types in the :ref:`main result list`. Fields used in the template :origin:`macro result_header `: url : :py:class:`str` Link URL of the result item. title : :py:class:`str` Link title of the result item. img_src, thumbnail : :py:class:`str` URL of a image or thumbnail that is displayed in the result item. .. _macro result_sub_header: ``result_sub_header`` --------------------- Execpt ``image.html`` and some others this macro is used in nearly all result types in the :ref:`main result list`. Fields used in the template :origin:`macro result_sub_header `: publishedDate : :py:obj:`datetime.datetime` The date on which the object was published. length: :py:obj:`datetime.timedelta` Playing duration in seconds. views: :py:class:`str` View count in humanized number format. author : :py:class:`str` Author of the title. metadata : :py:class:`str` Miscellaneous metadata. .. _engine_data: ``engine_data_form`` -------------------- The ``engine_data_form`` macro is used in :origin:`results,html ` in a HTML ``
`` element. The intention of this macro is to pass data of a engine from one :py:obj:`response ` to the :py:obj:`searx.search.SearchQuery` of the next :py:obj:`request `. To pass data, engine's response handler can append result items of typ ``engine_data``. This is by example used to pass a token from the response to the next request: .. code:: python def response(resp): ... results.append({ 'engine_data': token, 'key': 'next_page_token', }) ... return results def request(query, params): page_token = params['engine_data'].get('next_page_token') .. _main result list: Main Result List ================ The **media types** of the **main result type** are the template files in the :origin:`result_templates `. .. _template default: ``default.html`` ---------------- Displays result fields from: - :ref:`macro result_header` and - :ref:`macro result_sub_header` Additional fields used in the :origin:`default.html `: content : :py:class:`str` General text of the result item. iframe_src : :py:class:`str` URL of an embedded ``