Create your own private search engine.

Hello, everyone! Welcome back to ITS Tech Time. Today, we’re diving into the world of private web searching.

To achieve complete privacy while browsing, your best bet is to opt for a hosted open-source search engine or take the reins by self-hosting one. Mainstream search engines like Google tend to retain logs and might contain third-party trackers that share your browsing data through cookies. As well as constantly popping up advertisements.

One route you can take is using an open-source search engine hosted by a trusted entity. For instance, at ITS Tech Time, we’ve set up search.itstechtime.com, our self-hosted search engine powered by SearXNG. Alternatively, you can opt to host your very own.

Stay tuned as I guide you through the surprisingly simple steps to create your personal search engine.

Setup Host

To kick off the setup, your first task is establishing a host machine where your search engine will operate. SearXNG operates within Docker, so any machine with an internet connection will do the job just fine.

The beauty here is the flexibility. You can utilize your current machine, that extra laptop sitting idly in the corner, or opt for a cloud-hosted solution.

For this demonstration, I’ll demonstrate the process using Digital Ocean by creating a droplet. As a new user, you can take advantage of this referral link and receive $200 in credit over 60 days: https://m.do.co/c/732fbd1c3e4e

Once on Digital Ocean, I’ll click ‘Create a Droplet.’ The default region usually suffices, and my preference leans towards the latest LTS version of Ubuntu. Opting for the basic shared CPU and regular SSD will typically cost around $6 per month—a pretty cost-effective choice for browsing security.

Install Docker

For Ubuntu, you first need to update your existing list of packages.

sudo apt update

Next, install prerequisite packages that will let apt use packages over HTTPS.

sudo apt install apt-transport-https ca-certificates curl software-properties-common

Then, add the GPG key for the official Docker repository to your system.

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Add the Docker repository to APT sources.

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"

Now, install Docker.

sudo apt install docker-ce

Optional, Add yourself to the docker group.

sudo usermod -aG docker <user>

Now install docker-compose as well.

apt install docker-compose

Install SearXNG

Change the directory to where you want to store searXNG. For us, we are going to store it in /usr/local

root@localhost:~# cd /usr/local

Now we will clone the git docker project for searxng

git clone https://github.com/searxng/searxng-docker.git

Change the directory into the searxng-docker directory you just cloned, and open the .env file.

root@localhost:/usr/local# cd searxng-docker
root@localhost:/usr/local/searxng-docker# ll
total 84
drwxr-xr-x  4 root root  4096 Dec 14 21:30 ./
drwxr-xr-x 11 root root  4096 Dec 14 19:39 ../
-rw-r--r--  1 root root  3101 Dec 14 19:39 Caddyfile
-rw-r--r--  1 root root  1367 Dec 14 21:30 docker-compose.yaml
-rw-r--r--  1 root root   330 Dec 14 20:13 .env
drwxr-xr-x  8 root root  4096 Dec 14 19:39 .git/
-rw-r--r--  1 root root    55 Dec 14 19:39 .gitignore
-rw-r--r--  1 root root 34523 Dec 14 19:39 LICENSE
-rw-r--r--  1 root root  3230 Dec 14 19:39 README.md
drwxr-xr-x  2 root root  4096 Dec 14 21:11 searxng/
-rw-r--r--  1 root root   407 Dec 14 21:15 searxng-docker.service.template
root@localhost:/usr/local/searxng-docker# nano .env
# By default listen on https://localhost
# To change this:
# * uncomment SEARXNG_HOSTNAME, and replace <host> by the SearXNG hostname
# * uncomment LETSENCRYPT_EMAIL, and replace <email> by your email (require to create a Let's Encrypt certificate)

# SEARXNG_HOSTNAME=<host>
# LETSENCRYPT_EMAIL=<email>

You can edit the .env file to set a hostname; if you have one, you want to direct it to the search engine. It’s completely optional, and because it varies depending on the host provider, we will not go over directing the hostname here.

If you’ve set a hostname, you can also set an email so that it will use Let Encrypt to set SSL on your search engine.

You need to set up a secret key to keep your search engine private. Run this command to generate the key and paste it into your settings for you.

root@localhost:/usr/local/searxng-docker# sed -i "s|ultrasecretkey|$(openssl rand -hex 32)|g" searxng/settings.yml

Edit docker yaml

This is optional, but I edited the docker-compose.yaml so that the containers would restart on their own if the server restarted. It is a yaml file so be careful of the spacing if you do edit it.

root@search:/usr/local/searxng-docker# cat docker-compose.yaml
version: "3.7"

services:
  caddy:
    container_name: caddy
    restart: unless-stopped
    image: caddy:2-alpine
    network_mode: host
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy-data:/data:rw
      - caddy-config:/config:rw
    environment:
      - SEARXNG_HOSTNAME=${SEARXNG_HOSTNAME:-http://localhost:80}
      - SEARXNG_TLS=${LETSENCRYPT_EMAIL:-internal}
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE

  redis:
    container_name: redis
    restart: unless-stopped
    image: docker.io/library/redis:alpine
    command: redis-server --save 30 1 --loglevel warning
    networks:
      - searxng
    volumes:
      - redis-data:/data
    cap_drop:
      - ALL
    cap_add:
      - SETGID
      - SETUID
      - DAC_OVERRIDE

  searxng:
    container_name: searxng
    restart: unless-stopped
    image: searxng/searxng:latest
    networks:
      - searxng
    ports:
      - "127.0.0.1:8080:8080"
    volumes:
      - ./searxng:/etc/searxng:rw
    environment:
      - SEARXNG_BASE_URL=https://${SEARXNG_HOSTNAME:-localhost}/
    cap_drop:
      - ALL
    cap_add:
      - CHOWN
      - SETGID
      - SETUID
    logging:
      driver: "json-file"
      options:
        max-size: "1m"
        max-file: "1"

networks:
  searxng:
    ipam:
      driver: default

volumes:
  caddy-data:
  caddy-config:
  redis-data:
root@search:/usr/local/searxng-docker#

Testing Search Engine

Run docker-compose to start the docker containers. Then, navigate to your IP address or hostname. If you get a time-out error, make sure the firewall on your server is allowing HTTP and HTTPS.

root@search:/usr/local/searxng-docker# sudo docker-compose up -d

Stop and Rebuild SearXNG

If you need to make changes to SearXNG. You can stop and remove all three containers at once by running this command.

root@search:/usr/local/searxng-docker# sudo docker-compose down

Extra Features

With everything down, it’s a good time to edit the settings. Change directory into the searxng directory below, and nano the settings.yml file.

root@search:/usr/local/searxng-docker# cd searxng/
root@search:/usr/local/searxng-docker/searxng# ls
limiter.toml  settings.yml  uwsgi.ini
root@search:/usr/local/searxng-docker/searxng# nano settings.yml
# see https://docs.searxng.org/admin/settings/settings.html#settings-use-default-settings
use_default_settings: true
server:
  # base_url is defined in the SEARXNG_BASE_URL environment variable, see .env and docker-compose.yml
  secret_key: "SUPPER SECRET KEY IS HERE"  # change this!
  limiter: true  # can be disabled for a private instance
  image_proxy: true
  method: "GET"
ui:
  static_use_hash: true
redis:
  url: redis://redis:6379/0
general:
  debug: false
  instance_name: "Search ITS Tech Time"
  contact_url: true
   mailto:itstechtime@skiff.com
  enable_metrics: true
search:
  autocomplete: "duckduckgo"
brand:
  public_instances: https://search.itstechtime.com

The default is perfectly fine to use. Their official documentation lists all the changes that can be made: https://docs.searxng.org/admin/index.html

We changed the method to GET to address problems with the back button on some websites. We added a general section and set the instance name, enabled metrics, and added auto-complete with ducking go.

server:
  # base_url is defined in the SEARXNG_BASE_URL environment variable, see .env and docker-compose.yml
  secret_key: "SUPPER SECRET KEY IS HERE"  # change this!
  limiter: true  # can be disabled for a private instance
  image_proxy: true

  method: "GET"

general:
  debug: false
  instance_name: "Search ITS Tech Time"
  contact_url: false
   # mailto:itstechtime@skiff.com
  enable_metrics: true
search:
  autocomplete: "duckduckgo"

Here’s a quick look at our metrics page at the time of this writing.

Setting up SearXNG is a journey to establish your private search haven. It’s about taking control of your online presence, ensuring your browsing remains yours alone. If you want to test drive our SearXNG to see how it works first, you can visit search.itstechtime.com

Summary

If you have any questions or comments, you can reach out through our contact page: Contact Us

If you found this video helpful, give it a thumbs up, and don’t forget to share it with anyone who might benefit from learning about online privacy.

Thanks for watching, and as always, stay curious and stay secure. See you in the next video!

Scroll to Top