Docker Deployment

zhenghaozNovember 20, 2022About 5 min

Docker Deployment

Docker Compose or Docker Swarm is a good choice for managing small size microservice clusters. There are Docker images provided by the Gorse official:

Docker ImageVersionImage SizePulls
gorse-masteropen in new windowDocker Image Version (tag latest semver)gorse-masterDocker Pulls
gorse-serveropen in new windowDocker Image Version (tag latest semver)gorse-serverDocker Pulls
gorse-workeropen in new windowDocker Image Version (tag latest semver)gorse-workerDocker Pulls
gorse-in-oneopen in new windowDocker Image Version (tag latest semver)gorse-in-oneDocker Pulls

Tips

These images tagged with the nightly tag are built from the master branch. The tag should be fixed to a specified version in production.

Gorse-in-one on Docker

For single node deployment, the gorse-in-one Docker image can be used. There is an example of Gorse-in-one usage in docker-compose.yaml.

version: "3"
services:
  gorse:
    image: zhenghaoz/gorse-in-one
    restart: unless-stopped
    ports:
      - 8086:8086   # gRPC port
      - 8088:8088   # HTTP port
    environment:
      # Use Redis as cache storage backend.
      GORSE_CACHE_STORE: redis://redis:6379
      # Use MySQL as data storage backend.
      GORSE_DATA_STORE: mysql://gorse:gorse_pass@tcp(mysql:3306)/gorse?parseTime=true
    command: >
      -c /etc/gorse/config.toml 
      --log-path /var/log/gorse/master.log 
      --cache-path /var/lib/gorse/master_cache.data
    volumes:
      # Mount the configuration file.
      - ./config.toml:/etc/gorse/config.toml
    depends_on:
      - redis
      - mysql

Read Binary Deployment for the detailed information about command line flags of gorse-in-one.

Full Example
  1. Create docker-compose.yaml with the following content:
version: "3"
services:
  redis:
    image: redis
    restart: unless-stopped
    ports:
      - 6379:6379

  mysql:
    image: mysql/mysql-server
    restart: unless-stopped
    ports:
      - 3306:3306
    environment:
      MYSQL_ROOT_PASSWORD: root_pass
      MYSQL_DATABASE: gorse
      MYSQL_USER: gorse
      MYSQL_PASSWORD: gorse_pass

  gorse:
    image: zhenghaoz/gorse-in-one
    restart: unless-stopped
    ports:
      - 8086:8086   # gRPC port
      - 8088:8088   # HTTP port
    environment:
      # Use Redis as cache storage backend.
      GORSE_CACHE_STORE: redis://redis:6379
      # Use MySQL as data storage backend.
      GORSE_DATA_STORE: mysql://gorse:gorse_pass@tcp(mysql:3306)/gorse?parseTime=true
    command: >
      -c /etc/gorse/config.toml 
      --log-path /var/log/gorse/master.log 
      --cache-path /var/lib/gorse/master_cache.data
    volumes:
      # Mount the configuration file.
      - ./config.toml:/etc/gorse/config.toml
    depends_on:
      - redis
      - mysql
  1. Create a configuration file config.toml in the same directory based on config.toml.templateopen in new window.

  2. Start all services:

docker-compose up -d

Gorse Cluster on Docker

Gorse worker nodes and server nodes have horizontal scalability. Increase the number of server nodes to improve online recommendation throughput, while increase the number of worker nodes to improve offline recommendation throughput.

There is an example for a Gorse cluster with one master node, one server node and one worker node:

version: "3"
services:
  worker:
    image: zhenghaoz/gorse-worker
    restart: unless-stopped
    ports:
      - 8089:8089 # Prometheus metrics export port.
    command: >
      --master-host master --master-port 8086 
      --http-host 0.0.0.0 --http-port 8089
      --log-path /var/log/gorse/worker.log 
      --cache-path /var/lib/gorse/worker_cache.data
    volumes:
      - gorse_log:/var/log/gorse    # Mount log files in volume.
      - worker_data:/var/lib/gorse  # Mount cache files in volume.
    depends_on:
      - master

  server:
    image: zhenghaoz/gorse-server
    restart: unless-stopped
    ports:
      - 8087:8087 # RESTful APIs and Prometheus metrics export port.
    command: >
      --master-host master --master-port 8086 
      --http-host 0.0.0.0 --http-port 8087
      --log-path /var/log/gorse/server.log 
      --cache-path /var/lib/gorse/server_cache.data
    volumes:
      - gorse_log:/var/log/gorse    # Mount log files in volume.
      - server_data:/var/lib/gorse  # Mount cache files in volume.
    depends_on:
      - master

  master:
    image: zhenghaoz/gorse-master
    restart: unless-stopped
    ports:
      - 8086:8086 # HTTP port
      - 8088:8088 # gRPC port
    environment:
      # Use Redis as cache storage backend.
      GORSE_CACHE_STORE: redis://redis:6379
      # Use MySQL as data storage backend.
      GORSE_DATA_STORE: mysql://gorse:gorse_pass@tcp(mysql:3306)/gorse
    command: >
      -c /etc/gorse/config.toml 
      --log-path /var/log/gorse/master.log 
      --cache-path /var/lib/gorse/master_cache.data
    volumes:
      # Mount the configuration file.
      - ./config/config.toml.template:/etc/gorse/config.toml
      # Mount log files in volume.
      - gorse_log:/var/log/gorse  
      # Mount cache files in volume.
      - master_data:/var/lib/gorse
    depends_on:
      - redis
      - mysql

volumes:
  worker_data:
  server_data:
  master_data:
  gorse_log:
Full Example
  1. Create docker-compose.yaml with the following content:
version: "3"
services:
  redis:
    image: redis
    restart: unless-stopped
    ports:
      - 6379:6379

  mysql:
    image: mysql/mysql-server
    restart: unless-stopped
    ports:
      - 3306:3306
    environment:
      MYSQL_ROOT_PASSWORD: root_pass
      MYSQL_DATABASE: gorse
      MYSQL_USER: gorse
      MYSQL_PASSWORD: gorse_pass
    volumes:
      - mysql_data:/var/lib/mysql

  # postgres:
  #   image: postgres:10.0
  #   ports:
  #     - 5432:5432
  #   environment:
  #     POSTGRES_DB: gorse
  #     POSTGRES_USER: gorse
  #     POSTGRES_PASSWORD: gorse_pass
  #   volumes:
  #     - postgres_data:/var/lib/postgresql/data

  # mongo:
  #   image: mongo:4.0
  #   ports:
  #     - 27017:27017
  #   environment:
  #     MONGO_INITDB_DATABASE: gorse
  #     MONGO_INITDB_ROOT_USERNAME: root
  #     MONGO_INITDB_ROOT_PASSWORD: password
  #   volumes:
  #     - mongo_data:/data/db

  # clickhouse:
  #   image: yandex/clickhouse-server:21.10
  #   ports:
  #     - 8123:8123
  #   environment:
  #     CLICKHOUSE_DB: gorse
  #     CLICKHOUSE_USER: gorse
  #     CLICKHOUSE_PASSWORD: gorse_pass
  #   volumes:
  #     - clickhouse_data:/var/lib/clickhouse

  worker:
    image: zhenghaoz/gorse-worker
    restart: unless-stopped
    ports:
      - 8089:8089
    command: >
      --master-host master --master-port 8086 
      --http-host 0.0.0.0 --http-port 8089
      --log-path /var/log/gorse/worker.log 
      --cache-path /var/lib/gorse/worker_cache.data
    volumes:
      - gorse_log:/var/log/gorse
      - worker_data:/var/lib/gorse
    depends_on:
      - master

  server:
    image: zhenghaoz/gorse-server
    restart: unless-stopped
    ports:
      - 8087:8087
    command: >
      --master-host master --master-port 8086 
      --http-host 0.0.0.0 --http-port 8087
      --log-path /var/log/gorse/server.log 
      --cache-path /var/lib/gorse/server_cache.data
    volumes:
      - gorse_log:/var/log/gorse
      - server_data:/var/lib/gorse
    depends_on:
      - master

  master:
    image: zhenghaoz/gorse-master
    restart: unless-stopped
    ports:
      - 8086:8086
      - 8088:8088
    environment:
      GORSE_CACHE_STORE: redis://redis:6379
      GORSE_DATA_STORE: mysql://gorse:gorse_pass@tcp(mysql:3306)/gorse
      # GORSE_DATA_STORE: postgres://gorse:gorse_pass@postgres/gorse?sslmode=disable
      # GORSE_DATA_STORE: mongodb://root:password@mongo:27017/gorse?authSource=admin&connect=direct
      # GORSE_DATA_STORE: clickhouse://gorse:gorse_pass@clickhouse:8123/gorse
    command: >
      -c /etc/gorse/config.toml 
      --log-path /var/log/gorse/master.log 
      --cache-path /var/lib/gorse/master_cache.data
    volumes:
      - ./config/config.toml.template:/etc/gorse/config.toml
      - gorse_log:/var/log/gorse
      - master_data:/var/lib/gorse
    depends_on:
      - redis
      - mysql
      # - postgres
      # - mongo
      # - clickhouse

volumes:
  worker_data:
  server_data:
  master_data:
  gorse_log:
  mysql_data:
  # postgres_data:
  # mongo_data:
  # clickhouse_data:
  1. Create a configuration file config.toml in the same directory based on config.toml.templateopen in new window.

  2. Start all services:

docker-compose up -d

Command Line Flags

Command line flags of the master node:

FlagDefault ValueDescription
--cache-pathworker_cache.dataCache file path.
-c--configConfiguration file path.
--debugDebug log mode.
-h--helpHelp for gorse-master.
--log-pathLog file path.
-v--versionGorse version.

Command line flags of the server node:

FlagDefault ValueDescription
--cache-pathworker_cache.dataCache file path.
--debugDebug log mode.
-h--helpHelp for gorse-server.
--http-host127.0.0.1RESTful APIs and Prometheus
metrics export host
--http-port8087RESTful APIs and Prometheus
metrics export port
--log-pathLog file path.
--master-host127.0.0.1Master node host.
--master-port8086Master node port.
-v--versionGorse version.

Command line flags of the worker node:

FlagDefault ValueDescription
--cache-pathworker_cache.dataCache file path.
--debugDebug log mode.
-h--helpHelp for gorse-worker.
--http-host127.0.0.1Prometheus metrics export host.
--http-port8089Prometheus metrics export port.
-j--jobs1Number of working jobs.
--log-pathLog file path.
--master-host127.0.0.1Master node host.
--master-port8086Master node port
-v--versionGorse version.
Loading...