Docker Deployment

Docker Compose or Docker Swarm is a good choice for managing small-size microservice clusters. There are Docker images provided by the Gorse official:

Docker Image	Version	Image Size	Pulls
gorse-master
gorse-server
gorse-worker
gorse-in-one

Nightly Version

These images tagged with the nightly tag are built from the master branch.

Windows Container

These images tagged with the *-windowsservercore tag are built for the Windows container.

Gorse-in-one on Docker

For single node deployment, the gorse-in-one Docker image can be used.

1. Create docker-compose.yaml with the following content:

MySQL

version: "3"
services:
  mysql:
    image: mysql/mysql-server
    restart: unless-stopped
    ports:
      - 3306:3306
    environment:
      MYSQL_ROOT_PASSWORD: root_pass
      MYSQL_DATABASE: gorse
      MYSQL_USER: gorse
      MYSQL_PASSWORD: gorse_pass
    volumes:
      - mysql_data:/var/lib/mysql

  gorse:
    image: zhenghaoz/gorse-in-one
    restart: unless-stopped
    ports:
      - 8086:8086   # gRPC port
      - 8088:8088   # HTTP port
    environment:
      # Use MySQL as cache storage backend.
      GORSE_CACHE_STORE: mysql://gorse:gorse_pass@tcp(mysql:3306)/gorse?parseTime=true
      # Use MySQL as data storage backend.
      GORSE_DATA_STORE: mysql://gorse:gorse_pass@tcp(mysql:3306)/gorse?parseTime=true
    command: >
      -c /etc/gorse/config.toml 
      --log-path /var/log/gorse/master.log 
      --cache-path /var/lib/gorse/master_cache.data
    volumes:
      # Mount the configuration file.
      - ./config.toml:/etc/gorse/config.toml
    depends_on:
      - mysql

volumes:
  mysql_data:

PostgreSQL

version: "3"
services:
  postgres:
    image: postgres:10.0
    restart: unless-stopped
    ports:
      - 5432:5432
    environment:
      POSTGRES_DB: gorse
      POSTGRES_USER: gorse
      POSTGRES_PASSWORD: gorse_pass
    volumes:
      - postgres_data:/var/lib/postgresql/data

  gorse:
    image: zhenghaoz/gorse-in-one
    restart: unless-stopped
    ports:
      - 8086:8086   # gRPC port
      - 8088:8088   # HTTP port
    environment:
      # Use PostgreSQL as cache storage backend.
      GORSE_CACHE_STORE: postgres://gorse:gorse_pass@postgres/gorse?sslmode=disable
      # Use PostgreSQL as data storage backend.
      GORSE_DATA_STORE: postgres://gorse:gorse_pass@postgres/gorse?sslmode=disable
    command: >
      -c /etc/gorse/config.toml 
      --log-path /var/log/gorse/master.log 
      --cache-path /var/lib/gorse/master_cache.data
    volumes:
      # Mount the configuration file.
      - ./config.toml:/etc/gorse/config.toml
    depends_on:
      - postgres

volumes:
  postgres_data:

MongoDB

version: "3"
services:
  mongo:
    image: mongo:4.0
    restart: unless-stopped
    ports:
      - 27017:27017
    environment:
      MONGO_INITDB_DATABASE: gorse
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: password
    volumes:
      - mongo_data:/data/db

  gorse:
    image: zhenghaoz/gorse-in-one
    restart: unless-stopped
    ports:
      - 8086:8086   # gRPC port
      - 8088:8088   # HTTP port
    environment:
      # Use MongoDB as cache storage backend.
      GORSE_CACHE_STORE: mongodb://root:password@mongo:27017/gorse?authSource=admin&connect=direct
      # Use MongoDB as data storage backend.
      GORSE_DATA_STORE: mongodb://root:password@mongo:27017/gorse?authSource=admin&connect=direct
    command: >
      -c /etc/gorse/config.toml 
      --log-path /var/log/gorse/master.log 
      --cache-path /var/lib/gorse/master_cache.data
    volumes:
      # Mount the configuration file.
      - ./config.toml:/etc/gorse/config.toml
    depends_on:
      - mongo

volumes:
  mongo_data:

Read Binary Deployment for detailed information about command line flags of gorse-in-one.

2. Create a configuration file config.toml in the same directory based on the configuration template.

3. Start all services:

docker-compose up -d

Gorse Cluster on Docker

Gorse worker nodes and server nodes have horizontal scalability. Increase the number of server nodes to improve online recommendation throughput, while increase the number of worker nodes to improve offline recommendation throughput.

There is an example of a Gorse cluster with one master node, one server node and one worker node:

1. Create docker-compose.yaml with the following content:

MySQL

version: "3"
services:
  mysql:
    image: mysql/mysql-server
    restart: unless-stopped
    ports:
      - 3306:3306
    environment:
      MYSQL_ROOT_PASSWORD: root_pass
      MYSQL_DATABASE: gorse
      MYSQL_USER: gorse
      MYSQL_PASSWORD: gorse_pass
    volumes:
      - mysql_data:/var/lib/mysql

  worker:
    image: zhenghaoz/gorse-worker
    restart: unless-stopped
    ports:
      - 8089:8089
    command: >
      --master-host master --master-port 8086 
      --http-host 0.0.0.0 --http-port 8089
      --log-path /var/log/gorse/worker.log 
      --cache-path /var/lib/gorse/worker_cache.data
    volumes:
      - gorse_log:/var/log/gorse
      - worker_data:/var/lib/gorse
    depends_on:
      - master

  server:
    image: zhenghaoz/gorse-server
    restart: unless-stopped
    ports:
      - 8087:8087
    command: >
      --master-host master --master-port 8086 
      --http-host 0.0.0.0 --http-port 8087
      --log-path /var/log/gorse/server.log 
      --cache-path /var/lib/gorse/server_cache.data
    volumes:
      - gorse_log:/var/log/gorse
      - server_data:/var/lib/gorse
    depends_on:
      - master

  master:
    image: zhenghaoz/gorse-master
    restart: unless-stopped
    ports:
      - 8086:8086
      - 8088:8088
    environment:
      GORSE_CACHE_STORE: mysql://gorse:gorse_pass@tcp(mysql:3306)/gorse
      GORSE_DATA_STORE: mysql://gorse:gorse_pass@tcp(mysql:3306)/gorse
    command: >
      -c /etc/gorse/config.toml 
      --log-path /var/log/gorse/master.log 
      --cache-path /var/lib/gorse/master_cache.data
    volumes:
      - ./config/config.toml:/etc/gorse/config.toml
      - gorse_log:/var/log/gorse
      - master_data:/var/lib/gorse
    depends_on:
      - mysql

volumes:
  worker_data:
  server_data:
  master_data:
  gorse_log:
  mysql_data:

PostgreSQL

version: "3"
services:
  postgres:
    image: postgres:10.0
    restart: unless-stopped
    ports:
      - 5432:5432
    environment:
      POSTGRES_DB: gorse
      POSTGRES_USER: gorse
      POSTGRES_PASSWORD: gorse_pass
    volumes:
      - postgres_data:/var/lib/postgresql/data

  worker:
    image: zhenghaoz/gorse-worker
    restart: unless-stopped
    ports:
      - 8089:8089
    command: >
      --master-host master --master-port 8086 
      --http-host 0.0.0.0 --http-port 8089
      --log-path /var/log/gorse/worker.log 
      --cache-path /var/lib/gorse/worker_cache.data
    volumes:
      - gorse_log:/var/log/gorse
      - worker_data:/var/lib/gorse
    depends_on:
      - master

  server:
    image: zhenghaoz/gorse-server
    restart: unless-stopped
    ports:
      - 8087:8087
    command: >
      --master-host master --master-port 8086 
      --http-host 0.0.0.0 --http-port 8087
      --log-path /var/log/gorse/server.log 
      --cache-path /var/lib/gorse/server_cache.data
    volumes:
      - gorse_log:/var/log/gorse
      - server_data:/var/lib/gorse
    depends_on:
      - master

  master:
    image: zhenghaoz/gorse-master
    restart: unless-stopped
    ports:
      - 8086:8086
      - 8088:8088
    environment:
      GORSE_CACHE_STORE: postgres://gorse:gorse_pass@postgres/gorse?sslmode=disable
      GORSE_DATA_STORE: postgres://gorse:gorse_pass@postgres/gorse?sslmode=disable
    command: >
      -c /etc/gorse/config.toml 
      --log-path /var/log/gorse/master.log 
      --cache-path /var/lib/gorse/master_cache.data
    volumes:
      - ./config/config.toml:/etc/gorse/config.toml
      - gorse_log:/var/log/gorse
      - master_data:/var/lib/gorse
    depends_on:
      - postgres

volumes:
  worker_data:
  server_data:
  master_data:
  gorse_log:
  postgres_data:

MongoDB

version: "3"
services:
  mongo:
    image: mongo:4.0
    restart: unless-stopped
    ports:
      - 27017:27017
    environment:
      MONGO_INITDB_DATABASE: gorse
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: password
    volumes:
      - mongo_data:/data/db

  worker:
    image: zhenghaoz/gorse-worker
    restart: unless-stopped
    ports:
      - 8089:8089
    command: >
      --master-host master --master-port 8086 
      --http-host 0.0.0.0 --http-port 8089
      --log-path /var/log/gorse/worker.log 
      --cache-path /var/lib/gorse/worker_cache.data
    volumes:
      - gorse_log:/var/log/gorse
      - worker_data:/var/lib/gorse
    depends_on:
      - master

  server:
    image: zhenghaoz/gorse-server
    restart: unless-stopped
    ports:
      - 8087:8087
    command: >
      --master-host master --master-port 8086 
      --http-host 0.0.0.0 --http-port 8087
      --log-path /var/log/gorse/server.log 
      --cache-path /var/lib/gorse/server_cache.data
    volumes:
      - gorse_log:/var/log/gorse
      - server_data:/var/lib/gorse
    depends_on:
      - master

  master:
    image: zhenghaoz/gorse-master
    restart: unless-stopped
    ports:
      - 8086:8086
      - 8088:8088
    environment:
      GORSE_CACHE_STORE: mongodb://root:password@mongo:27017/gorse?authSource=admin&connect=direct
      GORSE_DATA_STORE: mongodb://root:password@mongo:27017/gorse?authSource=admin&connect=direct
    command: >
      -c /etc/gorse/config.toml 
      --log-path /var/log/gorse/master.log 
      --cache-path /var/lib/gorse/master_cache.data
    volumes:
      - ./config/config.toml:/etc/gorse/config.toml
      - gorse_log:/var/log/gorse
      - master_data:/var/lib/gorse
    depends_on:
      - mongo

volumes:
  worker_data:
  server_data:
  master_data:
  gorse_log:
  mongo_data:

2. Create a configuration file config.toml in the same directory based on the configuration template.

3. Start all services:

docker-compose up -d

Command Line Flags

Command line flags of the master node:

	Flag	Default Value	Description
	--cache-path	worker_cache.data	Cache file path.
-c	--config		Configuration file path.
	--debug		Debug log mode.
-h	--help		Help for gorse-master.
	--log-path		Path of log file.
	--log-max-size		Maximum size in megabytes of the log file.
	--log-max-age		Maximum number of days to retain old log files.
	--log-max-backups		Maximum number of old log files to retain.
-v	--version		Gorse version.

Command line flags of the server node:

	Flag	Default Value	Description
	--cache-path	worker_cache.data	Cache file path.
	--debug		Debug log mode.
-h	--help		Help for gorse-server.
	--http-host	127.0.0.1	RESTful APIs and Prometheus metrics export host
	--http-port	8087	RESTful APIs and Prometheus metrics export port
	--log-path		Path of log file.
	--log-max-size		Maximum size in megabytes of the log file.
	--log-max-age		Maximum number of days to retain old log files.
	--log-max-backups		Maximum number of old log files to retain.
	--master-host	127.0.0.1	Master node host.
	--master-port	8086	Master node port.
-v	--version		Gorse version.

Command line flags of the worker node:

	Flag	Default Value	Description
	--cache-path	worker_cache.data	Cache file path.
	--debug		Debug log mode.
-h	--help		Help for gorse-worker.
	--http-host	127.0.0.1	Prometheus metrics export host.
	--http-port	8089	Prometheus metrics export port.
-j	--jobs	1	Number of working jobs.
	--log-path		Path of log file.
	--log-max-size		Maximum size in megabytes of the log file.
	--log-max-age		Maximum number of days to retain old log files.
	--log-max-backups		Maximum number of old log files to retain.
	--master-host	127.0.0.1	Master node host.
	--master-port	8086	Master node port
-v	--version		Gorse version.