pacnpal/markov-discord
1
0
Fork 0
mirror of https://github.com/pacnpal/markov-discord.git synced 2025-12-20 03:01:04 -05:00
Code Issues Packages Projects Releases Wiki Activity

Update docs for 2.0.0 prep

Browse Source
This commit is contained in:
Charlie Laabs
2022-01-04 22:15:33 -06:00
parent fb2128b080
commit 947100282e
11 changed files with 306 additions and 189 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.dockerignore
View File

@@ -1,3 +1,5 @@
config
dist
node_modules
img
docs

33
.github/workflows/typedoc.yml vendored Normal file
View File

@@ -0,0 +1,33 @@
name: Publish Typedoc to Github Pages
on:
push:
branches:
- master
paths:
- Readme.md
- src/config/classes.ts
- .github/workflows/typedoc.yml
- package.json
jobs:
publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Setup Node.js for use with actions
uses: actions/setup-node@v2
with:
node-version: 'lts/*'
cache: 'npm'
- name: NPM install
run: npm ci
# Runs a single command using the runners shell
- name: Build and lint
run: npm run docs
- name: Deploy
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./docs

1
.gitignore vendored
View File

@@ -4,6 +4,7 @@ logs
npm-debug.log*
yarn-debug.log*
yarn-error.log*
docs
# Runtime data
pids

94
CHANGELOG.md Normal file
View File

@@ -0,0 +1,94 @@
# Changelog
All notable changes to this project will be documented in this file.
## Versions
### 2.0.0
#### Breaking Changes
* Config option `prefix` renamed to `messageCommandPrefix`
* Config option `game` renamed to `activity`
* Docker internal volume path moved from `/usr/src/markbot/config` to `/usr/app/config`
* Database changed from JSON files to a SQLite database. You'll need to retrain the bot to use it again.
* The bot must be explicitly granted permission to listen to a list of channels before using it. Configure it with `/listen`.
* Docker user changed from `root` to `node`
#### New Features
* Data is stored in a relational database to reduce memory and disk read/write usage, as well as to decrease latency
* The bot can be restricted to only learn/listen from a strict list of channels
* Bot responses can be seeded by a short phrase
* Discord slash command support
* Many more config options available at <https://claabs.github.io/markov-discord/classes/AppConfig.html>
* Config file supports [JSON5](https://json5.org/) (comments, trailing commas, etc)
### 0.7.3
* Fix crash when fetched messages is empty
* Update docs
* Update dependencies
### 0.7.2
* Fix @everyone replacement
### 0.7.1
* Readme updates
* Config loading fix
* Fix min score
* Add generator options to config
* Document Node 12 update
### 0.7.0
* Convert project to Typescript
* Optimize Docker build (smaller image)
* Load corpus from filesystem to reduce memory load
### 0.6.2
* Fix MarkovDB not loading on boot
### 0.6.1
* Fix bot crashing on scheduled regen
### 0.6.0
* Added Docker deploy functionality.
* Moved config and database to `./config` directory. Existing configs will be migrated.
* Config-less support via bot token located in an environment variable.
* Update dependencies.
* Change corpus regen time to 4 AM.
### 0.5.0
* Fixed bug where `!mark help` didn't work.
* Only admins can train.
* The bot responds when mentioned.
* The bot cannot mention @everyone.
* Added version number to help.
* Added `!mark tts` for a quieter TTS response.
* Readme overhaul.
* Simpler config loading.
### 0.4.0
* Huge refactor.
* Added `!mark debug` which sends debug info alongside the message.
* Converted the fetchMessages function to async/await (updating the requirement to Node.js 8).
* Updated module versions.
* Added faster unique-array-by-property function
* Added linting and linted the project.
### 0.3.0
* Added TTS support and random message attachments.
* Deleted messages no longer persist in the database longer than 24 hours.
### 0.2.0
* Updated training algorithm and data structure.

218
README.md
View File

@@ -4,213 +4,65 @@ A Markov chain bot using markov-strings.
## Usage
1. Configure what channels you want the bot to listen/learn from:
* User: `/listen modify`
* Bot: ![Select which channels your would like the bot to actively listen to](img/listen.png)
1. Train the bot in a lengthy text channel:
* User: `!mark train`
* Markbot: `@User, Finished training from past 76394 messages.`
* User: `/train`
* Bot: ![Parsing past messages from 5 channel(s).](img/train.png)
1. Ask the bot to say something:
* User: `!mark`
* Markbot: `This Shopko has a Linux release`
* User: `/mark`
* Bot: ![worms are not baby snakes, by the way](img/respond.png)
## Setup
First, create a [Discord bot application](https://discordapp.com/developers/applications/).
This bot stores your Discord server's entire message history, so a public instance to invite to your server is not available due to obvious data privacy concerns. Instead, you can host it yourself.
1. Create a [Discord bot application](https://discordapp.com/developers/applications/)
1. Under the "Bot" section, enable the "Message Content Intent", and copy the token for later.
1. Setup and configure the bot using one of the below methods:
### Docker
Running this bot in Docker is the easiest way to ensure it runs as expected and can easily recieve updates.
1. [Install Docker for your OS](https://docs.docker.com/get-docker/)
1. Open a command prompt and run:
```sh
docker run --rm -ti -v /my/host/dir:/usr/app/config ghcr.io/claabs/markov-discord:latest
```
Where `/my/host/dir` is a accessible path on your system.
1. The Docker container will create a default config file in your mounted volume (`/my/host/dir`). Open it and add your bot token. You may change any other values to your liking as well. Details for each configuration item can be found here: <https://claabs.github.io/markov-discord/classes/AppConfig.html>
1. Run the container again and use the invite link printed to the logs.
### Windows
#### Windows Requirements
* [Node.js 12.0+ (Current)](https://nodejs.org/en/download/)
* Installing with build tools is recommended
#### Windows Setup
1. Install Node.js 12.0 or newer.
1. Install [Node.js 16 or newer](https://nodejs.org/en/download/).
1. Download this repository using git in a command prompt
```cmd
git clone https://github.com/charlocharlie/markov-discord.git
git clone https://github.com/claabs/markov-discord.git
```
or by just downloading and extracting the [project zip](https://github.com/charlocharlie/markov-discord/archive/master.zip) from GitHub.
or by just downloading and extracting the [project zip](https://github.com/claabs/markov-discord/archive/master.zip) from GitHub.
1. Open a command prompt in the `markov-discord` folder.
```sh
# Install Windows build tools (if you didn't install build tools with Node)
npm install --global windows-build-tools
# NPM install non-development packages
npm install
npm ci --only=production
# Build the Typescript
npm run build
# Initialize the config
npm start
```
1. Create a file called `config.json` in the project directory with the contents:
```json
{
"prefix":"!mark",
"game":"\"!mark help\" for help",
"token":"k5NzE2NDg1MTIwMjc0ODQ0Nj.DSnXwg.ttNotARealToken5p3WfDoUxhiH"
}
```
Feel free to change the command prefix, game display. Add your bot token.
1. The program will create a `config/config.json` in the project folder. Open it and add your bot token. You may change any other values to your liking as well. Details for each configuration item can be found here: <https://claabs.github.io/markov-discord/classes/AppConfig.html>
1. Run the bot:
```sh
npm start
```
### Debian Linux
#### Debian Requirements
* Node.js 12.0+
* Python 2.7 (for erlpack)
* C++ build tools (for erlpack)
#### Download
```sh
# Clone this repository
git clone https://github.com/charlocharlie/markov-discord.git
cd markov-discord
```
#### Configure
Create a file called `config.json` in the project directory with the contents:
```json
{
"prefix":"!mark",
"game":"\"!mark help\" for help",
"token":"k5NzE2NDg1MTIwMjc0ODQ0Nj.DSnXwg.ttNotARealToken5p3WfDoUxhiH"
}
```
Feel free to change the command prefix, game display. Add your bot token.
#### Install and Run
```sh
# Install Node.js if you haven't already
wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.33.11/install.sh | bash
nvm install node
# NPM install non-development packages
npm install
# If you run into build errors, install the following packages:
sudo apt-get install python -y
sudo apt-get install build-essential -y
# Build the Typescript
npm run build
# Start the program
npm start
```
### Docker
#### Setup with Docker Hub image
1. Install Docker for your OS.
1. Open a command prompts and run:
```sh
docker pull charlocharlie/markov-discord
docker run --rm -d charlocharlie/markov-discord:latest
```
#### Setup with source
1. Install Docker for your OS.
1. Download this repository using git in a command prompt
```sh
git clone https://github.com/charlocharlie/markov-discord.git
```
or by just downloading and extracting the [project zip](https://github.com/charlocharlie/markov-discord/archive/master.zip) from GitHub.
1. Open a command prompt in the markov-discord folder and run this one-liner:
```sh
docker run --rm -e TOKEN=YOUR.BOT.TOKEN -v config:/usr/src/markbot/config -it $(docker build -q .)
# Be patient as the build output is suppressed
```
## Changelog
### 0.7.3
* Fix crash when fetched messages is empty
* Update docs
* Update dependencies
### 0.7.2
* Fix @everyone replacement
### 0.7.1
* Readme updates
* Config loading fix
* Fix min score
* Add generator options to config
* Document Node 12 update
### 0.7.0
* Convert project to Typescript
* Optimize Docker build (smaller image)
* Load corpus from filesystem to reduce memory load
### 0.6.2
* Fix MarkovDB not loading on boot
### 0.6.1
* Fix bot crashing on scheduled regen
### 0.6.0
* Added Docker deploy functionality.
* Moved config and database to `./config` directory. Existing configs will be migrated.
* Config-less support via bot token located in an environment variable.
* Update dependencies.
* Change corpus regen time to 4 AM.
### 0.5.0
* Fixed bug where `!mark help` didn't work.
* Only admins can train.
* The bot responds when mentioned.
* The bot cannot mention @everyone.
* Added version number to help.
* Added `!mark tts` for a quieter TTS response.
* Readme overhaul.
* Simpler config loading.
### 0.4.0
* Huge refactor.
* Added `!mark debug` which sends debug info alongside the message.
* Converted the fetchMessages function to async/await (updating the requirement to Node.js 8).
* Updated module versions.
* Added faster unique-array-by-property function
* Added linting and linted the project.
### 0.3.0
* Added TTS support and random message attachments.
* Deleted messages no longer persist in the database longer than 24 hours.
### 0.2.0
* Updated training algorithm and data structure.
## Thanks
Thanks to [BotMaker-for-Discord](https://github.com/CorySanin/BotMaker-for-Discord) which I used as a reference when during development.
And use the invite link printed to the logs.

BIN
img/listen.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 46 KiB

BIN
img/respond.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 32 KiB

BIN
img/train.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 47 KiB

124
package-lock.json generated
View File

@@ -46,6 +46,7 @@
"prettier": "^2.5.1",
"rimraf": "^3.0.2",
"ts-node": "^10.4.0",
"typedoc": "^0.22.10",
"types-package-json": "^2.0.39",
"typescript": "^4.5.4"
},
@@ -2561,6 +2562,12 @@
"node": ">=6"
}
},
"node_modules/jsonc-parser": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/jsonc-parser/-/jsonc-parser-3.0.0.tgz",
"integrity": "sha512-fQzRfAbIBnR0IQvftw9FJveWiHp72Fg20giDrHz6TdfB12UH/uue0D3hm57UB5KgAVuniLMCaS8P1IMj9NR7cA==",
"dev": true
},
"node_modules/jsonfile": {
"version": "6.1.0",
"resolved": "https://registry.npmjs.org/jsonfile/-/jsonfile-6.1.0.tgz",
@@ -2628,12 +2635,30 @@
"node": ">=10"
}
},
"node_modules/lunr": {
"version": "2.3.9",
"resolved": "https://registry.npmjs.org/lunr/-/lunr-2.3.9.tgz",
"integrity": "sha512-zTU3DaZaF3Rt9rhN3uBMGQD3dD2/vFQqnvZCDv4dl5iOzq2IZQqTxu90r4E5J+nP70J3ilqVCrbho2eWaeW8Ow==",
"dev": true
},
"node_modules/make-error": {
"version": "1.3.6",
"resolved": "https://registry.npmjs.org/make-error/-/make-error-1.3.6.tgz",
"integrity": "sha512-s8UhlNe7vPKomQhC1qFelMokr/Sc3AgNbso3n74mVPA5LTZwkB9NlXf4XPamLxJE8h0gh73rM94xvwRT2CVInw==",
"dev": true
},
"node_modules/marked": {
"version": "3.0.8",
"resolved": "https://registry.npmjs.org/marked/-/marked-3.0.8.tgz",
"integrity": "sha512-0gVrAjo5m0VZSJb4rpL59K1unJAMb/hm8HRXqasD8VeC8m91ytDPMritgFSlKonfdt+rRYYpP/JfLxgIX8yoSw==",
"dev": true,
"bin": {
"marked": "bin/marked"
},
"engines": {
"node": ">= 12"
}
},
"node_modules/markov-strings-db": {
"version": "4.1.4",
"resolved": "https://registry.npmjs.org/markov-strings-db/-/markov-strings-db-4.1.4.tgz",
@@ -3485,6 +3510,17 @@
"node": ">=8"
}
},
"node_modules/shiki": {
"version": "0.9.15",
"resolved": "https://registry.npmjs.org/shiki/-/shiki-0.9.15.tgz",
"integrity": "sha512-/Y0z9IzhJ8nD9nbceORCqu6NgT9X6I8Fk8c3SICHI5NbZRLdZYFaB233gwct9sU0vvSypyaL/qaKvzyQGJBZSw==",
"dev": true,
"dependencies": {
"jsonc-parser": "^3.0.0",
"vscode-oniguruma": "^1.6.1",
"vscode-textmate": "5.2.0"
}
},
"node_modules/side-channel": {
"version": "1.0.4",
"resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.0.4.tgz",
@@ -3948,6 +3984,28 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/typedoc": {
"version": "0.22.10",
"resolved": "https://registry.npmjs.org/typedoc/-/typedoc-0.22.10.tgz",
"integrity": "sha512-hQYZ4WtoMZ61wDC6w10kxA42+jclWngdmztNZsDvIz7BMJg7F2xnT+uYsUa7OluyKossdFj9E9Ye4QOZKTy8SA==",
"dev": true,
"dependencies": {
"glob": "^7.2.0",
"lunr": "^2.3.9",
"marked": "^3.0.8",
"minimatch": "^3.0.4",
"shiki": "^0.9.12"
},
"bin": {
"typedoc": "bin/typedoc"
},
"engines": {
"node": ">= 12.10.0"
},
"peerDependencies": {
"typescript": "4.0.x || 4.1.x || 4.2.x || 4.3.x || 4.4.x || 4.5.x"
}
},
"node_modules/typeorm": {
"version": "0.2.41",
"resolved": "https://registry.npmjs.org/typeorm/-/typeorm-0.2.41.tgz",
@@ -4154,6 +4212,18 @@
"node": ">= 0.10"
}
},
"node_modules/vscode-oniguruma": {
"version": "1.6.1",
"resolved": "https://registry.npmjs.org/vscode-oniguruma/-/vscode-oniguruma-1.6.1.tgz",
"integrity": "sha512-vc4WhSIaVpgJ0jJIejjYxPvURJavX6QG41vu0mGhqywMkQqulezEqEQ3cO3gc8GvcOpX6ycmKGqRoROEMBNXTQ==",
"dev": true
},
"node_modules/vscode-textmate": {
"version": "5.2.0",
"resolved": "https://registry.npmjs.org/vscode-textmate/-/vscode-textmate-5.2.0.tgz",
"integrity": "sha512-Uw5ooOQxRASHgu6C7GVvUxisKXfSgW4oFlO+aa+PAkgmH89O3CXxEEzNRNtHSqtXFTl0nAC1uYj0GMSH27uwtQ==",
"dev": true
},
"node_modules/webidl-conversions": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz",
@@ -6255,6 +6325,12 @@
"minimist": "^1.2.5"
}
},
"jsonc-parser": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/jsonc-parser/-/jsonc-parser-3.0.0.tgz",
"integrity": "sha512-fQzRfAbIBnR0IQvftw9FJveWiHp72Fg20giDrHz6TdfB12UH/uue0D3hm57UB5KgAVuniLMCaS8P1IMj9NR7cA==",
"dev": true
},
"jsonfile": {
"version": "6.1.0",
"resolved": "https://registry.npmjs.org/jsonfile/-/jsonfile-6.1.0.tgz",
@@ -6308,12 +6384,24 @@
"yallist": "^4.0.0"
}
},
"lunr": {
"version": "2.3.9",
"resolved": "https://registry.npmjs.org/lunr/-/lunr-2.3.9.tgz",
"integrity": "sha512-zTU3DaZaF3Rt9rhN3uBMGQD3dD2/vFQqnvZCDv4dl5iOzq2IZQqTxu90r4E5J+nP70J3ilqVCrbho2eWaeW8Ow==",
"dev": true
},
"make-error": {
"version": "1.3.6",
"resolved": "https://registry.npmjs.org/make-error/-/make-error-1.3.6.tgz",
"integrity": "sha512-s8UhlNe7vPKomQhC1qFelMokr/Sc3AgNbso3n74mVPA5LTZwkB9NlXf4XPamLxJE8h0gh73rM94xvwRT2CVInw==",
"dev": true
},
"marked": {
"version": "3.0.8",
"resolved": "https://registry.npmjs.org/marked/-/marked-3.0.8.tgz",
"integrity": "sha512-0gVrAjo5m0VZSJb4rpL59K1unJAMb/hm8HRXqasD8VeC8m91ytDPMritgFSlKonfdt+rRYYpP/JfLxgIX8yoSw==",
"dev": true
},
"markov-strings-db": {
"version": "4.1.4",
"resolved": "https://registry.npmjs.org/markov-strings-db/-/markov-strings-db-4.1.4.tgz",
@@ -6945,6 +7033,17 @@
"integrity": "sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A==",
"dev": true
},
"shiki": {
"version": "0.9.15",
"resolved": "https://registry.npmjs.org/shiki/-/shiki-0.9.15.tgz",
"integrity": "sha512-/Y0z9IzhJ8nD9nbceORCqu6NgT9X6I8Fk8c3SICHI5NbZRLdZYFaB233gwct9sU0vvSypyaL/qaKvzyQGJBZSw==",
"dev": true,
"requires": {
"jsonc-parser": "^3.0.0",
"vscode-oniguruma": "^1.6.1",
"vscode-textmate": "5.2.0"
}
},
"side-channel": {
"version": "1.0.4",
"resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.0.4.tgz",
@@ -7291,6 +7390,19 @@
"integrity": "sha512-Ne+eE4r0/iWnpAxD852z3A+N0Bt5RN//NjJwRd2VFHEmrywxf5vsZlh4R6lixl6B+wz/8d+maTSAkN1FIkI3LQ==",
"dev": true
},
"typedoc": {
"version": "0.22.10",
"resolved": "https://registry.npmjs.org/typedoc/-/typedoc-0.22.10.tgz",
"integrity": "sha512-hQYZ4WtoMZ61wDC6w10kxA42+jclWngdmztNZsDvIz7BMJg7F2xnT+uYsUa7OluyKossdFj9E9Ye4QOZKTy8SA==",
"dev": true,
"requires": {
"glob": "^7.2.0",
"lunr": "^2.3.9",
"marked": "^3.0.8",
"minimatch": "^3.0.4",
"shiki": "^0.9.12"
}
},
"typeorm": {
"version": "0.2.41",
"resolved": "https://registry.npmjs.org/typeorm/-/typeorm-0.2.41.tgz",
@@ -7392,6 +7504,18 @@
"resolved": "https://registry.npmjs.org/validator/-/validator-13.7.0.tgz",
"integrity": "sha512-nYXQLCBkpJ8X6ltALua9dRrZDHVYxjJ1wgskNt1lH9fzGjs3tgojGSCBjmEPwkWS1y29+DrizMTW19Pr9uB2nw=="
},
"vscode-oniguruma": {
"version": "1.6.1",
"resolved": "https://registry.npmjs.org/vscode-oniguruma/-/vscode-oniguruma-1.6.1.tgz",
"integrity": "sha512-vc4WhSIaVpgJ0jJIejjYxPvURJavX6QG41vu0mGhqywMkQqulezEqEQ3cO3gc8GvcOpX6ycmKGqRoROEMBNXTQ==",
"dev": true
},
"vscode-textmate": {
"version": "5.2.0",
"resolved": "https://registry.npmjs.org/vscode-textmate/-/vscode-textmate-5.2.0.tgz",
"integrity": "sha512-Uw5ooOQxRASHgu6C7GVvUxisKXfSgW4oFlO+aa+PAkgmH89O3CXxEEzNRNtHSqtXFTl0nAC1uYj0GMSH27uwtQ==",
"dev": true
},
"webidl-conversions": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz",

6
package.json
View File

@@ -4,13 +4,14 @@
"description": "A conversational Markov chain bot for Discord",
"main": "dist/index.js",
"scripts": {
"start": "node dist/index.js",
"start": "NODE_ENV=production node dist/index.js",
"start:ts": "ts-node src/index.ts",
"build": "rimraf dist && tsc",
"lint": "tsc --noEmit && eslint **/*.ts *.js",
"docker:build": "docker build . -t charlocharlie/markov-discord:latest --target deploy",
"docker:run": "docker run --rm -ti -v $(pwd)/config:/usr/app/config charlocharlie/markov-discord:latest",
"typeorm": "node --require ts-node/register ./node_modules/typeorm/cli.js"
"typeorm": "node --require ts-node/register ./node_modules/typeorm/cli.js",
"docs": "typedoc --out docs src/config/classes.ts"
},
"repository": "https://github.com/claabs/markov-discord.git",
"keywords": [
@@ -67,6 +68,7 @@
"prettier": "^2.5.1",
"rimraf": "^3.0.2",
"ts-node": "^10.4.0",
"typedoc": "^0.22.10",
"types-package-json": "^2.0.39",
"typescript": "^4.5.4"
},

15
src/config/classes.ts
View File

@@ -1,7 +1,15 @@
/* eslint-disable @typescript-eslint/no-empty-function, no-useless-constructor, max-classes-per-file */
import 'reflect-metadata';
import { Type } from 'class-transformer';
import { IsString, IsOptional, IsEnum, IsArray, IsInt, IsDefined } from 'class-validator';
import {
IsString,
IsOptional,
IsEnum,
IsArray,
IsInt,
IsDefined,
IsNotEmpty,
} from 'class-validator';
export enum LogLevel {
SILENT = 'silent',
@@ -31,7 +39,8 @@ export class AppConfig {
*/
@IsDefined()
@IsString()
token = process.env.TOKEN as string;
@IsNotEmpty()
token = process.env.TOKEN || '';
/**
* The command prefix used to trigger the bot commands (when not using slash commands)
@@ -65,7 +74,7 @@ export class AppConfig {
/**
* A list of Discord user IDs that have owner permissions for the bot
* @example ['82684276755136512']
* @example ["82684276755136512"]
* @default []
* @env OWNER_IDS (comma separated)
*/