Telegram message forwarder (client API)

Functionality

Live forwarding and updating messages
Source and target channels/chats/topics (one-to-one, many-to-one, many-to-many) mapping
Incoming message filters: word replacing, forward formating, skipping by keyword and more

Prepare

Create Telegram App
Obtain API_ID and API_HASH
Get SESSION_STRING

SESSION_STRING can be obtained by running login.py with provided API_ID and API_HASH environment variables. ❗ DON'T USE your own account.
Setup Postgres database or use InMemoryDatabase with USE_MEMORY_DB=true parameter in .env file

Fill .env with your data

.env-example contains the minimum environment configuration to run with an in-memory database.

.env overview

###########################
#    App configuration    #
###########################

# Telegram app ID
API_ID=test
# Telegram app hash
API_HASH=test
# Telegram session string (telethon session, see login.py in root directory)
SESSION_STRING=test
# Use an in-memory database instead of Postgres DB (true or false). Defaults to false
USE_MEMORY_DB=false
# Postgres credentials
DATABASE_URL=postgres://user:pass@host/dbname
# or
DB_NAME=test
DB_USER=test
DB_HOST=test
DB_PASS=test
# Logging level (debug, info, warning, error or critical). Defaults to info
LOG_LEVEL=info

Setup mirror forwarding config:

mirror.config.yml-example contains an example config: to set up a filter, specify the name of the filter (should be accessable from ./telemirror/messagefilters) and pass named parameters with the same types but in yaml syntax. Example:

- ForwardFormatFilter:  # Filter name under telemirror/messagefilters
    # Filter string argument
    format: "{message_text}\n\nForwarded from [{channel_name}]({message_link})"

- SkipUrlFilter:
    skip_mention: false # Filter bool argument

- UrlMessageFilter:
    blacklist: !!set    # Filter set argument
        ? t.me
        ? google.com

./.configs/mirror.config.yml or .env (limited) can be used to configure mirroring:

./.configs/mirror.config.yml mirroring config overview

# (Optional) Global filters, will be applied in order
filters:
  - ForwardFormatFilter: # Filter name under ./telemirror/messagefilters
      format: ""           # Filters arguments
  - EmptyMessageFilter
  - UrlMessageFilter:
      blacklist: !!set
        ? t.me
  - SkipUrlFilter:
      skip_mention: false

# (Optional) Global settings
disable_edit: true
disable_delete: true
mode: copy # or forward

# (Required) Mirror directions
directions:
  - from: [-1001, -1002, -1003]
    to: [-100203]

  - from: [-1000#3] # forwards from topic to topic
    to: [-1001#4]

  - from: [-100226]
    to: [-1006, -1008]
    # Overwrite global settings
    disable_edit: false
    disable_delete: false
    mode: forward
    # Overwrite global filters
    filters:
      - UrlMessageFilter:
          blacklist: !!set
            ? t.me
      - KeywordReplaceFilter:
          keywords:
            "google.com": "bing.com"
          regex: false # treat keywords as plain text
      - SkipWithKeywordsFilter:
          keywords: !!set
            ? "stopword"
          regex: true # treat keywords as regex expr

.env mirroring config overview

###############################################
#    Setup directions and filters from env    #
###############################################

# Mapping between source and target channels/chats
# Channel/chat id can be fetched by using @messageinformationsbot telegram bot
# Channel id should be prefixed with -100
# [id1, id2, id3:id4] means send messages from id1, id2, id3 to id4
# id5:id6 means send messages from id5 to id6
# [id1, id2, id3:id4];[id5:id6] semicolon means AND
CHAT_MAPPING=[-100999999,-100999999,-100999999:-1009999999];

# (Optional) YAML filter configuration thru single-lined env string (new lines (\n) should be replaced to \\n), other filter settings from env will be ignored
YAML_CONFIG_ENV=

# Remove URLs from incoming messages (true or false). Defaults to false
REMOVE_URLS=false
# Comma-separated list of URLs to remove (reddit.com,youtube.com)
REMOVE_URLS_LIST=google.com,twitter.com
# Comma-separated list of URLs to exclude from removal (google.com,twitter.com).
# Will be applied after the REMOVE_URLS_LIST
REMOVE_URLS_WL=youtube.com,youtu.be,vk.com,twitch.tv,instagram.com
# Disable mirror message deleting (true or false). Defaults to false
DISABLE_DELETE=false
# Disable mirror message editing (true or false). Defaults to false
DISABLE_EDIT=false

Channel mirroring config priority:

YAML_CONFIG_ENV from .env
from ./.configs/mirror.config.yml file
CHAT_MAPPING, REMOVE_URLS, REMOVE_URLS_LIST, REMOVE_URLS_WL, DISABLE_DELETE, DISABLE_EDIT from .env file

❓ Channels ID can be fetched by using Telegram bot.

❗ Note: never push your .env/.yml files with real crendential to a public repo. Use a separate branch (eg, heroku-branch) with .env/.yml files to push to git-based deployment system like Heroku.

Make sure the account has joined source and target channels
Be careful with forwards from channels with RESTRICTED SAVING CONTENT. It may lead to an account ban

Deploy

Heroku

or via CLI:

Clone project

git clone https://github.com/khoben/telemirror.git

Create new heroku app within Heroku CLI
```
heroku create {your app name}
```
Add heroku remote
```
heroku git:remote -a {your app name}
```
Set environment variables to your heroku app from .env by running bash script
```
./set_heroku_env.bash
```
Upload on heroku host
```
git push heroku master
```
Start heroku app
```
heroku ps:scale web=1
```

Keep up-to-date with Heroku

If you deployed manually, move to step 2.

Get project to your PC:
```
heroku git:clone -a {your app name}
```

Init upstream repo (this repository or its fork)

git remote add origin https://github.com/khoben/telemirror

Get latest changes
```
git pull origin master
```
Push latest changes to heroku
```
git push heroku master -f
```

Locally:

Create and activate python virtual environment

python -m venv venv
source ./venv/Scripts/activate # linux
venv\Scripts\activate # windows

Install dependencies
```
pip install -r requirements.txt
```
Run
```
python main.py
```

telemirror

Commits

Update dependencies

Update README.md

Add surrogates to filter replacement words