main
This commit is contained in:
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,193 @@
|
||||
# Mastodon Email Blocker
|
||||
|
||||
Small maintenance tool for Mastodon instances that keeps the local `EmailDomainBlock` table in sync with public disposable-email domain lists.
|
||||
|
||||
It supports two existing Mastodon layouts:
|
||||
|
||||
- a normal source installation where `bin/rails` and `bin/tootctl` are available on the host;
|
||||
- an existing Docker Compose installation, where the commands are executed inside the Mastodon application container with `docker compose exec`.
|
||||
|
||||
The tool does not use the Mastodon Admin API, so importing a large list does not generate one HTTP request per domain and does not run into API rate limits.
|
||||
|
||||
## What it changes
|
||||
|
||||
The blocklists are downloaded, normalized and merged. Domains already present in Mastodon are skipped; missing domains are added with Mastodon's own `tootctl email-domain-blocks add` command.
|
||||
|
||||
By default the tool also turns existing `allow_with_approval` email-domain records into hard deny records. Use `--no-force-deny` if that is not wanted.
|
||||
|
||||
MX records are never blocked automatically. `--mx-mode report` can be used for occasional DNS analysis, but the result is informational only. Shared mail infrastructure is common and blocking an MX hostname can affect unrelated, legitimate domains.
|
||||
|
||||
## Sources
|
||||
|
||||
The default set is built from:
|
||||
|
||||
- mehrtat/disposable-email-domain
|
||||
- FFraud-com/disposable-email-domains
|
||||
- disposable/disposable-email-domains
|
||||
|
||||
If a source is temporarily unavailable, the other successful sources are still used. If every source fails, the run stops before changing Mastodon.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Python 3.10 or newer
|
||||
- access to the Mastodon administration commands
|
||||
- `dnspython` only when MX reporting is enabled
|
||||
|
||||
Create a virtual environment:
|
||||
|
||||
```sh
|
||||
python3 -m venv /opt/mastodon-email-blocker/venv
|
||||
/opt/mastodon-email-blocker/venv/bin/pip install -r requirements.txt
|
||||
```
|
||||
|
||||
Install the script:
|
||||
|
||||
```sh
|
||||
install -d /opt/mastodon-email-blocker
|
||||
install -m 0755 bin/mastodon-email-blocker.py /opt/mastodon-email-blocker/mastodon-email-blocker.py
|
||||
install -d /etc/mastodon-email-blocker /var/lib/mastodon-email-blocker
|
||||
install -m 0644 config/allowlist.example.txt /etc/mastodon-email-blocker/allowlist.txt
|
||||
```
|
||||
|
||||
The account running the tool must be able to run the Mastodon commands and write to `/var/lib/mastodon-email-blocker`.
|
||||
|
||||
## Source installation
|
||||
|
||||
Run a dry run first:
|
||||
|
||||
```sh
|
||||
/opt/mastodon-email-blocker/venv/bin/python \
|
||||
/opt/mastodon-email-blocker/mastodon-email-blocker.py \
|
||||
--backend native \
|
||||
--mastodon-dir /path/to/mastodon/live \
|
||||
--ruby-bin /path/to/ruby/bin \
|
||||
--mx-mode off \
|
||||
--dry-run
|
||||
```
|
||||
|
||||
`--ruby-bin` can be omitted when the correct Ruby is already in `PATH`.
|
||||
|
||||
Apply the changes:
|
||||
|
||||
```sh
|
||||
/opt/mastodon-email-blocker/venv/bin/python \
|
||||
/opt/mastodon-email-blocker/mastodon-email-blocker.py \
|
||||
--backend native \
|
||||
--mastodon-dir /path/to/mastodon/live \
|
||||
--ruby-bin /path/to/ruby/bin \
|
||||
--mx-mode off
|
||||
```
|
||||
|
||||
Do not source Mastodon's `.env.production` from this script. Rails should load the application's production configuration in the same way it does for normal Mastodon commands.
|
||||
|
||||
## Existing Docker Compose installation
|
||||
|
||||
Nothing is installed inside the Mastodon image. The Python tool runs on the Docker host and invokes Rails/tootctl in the existing application service.
|
||||
|
||||
For the standard Mastodon Compose layout the application service is normally named `web`. If the local Compose file uses another service name, pass it with `--compose-service`.
|
||||
|
||||
Dry run:
|
||||
|
||||
```sh
|
||||
/opt/mastodon-email-blocker/venv/bin/python \
|
||||
/opt/mastodon-email-blocker/mastodon-email-blocker.py \
|
||||
--backend docker \
|
||||
--compose-dir /path/to/mastodon-compose \
|
||||
--compose-service web \
|
||||
--mx-mode off \
|
||||
--dry-run
|
||||
```
|
||||
|
||||
Apply the changes:
|
||||
|
||||
```sh
|
||||
/opt/mastodon-email-blocker/venv/bin/python \
|
||||
/opt/mastodon-email-blocker/mastodon-email-blocker.py \
|
||||
--backend docker \
|
||||
--compose-dir /path/to/mastodon-compose \
|
||||
--compose-service web \
|
||||
--mx-mode off
|
||||
```
|
||||
|
||||
For installations still using the old standalone binary, use:
|
||||
|
||||
```sh
|
||||
--compose-command "docker-compose"
|
||||
```
|
||||
|
||||
The Docker user running the tool must have permission to use Docker. No database password, Mastodon secret, `.env.production` file or API token is read by the Python program; the existing application container already has its normal environment.
|
||||
|
||||
## Allowlist
|
||||
|
||||
Put exceptions in `/etc/mastodon-email-blocker/allowlist.txt`, one domain per line:
|
||||
|
||||
```text
|
||||
example.org
|
||||
mail.example.net
|
||||
```
|
||||
|
||||
The allowlist only prevents the tool from adding those domains. It does not delete a block that already exists in Mastodon.
|
||||
|
||||
## MX report
|
||||
|
||||
DNS/MX analysis is intentionally disabled for scheduled runs because resolving a large domain set is slow and MX providers are often shared.
|
||||
|
||||
Run it manually when useful:
|
||||
|
||||
```sh
|
||||
/opt/mastodon-email-blocker/venv/bin/python \
|
||||
/opt/mastodon-email-blocker/mastodon-email-blocker.py \
|
||||
--backend native \
|
||||
--mastodon-dir /path/to/mastodon/live \
|
||||
--mx-mode report
|
||||
```
|
||||
|
||||
or, for Compose:
|
||||
|
||||
```sh
|
||||
/opt/mastodon-email-blocker/venv/bin/python \
|
||||
/opt/mastodon-email-blocker/mastodon-email-blocker.py \
|
||||
--backend docker \
|
||||
--compose-dir /path/to/mastodon-compose \
|
||||
--compose-service web \
|
||||
--mx-mode report
|
||||
```
|
||||
|
||||
The report is written to `/var/lib/mastodon-email-blocker/last-report.json`. MX entries in that file have `REPORT_ONLY` status and are not sent to Mastodon.
|
||||
|
||||
## systemd
|
||||
|
||||
Examples are in `systemd/`. Copy the matching service and the common timer, then create the corresponding environment file from `config/native.env.example` or `config/docker.env.example`.
|
||||
|
||||
For example:
|
||||
|
||||
```sh
|
||||
install -m 0644 systemd/mastodon-email-blocker-docker.service /etc/systemd/system/mastodon-email-blocker.service
|
||||
install -m 0644 systemd/mastodon-email-blocker.timer /etc/systemd/system/mastodon-email-blocker.timer
|
||||
install -m 0644 config/docker.env.example /etc/mastodon-email-blocker/docker.env
|
||||
```
|
||||
|
||||
Edit `/etc/mastodon-email-blocker/docker.env`, then enable the timer:
|
||||
|
||||
```sh
|
||||
systemctl daemon-reload
|
||||
systemctl enable --now mastodon-email-blocker.timer
|
||||
```
|
||||
|
||||
A manual service run is useful before enabling the timer:
|
||||
|
||||
```sh
|
||||
systemctl start mastodon-email-blocker.service
|
||||
journalctl -u mastodon-email-blocker.service -n 100 --no-pager
|
||||
```
|
||||
|
||||
## Interrupting and re-running
|
||||
|
||||
A run can be interrupted with `Ctrl+C`. Completed batches stay in Mastodon. On the next run the existing records are read again, so already-added domains are skipped and only the remaining ones are submitted.
|
||||
|
||||
## Files written
|
||||
|
||||
- `/var/lib/mastodon-email-blocker/last-report.json` — last run report
|
||||
- `/var/lib/mastodon-email-blocker/state.json` — small state summary
|
||||
|
||||
The tool does not store Mastodon credentials or API tokens.
|
||||
Executable
+421
@@ -0,0 +1,421 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import argparse
|
||||
import concurrent.futures
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shlex
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import urllib.request
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import dns.resolver
|
||||
except ImportError:
|
||||
dns = None
|
||||
|
||||
VERSION = "1.0.0"
|
||||
|
||||
SOURCES = [
|
||||
"https://raw.githubusercontent.com/mehrtat/disposable-email-domain/main/domains.txt",
|
||||
"https://raw.githubusercontent.com/FFraud-com/disposable-email-domains/main/disposable-email-domains.txt",
|
||||
"https://disposable.github.io/disposable-email-domains/domains.txt",
|
||||
]
|
||||
|
||||
DOMAIN_RE = re.compile(
|
||||
r"^(?=.{1,253}$)(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+"
|
||||
r"[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$"
|
||||
)
|
||||
|
||||
|
||||
def log(message=""):
|
||||
print(message, flush=True)
|
||||
|
||||
|
||||
def normalize_domain(value):
|
||||
if not value:
|
||||
return None
|
||||
value = value.strip().lower().rstrip(".")
|
||||
if not value or value.startswith("#"):
|
||||
return None
|
||||
if value.startswith("@"):
|
||||
value = value[1:]
|
||||
if value.startswith("*."):
|
||||
value = value[2:]
|
||||
try:
|
||||
value = value.encode("idna").decode("ascii")
|
||||
except Exception:
|
||||
return None
|
||||
return value if DOMAIN_RE.match(value) else None
|
||||
|
||||
|
||||
def read_domain_file(filename):
|
||||
path = Path(filename)
|
||||
if not path.exists():
|
||||
return set()
|
||||
result = set()
|
||||
for line in path.read_text(encoding="utf-8", errors="ignore").splitlines():
|
||||
domain = normalize_domain(line)
|
||||
if domain:
|
||||
result.add(domain)
|
||||
return result
|
||||
|
||||
|
||||
def download_source(url, timeout=120):
|
||||
request = urllib.request.Request(
|
||||
url,
|
||||
headers={"User-Agent": f"mastodon-email-blocker/{VERSION}"},
|
||||
)
|
||||
with urllib.request.urlopen(request, timeout=timeout) as response:
|
||||
content = response.read().decode("utf-8", errors="ignore")
|
||||
result = set()
|
||||
for line in content.splitlines():
|
||||
domain = normalize_domain(line)
|
||||
if domain:
|
||||
result.add(domain)
|
||||
return result
|
||||
|
||||
|
||||
def chunks(values, size):
|
||||
values = list(values)
|
||||
for i in range(0, len(values), size):
|
||||
yield values[i:i + size]
|
||||
|
||||
|
||||
def save_json_atomic(filename, value):
|
||||
filename = Path(filename)
|
||||
filename.parent.mkdir(parents=True, exist_ok=True)
|
||||
fd, temporary = tempfile.mkstemp(prefix=filename.name + ".", dir=str(filename.parent))
|
||||
try:
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as handle:
|
||||
json.dump(value, handle, indent=2, sort_keys=True)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
os.replace(temporary, filename)
|
||||
finally:
|
||||
if os.path.exists(temporary):
|
||||
os.unlink(temporary)
|
||||
|
||||
|
||||
class MastodonBackend:
|
||||
def run_rails(self, code):
|
||||
raise NotImplementedError
|
||||
|
||||
def run_tootctl_add(self, domains):
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
class NativeBackend(MastodonBackend):
|
||||
def __init__(self, mastodon_dir, ruby_bin=None):
|
||||
self.mastodon_dir = Path(mastodon_dir)
|
||||
self.ruby_bin = ruby_bin
|
||||
if not (self.mastodon_dir / "bin/rails").exists():
|
||||
raise RuntimeError(f"bin/rails not found in {self.mastodon_dir}")
|
||||
if not (self.mastodon_dir / "bin/tootctl").exists():
|
||||
raise RuntimeError(f"bin/tootctl not found in {self.mastodon_dir}")
|
||||
|
||||
def _env(self):
|
||||
env = os.environ.copy()
|
||||
env["RAILS_ENV"] = "production"
|
||||
if self.ruby_bin:
|
||||
env["PATH"] = f"{self.ruby_bin}:{env.get('PATH', '/usr/local/bin:/usr/bin:/bin')}"
|
||||
return env
|
||||
|
||||
def _run(self, command):
|
||||
process = subprocess.run(
|
||||
command,
|
||||
cwd=str(self.mastodon_dir),
|
||||
env=self._env(),
|
||||
text=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
)
|
||||
if process.returncode != 0:
|
||||
if process.stdout:
|
||||
sys.stderr.write(process.stdout)
|
||||
if process.stderr:
|
||||
sys.stderr.write(process.stderr)
|
||||
raise RuntimeError("Command failed: " + " ".join(command))
|
||||
return process.stdout
|
||||
|
||||
def run_rails(self, code):
|
||||
return self._run(["bin/rails", "runner", code])
|
||||
|
||||
def run_tootctl_add(self, domains):
|
||||
return self._run(["bin/tootctl", "email-domain-blocks", "add", *domains])
|
||||
|
||||
|
||||
class DockerBackend(MastodonBackend):
|
||||
def __init__(self, compose_dir, compose_command="docker compose", service="web"):
|
||||
self.compose_dir = Path(compose_dir)
|
||||
self.compose_command = shlex.split(compose_command)
|
||||
self.service = service
|
||||
if not self.compose_dir.exists():
|
||||
raise RuntimeError(f"Compose directory not found: {self.compose_dir}")
|
||||
|
||||
def _run(self, command):
|
||||
full_command = [
|
||||
*self.compose_command,
|
||||
"exec",
|
||||
"-T",
|
||||
"-e", "RAILS_ENV=production",
|
||||
self.service,
|
||||
*command,
|
||||
]
|
||||
process = subprocess.run(
|
||||
full_command,
|
||||
cwd=str(self.compose_dir),
|
||||
text=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
)
|
||||
if process.returncode != 0:
|
||||
if process.stdout:
|
||||
sys.stderr.write(process.stdout)
|
||||
if process.stderr:
|
||||
sys.stderr.write(process.stderr)
|
||||
raise RuntimeError("Command failed: " + " ".join(full_command))
|
||||
return process.stdout
|
||||
|
||||
def run_rails(self, code):
|
||||
return self._run(["bin/rails", "runner", code])
|
||||
|
||||
def run_tootctl_add(self, domains):
|
||||
return self._run(["bin/tootctl", "email-domain-blocks", "add", *domains])
|
||||
|
||||
|
||||
def get_blocks(backend):
|
||||
code = r'''
|
||||
rows = EmailDomainBlock.order(:id).pluck(:id, :domain, :allow_with_approval, :parent_id)
|
||||
STDOUT.write("__JSON__" + JSON.generate(rows))
|
||||
'''
|
||||
output = backend.run_rails(code)
|
||||
marker = "__JSON__"
|
||||
pos = output.rfind(marker)
|
||||
if pos < 0:
|
||||
raise RuntimeError("Could not read EmailDomainBlock from Rails")
|
||||
return json.loads(output[pos + len(marker):])
|
||||
|
||||
|
||||
def force_deny(backend):
|
||||
code = r'''
|
||||
count = EmailDomainBlock.where(allow_with_approval: true).update_all(
|
||||
allow_with_approval: false,
|
||||
updated_at: Time.current
|
||||
)
|
||||
STDOUT.write("__COUNT__#{count}")
|
||||
'''
|
||||
output = backend.run_rails(code)
|
||||
marker = "__COUNT__"
|
||||
pos = output.rfind(marker)
|
||||
if pos < 0:
|
||||
raise RuntimeError("Could not convert approval blocks to deny blocks")
|
||||
return int(output[pos + len(marker):].strip())
|
||||
|
||||
|
||||
def resolve_mx(domain, timeout):
|
||||
resolver = dns.resolver.Resolver(configure=True)
|
||||
resolver.timeout = timeout
|
||||
resolver.lifetime = timeout
|
||||
try:
|
||||
answers = resolver.resolve(domain, "MX")
|
||||
except Exception:
|
||||
return domain, []
|
||||
mx_records = set()
|
||||
for answer in answers:
|
||||
mx = normalize_domain(str(answer.exchange))
|
||||
if mx:
|
||||
mx_records.add(mx)
|
||||
return domain, sorted(mx_records)
|
||||
|
||||
|
||||
def build_backend(args):
|
||||
if args.backend == "native":
|
||||
if not args.mastodon_dir:
|
||||
raise RuntimeError("--mastodon-dir is required for the native backend")
|
||||
return NativeBackend(args.mastodon_dir, args.ruby_bin)
|
||||
|
||||
if not args.compose_dir:
|
||||
raise RuntimeError("--compose-dir is required for the docker backend")
|
||||
return DockerBackend(args.compose_dir, args.compose_command, args.compose_service)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Maintain Mastodon disposable email domain blocks")
|
||||
parser.add_argument("--backend", choices=["native", "docker"], required=True)
|
||||
parser.add_argument("--mastodon-dir")
|
||||
parser.add_argument("--ruby-bin", help="Directory containing ruby, only needed when Ruby is not already in PATH")
|
||||
parser.add_argument("--compose-dir")
|
||||
parser.add_argument("--compose-command", default="docker compose")
|
||||
parser.add_argument("--compose-service", default="web")
|
||||
parser.add_argument("--allowlist", default="/etc/mastodon-email-blocker/allowlist.txt")
|
||||
parser.add_argument("--report", default="/var/lib/mastodon-email-blocker/last-report.json")
|
||||
parser.add_argument("--state", default="/var/lib/mastodon-email-blocker/state.json")
|
||||
parser.add_argument("--batch-size", type=int, default=250)
|
||||
parser.add_argument("--mx-mode", choices=["off", "report"], default="off")
|
||||
parser.add_argument("--mx-workers", type=int, default=40)
|
||||
parser.add_argument("--mx-timeout", type=float, default=4.0)
|
||||
parser.add_argument("--dry-run", action="store_true")
|
||||
parser.add_argument("--no-force-deny", action="store_true")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.mx_mode == "report" and dns is None:
|
||||
raise RuntimeError("MX reporting requires dnspython: pip install dnspython")
|
||||
|
||||
backend = build_backend(args)
|
||||
allowlist = read_domain_file(args.allowlist)
|
||||
|
||||
log(f"mastodon-email-blocker {VERSION}")
|
||||
log(f"backend: {args.backend}")
|
||||
log(f"mx mode: {args.mx_mode}")
|
||||
log()
|
||||
|
||||
disposable = set()
|
||||
source_stats = {}
|
||||
successful_sources = 0
|
||||
|
||||
for url in SOURCES:
|
||||
log(f"[source] {url}")
|
||||
try:
|
||||
domains = download_source(url)
|
||||
source_stats[url] = len(domains)
|
||||
successful_sources += 1
|
||||
disposable |= domains
|
||||
log(f" {len(domains):,} domains")
|
||||
except Exception as exc:
|
||||
source_stats[url] = f"ERROR: {exc}"
|
||||
log(f" ERROR: {exc}")
|
||||
|
||||
if successful_sources == 0:
|
||||
raise RuntimeError("All blocklist sources failed; refusing to change Mastodon")
|
||||
|
||||
disposable -= allowlist
|
||||
log(f"[source] union: {len(disposable):,} disposable domains")
|
||||
|
||||
log("[mastodon] reading existing blocks")
|
||||
rows = get_blocks(backend)
|
||||
approvals = [row for row in rows if row[2] is True]
|
||||
|
||||
converted = 0
|
||||
if approvals and not args.no_force_deny:
|
||||
if args.dry_run:
|
||||
log(f"[dry-run] would convert {len(approvals):,} approval blocks to deny")
|
||||
else:
|
||||
converted = force_deny(backend)
|
||||
log(f"[mastodon] converted {converted:,} approval blocks to deny")
|
||||
rows = get_blocks(backend)
|
||||
|
||||
existing = {normalize_domain(row[1]) for row in rows}
|
||||
existing.discard(None)
|
||||
missing = sorted(disposable - existing)
|
||||
|
||||
log(f"[domains] existing blocks: {len(existing):,}")
|
||||
log(f"[domains] disposable domains to add: {len(missing):,}")
|
||||
|
||||
domain_batches = list(chunks(missing, args.batch_size))
|
||||
for number, batch in enumerate(domain_batches, 1):
|
||||
log(f"[domains] batch {number}/{len(domain_batches)} ({len(batch)})")
|
||||
if args.dry_run:
|
||||
continue
|
||||
backend.run_tootctl_add(batch)
|
||||
|
||||
mx_to_domains = defaultdict(set)
|
||||
domains_with_mx = 0
|
||||
domains_without_mx = 0
|
||||
|
||||
if args.mx_mode == "report":
|
||||
all_domains = sorted(disposable)
|
||||
log(f"[mx] resolving {len(all_domains):,} domains; MX records are report-only")
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=args.mx_workers) as executor:
|
||||
futures = {
|
||||
executor.submit(resolve_mx, domain, args.mx_timeout): domain
|
||||
for domain in all_domains
|
||||
}
|
||||
completed = 0
|
||||
for future in concurrent.futures.as_completed(futures):
|
||||
try:
|
||||
domain, records = future.result()
|
||||
except Exception:
|
||||
domain, records = futures[future], []
|
||||
if records:
|
||||
domains_with_mx += 1
|
||||
else:
|
||||
domains_without_mx += 1
|
||||
for mx in records:
|
||||
mx_to_domains[mx].add(domain)
|
||||
completed += 1
|
||||
if completed % 5000 == 0 or completed == len(all_domains):
|
||||
log(f"[mx] {completed:,}/{len(all_domains):,}")
|
||||
|
||||
top_mx = sorted(
|
||||
((mx, len(domains)) for mx, domains in mx_to_domains.items()),
|
||||
key=lambda item: (-item[1], item[0]),
|
||||
)[:100]
|
||||
|
||||
final_rows = rows if args.dry_run else get_blocks(backend)
|
||||
final_deny = sum(1 for row in final_rows if row[2] is False)
|
||||
final_approval = sum(1 for row in final_rows if row[2] is True)
|
||||
|
||||
report = {
|
||||
"version": VERSION,
|
||||
"timestamp": int(time.time()),
|
||||
"backend": args.backend,
|
||||
"dry_run": args.dry_run,
|
||||
"sources": source_stats,
|
||||
"successful_sources": successful_sources,
|
||||
"disposable_domains": len(disposable),
|
||||
"allowlist_entries": len(allowlist),
|
||||
"existing_before": len(rows),
|
||||
"approval_before": len(approvals),
|
||||
"approval_converted": converted,
|
||||
"missing_domains": len(missing),
|
||||
"domains_added_or_planned": len(missing),
|
||||
"mx_mode": args.mx_mode,
|
||||
"mx_autoblock": False,
|
||||
"domains_with_mx": domains_with_mx,
|
||||
"domains_without_mx": domains_without_mx,
|
||||
"mx_unique": len(mx_to_domains),
|
||||
"final_deny": final_deny,
|
||||
"final_approval": final_approval,
|
||||
"top_mx": [
|
||||
{"mx": mx, "disposable_domains": count, "action": "REPORT_ONLY"}
|
||||
for mx, count in top_mx
|
||||
],
|
||||
}
|
||||
|
||||
save_json_atomic(args.report, report)
|
||||
save_json_atomic(args.state, {
|
||||
"version": VERSION,
|
||||
"last_run": int(time.time()),
|
||||
"backend": args.backend,
|
||||
"disposable_domains": len(disposable),
|
||||
"mx_unique": len(mx_to_domains),
|
||||
"mx_autoblock": False,
|
||||
"last_report": args.report,
|
||||
})
|
||||
|
||||
log()
|
||||
log("Result")
|
||||
log(f" disposable domains : {len(disposable):,}")
|
||||
log(f" new domains : {len(missing):,}")
|
||||
log(f" MX observed : {len(mx_to_domains):,}")
|
||||
log(" MX blocked : 0")
|
||||
log(f" deny blocks : {final_deny:,}")
|
||||
log(f" approval blocks : {final_approval:,}")
|
||||
log(f" report : {args.report}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
main()
|
||||
except KeyboardInterrupt:
|
||||
log("\nInterrupted")
|
||||
sys.exit(130)
|
||||
except Exception as exc:
|
||||
log(f"\nERROR: {exc}")
|
||||
sys.exit(1)
|
||||
@@ -0,0 +1,4 @@
|
||||
# One domain per line.
|
||||
# Entries here are removed from the aggregated disposable-domain set.
|
||||
#
|
||||
# example.org
|
||||
@@ -0,0 +1,5 @@
|
||||
# Used by the systemd example for an existing Docker Compose Mastodon installation.
|
||||
# COMPOSE_DIR is the directory containing compose.yml/docker-compose.yml.
|
||||
COMPOSE_DIR=/path/to/mastodon-compose
|
||||
COMPOSE_SERVICE=web
|
||||
COMPOSE_COMMAND=docker compose
|
||||
@@ -0,0 +1,4 @@
|
||||
# Used by the systemd example for a source/native Mastodon installation.
|
||||
# Edit these paths for the local installation.
|
||||
MASTODON_DIR=/path/to/mastodon/live
|
||||
RUBY_BIN=/path/to/ruby/bin
|
||||
@@ -0,0 +1,21 @@
|
||||
# Docker Compose notes
|
||||
|
||||
This integration is for an existing Mastodon deployment. It does not create a Mastodon stack and does not add a new long-running application container.
|
||||
|
||||
The host-side script calls commands equivalent to:
|
||||
|
||||
```sh
|
||||
docker compose exec -T -e RAILS_ENV=production web bin/rails runner '...'
|
||||
docker compose exec -T -e RAILS_ENV=production web bin/tootctl email-domain-blocks add ...
|
||||
```
|
||||
|
||||
The official Mastodon Compose file uses a `web` service for the Rails application. Other distributions and custom Compose files may use a different service name; set `--compose-service` accordingly.
|
||||
|
||||
Before running the blocker, these two checks should succeed from the Compose directory:
|
||||
|
||||
```sh
|
||||
docker compose exec -T web bin/rails runner 'puts EmailDomainBlock.count'
|
||||
docker compose exec -T web bin/tootctl email-domain-blocks list --help
|
||||
```
|
||||
|
||||
If the local deployment uses a wrapper, a different Compose project directory, or the old `docker-compose` binary, adjust `--compose-dir` and `--compose-command`. The Python program does not need access to the database container and should not be given database credentials.
|
||||
@@ -0,0 +1 @@
|
||||
dnspython>=2.6,<3
|
||||
@@ -0,0 +1,12 @@
|
||||
[Unit]
|
||||
Description=Update Mastodon disposable email domain blocks through Docker Compose
|
||||
After=network-online.target docker.service
|
||||
Wants=network-online.target docker.service
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
EnvironmentFile=/etc/mastodon-email-blocker/docker.env
|
||||
ExecStart=/opt/mastodon-email-blocker/venv/bin/python /opt/mastodon-email-blocker/mastodon-email-blocker.py --backend docker --compose-dir ${COMPOSE_DIR} --compose-command ${COMPOSE_COMMAND} --compose-service ${COMPOSE_SERVICE} --mx-mode off
|
||||
ReadWritePaths=/var/lib/mastodon-email-blocker
|
||||
ReadOnlyPaths=/etc/mastodon-email-blocker
|
||||
TimeoutStartSec=infinity
|
||||
@@ -0,0 +1,12 @@
|
||||
[Unit]
|
||||
Description=Update Mastodon disposable email domain blocks
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
EnvironmentFile=/etc/mastodon-email-blocker/native.env
|
||||
ExecStart=/opt/mastodon-email-blocker/venv/bin/python /opt/mastodon-email-blocker/mastodon-email-blocker.py --backend native --mastodon-dir ${MASTODON_DIR} --ruby-bin ${RUBY_BIN} --mx-mode off
|
||||
ReadWritePaths=/var/lib/mastodon-email-blocker
|
||||
ReadOnlyPaths=/etc/mastodon-email-blocker
|
||||
TimeoutStartSec=infinity
|
||||
@@ -0,0 +1,10 @@
|
||||
[Unit]
|
||||
Description=Run Mastodon disposable email blocker daily
|
||||
|
||||
[Timer]
|
||||
OnCalendar=*-*-* 04:20:00
|
||||
RandomizedDelaySec=30m
|
||||
Persistent=true
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
Reference in New Issue
Block a user