Self‑Hosting an Archive.is Alternative: Complete Guide to Building Your Own Web‑Page Snapshots Service

The Community Spark #

The r/selfhosted thread “Self hosted archive.is type solution” blew up last week, with over 2 k up‑votes and dozens of replies. Users are tired of the occasional downtime of archive.is and want a privacy‑first, always‑available snapshot service they can run on a modest VPS. The core question is simple: What open‑source stack can replace archive.is, keep snapshots forever, and be easy enough for a hobbyist to maintain?

Synthesized Community Perspectives #

Consensus: Most users agree that a Docker‑based deployment lowers the barrier to entry, while the community values long‑term format stability (WARC) and an easy‑to‑use web UI. The debate centers on complexity vs. performance: pywb shines at serving millions of captures but needs a separate storage backend; ArchiveBox is simpler but can become sluggish at scale.

Deep‑Dive Actionable Guide: Building a DIY Archive.is Clone #

Below is a step‑by‑step tutorial that merges the most popular community suggestions into a single, production‑ready stack. It assumes a fresh Ubuntu 22.04 VPS (2 vCPU, 4 GB RAM, 100 GB SSD).

1. Prerequisites #

# Update and install Docker + Docker‑compose
sudo apt update && sudo apt upgrade -y
sudo apt install -y ca-certificates curl gnupg lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] \
https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update && sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
sudo usermod -aG docker $USER   # log out/in after this

2. Clone the “ArchiveBox‑Hybrid” repo (community‑maintained) #

git clone https://github.com/archiverhq/archivebox-hybrid.git
cd archivebox-hybrid

The repo bundles three services:

• archivebox – crawls URLs, stores WARC, screenshots, PDFs.
• pywb – fast WARC replay server.
• caddy – reverse‑proxy with HTTPS (Let’s Encrypt).

3. Configure environment variables (.env) #

# ArchiveBox
AB_OUTPUT_DIR=/data/archive
AB_MAX_THREADS=4

# pywb
PYWB_ROOT=/data/archive
PYWB_STATIC_ROOT=/data/static

# Caddy
DOMAIN=snap.yourdomain.com
EMAIL=you@example.com

Create the data directories and set permissions:

mkdir -p /data/archive /data/static
sudo chown -R 1000:1000 /data   # Docker containers run as UID 1000 by default

4. Launch the stack #

docker compose up -d

Docker will pull three images (archivebox/archivebox, webrecorder/pywb, caddy). After a minute, visit https://snap.yourdomain.com – you’ll see ArchiveBox’s UI for submitting URLs.

5. Capturing a page (the “archive.is” experience) #

1. Paste the target URL into ArchiveBox’s “Add URL” form.
2. Choose WARC + Screenshot + PDF (default).
3. Click Save – the system spawns a worker that runs wget, singlefile, and chromium headless to generate artifacts.
4. Once done, the snapshot appears in ArchiveBox’s list. Clicking View redirects to the pywb replay endpoint (e.g., https://snap.yourdomain.com/pywb/*/2024-07-19/http://example.com/).

6. Automating periodic archiving (optional) #

Add a cron job inside the ArchiveBox container:

docker exec -it archivebox /bin/bash
crontab -e   # add line:
0 3 * * * /app/archivebox schedule --interval daily

7. Back‑ups & Scaling #

• Back‑up: rsync -avz /data/archive /mnt/backup/ (run nightly).
• Scaling: Increase AB_MAX_THREADS or spin a second ArchiveBox worker container. pywb can read from a shared NFS or S3 bucket for massive archives.

Pros & Cons / Comparative Table #

The Verdict / Expert Advice #

• For most self‑hosters (single‑person projects, personal blogs, research notes), ArchiveBox alone is the sweet spot: quick to install, rich UI, and all‑in‑one storage.
• If you anticipate >10 k captures or need lightning‑fast replay, add pywb (the hybrid approach). The extra complexity pays off in speed and standards compliance.
• When VPS resources are scarce (e.g., 1 vCPU, 512 MB RAM), Starch offers a minimalist archive.is clone without the overhead of Docker, but you lose the visual UI.
• Never ignore backups – WARC files are immutable, but a corrupted disk wipes everything. Mirror to an S3 bucket or another VPS monthly.

Frequently Asked Questions #

Q1: Can I run the stack on a low‑cost $5/month VPS?
Yes. ArchiveBox’s Docker image runs comfortably on 1 vCPU/2 GB RAM. Reduce AB_MAX_THREADS to 2 and disable optional PDF generation to stay within memory limits.

Q2: How do I serve snapshots over HTTPS without buying a certificate?
The bundled Caddy server automatically obtains free Let’s Encrypt certificates for any domain you set in DOMAIN. Just ensure ports 80/443 are open.

Q3: Are the captured pages truly permanent?
ArchiveBox stores files in WARC, which is an ISO‑standard archival format. As long as the storage medium survives, the captures are immutable. For long‑term preservation, consider periodic integrity checks with warcio validate.

Q4: How does this differ from the official archive.is service?
Your self‑hosted stack gives you full control over data, no third‑party tracking, and the ability to add custom processing (e.g., OCR, metadata extraction). Archive.is is a public service that may delete or censor content; you own every byte on your server.