Docs/Guides/Backup & Restore

Backup & restore

This guide covers how to back up your Ptolemy workspace data using the Export API and restore it using the Import API. Regular backups are a good operational practice, particularly before large schema changes.

Backup strategy

A complete workspace backup requires exporting each resource type separately. A recommended backup order (respecting foreign key dependencies):

1. Reference data

Instruments and markets first — they have no dependencies on other exportable types.

2. Market listings

Depends on instruments and markets existing.

3. Time series

Depends on instruments and series types.

Type schemas (instrument types, market types, time series types) and their field definitions cannot be exported or imported via the API — they must be recreated manually via the dashboard if you’re migrating to a new workspace.

Running a full backup

Submit export jobs for each resource type and wait for all to complete:

import time

RESOURCE_TYPES = [
  "instruments",
  "markets",
  "market_listings",
  "market_listing_suspensions",
  "time_series_values",
  "composite_time_series_values",
}

# Submit all export jobs
jobs = []
for rt in RESOURCE_TYPES:
  job = client.post("/exports", json={
    "resource_type": rt,
    "format": "json",
    "include_archived": True
  }).json()
  jobs.append(job)

# Wait for all to complete
while any(j["status"] in ("queued", "processing") for j in jobs):
  time.sleep(10)
  jobs = [client.get(f"/exports/{j['id']}").json() for j in jobs]

# Download all files
import requests
for job in jobs:
  if job["status"] == "completed":
    data = requests.get(job["download_url"]).content
    with open(f"{job['resource_type']}.json", "wb") as f:
      f.write(data)

Restoring from backup

To restore, import each resource type in dependency order. Import instruments and markets first, then listings, then time series data.

import time

RESTORE_ORDER = [
  "instruments",
  "markets",
  "market_listings",
  "market_listing_suspensions",
  "time_series_values",
  "composite_time_series_values",
}

for rt in RESTORE_ORDER:
  with open(f"{rt}.json", "rb") as f:
    resp = client.post("/imports",
      files={ "file": (f"{rt}.json", f, "application/json") },
      data={ "resource_type": rt, "on_error": "continue" }
    ).json()
    job_id = resp["id"]
  
  # Wait for import to complete
  while True:
    job = client.get(f"/imports/{job_id}").json()
    if job["status"] in ("completed", "failed"): break
    time.sleep(5)
  
  print(f"{rt}: {job['rows_imported']} imported, {job['rows_failed']} failed")

Restoring into an existing workspace with data may produce 409 Conflict errors for duplicate identifiers. Set on_error: "continue" to skip duplicates and import the rest.

Scheduling automated backups

For production workspaces, run backups on a schedule. A simple approach using a cron job:

Daily backup cron (bash)

# Run at 2 AM daily
0 2 * * * /usr/bin/python3 /opt/scripts/backup_ptolemy.py >>  /var/log/ptolemy_backup.log 2>&1

Store backups externally

Upload exported files to S3, GCS, or your backup service immediately after download. Don’t rely on the pre-signed Ptolemy URL — it expires in 15 minutes.

Test your restore

Periodically restore a backup to a test workspace to verify your backup files are valid and complete.