Historical Backfill

Use historical backfill when you already have usage data in logs, billing systems, event streams, or another metering tool and want Open Spanner to become the queryable source going forward.

Model

Use the same meter definitions you plan to use for live ingestion. Backfilled events should look like normal usage events, with original timestamps and stable idempotency keys.

Useful dimensions depend on the source meter, but common fields include:

• source
• import_batch
• region
• plan

How To Use It

Generate deterministic idempotency keys from the original event identity, such as source system, customer, meter, timestamp, and source row ID. This lets you retry imports without double-counting usage.

This lets you:

• Rebuild usage history after migrating to Open Spanner.
• Retry imports safely.
• Compare old billing exports against new usage queries.
• Keep historical reporting continuous for customers.

Example

Create or reuse the same meter you will use for live traffic:

curl -X POST "$OPEN_SPANNER_BASE_URL/v1/meters" \
  -H "Authorization: Bearer $OPEN_SPANNER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "api_requests",
    "description": "Accepted API requests",
    "unit": "request",
    "aggregation": "sum",
    "event_retention_days": 365,
    "dimensions": [
      { "name": "source", "type": "string" },
      { "name": "import_batch", "type": "string" },
      { "name": "region", "type": "string" },
      { "name": "plan", "type": "string" }
    ]
  }'

Backfill events with original timestamps and deterministic idempotency keys:

curl -X POST "$OPEN_SPANNER_BASE_URL/v1/usages" \
  -H "Authorization: Bearer $OPEN_SPANNER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "idempotency_key": "legacy_billing:org_123:api_requests:row_98765",
    "subject": "org_123",
    "meter": "api_requests",
    "quantity": 42,
    "timestamp": "2026-01-15T12:00:00Z",
    "metadata": {
      "source": "legacy_billing",
      "import_batch": "2026_q1_backfill",
      "region": "us-east",
      "plan": "enterprise"
    }
  }'

For larger imports, use bulk ingestion with a stable batch-level Idempotency-Key header and stable per-event idempotency_key values.