附录:随书实验完整代码
以下文件在构建时从随书实验与测试目录读取,与下载区实验包同源。它们是本书教学实现;实际验证范围以实验章节和验证记录为准。
lab/README.md
# 真实 CDC 实验
需要 Linux/macOS Docker Compose、Python3.12、至少约4GB可用内存、可访问镜像与Python依赖源。端口23316/29092/28083/25436必须空闲。凭据仅限本地教学;不要在已有业务数据库上执行初始化SQL。
```sh
export CDC_COMPOSE_PROJECT=cdc-book-learning-001
sh lab/run.sh
```
入口安装固定依赖、创建项目独占容器、保存实际镜像摘要与版本,执行真实MySQL→Debezium→Kafka→PostgreSQL集成,最后只清理指定Compose项目及其卷。不要并行运行两份(端口固定)。失败时evidence保留检查与容器日志,不会skip缺失依赖。重跑应使用空的evidence目录和新的项目身份,避免旧kill-ready.json污染控制信号;正式CI在独立目录执行。
实验依次验证快照期间真实写入、r/c/u/d、重复重放、三种绑定误用拒绝、DB提交而broker未提交时实际SIGKILL/重启、停机100次更新追平、删除、真实ALTER与显式投影默认值迁移、新connector全量重建、六次逐行逐字段对账。runtime.json包含22项检查及实际环境。数据是2000行合成商品;固定镜像digest与依赖见compose.yml和requirements.txt。
消费者是有限实验运行器,读到数据后空闲3秒结束,再由对账判定收敛;不是生产持续服务。单broker/单分区,未验证多机故障、日志过期、跨表原子可见性、任意DDL、权限撤回。PostgreSQL投影不是全文搜索引擎。数据库事务与Kafka位点分开提交,保证为至少一次交付下的幂等收敛,不是端到端exactly-once。
ZIP只包含源码、SQL、依赖、Compose和说明,不携带运行evidence/marker。出版CI会在独立解压目录从空容器环境重跑同一入口。独立证据通过发布artifact和下载区validation.json提供。
lab/compose.yml
services:
mysql:
image: mysql@sha256:0f51e63e537a1f176423fa03ef559aabe6fa4372a53160a44bc7e75da7d15364
command: [--server-id=184054, --log-bin=mysql-bin, --binlog-format=ROW, --binlog-row-image=FULL, --binlog-expire-logs-seconds=86400]
environment:
MYSQL_ROOT_PASSWORD: cdc-book-local
MYSQL_ROOT_HOST: '%'
ports: ['127.0.0.1:23316:3306']
kafka:
image: apache/kafka@sha256:4ceccc577f03f51f6af8dbfda55194d0d892f4fa7913ffbded567ce3895622ed
environment:
KAFKA_NODE_ID: 1
KAFKA_PROCESS_ROLES: broker,controller
KAFKA_LISTENERS: PLAINTEXT://:9092,CONTROLLER://:9093,HOST://:29092
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,HOST://127.0.0.1:29092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,HOST:PLAINTEXT
KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
ports: ['127.0.0.1:29092:29092']
connect:
image: quay.io/debezium/connect@sha256:1290a7abb0c1fe957f8d732e43efbea24201d9ba0fa5ba79582f76438c05d0ba
depends_on: [mysql, kafka]
environment:
BOOTSTRAP_SERVERS: kafka:9092
GROUP_ID: cdc-book-connect
CONFIG_STORAGE_TOPIC: book_connect_configs
OFFSET_STORAGE_TOPIC: book_connect_offsets
STATUS_STORAGE_TOPIC: book_connect_status
CONFIG_STORAGE_REPLICATION_FACTOR: 1
OFFSET_STORAGE_REPLICATION_FACTOR: 1
STATUS_STORAGE_REPLICATION_FACTOR: 1
KEY_CONVERTER: org.apache.kafka.connect.json.JsonConverter
VALUE_CONVERTER: org.apache.kafka.connect.json.JsonConverter
KEY_CONVERTER_SCHEMAS_ENABLE: 'false'
VALUE_CONVERTER_SCHEMAS_ENABLE: 'false'
REST_ADVERTISED_HOST_NAME: connect
ports: ['127.0.0.1:28083:8083']
postgres:
image: postgres@sha256:a63c1ea3f6a3a5a1621f97d2aba3b30faecfab4224c1b0000cc85ced57bb1939
environment:
POSTGRES_USER: book
POSTGRES_PASSWORD: cdc-book-local
POSTGRES_DB: book
ports: ['127.0.0.1:25436:5432']lab/consumer.py
#!/usr/bin/env python3
"""Real Kafka consumer; database transaction commits before broker offset commit."""
import argparse
import json
import os
import signal
import time
from pathlib import Path
import psycopg
from confluent_kafka import Consumer, KafkaError
DSN = 'postgresql://book:cdc-book-local@127.0.0.1:25436/book'
def apply(db, group, generation, message):
value = json.loads(message.value()) if message.value() is not None else None
key = json.loads(message.key()) if message.key() else None
# Kafka tombstones are compaction markers; the preceding Debezium delete
# envelope, not the null value, carries the source position and operation.
with db.transaction():
fresh = db.execute('INSERT INTO delivery_history VALUES (%s,%s,%s,%s) ON CONFLICT DO NOTHING RETURNING 1',
(group, message.topic(), message.partition(), message.offset())).fetchone()
if fresh and value:
assert value['op'] in ('r', 'c', 'u', 'd'), value
after = value['after']
row = after if after is not None else value['before']
assert key['id'] == row['id']
document = {k: row[k] for k in ('id', 'sku', 'price_cents', 'active', 'revision')}
if 'note' in row:
document['note'] = row['note']
# Same primary key is always on one partition; no cross-partition
# numeric offset comparison is claimed here.
db.execute('''INSERT INTO products_projection VALUES (%s,%s,%s,%s,%s,%s)
ON CONFLICT(generation,id) DO UPDATE SET document=EXCLUDED.document,
deleted=EXCLUDED.deleted,source_partition=EXCLUDED.source_partition,
source_offset=EXCLUDED.source_offset
WHERE products_projection.source_partition=EXCLUDED.source_partition
AND products_projection.source_offset<EXCLUDED.source_offset''',
(generation, row['id'], psycopg.types.json.Jsonb(document), value['op'] == 'd', message.partition(), message.offset()))
db.execute('''INSERT INTO delivery_checkpoint VALUES (%s,%s,%s,%s)
ON CONFLICT(consumer_group,topic,partition) DO UPDATE
SET next_offset=GREATEST(delivery_checkpoint.next_offset,EXCLUDED.next_offset)''',
(group, message.topic(), message.partition(), message.offset()+1))
return bool(fresh), value
def bind(db, generation, group, topic):
with db.transaction():
db.execute('INSERT INTO consumer_binding VALUES (%s,%s,%s) ON CONFLICT DO NOTHING', (generation,group,topic))
actual = db.execute('SELECT consumer_group,topic FROM consumer_binding WHERE generation=%s', (generation,)).fetchone()
if actual != (group,topic):
raise ValueError('Generation must bind to one unique consumer group and topic')
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--group', required=True)
parser.add_argument('--generation', required=True)
parser.add_argument('--topic', required=True)
parser.add_argument('--crash-after-db', action='store_true')
parser.add_argument('--ready-file')
parser.add_argument('--replay', action='store_true')
parser.add_argument('--idle-seconds', type=float, default=3)
args = parser.parse_args()
db = psycopg.connect(DSN, autocommit=True)
bind(db, args.generation, args.group, args.topic)
consumer = Consumer({'bootstrap.servers': '127.0.0.1:29092', 'group.id': args.group,
'enable.auto.commit': False, 'auto.offset.reset': 'earliest',
'enable.auto.offset.store': False})
def assigned(c, partitions):
if args.replay:
for partition in partitions:
partition.offset = 0
c.assign(partitions)
consumer.subscribe([args.topic], on_assign=assigned)
totals = {'messages': 0, 'fresh': 0, 'duplicates': 0, 'ops': {}, 'source_positions': []}
last = time.monotonic()
try:
while True:
msg = consumer.poll(0.5)
if msg is None:
if totals['messages'] and time.monotonic()-last > args.idle_seconds:
break
if time.monotonic()-last > 60:
raise TimeoutError('No messages available; integration may not be skipped')
continue
if msg.error():
raise RuntimeError(str(msg.error()))
fresh, value = apply(db, args.group, args.generation, msg)
totals['messages'] += 1
totals['fresh' if fresh else 'duplicates'] += 1
if value:
totals['ops'][value['op']] = totals['ops'].get(value['op'], 0)+1
if len(totals['source_positions']) < 8:
totals['source_positions'].append(value['source'])
if args.crash_after_db:
if args.ready_file:
Path(args.ready_file).write_text(json.dumps({'topic': msg.topic(), 'partition': msg.partition(), 'offset': msg.offset()}))
print('database committed; waiting for controller SIGKILL', flush=True)
while True:
time.sleep(1)
consumer.commit(message=msg, asynchronous=False)
last = time.monotonic()
finally:
consumer.close()
db.close()
print(json.dumps(totals, ensure_ascii=False), flush=True)
if __name__ == '__main__':
main()lab/projection.sql
CREATE TABLE products_projection (
generation TEXT NOT NULL,
id BIGINT NOT NULL,
document JSONB,
deleted BOOLEAN NOT NULL,
source_partition INTEGER NOT NULL,
source_offset BIGINT NOT NULL,
PRIMARY KEY(generation,id)
);
CREATE TABLE delivery_checkpoint (
consumer_group TEXT NOT NULL,
topic TEXT NOT NULL,
partition INTEGER NOT NULL,
next_offset BIGINT NOT NULL,
PRIMARY KEY(consumer_group,topic,partition)
);
CREATE TABLE delivery_history (
consumer_group TEXT NOT NULL,
topic TEXT NOT NULL,
partition INTEGER NOT NULL,
offset_value BIGINT NOT NULL,
PRIMARY KEY(consumer_group,topic,partition,offset_value)
);
CREATE TABLE consumer_binding (
generation TEXT PRIMARY KEY,
consumer_group TEXT NOT NULL UNIQUE,
topic TEXT NOT NULL
);lab/requirements.txt
PyMySQL[rsa]==1.1.1
confluent-kafka==2.10.0
psycopg[binary]==3.2.9
lab/run.py
#!/usr/bin/env python3
"""Fail-closed integration against real Docker MySQL/Debezium/Kafka/Postgres."""
import argparse
import hashlib
import json
import os
from pathlib import Path
import signal
import subprocess
import sys
import time
import urllib.request
import pymysql
import psycopg
from confluent_kafka import Consumer, TopicPartition
from confluent_kafka.admin import AdminClient
ROOT = Path(__file__).resolve().parents[1]
LAB = ROOT/'lab'
EVIDENCE = Path(os.environ.get('CDC_EVIDENCE_DIR', str(LAB/'evidence')))
EVIDENCE.mkdir(parents=True, exist_ok=True)
checks = []
def check(name, condition, details=None):
record = {'name': name, 'passed': bool(condition), 'details': details}
checks.append(record)
(EVIDENCE/'checks.json').write_text(json.dumps(checks, ensure_ascii=False, indent=2))
if not condition:
raise AssertionError(record)
print('PASS', name, flush=True)
def eventually(fn, seconds=180):
deadline = time.monotonic()+seconds
last = None
while time.monotonic() < deadline:
try:
result = fn()
if result:
return result
except Exception as exc:
last = str(exc)
time.sleep(1)
raise TimeoutError(f'Integration dependency did not become ready: {last}')
def mysql():
return pymysql.connect(host='127.0.0.1', port=23316, user='root', password='cdc-book-local', autocommit=True,
cursorclass=pymysql.cursors.DictCursor)
def sql(connection, query, args=None):
with connection.cursor() as cursor:
cursor.execute(query, args)
return cursor.fetchall()
def request(path, payload=None):
data = json.dumps(payload).encode() if payload is not None else None
req = urllib.request.Request('http://127.0.0.1:28083'+path, data=data,
headers={'Content-Type': 'application/json'})
with urllib.request.urlopen(req, timeout=15) as response:
return json.load(response)
def connector(name, prefix, slow=False):
cfg = {'connector.class': 'io.debezium.connector.mysql.MySqlConnector', 'tasks.max': '1',
'key.converter': 'org.apache.kafka.connect.json.JsonConverter', 'key.converter.schemas.enable': 'false',
'value.converter': 'org.apache.kafka.connect.json.JsonConverter', 'value.converter.schemas.enable': 'false',
'database.hostname': 'mysql', 'database.port': '3306', 'database.user': 'cdc',
'database.password': 'cdc-book-local', 'database.server.id': '184055' if slow else '184056',
'topic.prefix': prefix, 'database.include.list': 'inventory', 'table.include.list': 'inventory.products',
'schema.history.internal.kafka.bootstrap.servers': 'kafka:9092',
'schema.history.internal.kafka.topic': prefix+'.schema-history', 'snapshot.mode': 'initial',
'include.schema.changes': 'false', 'topic.creation.default.replication.factor': '1',
'topic.creation.default.partitions': '1', 'heartbeat.interval.ms': '1000'}
if slow:
cfg['snapshot.select.statement.overrides'] = 'inventory.products'
cfg['snapshot.select.statement.overrides.inventory.products'] = 'SELECT * FROM inventory.products WHERE SLEEP(0.003)=0'
request('/connectors', {'name': name, 'config': cfg})
return cfg
def worker(group, generation, topic, extra=()):
admin = AdminClient({'bootstrap.servers':'127.0.0.1:29092'})
eventually(lambda: topic in admin.list_topics(timeout=10).topics)
command = [sys.executable, str(LAB/'consumer.py'), '--group', group, '--generation', generation,
'--topic', topic, *extra]
result = subprocess.run(command, capture_output=True, text=True, timeout=150)
(EVIDENCE/(group+('-replay' if '--replay' in extra else '')+'.log')).write_text(result.stdout+'\n'+result.stderr)
if result.returncode:
raise RuntimeError(f'Consumer failed ({group}): {result.stdout}\n{result.stderr}')
return json.loads(result.stdout.strip().splitlines()[-1])
def source_rows(source):
rows = sql(source, 'SELECT * FROM inventory.products ORDER BY id')
return rows
def projection_rows(db, generation):
return [row[0] for row in db.execute('SELECT document FROM products_projection WHERE generation=%s AND NOT deleted ORDER BY id', (generation,)).fetchall()]
def reconcile(source, db, generation, label):
a = source_rows(source)
b = projection_rows(db, generation)
dump = lambda rows: json.dumps(rows, ensure_ascii=False, sort_keys=True, separators=(',', ':')).encode()
evidence = {'source_rows': len(a), 'projection_rows': len(b), 'source_sha256': hashlib.sha256(dump(a)).hexdigest(),
'projection_sha256': hashlib.sha256(dump(b)).hexdigest(), 'all_rows_all_fields_equal': a == b}
(EVIDENCE/(label+'-reconciliation.json')).write_text(json.dumps(evidence, indent=2))
check(label+' exact full-row reconciliation', a == b, evidence)
return evidence
def main():
start = time.time()
source = eventually(mysql)
pg = eventually(lambda: psycopg.connect('postgresql://book:cdc-book-local@127.0.0.1:25436/book', autocommit=True))
eventually(lambda: request('/connector-plugins'))
versions = {'mysql': sql(source, 'SELECT VERSION() AS version')[0]['version'],
'postgres': pg.execute('SELECT version()').fetchone()[0],
'connect_plugins': request('/connector-plugins'), 'python': sys.version,
'kafka': (EVIDENCE/'kafka-version.txt').read_text().strip(),
'images': {name:json.loads((EVIDENCE/('image-'+name+'.json')).read_text()) for name in ('mysql','kafka','connect','postgres')}}
(EVIDENCE/'versions.json').write_text(json.dumps(versions, indent=2))
for statement in (LAB/'schema.sql').read_text().split(';'):
if statement.strip():
sql(source, statement)
with source.cursor() as cursor:
cursor.executemany('INSERT INTO inventory.products VALUES (%s,%s,%s,%s,%s)',
[(i, 'sku-'+str(i), i*17, 1, 1) for i in range(1,2001)])
pg.execute((LAB/'projection.sql').read_text())
cfg = connector('book-source', 'book', slow=True)
(EVIDENCE/'connector-config.json').write_text(json.dumps(cfg, indent=2))
def active_scan():
return [r for r in sql(source, 'SHOW FULL PROCESSLIST') if r.get('Info') and 'SELECT * FROM inventory.products WHERE SLEEP' in r['Info']]
before = eventually(active_scan)
timestamp_before = sql(source, 'SELECT UTC_TIMESTAMP(6) AS t')[0]['t'].isoformat()
sql(source, "INSERT INTO inventory.products VALUES (3001,'snapshot-concurrent-insert',321,1,1)")
sql(source, "UPDATE inventory.products SET price_cents=777,revision=2 WHERE id=1")
sql(source, 'DELETE FROM inventory.products WHERE id=2')
timestamp_after = sql(source, 'SELECT UTC_TIMESTAMP(6) AS t')[0]['t'].isoformat()
after = active_scan()
overlap = {'before': before, 'after': after, 'write_start_mysql_utc': timestamp_before,
'write_end_mysql_utc': timestamp_after, 'write_ids': [3001,1,2]}
(EVIDENCE/'snapshot-write-overlap.json').write_text(json.dumps(overlap, default=str, indent=2))
check('writes occurred while real snapshot SELECT remained active', bool({r['Id'] for r in before} & {r['Id'] for r in after}), overlap)
first = worker('main', 'live', 'book.inventory.products')
(EVIDENCE/'initial-consume.json').write_text(json.dumps(first, indent=2))
check('real snapshot and binlog r/c/u/d captured', all(first['ops'].get(op,0)>0 for op in ('r','c','u','d')), first['ops'])
check('real binlog file and position captured', any(p.get('file') and p.get('pos') for p in first['source_positions']))
reconcile(source, pg, 'live', 'snapshot-plus-binlog')
replay = worker('main', 'live', 'book.inventory.products', ['--replay'])
check('duplicate replay detected in transactional inbox', replay['duplicates'] == first['messages'] and replay['fresh'] == 0, replay)
reconcile(source, pg, 'live', 'duplicate-replay')
before_invalid = (pg.execute('SELECT count(*) FROM products_projection').fetchone()[0], pg.execute('SELECT count(*) FROM delivery_history').fetchone()[0])
rejections=[]
for group,generation,topic in [('main','forbidden-new','book.inventory.products'),('other','live','book.inventory.products'),('main','live','wrong-topic')]:
bad=subprocess.run([sys.executable,str(LAB/'consumer.py'),'--group',group,'--generation',generation,'--topic',topic],capture_output=True,text=True,timeout=15)
rejections.append({'group':group,'generation':generation,'topic':topic,'returncode':bad.returncode,'explicit_binding_error':'Generation must bind' in bad.stderr})
after_invalid = (pg.execute('SELECT count(*) FROM products_projection').fetchone()[0], pg.execute('SELECT count(*) FROM delivery_history').fetchone()[0])
check('generation group topic mismatch rejected without pollution', all(r['returncode']!=0 and r['explicit_binding_error'] for r in rejections) and before_invalid==after_invalid and pg.execute('SELECT count(*) FROM consumer_binding').fetchone()[0]==1, rejections)
# Fresh group: one durable DB transaction, then forcibly kill the real process
# while its code is blocked before consumer.commit().
marker = EVIDENCE/'kill-ready.json'
command = [sys.executable, str(LAB/'consumer.py'), '--group', 'crash', '--generation', 'crash-gen',
'--topic', 'book.inventory.products', '--crash-after-db', '--ready-file', str(marker)]
with (EVIDENCE/'crashed-process.log').open('w') as log:
child = subprocess.Popen(command, stdout=log, stderr=log)
eventually(lambda: marker.exists(), 60)
position = json.loads(marker.read_text())
check('database committed before process kill', pg.execute('SELECT count(*) FROM delivery_history WHERE consumer_group=%s', ('crash',)).fetchone()[0] == 1)
probe = Consumer({'bootstrap.servers':'127.0.0.1:29092','group.id':'crash','enable.auto.commit':False})
committed = probe.committed([TopicPartition(position['topic'],position['partition'])], timeout=10)[0].offset
probe.close()
check('broker offset uncommitted at kill barrier', committed < 0 or committed <= position['offset'], {'broker_next_offset':committed, **position})
child.kill()
child.wait(timeout=10)
check('real subprocess terminated by SIGKILL', child.returncode == -signal.SIGKILL)
restarted = worker('crash', 'crash-gen', 'book.inventory.products')
check('restart re-delivers committed-database event', restarted['duplicates'] >= 1, restarted)
reconcile(source, pg, 'crash-gen', 'crash-restart')
# No consumer runs while these real source transactions create backlog.
for i in range(100):
sql(source, 'UPDATE inventory.products SET price_cents=price_cents+1,revision=revision+1 WHERE id=%s', (10+i,))
sql(source, 'DELETE FROM inventory.products WHERE id=10')
sql(source, "INSERT INTO inventory.products VALUES (4001,'after-outage',222,1,1)")
stale = source_rows(source) != projection_rows(pg, 'live')
check('stopped consumer leaves projection observably stale', stale)
caught = worker('main', 'live', 'book.inventory.products')
check('backlog drained after consumer restart', caught['ops'].get('u',0)>=100 and caught['ops'].get('d',0)>=1, caught)
reconcile(source, pg, 'live', 'backlog-catchup')
sql(source, "ALTER TABLE inventory.products ADD COLUMN note VARCHAR(80) NULL")
sql(source, "UPDATE inventory.products SET note='new schema field',revision=revision+1 WHERE id=1")
# Existing untouched rows receive the schema default on rebuild; live schema
# migration normalizes the same null default before reading new events.
pg.execute("UPDATE products_projection SET document=document || '{\"note\":null}'::jsonb WHERE generation='live'")
evolved = worker('main', 'live', 'book.inventory.products')
check('real ALTER and subsequent binlog update consumed', evolved['ops'].get('u',0)>=1)
check('added field reaches query document', pg.execute("SELECT document->>'note' FROM products_projection WHERE generation='live' AND id=1").fetchone()[0] == 'new schema field')
reconcile(source, pg, 'live', 'schema-evolution')
connector('book-rebuild', 'rebuild', slow=False)
rebuilt = worker('rebuild', 'new-generation', 'rebuild.inventory.products')
check('isolated new snapshot rebuild consumed', rebuilt['ops'].get('r',0)==len(source_rows(source)), rebuilt['ops'])
reconcile(source, pg, 'new-generation', 'full-rebuild')
check('live generation preserved during rebuild', projection_rows(pg,'live') == projection_rows(pg,'new-generation'))
statuses = {name: request('/connectors/'+name+'/status') for name in ('book-source','book-rebuild')}
check('both actual connectors and tasks RUNNING', all(s['connector']['state']=='RUNNING' and len(s['tasks'])==1 and all(t['state']=='RUNNING' for t in s['tasks']) for s in statuses.values()), statuses)
result = {'status':'PASS','skipped':0,'checks':checks,'check_count':len(checks),'duration_seconds':round(time.time()-start,3),
'versions':versions,'run_url':os.environ.get('BOOK_RUN_URL'),'source_revision':os.environ.get('BOOK_SOURCE_SHA'),
'boundaries':['single Kafka broker; one data partition','real Debezium initial snapshot and MySQL binlog',
'local teaching credentials only','no multi-node failover or binlog-loss proof',
'cross-store delivery is at-least-once with projection convergence; not end-to-end exactly-once']}
(EVIDENCE/'runtime.json').write_text(json.dumps(result, ensure_ascii=False, indent=2))
print(json.dumps({'status':'PASS','checks':len(checks),'seconds':result['duration_seconds']}))
if __name__ == '__main__':
main()lab/run.sh
#!/bin/sh
set -eu
cd "$(dirname "$0")/.."
: "${CDC_COMPOSE_PROJECT:?Set a dedicated CDC_COMPOSE_PROJECT before running; this lab owns only the dedicated project resources}"
case "$CDC_COMPOSE_PROJECT" in cdc-book-*) ;; *) echo 'Project must start cdc-book-' >&2; exit 2;; esac
mkdir -p "${CDC_EVIDENCE_DIR:-lab/evidence}"
if [ -f "${CDC_EVIDENCE_DIR:-lab/evidence}/kill-ready.json" ]; then
echo "Refusing stale crash marker; use a fresh evidence directory" >&2
exit 2
fi
cleanup() {
docker compose -p "$CDC_COMPOSE_PROJECT" -f lab/compose.yml logs --no-color > "${CDC_EVIDENCE_DIR:-lab/evidence}/containers.log" 2>&1 || true
docker compose -p "$CDC_COMPOSE_PROJECT" -f lab/compose.yml down --volumes --remove-orphans
}
trap cleanup EXIT
python3 -m pip install -r lab/requirements.txt
docker compose -p "$CDC_COMPOSE_PROJECT" -f lab/compose.yml up -d
for task_service in mysql kafka connect postgres; do
task_container=$(docker compose -p "$CDC_COMPOSE_PROJECT" -f lab/compose.yml ps -q "$task_service")
task_image=$(docker inspect --format '{{.Image}}' "$task_container")
docker image inspect --format '{{json .RepoDigests}}' "$task_image" > "${CDC_EVIDENCE_DIR:-lab/evidence}/image-$task_service.json"
done
docker compose -p "$CDC_COMPOSE_PROJECT" -f lab/compose.yml exec -T kafka /opt/kafka/bin/kafka-topics.sh --version > "${CDC_EVIDENCE_DIR:-lab/evidence}/kafka-version.txt"
python3 lab/run.pylab/schema.sql
CREATE DATABASE inventory;
CREATE USER 'cdc'@'%' IDENTIFIED BY 'cdc-book-local';
GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT, LOCK TABLES ON *.* TO 'cdc'@'%';
CREATE TABLE inventory.products (
id BIGINT PRIMARY KEY,
sku VARCHAR(64) NOT NULL,
price_cents INT NOT NULL,
active TINYINT NOT NULL,
revision INT NOT NULL
);