Kafka consumer, topic = bounded or unbounded SQL table.
Why it matters: Streaming-as-table is the headline value prop of Materialize / RisingWave / Arroyo. "SELECT count(*) FROM orders_topic WHERE …" with a tail-mode option turns dbfy into a real ops/log-analytics tool, not just a read-once federation.
Scope (v1): dbfy_kafka('broker:9092', topic := 'orders', format := 'json', start := 'earliest', end := 'latest'). Schema inference via the parser layer (jsonl/avro/protobuf). tail := true for unbounded streaming via the streaming SQL operator (when DataFusion's stream support stabilises).
Effort: 5-7 days (the bounded vs unbounded distinction + offset management is the real complexity, not the consumer wire).
Kafka consumer, topic = bounded or unbounded SQL table.
Why it matters: Streaming-as-table is the headline value prop of Materialize / RisingWave / Arroyo. "SELECT count(*) FROM orders_topic WHERE …" with a tail-mode option turns dbfy into a real ops/log-analytics tool, not just a read-once federation.
Scope (v1):
dbfy_kafka('broker:9092', topic := 'orders', format := 'json', start := 'earliest', end := 'latest'). Schema inference via the parser layer (jsonl/avro/protobuf).tail := truefor unbounded streaming via the streaming SQL operator (when DataFusion's stream support stabilises).Effort: 5-7 days (the bounded vs unbounded distinction + offset management is the real complexity, not the consumer wire).