You defintiely can use S3 for appending to a WAL (I've done it), they have read-... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		dangoodmanUT on Nov 15, 2023 \| parent \| context \| favorite \| on: The Architecture of Serverless Data Systems You defintiely can use S3 for appending to a WAL (I've done it), they have read-after-write consistency

ddorian43 on Nov 15, 2023 | [–]

You're doing it for large OLAP writes. But as soon as you do OLTP or small writes it will become very slow & expensive.

See warpstream as example https://news.ycombinator.com/item?id=37036291

rowls66 on Nov 15, 2023 | [–]

How about using Kafka for the WAL? Anybody tried that?

ddorian43 on Nov 16, 2023 | [–]

LogDevice or apache bookeeper is better for distributed log.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact