Turso With File Storage

[mcmah309]

Have there been any benchmarks or is there any insight on using turso with file storage?

We would like to store files of size 1 MB to around 500 MB (as a BLOB) along with the metadata etc. in the same row. The reason behind this is having everything in one place and available through sync.

Is this advisable?

Or if not, what is the best way to store the metadata with turso and properly sync files that turso may reference (an id or path entry in column)?

[apoorva-01]

I'd keep files that size out of the row. Turso has no incremental blob I/O yet. Every sqlite3_blob_* API is stubbed in COMPAT.md, so you can't stream a blob, you read or write the whole value at once. A 500MB row means 500MB materialized in memory on every access, and with sync you're shipping the entire blob again on every change to that row. SQLite's default cap on a single value is around 1GB too, so 500MB is already near the ceiling.

Cleaner shape: keep the metadata in Turso and store the actual bytes outside it (object storage or the filesystem), with a path/key column pointing at them. You still get one queryable source of truth for the metadata, and sync only moves small rows.

If everything genuinely has to live in the DB for sync to cover it, chunk each file into fixed-size rows (say 1–4MB each, file_id + seq + data) and reassemble on read, so no single value is huge and a change only re-syncs the chunks that moved. But external blob store plus a reference is the option I'd reach for first.

[mcmah309]

Cleaner shape: keep the metadata in Turso and store the actual bytes outside it (object storage or the filesystem), with a path/key column pointing at them. You still get one queryable source of truth for the metadata, and sync only moves small rows.

The issue is now one has to maintain an entire separate reconciliation process for deleting data that lives outside (e.g. like in an AWS S3 bucket). There are so many edge cases that could go wrong for durability here. So This would likely have to be accounted for by modifying the schema to handle this. It's simpler just to keep it in the db.

If everything genuinely has to live in the DB for sync to cover it, chunk each file into fixed-size rows (say 1–4MB each, file_id + seq + data) and reassemble on read, so no single value is huge and a change only re-syncs the chunks that moved. But external blob store plus a reference is the option I'd reach for first.

This only seems useful if data should change chunk by chunck, but most file data does not follow this. e.g. If I am working with an txt and I add one character to the start of the file, all chunks have to shift. So unless it is a very special case, I don't see this as being useful.