Write Performance

[dstanesc]

Thank you @jchris and all other shoulders for the great work in this area. I believe Fireproof and in general all concepts Fireproof builds upon are disruptive and part of the next stories to be written. I am however interested today in the roadmap for large(er) data. I have two topics:

• Just did some quick benchmarks and was wondering if there are easy ways of boosting write performance. In the future would be great if Fireproof could expose a way to publish multiple documents efficiently (ie. a bulk API), ideally in a transactional fashion

• Any plan for large(er) documents, ie. offer clients the ability to load and modify fragments of documents which otherwise would be inefficient to load or even cannot fit on the client?

[jchris]

Great question, on the question of larger documents, see the File Attachments section in the Concept Guide we just shipped. Feedback welcome!

For your benchmarks, it would be interesting to see what the performance of a second query on the same index is. I've been focusing on CRDT merge performance with multiple writers, but raw write performance is a good next target.

[jchris]

There is a community benchmark here, we do about 250 documents per second on a MacBook: https://github.com/dstanesc/fireproof-playground

Video of benchmark https://twitter.com/FireproofStorge/status/1750564015653388515

[jchris]

We've had another order of magnitude or more speedup since this post. I think we are ready for a bigger benchmark.