Are you getting
Error: 3 INVALID_ARGUMENT: maximum 500 writes allowed per request from Firestore? We have a one-liner solution for you!
Firebase and Firestore enable building modern, ‘live’ web applications without having to think about schemas, migrations and infrastructure. While this makes it easier to focus on what's important - delivering user value - you might sometimes find yourself missing the ‘good old SQL transaction’.
Firestore offers two types of batch operations – transactions (which allow both reads and writes) and batched writes (for writes only). However, both of those however have hard limit of writing to 500 documents. When you start building your app, you don't think you’ll ever have to update more than that.
And then you find yourself staring at
Error: 3 INVALID_ARGUMENT: maximum 500 writes allowed per request in your Sentry dashboard (because you are responsible and you are tracking your functions errors). What should you do now?
The easy solution: use our firestore-big-batch library and replace all calls to
npm install @qualdesk/firestore-big-batch --save or
yarn add @qualdesk/firestore-big-batch
How it works
It is a very simple, but powerful concept. All the
BigBatch class does is keep track of how many operations you called, and when you reach the limit (defined at 499, just in case), it automatically creates another Firestore batch for you and adds it to an array of batches:
Yes, that brings us to the biggest warning about this library. It will create multiple Firestore batches if you have more than 499 operations in your BigBatch. Unfortunately, this is the closest we can get to making big transactions. If anything changes in Firestore we will update the library accordingly.
When you call
batch.commit() it uses
Promise.all() to convert array of promise to one promise that you can
await on in your code.
We know this library is not perfect, but it does the job for us at Qualdesk. There are a few improvements that we would like to introduce to it:
- better error handling when batches fail (
Promise.all()is not that great)
- see if we can support
- write tests!
We hope you find it useful! If you want to help us with the library (whether it's improvements mentioned above or anything else), all PRs are welcome qualdesk/firestore-big-batch
If you have questions or comments about this post, please tweet us @qualdesk and let us know how you get on.