Motivation:
This article is useful if you want to batch inserts concurrently. Having a huge iterable (e.g., List) of entities, it is much faster to empower batch inserts concurrently instead of the sequential approach.
150+ PERSISTENCE PERFORMANCE ITEMS
THAT WILL ROCK YOUR APPS
Description:
This application is a sample of employing CompletableFuture for performing batching inserts in a concurrent approach. The used CompletableFuture uses an Executor that it is initialized with a number of threads equal with the number of your computer cores.
The following implementation facilitiates the usage of concurrent batch inserts in Spring Boot style via a repository and a method called saveInBatch().
Key points:
The implementation starts by creating an interface that contains the saveInBatch() method signature (we pass to this method the List of entities that should inserted in the database):
At the next step, we extend the built-in SimpleJpaRepository and implement our BatchRepository interface. Don't wory about the fact that we set transaction propagation tp NEVER! This is needed because the saveInBatch() method relies on another Spring component (listed soon) that is actually performing the batching, therefore there is no need to start a transaction now. As a rule of thumb, we always must strive for having short transactions in order to maintain a high-performance of the persistence layer. Moreover, we want to run a new transaction per thread, therefore starting a transaction here is pointless. Without setting the transaction propagation to NEVER, Spring Boot will start a transaction in our behalf when we call saveInBatch(), so we avoid this behavior. Using any of the built-in query-methods (e.g., findAll()) will work exactly as they always do. Our setting doesn't affect their transactional context.
Next, we create the BatchExecutor that uses CompletableFuture. This is a Spring Boot component that perform the concurrenct batching. Each batch run in a new transaction, therefore each thread uses its own transaction:
Testing time:
Usage is straightforward and follows Spring Boot style. First, we define a repository for an entity named Author:
Further, we can call saveInBatch() from a service-method as below:
Time-performance trend graphic:
(1000, 5000 and 10000 inserts via 1, 4 threads, and 8 threads)
Tam Ta Da Dam! :) The complete application is available on GitHub.
If you need a deep dive into the performance recipes exposed in this repository then I am sure that you will love my book "Spring Boot Persistence Best Practices".
Kommentare