An example: Automated order fulfillment system using JobRunr
To illustrate how JobRunr works, let’s simulate an order fulfillment system. Order fulfillment involves tasks required to completely satisfy a customer's order. It's a process where each step can affect another, which happens on several levels. For example, a confirmed order reduces stock, and to avoid stock-outs it is important to monitor and replenish the inventory.
The order fulfillment workflow has a lot of potential for automation where events can trigger jobs, which in turn trigger events until all steps are completed. As some of the workflow happens in the background, it’s a good opportunity to illustrate JobRunr! The setup of this example is as follows:
We’ll create an order confirmation that will trigger three tasks that can run asynchronously: 1) send a confirmation email to the customer, 2) notify the warehouse of the arrival of the order, and 3) initiate shipment by calling the preferred carrier’s API.
Some tasks are recurring. We illustrate this using two tasks: 1) a job that generates sales reports monthly, and 2) a job that monitors the inventory on a daily basis.
Something may go wrong while executing the tasks, in those cases, we want to trigger an alert.
This setup will allow to learn:
How to setup JobRunr in Spring Boot application
How to enqueue jobs using a Java 8 lambda configured via the
@Jobannotation and theJobBuilderHow to create recurring jobs using the
@RecurringannotationHow to hook into a job lifecycle using a
JobFilter
Please note that this example focuses on JobRunr side of things, we leave the implementation of object such as the Inventory, the
Order, theProduct, etc. as an exercise. Also the running of tasks is simulated byThread.sleepwhich implies explicit handling ofInterruptedException, you probably won't need those in an actual application. Although if you encounter such a case in your project it’s a best practice to throw the InterruptedException, JobRunr will handle it.
Setting up JobRunr in a Spring Boot application
Adding the JobRunr Spring Boot starter dependency
The setup is very easy for Spring Boot, it often consists of adding JobRunr’s Spring Boot starter artifact as a dependency. The following snippet shows how to include it when using Maven as a build tool.
<dependency>
<groupId>org.jobrunr</groupId>
<artifactId>jobrunr-spring-boot-3-starter</artifactId>
<version>${jobrunr.version}</version>
</dependency>
JobRunr requires adding database dependencies. We can skip this step as this is usually done when initializing the Spring Boot application. Nonetheless, it’s worth noting that JobRunr supports several databases. Another DB related feature is that all JobRunr related migrations are automatically handled by default. You can always decide otherwise and take control over the DB setup.
JobRunr also needs a JSON processing library, but as Spring Boot by default comes with Jackson support this is already covered.
Note: JobRunr also supports the older Spring Boot 2 and other frameworks such as Quarkus and Micronaut. The library can be found on Maven Central, feel free to use your preferred build tool.
Configuring JobRunr
JobRunr can be configured using application.properties. Job processing and the dashboard are disabled by default. To enable them, we’ll add the following to the configuration file:
org.jobrunr.dashboard.enabled=true
org.jobrunr.background-job-server.enabled=true
While we’re still in application.properties we will also add:
monthly-sales-report.cron=0 0 1 * *
daily-resupply.cron=0 0 * * *
stock-locations=Brussels,Antwerp,Bruges,Liege
Those will be useful to our recurring jobs as we’ll see later!
JobRunr offers advanced configuration options allowing to fine tune the application. When using the JobRunr’s Spring Boot starters or the other frameworks integrations, it’s also possible to override the predefined Beans.
Implementing the order fulfillment system
Now that the set up is done, we can start implementing our simulated order fulfillment process. We’ll focus on the JobRunr sides of things, i.e., enqueueing jobs, creating recurring jobs and hooking into a job lifecycle.
Triggering tasks on order confirmation
Once an order is confirmed, our system will trigger three tasks:
Send the order confirmation to the customer.
Notify the warehouse of the arrival of the order so work on the packaging can be swiftly done. This step may also involve generating and optimizing packing slips.
Initiate shipment by making a call to the selected carrier's booking API.
To automate these tasks with JobRunr we will create the following classes: a Spring’s RestController (namely OrderFulfillmentController), an OrderFulfillmentService, and an OrderFulfillmentTasks.
The OrderFulfillmentController will provide an endpoint to enqueue jobs. These jobs’ logic are implemented by the OrderFulfillmentService. We could stop here, but we would like the business logic to be separated from the job processing logic which will include features from JobRunr. Thus the introduction of OrderFulfillmentTasks.
Here is initial implementation of the OrderFulfillmentService class, we'll add more to it later.
@Service
public class OrderFulfillmentService {
private static final Logger LOGGER = LoggerFactory.getLogger(OrderFulfillmentService.class);
public void sendOrderConfirmation(UUID orderId) throws InterruptedException {
// TODO retrieve order from database, generate an invoice and send the confirmation to the customer
LOGGER.info("Order {}: sending order confirmation%n", orderId);
Thread.sleep(2000);
}
public void notifyWarehouse(UUID orderId) throws InterruptedException {
// TODO notify the warehouse of the arrival of a new order after retrieving/computing the necessary data using the orderId
LOGGER.info("Order {}: notifying the warehouse", orderId);
Thread.sleep(1000);
}
public void initiateShipment(UUID orderId) throws InterruptedException {
// TODO call the carrier's shipment initiation endpoint after retrieving/computing the necessary data using the orderId
LOGGER.info("Order {}: initiating shipment", orderId);
Thread.sleep(5000);
}
}
Note that we use Thread.sleep to simulate work, this forces us to explicitly handle InterruptedException, in an actual application, it’s probably not needed.
The OrderFulfillmentTasks class makes use of the OrderFulfillmentService. Notice the additional @Job, which allows setting values to a job’s attributes. This annotation is very handy! You may also use the alternative: the JobBuilder. We're doing exactly that in the body of the method, the jobs are configured using the JobBuilder and then saved atomically. This method is essentially a job that creates other jobs, so it also benefits from JobRunr's fault tolerance capabilities.
@Service
public class OrderFulfillmentTasks {
private final OrderFulfillmentService orderFulfillmentService;
public OrderFulfillmentTasks(OrderFulfillmentService orderFulfillmentService) {
this.orderFulfillmentService = orderFulfillmentService;
}
@Job(name = "order-%0")
public void enqueueConfirmedOrderTasks(UUID orderId) {
BackgroundJob.create(of(
aJob()
.withName(format("order-%s-confirmation", orderId))
.withDetails(() -> orderFulfillmentService.sendOrderConfirmation(orderId)),
aJob()
.withName(format("order-%s-warehouse-notification", orderId))
.withAmountOfRetries(20)
.withDetails(() -> orderFulfillmentService.notifyWarehouse(orderId)),
aJob()
.withName(format("order-%s-shipment-initiation", orderId))
.withDetails(() -> orderFulfillmentService.initiateShipment(orderId))
));
}
}
Note the use of withAmountOfRetries(20) for the warehouse notification task, we have to increase the amount of retries since our internal service is quite unstable.
The OrderFulfillmentController is quite simple as it exposes a single endpoint that requests JobRunr to enqueue jobs. Once the metadata of these jobs are saved in the database, the server workers will process them asynchronously when they are ready.
@RestController
public class OrderFulfillmentController {
@GetMapping("/confirm-order")
public String confirmOrder() {
UUID orderId = UUID.randomUUID();
BackgroundJob.<OrderFulfillmentTasks>enqueue(x -> x.enqueueConfirmedOrderTasks(orderId));
return "Done";
}
}
Creating reccuring tasks
The first part of our order fulfillment system is done. Now, we’ll add recurring tasks to know how well the company is doing and to make sure we never go out-of-stock!
Let’s update the OrderFulfillmentService to add the logic for these two tasks.
@Service
public class OrderFulfillmentService {
// ...
// Add the following methods to the class
public void generateMonthlySalesReport(YearMonth month) throws InterruptedException {
// TODO aggregate monthly sales, generate PDF and send it to managers
LOGGER.info("Sales report: generating monthly sales report on {} for month {}", Instant.now(), month);
Thread.sleep(10000);
}
public void resupply(String stockLocation) throws InterruptedException {
// TODO check the stock and contact supplier if needed
LOGGER.info("Inventory resupply: resupplying the inventory on {} for stock {}", Instant.now(), stockLocation);
Thread.sleep(5000);
}
}
We now update the OrderFulfillmentTasks to register those tasks. As they are recurring, we annotate them with @Recurring and a few attributes.
@Service
public class OrderFulfillmentTasks {
// Add the following attribute to the class
@Value("${stock-locations}")
private List<String> stockLocations;
// ...
// Add the following methods to the class
@Recurring(id = "monthly-sales-report", cron = "${monthly-sales-report.cron}", zoneId = "Europe/Brussels")
public void generateMonthlySalesReport() throws InterruptedException {
YearMonth previousMonth = YearMonth.now().minusMonths(1);
orderFulfillmentService.generateMonthlySalesReport(previousMonth);
}
@Recurring(id = "daily-resupply", cron = "${daily-resupply.cron}", zoneId = "Europe/Brussels")
public void resupply(JobContext jobContext) throws InterruptedException {
JobDashboardProgressBar jobDashboardProgressBar = jobContext.progressBar(stockLocations.size());
for(String stockLocation : stockLocations) {
orderFulfillmentService.resupply(stockLocation);
jobDashboardProgressBar.increaseByOne();
jobContext.logger().info(format("Resupplied stock %s", stockLocation));
}
}
}
JobRunr will automatically register methods annotated with @Reccuring and schedule them for execution at the specified times!
Note the use of application properties, we promised to come back to them. We use them to configure the CRON expressions of our recurring jobs. The last property is used to provide the locations of our different warehouses. Here we have assumed that our imaginary company operates from Belgium and set the zoneId accordingly.
Triggering an alert on task failure
The system may encounter an issue that causes jobs to fail during execution. In our scenario, it’s important to alert the operations team so they can make sure the failed task is one way or another completed.
JobRunr allows us to hook into a job lifecycle using Job Filters. The following code simulates the idea of sending a notification when all retries have been exhausted.
@Component
public class OrderFulfilmentTasksFilter implements JobServerFilter {
private static final Logger LOGGER = LoggerFactory.getLogger(OrderFulfilmentTasksFilter.class);
@Override
public void onFailedAfterRetries(Job job) {
// TODO alert operational team of Job failure
LOGGER.info("All retries failed for Job {}", job.getJobName());
}
}
JobRunr does not automatically register custom job filters. We need additional code to make sure our hook will be called by JobRunr’s background job servers. We can achieve this by overriding the BackgroundJobServer bean. Here, we illustrate another approach using Spring’s BeanPostProcessor.
@Component
public class BackgroundJobServerBeanPostProcessor implements BeanPostProcessor {
private final OrderFulfilmentTasksFilter orderFulfilmentTasksFilter;
public BackgroundJobServerBeanPostProcessor(OrderFulfilmentTasksFilter orderFulfilmentTasksFilter) {
this.orderFulfilmentTasksFilter = orderFulfilmentTasksFilter;
}
@Override
public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
if(bean instanceof BackgroundJobServer backgroundJobServer) {
backgroundJobServer.setJobFilters(Collections.singletonList(orderFulfilmentTasksFilter));
}
return bean;
}
}
Running the application
And we’re done, let’s enjoy the results! Start the application and head over to http://localhost:8000/dashboard. You should land on a beautiful dashboard, feel free to visit other pages.
Once you’re satisfied, visit http://localhost:8080/confirm-order to trigger the order fulfillment tasks. Head back to the dashboard, you can observe the jobs moving from one state to another until they succeed.
If you haven’t done so yet, visit http://localhost:8000/reccuring-jobs to see the recurring jobs. Have fun triggering them!
Going further
We could not include all the features of JobRunr in our example. We encourage interested developers to explore the library. You can fork the example project and make it your own! We also have a few more examples to get you started: example-spring, example-quarkus, example-micronaut. If you're having trouble, you can find most of the information you need in the documentation or start a discussion on Github to get help.
Here is a non-exhaustive list of potential things to explore:
Distributed processing: if our imaginary company gets successful a single machine will probably not be enough to handle all the tasks. See what happens when you run two or more of the example applications in parallel. Note: it’ll not work with an in memory database.
Scheduling jobs: technically recurring jobs are scheduled (unless they are triggered manually). You could also create non recurring scheduled tasks (a.k.a., delayed jobs).
Retries: It may be necessary to change the amount of retries as some jobs should not be retried at all. In the example we do so using the
JobBuilder, the same can be achieved via the@Jobannotation.Dashboard: delete or requeue jobs, check servers statuses on the servers page, notifications on the dashboard page (if any)
Best practices: read on the best practices to follow when using JobRunr
Logging job progress: keep an eye on the progress of a job by adding a progress bar. If you want to see this live in action, trigger the recurring job that with id
daily-resupply, find the job on the jobs page, click on the job id and wait until it starts processing 👀Different ways of configuring jobs: check out ‘Enqueueing jobs‘ for some usage examples on how to enqueue jobs configured using annotations or builders.
Conclusion
JobRunr is a great solution for processing your background jobs. It can be run in a distributed system and has built-in monitoring. The library is actively maintained, which means it's always getting new features and improving existing ones. That’s in part thanks to the great community helping us by contributing and providing feedback. We'd love for you to contribute or provide feedback on GitHub.
To make open source development sustainable, we develop and maintain JobRunr Pro. It offers Enterprise grade features such as Batches, Rate Limiters, Dashboard Single Sign-On (SSO), and many more. If you need those extra features, consider taking a subscription. You’ll not only support open source development but also our planet.
Resources
The complete source code for the example is available over on GitHub.
Check out the video of Josh Long's review of JobRunr.
Top comments (0)