Spring Builders: Donatavict

Task schedulers in Java: modern alternatives to Quartz Scheduler

Donatavict — Fri, 08 Nov 2024 14:05:07 +0000

Quartz is often considered the standard job scheduling library in Java, which can lead developers to overlook more modern alternatives.

For a long time, Quartz, also known as Quartz Scheduler, was the only viable open source task scheduler in Java. In fact, the official Quartz documentation still suggests that there is no real alternative. In this article, we will list a few different Quartz Scheduler alternatives that offer a similar set of features while being easier and more enjoyable to use.

The Quartz Scheduler: how good is it really?

Similar to its alternative, the Quartz Scheduler can be integrated into almost any Java application to schedule and process any task implemented in Java. In its decades of existence, Quartz has evolved into a feature-rich framework for persisting task states, automatically retrying tasks on failure, defining complex schedules, prioritizing tasks, achieving distributed processing, and more.

The fact that Quartz has been around for so long has two major advantages: the library is battle-tested and has a large community, which makes it easier to find help. All these elements together make it the most popular job scheduling library in Java. Despite all these strengths, choosing Quartz is no longer so obvious due to some glaring issues.

Limitations of Quartz

Quartz is showing its age

The Quartz scheduler has several issues that make it less of an obvious choice, especially when there are several really good alternatives.

The age of Quartz, while an advantage, is also a disadvantage, as the Quartz API and architecture are also getting older. The Quartz API can be considered verbose by today’s standards. This may not seem like a big deal, but it’s important for a library or language to remain attractive to the next generation of developers. Reduced verbosity is one of the main factors contributing to the popularity of Kotlin, and Java itself is moving toward becoming more beginner-friendly.

Another place where Quartz shows its age is in its architecture. When using an RDBMS, the scheduler requires the creation of more than 10 tables, which is probably overkill for most applications where the actual business logic requires fewer tables.

We have written an article that compares Quartz to a more modern solution, namely JobRunr. This article gives a feel for how complex and verbose Quartz is to use.

Performs worse than modern alternatives

When compared to more modern alternatives, Quartz is reported as having significantly lower performance. This performance issue may be caused by the use of row level lock and the complexity of the configuration making the Quartz Scheduler harder to fine-tune.

Lack of built-in monitoring

Quartz Scheduler lacks an intuitive dashboard or UI for tracking and monitoring jobs. It also doesn’t provide a way to interface with external monitoring tools like Grafana or Jaeger. Quartz users cannot actively monitor the system to proactively address issues such as job failures.

Distributed scheduling is opt-in

Quartz has the ability to handle the tasks in a distributed fashion, but this feature is opt-in. With tools like Kubernetes now allowing a pod to spin up on demand, such a feature should not be an afterthought, but rather the default.

Sporadic maintenance

The last non-beta release of Quartz dates back to October 23, 2019. While development was on hold, the ecosystem around it didn’t stop evolving. This hiatus led to a pile of unresolved issues. The users of the library are still suffering from the javax to jakarta namespace change, unless they use the latest release candidate, along with many bugs and security issues. They also cannot use newest Java features such as virtual threads, that may benefit background job processing, for instance to increase throughput…

There are good news for Quartz users. the recent acquisition by IBM led to a spark in activity, with the community helping to fix all the major issues. It is still unclear what this acquisition means for the future of Quartz…

Modern alternatives to Quartz Scheduler

If you’re looking for a Quartz alternative for your Java application, there are several modern open source schedulers that offer similar features, more developer-friendly APIs, and more robust support for distributed and cloud-based environments.

We distinguish between two types of schedulers:

The first class consists of Java job scheduling libraries. Similar to Quartz, they can be included in any Java application. They live inside your application.
The second class are workflow engines, which target a broader audience. These are standalone services that your application will need to communicate with.

Alternatives to Quartz in Java

There are two great alternatives to Quartz for scheduling jobs in Java, namely JobRunr and db-scheduler. They can be added as a dependency to any Java application to provide the needed features for distributed background task processing and persisted task scheduling.

These libraries keep the number of third-party dependencies to a minimum, providing better security insurance. They are available as standard Java jars on Maven Central.

JobRunr

JobRunr is a modern, actively maintained, open source Java library designed for background job processing. JobRunr offers an easy-to-use API that simplifies job scheduling. Unlike Quartz, JobRunr was designed with cloud-native applications in mind and provides support for distributed job scheduling.

JobRunr aims to be developer friendly by providing a simple, flexible and straightforward API, automatic retries on failure, and seamless integration with your existing infrastructure. In fact, all you need to create a background job is a Java 8 lambda. JobRunr also supports all major SQL databases and popular NoSQL databases.

A feature of JobRunr that neither Quartz nor db-scheduler provide out of the box is a dashboard. Thanks to the built-in dashboard, it’s easy to monitor the system to see why a job failed or to perform actions such as requeuing or deleting a job.

Used and trusted by industry leaders, the open source version of JobRunr provides the essential features you need to schedule tasks in Java. You can expect features similar to those in the Quartz Scheduler, but much easier to use. The JobRunr team also develop and maintain JobRunr Pro which offers Enterprise grade features, making JobRunr Pro the best job scheduling library in Java!

Get started with JobRunr or watch Ronald present the motivation, design and code of JobRunr at Spring I/O 2022!

DB Scheduler

Another good alternative to Quartz is db-scheduler. The library was originally designed to be a simpler alternative to Quartz. This design reduces the number of required tables to one (compared to Quartz’s 11 tables!) and provides a much simpler API.

Similar to JobRunr, db-scheduler is distributed by default and guarantees that the same job will not be run concurrently. This guarantee is achieved by using the same centralized DB; in the case of db-scheduler, a relational database is typically required.

DB Scheduler provides all the essential features for background job scheduling, it can handle fire-and-forget jobs as well as scheduled (a.k.a. delayed) and recurring jobs. The library provides some simple monitoring via Micrometer. It can also perform an automatic retry if a task fails.

The library is well maintained and growing in features to match the capabilities of Quartz Scheduler, without the extra complexity.

Other job scheduling tools

There are several job scheduling alternatives to Quartz that are aimed at a broader audience; we could probably not list them all, even if we wanted to. These alternatives come as standalone services, often referred to as workflow engines, that communicates with your application.

Using these tools means you’ll have to get used to the terminology they use. In particular, you’ll have to accept that every task, even the simplest, must be defined as a workflow.

Here we’ll briefly introduce two such workflow engines that work well with Java applications: Temporal and Kestra.

Temporal

Temporal is an open source workflow engine designed to make applications more resilient. The engine is written in Go and can be interfaced with your Java application using the provided Java SDK.

Very similar to pure Java-based solutions like JobRunr or Quartz, Temporal provides the essential features that help developers to focus on business logic instead of worrying about the complex mechanisms behind a resilient and distributed scheduler.

In addition to providing features such as retries, scalability, delayed and recurring executions, Temporal also provides a user interface that allows you to monitor workflow execution to detect failed tasks.

Kestra

Kestra is an open-source workflow automation platform that makes job scheduling easy. The engine is written in Java but aims at running any task in any programming language. Kestra is a great low-code alternative to Quartz. It provides several hundreds of plugin allowing to extract data from any database, cloud storage, or API, and run scripts in any language.

Kestra provides a UI for writing or configuring workflows. This UI can also be used to monitor the system and track workflow state changes.

Kestra also has all the essential features required for job scheduling: reliable distributed system, retries, delayed and recurring executions, etc.

Conclusion

For a long time, there was no real alternative to Quartz for job scheduling in Java. This is no longer the case, as there are several modern tools available to developers. These tools are simpler to use while providing the essential building blocks to achieve reliable and distributed scheduling.

You may be looking for a persistent task scheduling library that integrates seamlessly into a Java application. In this case, we highly recommend trying out our solution JobRunr. DB Scheduler is another option that can give you satisfactory results.

You can also take a look at workflow engines such as Temporal or Kestra. These are especially useful if your tasks are written in different programming languages.

With these diverse tools at their disposal, Java developers can now select the one that best meets their needs, rather than relying on a single option.

Getting started with JobRunr - use case example (pt. II)

Donatavict — Tue, 28 May 2024 10:53:08 +0000

Authors: @auloin @rdehuyss .

Read the Part I

An example: Automated order fulfillment system using JobRunr

To illustrate how JobRunr works, let’s simulate an order fulfillment system. Order fulfillment involves tasks required to completely satisfy a customer's order. It's a process where each step can affect another, which happens on several levels. For example, a confirmed order reduces stock, and to avoid stock-outs it is important to monitor and replenish the inventory.

The order fulfillment workflow has a lot of potential for automation where events can trigger jobs, which in turn trigger events until all steps are completed. As some of the workflow happens in the background, it’s a good opportunity to illustrate JobRunr! The setup of this example is as follows:

We’ll create an order confirmation that will trigger three tasks that can run asynchronously: 1) send a confirmation email to the customer, 2) notify the warehouse of the arrival of the order, and 3) initiate shipment by calling the preferred carrier’s API.
Some tasks are recurring. We illustrate this using two tasks: 1) a job that generates sales reports monthly, and 2) a job that monitors the inventory on a daily basis.
Something may go wrong while executing the tasks, in those cases, we want to trigger an alert.

This setup will allow to learn:

How to setup JobRunr in Spring Boot application
How to enqueue jobs using a Java 8 lambda configured via the @Job annotation and the JobBuilder
How to create recurring jobs using the @Recurring annotation
How to hook into a job lifecycle using a JobFilter

Please note that this example focuses on JobRunr side of things, we leave the implementation of object such as the Inventory, the Order, the Product, etc. as an exercise. Also the running of tasks is simulated by Thread.sleep which implies explicit handling of InterruptedException, you probably won't need those in an actual application. Although if you encounter such a case in your project it’s a best practice to throw the InterruptedException, JobRunr will handle it.

Setting up JobRunr in a Spring Boot application

Adding the JobRunr Spring Boot starter dependency

The setup is very easy for Spring Boot, it often consists of adding JobRunr’s Spring Boot starter artifact as a dependency. The following snippet shows how to include it when using Maven as a build tool.

<dependency> 
    <groupId>org.jobrunr</groupId> 
    <artifactId>jobrunr-spring-boot-3-starter</artifactId>
    <version>${jobrunr.version}</version> 
</dependency>

JobRunr requires adding database dependencies. We can skip this step as this is usually done when initializing the Spring Boot application. Nonetheless, it’s worth noting that JobRunr supports several databases. Another DB related feature is that all JobRunr related migrations are automatically handled by default. You can always decide otherwise and take control over the DB setup.

JobRunr also needs a JSON processing library, but as Spring Boot by default comes with Jackson support this is already covered.

Note: JobRunr also supports the older Spring Boot 2 and other frameworks such as Quarkus and Micronaut. The library can be found on Maven Central, feel free to use your preferred build tool.

Configuring JobRunr

JobRunr can be configured using application.properties. Job processing and the dashboard are disabled by default. To enable them, we’ll add the following to the configuration file:

org.jobrunr.dashboard.enabled=true
org.jobrunr.background-job-server.enabled=true

While we’re still in application.properties we will also add:

monthly-sales-report.cron=0 0 1 * *
daily-resupply.cron=0 0 * * *

stock-locations=Brussels,Antwerp,Bruges,Liege

Those will be useful to our recurring jobs as we’ll see later!

JobRunr offers advanced configuration options allowing to fine tune the application. When using the JobRunr’s Spring Boot starters or the other frameworks integrations, it’s also possible to override the predefined Beans.

Implementing the order fulfillment system

Now that the set up is done, we can start implementing our simulated order fulfillment process. We’ll focus on the JobRunr sides of things, i.e., enqueueing jobs, creating recurring jobs and hooking into a job lifecycle.

Triggering tasks on order confirmation

Once an order is confirmed, our system will trigger three tasks:

Send the order confirmation to the customer.
Notify the warehouse of the arrival of the order so work on the packaging can be swiftly done. This step may also involve generating and optimizing packing slips.
Initiate shipment by making a call to the selected carrier's booking API.

To automate these tasks with JobRunr we will create the following classes: a Spring’s RestController (namely OrderFulfillmentController), an OrderFulfillmentService, and an OrderFulfillmentTasks.

The OrderFulfillmentController will provide an endpoint to enqueue jobs. These jobs’ logic are implemented by the OrderFulfillmentService. We could stop here, but we would like the business logic to be separated from the job processing logic which will include features from JobRunr. Thus the introduction of OrderFulfillmentTasks.

Here is initial implementation of the OrderFulfillmentService class, we'll add more to it later.

@Service
public class OrderFulfillmentService {
    private static final Logger LOGGER = LoggerFactory.getLogger(OrderFulfillmentService.class);
    public void sendOrderConfirmation(UUID orderId) throws InterruptedException {
        // TODO retrieve order from database, generate an invoice and send the confirmation to the customer
        LOGGER.info("Order {}: sending order confirmation%n", orderId);
        Thread.sleep(2000);
    }

    public void notifyWarehouse(UUID orderId) throws InterruptedException {
        // TODO notify the warehouse of the arrival of a new order after retrieving/computing the necessary data using the orderId
        LOGGER.info("Order {}: notifying the warehouse", orderId);
        Thread.sleep(1000);
    }

    public void initiateShipment(UUID orderId) throws InterruptedException {
        // TODO call the carrier's shipment initiation endpoint after retrieving/computing the necessary data using the orderId
        LOGGER.info("Order {}: initiating shipment", orderId);
        Thread.sleep(5000);
    }
}

Note that we use Thread.sleep to simulate work, this forces us to explicitly handle InterruptedException, in an actual application, it’s probably not needed.

The OrderFulfillmentTasks class makes use of the OrderFulfillmentService. Notice the additional @Job, which allows setting values to a job’s attributes. This annotation is very handy! You may also use the alternative: the JobBuilder. We're doing exactly that in the body of the method, the jobs are configured using the JobBuilder and then saved atomically. This method is essentially a job that creates other jobs, so it also benefits from JobRunr's fault tolerance capabilities.

@Service
public class OrderFulfillmentTasks {

    private final OrderFulfillmentService orderFulfillmentService;

    public OrderFulfillmentTasks(OrderFulfillmentService orderFulfillmentService) {
        this.orderFulfillmentService = orderFulfillmentService;
    }

    @Job(name = "order-%0")
    public void enqueueConfirmedOrderTasks(UUID orderId) {
        BackgroundJob.create(of(
                aJob()
                        .withName(format("order-%s-confirmation", orderId))
                        .withDetails(() -> orderFulfillmentService.sendOrderConfirmation(orderId)),
                aJob()
                        .withName(format("order-%s-warehouse-notification", orderId))
                        .withAmountOfRetries(20)
                        .withDetails(() -> orderFulfillmentService.notifyWarehouse(orderId)),
                aJob()
                        .withName(format("order-%s-shipment-initiation", orderId))
                        .withDetails(() -> orderFulfillmentService.initiateShipment(orderId))
        ));
    }
}

Note the use of withAmountOfRetries(20) for the warehouse notification task, we have to increase the amount of retries since our internal service is quite unstable.

The OrderFulfillmentController is quite simple as it exposes a single endpoint that requests JobRunr to enqueue jobs. Once the metadata of these jobs are saved in the database, the server workers will process them asynchronously when they are ready.

@RestController
public class OrderFulfillmentController {

    @GetMapping("/confirm-order")
    public String confirmOrder() {
        UUID orderId = UUID.randomUUID();
        BackgroundJob.<OrderFulfillmentTasks>enqueue(x -> x.enqueueConfirmedOrderTasks(orderId));

        return "Done";
    }
}

Creating reccuring tasks

The first part of our order fulfillment system is done. Now, we’ll add recurring tasks to know how well the company is doing and to make sure we never go out-of-stock!

Let’s update the OrderFulfillmentService to add the logic for these two tasks.

@Service
public class OrderFulfillmentService {
    // ...

    // Add the following methods to the class

    public void generateMonthlySalesReport(YearMonth month) throws InterruptedException {
        // TODO aggregate monthly sales, generate PDF and send it to managers
        LOGGER.info("Sales report: generating monthly sales report on {} for month {}", Instant.now(), month);
        Thread.sleep(10000);
    }

    public void resupply(String stockLocation) throws InterruptedException {
        // TODO check the stock and contact supplier if needed
        LOGGER.info("Inventory resupply: resupplying the inventory on {} for stock {}", Instant.now(), stockLocation);
        Thread.sleep(5000);
    }
}

We now update the OrderFulfillmentTasks to register those tasks. As they are recurring, we annotate them with @Recurring and a few attributes.

@Service
public class OrderFulfillmentTasks {

   // Add the following attribute to the class

    @Value("${stock-locations}")
    private List<String> stockLocations;

    // ...

    // Add the following methods to the class

    @Recurring(id = "monthly-sales-report", cron = "${monthly-sales-report.cron}", zoneId = "Europe/Brussels")
    public void generateMonthlySalesReport() throws InterruptedException {
        YearMonth previousMonth = YearMonth.now().minusMonths(1);
        orderFulfillmentService.generateMonthlySalesReport(previousMonth);
    }

    @Recurring(id = "daily-resupply", cron = "${daily-resupply.cron}", zoneId = "Europe/Brussels")
    public void resupply(JobContext jobContext) throws InterruptedException {
        JobDashboardProgressBar jobDashboardProgressBar = jobContext.progressBar(stockLocations.size());
        for(String stockLocation : stockLocations) {
            orderFulfillmentService.resupply(stockLocation);
            jobDashboardProgressBar.increaseByOne();
            jobContext.logger().info(format("Resupplied stock %s", stockLocation));
        }
    }

}

JobRunr will automatically register methods annotated with @Reccuring and schedule them for execution at the specified times!

Note the use of application properties, we promised to come back to them. We use them to configure the CRON expressions of our recurring jobs. The last property is used to provide the locations of our different warehouses. Here we have assumed that our imaginary company operates from Belgium and set the zoneId accordingly.

Triggering an alert on task failure

The system may encounter an issue that causes jobs to fail during execution. In our scenario, it’s important to alert the operations team so they can make sure the failed task is one way or another completed.

JobRunr allows us to hook into a job lifecycle using Job Filters. The following code simulates the idea of sending a notification when all retries have been exhausted.

@Component
public class OrderFulfilmentTasksFilter implements JobServerFilter {
    private static final Logger LOGGER = LoggerFactory.getLogger(OrderFulfilmentTasksFilter.class);

    @Override
    public void onFailedAfterRetries(Job job) {
        // TODO alert operational team of Job failure
        LOGGER.info("All retries failed for Job {}", job.getJobName());
    }
}

JobRunr does not automatically register custom job filters. We need additional code to make sure our hook will be called by JobRunr’s background job servers. We can achieve this by overriding the BackgroundJobServer bean. Here, we illustrate another approach using Spring’s BeanPostProcessor.

@Component
public class BackgroundJobServerBeanPostProcessor implements BeanPostProcessor {
    private final OrderFulfilmentTasksFilter orderFulfilmentTasksFilter;

    public BackgroundJobServerBeanPostProcessor(OrderFulfilmentTasksFilter orderFulfilmentTasksFilter) {
        this.orderFulfilmentTasksFilter = orderFulfilmentTasksFilter;
    }

    @Override
    public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
        if(bean instanceof BackgroundJobServer backgroundJobServer) {
            backgroundJobServer.setJobFilters(Collections.singletonList(orderFulfilmentTasksFilter));
        }
        return bean;
    }
}

Running the application

And we’re done, let’s enjoy the results! Start the application and head over to http://localhost:8000/dashboard. You should land on a beautiful dashboard, feel free to visit other pages.
Once you’re satisfied, visit http://localhost:8080/confirm-order to trigger the order fulfillment tasks. Head back to the dashboard, you can observe the jobs moving from one state to another until they succeed.

If you haven’t done so yet, visit http://localhost:8000/reccuring-jobs to see the recurring jobs. Have fun triggering them!

Going further

We could not include all the features of JobRunr in our example. We encourage interested developers to explore the library. You can fork the example project and make it your own! We also have a few more examples to get you started: example-spring, example-quarkus, example-micronaut. If you're having trouble, you can find most of the information you need in the documentation or start a discussion on Github to get help.

Here is a non-exhaustive list of potential things to explore:

Distributed processing: if our imaginary company gets successful a single machine will probably not be enough to handle all the tasks. See what happens when you run two or more of the example applications in parallel. Note: it’ll not work with an in memory database.
Scheduling jobs: technically recurring jobs are scheduled (unless they are triggered manually). You could also create non recurring scheduled tasks (a.k.a., delayed jobs).
Retries: It may be necessary to change the amount of retries as some jobs should not be retried at all. In the example we do so using the JobBuilder, the same can be achieved via the @Job annotation.
Dashboard: delete or requeue jobs, check servers statuses on the servers page, notifications on the dashboard page (if any)
Best practices: read on the best practices to follow when using JobRunr
Logging job progress: keep an eye on the progress of a job by adding a progress bar. If you want to see this live in action, trigger the recurring job that with id daily-resupply, find the job on the jobs page, click on the job id and wait until it starts processing 👀
Different ways of configuring jobs: check out ‘Enqueueing jobs‘ for some usage examples on how to enqueue jobs configured using annotations or builders.

Conclusion

JobRunr is a great solution for processing your background jobs. It can be run in a distributed system and has built-in monitoring. The library is actively maintained, which means it's always getting new features and improving existing ones. That’s in part thanks to the great community helping us by contributing and providing feedback. We'd love for you to contribute or provide feedback on GitHub.

To make open source development sustainable, we develop and maintain JobRunr Pro. It offers Enterprise grade features such as Batches, Rate Limiters, Dashboard Single Sign-On (SSO), and many more. If you need those extra features, consider taking a subscription. You’ll not only support open source development but also our planet.

Resources

The complete source code for the example is available over on GitHub.
Check out the video of Josh Long's review of JobRunr.

Getting started with JobRunr - Introduction (pt. I)

Donatavict — Fri, 17 May 2024 12:31:19 +0000

Task scheduler in Java

JobRunr is an open-source Java library for task scheduling and distributed background job processing. It offers an effortless way to perform background tasks using only Java 8 lambdas.

Whether it is a fire-and-forget, scheduled or recurring job, JobRunr analyzes the lambda and stores the metadata needed to process the job in a database. This simple architecture, with a few other mechanisms, allows a job to be executed on any available server running your application (👋 k8s).

In the event of a failure, JobRunr automatically retries the job. Thanks to the built-in dashboard, it's easy to monitor the system to see why a job failed or to perform actions such as requeuing or deleting a job.

The software integrates easily into any Java application by adding JobRunr as a dependency. It also works seamlessly with frameworks like Spring Boot, Quarkus, and Micronaut.

JobRunr use cases in Java

The applications of JobRunr are almost limitless: anything that can be done in Java can be automated and scheduled with JobRunr. Would you like to send mass notifications, fire webhooks, process images, or videos? JobRunr can do it all for you, so you can focus on solving the business problems at hand.

JobRunr is used in various industries from retail to healthcare to marketing, no matter the size of your company. We have written a few articles about how JobRunr Pro is used in those industries.

Core task scheduler features for Developers:

Developer friendly: the API of JobRunr is simple, flexible and straightforward - with a simple, yet extensive set up.
Simple adoption: JobRunr fits into any software architecture and requires little changes to your codebase while requiring a low amount of dependencies.
Framework integration: JobRunr supports the most popular Java frameworks: Spring, Quarkus and Micronaut.
Various storage options: persistent storage is done via either RDBMS (e.g. Postgres, MariaDB/MySQL, Oracle, SQL Server, DB2, and SQLite) or NoSQL (ElasticSearch, MongoDB and Redis)
Cloud-native and cloud agnostic: deployable anywhere, either on your preferred cloud provider or on-premise
Distributed processing: scale is not an issue - JobRunr supports job distribution across a cluster.
Fault tolerance: JobRunr is also fault-tolerant - is an external web service down? No worries, the job is automatically retried 10-times with a smart back-off policy. The retry policy is of course configurable!
Real-time monitoring: JobRunr provides a dashboard that allows users to monitor jobs, trigger or delete recurring jobs, and requeue or delete jobs.
Virtual threads: JobRunr supports virtual threads to allow for increased throughput of I/O jobs.
Comprehensive documentation: the documentation covers everything from setup to advanced features, easing the learning curve.

Stay tuned for more of getting started with JobRunr and its implementation!😉

Authors: Ismaila Abdoulahi, Ronald Dehuysser, Donata Petkeviciute