By default, Quartz only provides support for the traditional relational databases. Browsing through, I stumbled upon this github repository by Michael Klishin which provides a MongoDB implementation of the Quartz library in a clustered environment.
We will be using a Spring boot application to show you how we can integrate the Quartz library for scheduling in a clustered environment using MongoDB.
The GitHub repository with the code shown in this article can be found here.
All quartz related configuration is stored in a property file. The attributes we will be using are as follows;
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~ # Quartz Job Scheduling # ~~~~~~~~~~~~~~~~~~~~~~~~~~~ # Use the MongoDB store org.quartz.jobStore.class=com.quartz.mongo.intro.quartzintro.scheduler.CustomMongoQuartzSchedulerJobStore # --- # Note that all the mongo db configuration are set in the CustomMongoQuartzSchedulerJobStore.java class --- # MongoDB URI (optional if 'org.quartz.jobStore.addresses' is set) #org.quartz.jobStore.mongoUri=mongodb://localhost:27017 # Comma separated list of mongodb hosts/replica set seeds (optional if 'org.quartz.jobStore.mongoUri' is set) #org.quartz.jobStore.addresses=localhost # Will be used to create collections like quartz_jobs, quartz_triggers, quartz_calendars, quartz_locks org.quartz.jobStore.collectionPrefix=quartz_ # Thread count setting is ignored by the MongoDB store but Quartz requires it org.quartz.threadPool.threadCount=1 # Skip running a web request to determine if there is an updated version of Quartz available for download org.quartz.scheduler.skipUpdateCheck=true org.quartz.jobStore.isClustered=true #The instance ID will be auto generated by Quartz for all nodes running in a cluster. org.quartz.scheduler.instanceId=AUTO org.quartz.scheduler.instanceName=quartzMongoInstance
Let us look at some of these properties. Others are self-explanatory with the comments provided.
- org.quartz.jobStore.class : This defines the job store class which will handle storing the job related details in the database. By default, with the GitHub project mentioned before, we are provided with the MongoDBJobStore. For the purposes of this article however, we will extend the functionality provided by this class with our own implementation which will handle the MongoDB configuration based on Spring profiles.
- org.quartz.jobStore.mongoUri : You will define the comma separated MongoDB URI's here if you wanted to use the default MongoDBJobStore class. On this implementation however, since we are defining a custom job store, we will not be using this property. An example of how you would define this would be mongodb://<ip1>:<port>,<ip2>:<port>
- org.quartz.jobStore.collectionPrefix : This property defines the prefix for the collections created for the purposes of storing quartz specific details.
Let us first see how our JobStore configuration class looks like;
package com.quartz.mongo.intro.quartzintro.scheduler;
import org.apache.commons.lang3.StringUtils;
import org.quartz.impl.StdSchedulerFactory;
import org.springframework.beans.factory.config.YamlPropertiesFactoryBean;
import org.springframework.core.io.ClassPathResource;
import com.novemberain.quartz.mongodb.MongoDBJobStore;
import com.quartz.mongo.intro.quartzintro.constants.SchedulerConstants;
import com.quartz.mongo.intro.quartzintro.constants.SystemProperties;
/**
*
* <p>
* We extend the {@link MongoDBJobStore} because we need to set the custom mongo
* db parameters. Some of the configuration comes from system properties set via
* docker and the others come via the application.yml files we have for each
* environment.
* </p>
*
* < These are set as part of initialization. This class is initialized by
* {@link StdSchedulerFactory} and defined in the quartz.properties file.
*
* </p>
*
* @author dinuka
*
*/
public class CustomMongoQuartzSchedulerJobStore extends MongoDBJobStore {
private static String mongoAddresses;
private static String userName;
private static String password;
private static String dbName;
private static boolean isSSLEnabled;
private static boolean isSSLInvalidHostnameAllowed;
public CustomMongoQuartzSchedulerJobStore() {
super();
initializeMongo();
setMongoUri("mongodb://" + mongoAddresses);
setUsername(userName);
setPassword(password);
setDbName(dbName);
setMongoOptionEnableSSL(isSSLEnabled);
setMongoOptionSslInvalidHostNameAllowed(isSSLInvalidHostnameAllowed);
}
/**
* <p>
* This method will initialize the mongo instance required by the Quartz
* scheduler.
*
* The use case here is that we have two profiles;
* </p>
*
* <ul>
* <li>Development</li>
* <li>Production</li>
* </ul>
*
* <p>
* So when constructing the mongo instance to be used for the Quartz
* scheduler, we need to read the various properties set within the system
* to determine which would be appropriate depending on which spring profile
* is active.
* </p>
*
*/
private static void initializeMongo() {
/**
* The use case here is that when we run our application, the property
* spring.profiles.active is set as a system property during production.
* But it will not be set in a development environment.
*/
String env = System.getProperty(SystemProperties.ENVIRONMENT);
env = StringUtils.isNotBlank(env) ? env : "dev";
YamlPropertiesFactoryBean commonProperties = new YamlPropertiesFactoryBean();
commonProperties.setResources(new ClassPathResource("application.yml"));
/**
* The mongo DB user name and password are only password as command line
* parameters in the production environment and for the development
* environment it will be null which is why we use
* StringUtils#trimToEmpty so we can pass empty strings for the user
* name and password in the development environment since we do not have
* authentication on the development environment.s
*/
userName = StringUtils.trimToEmpty(commonProperties.getObject().getProperty(SystemProperties.SERVER_NAME));
password = StringUtils.trimToEmpty(System.getProperty(SystemProperties.MONGO_PASSWORD));
dbName = commonProperties.getObject().getProperty(SchedulerConstants.QUARTZ_SCHEDULER_DB_NAME);
YamlPropertiesFactoryBean environmentSpecificProperties = new YamlPropertiesFactoryBean();
userName = commonProperties.getObject().getProperty(SystemProperties.SERVER_NAME);
switch (env) {
case "prod":
environmentSpecificProperties.setResources(new ClassPathResource("application-prod.yml"));
/**
* By deafult, in the production mongo instance, SSL is enabled and
* SSL invalid host name allowed property is set.
*/
isSSLEnabled = true;
isSSLInvalidHostnameAllowed = true;
mongoAddresses = environmentSpecificProperties.getObject().getProperty(SystemProperties.MONGO_URI);
break;
case "dev":
/**
* For the development profile, we just read the mongo URI that is
* set.
*/
environmentSpecificProperties.setResources(new ClassPathResource("application-dev.yml"));
mongoAddresses = environmentSpecificProperties.getObject().getProperty(SystemProperties.MONGO_URI);
break;
}
}
}
In this above implementation, we have retrieved the MongoDB details pertaining to the active profile. If no profile is defined it defaults to the development profile. We have used the YamlPropertiesFactoryBean here to read off the application properties pertaining to different environments.
Moving on, we then need to let Spring manage the creation of the Quartz configuration using the SchedulerFactoryBean.
package com.quartz.mongo.intro.quartzintro.config;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;
import org.springframework.scheduling.quartz.SchedulerFactoryBean;
/**
* This class will configure and setup quartz using the
* {@link SchedulerFactoryBean}
*
* @author dinuka
*
*/
@Configuration
public class QuartzConfiguration {
/**
* Here we integrate quartz with Spring and let Spring manage initializing
* quartz as a spring bean.
*
* @return an instance of {@link SchedulerFactoryBean} which will be managed
* by spring.
*/
@Bean
public SchedulerFactoryBean schedulerFactoryBean() {
SchedulerFactoryBean scheduler = new SchedulerFactoryBean();
scheduler.setApplicationContextSchedulerContextKey("applicationContext");
scheduler.setConfigLocation(new ClassPathResource("quartz.properties"));
scheduler.setWaitForJobsToCompleteOnShutdown(true);
return scheduler;
}
}
We define this as a Configuration class so that it will be picked up when we run the Spring boot application.
The call to setApplicationContextSchedulerContextKey method here is in order to get a reference to the Spring application context within our job class which is as follows;
package com.quartz.mongo.intro.quartzintro.scheduler.jobs;
import org.quartz.DisallowConcurrentExecution;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;
import org.quartz.PersistJobDataAfterExecution;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.ApplicationContext;
import org.springframework.core.env.Environment;
import org.springframework.scheduling.quartz.QuartzJobBean;
import org.springframework.scheduling.quartz.SchedulerFactoryBean;
import com.quartz.mongo.intro.quartzintro.config.JobConfiguration;
import com.quartz.mongo.intro.quartzintro.config.QuartzConfiguration;
/**
*
* This is the job class that will be triggered based on the job configuration
* defined in {@link JobConfiguration}
*
* @author dinuka
*
*/
@PersistJobDataAfterExecution
@DisallowConcurrentExecution
public class SampleJob extends QuartzJobBean {
private static Logger log = LoggerFactory.getLogger(SampleJob.class);
private ApplicationContext applicationContext;
/**
* This method is called by Spring since we set the
* {@link SchedulerFactoryBean#setApplicationContextSchedulerContextKey(String)}
* in {@link QuartzConfiguration}
*
* @param applicationContext
*/
public void setApplicationContext(ApplicationContext applicationContext) {
this.applicationContext = applicationContext;
}
/**
* This is the method that will be executed each time the trigger is fired.
*/
@Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
log.info("This is the sample job, executed by {}", applicationContext.getBean(Environment.class));
}
}
As you can see, we get a reference to the application context when the SchedulerFactoryBean is initialised. The part of the Spring documentation I would like to draw you attention to is as follows;
In case of a QuartzJobBean, the reference will be applied to the Jobinstance as bean property. An "applicationContext" attribute willcorrespond to a "setApplicationContext" method in that scenario.
Next up, we go on to configure the job to be run with the frequency by which to run the scheduled activity.
package com.quartz.mongo.intro.quartzintro.config;
import static org.quartz.TriggerBuilder.newTrigger;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.util.Date;
import javax.annotation.PostConstruct;
import org.quartz.JobDetail;
import org.quartz.JobKey;
import org.quartz.SimpleScheduleBuilder;
import org.quartz.Trigger;
import org.quartz.TriggerKey;
import org.quartz.impl.JobDetailImpl;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.quartz.SchedulerFactoryBean;
import com.quartz.mongo.intro.quartzintro.constants.SchedulerConstants;
import com.quartz.mongo.intro.quartzintro.scheduler.jobs.SampleJob;
/**
*
* This will configure the job to run within quartz.
*
* @author dinuka
*
*/
@Configuration
public class JobConfiguration {
@Autowired
private SchedulerFactoryBean schedulerFactoryBean;
@PostConstruct
private void initialize() throws Exception {
schedulerFactoryBean.getScheduler().addJob(sampleJobDetail(), true, true);
if (!schedulerFactoryBean.getScheduler().checkExists(new TriggerKey(
SchedulerConstants.SAMPLE_JOB_POLLING_TRIGGER_KEY, SchedulerConstants.SAMPLE_JOB_POLLING_GROUP))) {
schedulerFactoryBean.getScheduler().scheduleJob(sampleJobTrigger());
}
}
/**
* <p>
* The job is configured here where we provide the job class to be run on
* each invocation. We give the job a name and a value so that we can
* provide the trigger to it on our method {@link #sampleJobTrigger()}
* </p>
*
* @return an instance of {@link JobDetail}
*/
private static JobDetail sampleJobDetail() {
JobDetailImpl jobDetail = new JobDetailImpl();
jobDetail.setKey(
new JobKey(SchedulerConstants.SAMPLE_JOB_POLLING_JOB_KEY, SchedulerConstants.SAMPLE_JOB_POLLING_GROUP));
jobDetail.setJobClass(SampleJob.class);
jobDetail.setDurability(true);
return jobDetail;
}
/**
* <p>
* This method will define the frequency with which we will be running the
* scheduled job which in this instance is every minute three seconds after
* the start up.
* </p>
*
* @return an instance of {@link Trigger}
*/
private static Trigger sampleJobTrigger() {
return newTrigger().forJob(sampleJobDetail())
.withIdentity(SchedulerConstants.SAMPLE_JOB_POLLING_TRIGGER_KEY,
SchedulerConstants.SAMPLE_JOB_POLLING_GROUP)
.withPriority(50).withSchedule(SimpleScheduleBuilder.repeatMinutelyForever())
.startAt(Date.from(LocalDateTime.now().plusSeconds(3).atZone(ZoneId.systemDefault()).toInstant()))
.build();
}
}
There are many ways you can configure your scheduler including cron configuration. For the purposes of this article, we will define a simple trigger to run every minute, three seconds after start up. We define this as a Configuration class so that it will be picked up when we run the Spring boot application.
That is about it. When you now run the Spring Boot application class found in the GitHub repository with a running MongoDB instance, you will see the following collections created;
- quartz_calendars
- quartz_jobs
- quartz_locks
- quartz_schedulers
- quartz_triggers
Thank you for reading and if there are any comments, improvements, suggestions, do kindly leave by a comment which is always appreciated.
