Using Hibernate Search with Spring Boot
Spring Boot is a framework, that makes it much easier to develop Spring based applications, by following a convention over configuration principle (while in contrast Spring critics claim that the framework’s principle is rather configuration over everything). In this article, I am going to provide an example how to achieve the following:
- Create a simple Web application based on Spring Boot
- Persist and access data with Hibernate
- Make it searchable with Hibernate Search (Lucine)
I use Eclipse with a Gradle plugin for convenience. MySQL will be our back-end for storing the data. The full example can be obtained from my [Github Repository][1].
Bootstrapping: Create a Simple Spring Boot Webapp
The easiest way to start with Spring Boot is heading over to [start.spring.io][2] and create a new project. In this example, I will use Gradle for building the application and handling the dependencies and I add Web and JPA starters.
[][3]
Download the archive to your local drive and extract it to a folder. I called the project SearchaRoo.
Import the Project with Eclipse
Import it as an existing Gradle Project in Eclipse by using the default settings. You will end up with a nice little project structure as shown below:
[][4]
We have a central application starter class denoted SearchaRooAppication.java, package definitions, application properties and even test classes. The great thing with Spring Boot is that it is very simple to start and that you can debug it as every other local Java application. There is no need for remote debugging or complex application server setups.
Prepare the Database
We need a few permissions on our MySQL instance before we can start.
CREATE DATABASE spring_employees;
CREATE USER 'dev'@'localhost' IDENTIFIED BY 'sEcReT';
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, FILE, REFERENCES, INDEX,
ALTER, SHOW DATABASES, SUPER, LOCK TABLES, CREATE VIEW, SHOW VIEW
on spring_employees.* TO 'dev'@'localhost';
GRANT RELOAD on *.* TO 'dev'@'localhost';
FLUSH PRIVILEGES;```
We can then add the connection details into the application.properties file. We will edit this file several times when the complexity of this project increases.
===============================
= JPA / HIBERNATE
===============================
Specify the DBMS
spring.jpa.database = MYSQL
Show or not log for each sql query
spring.jpa.show-sql = true spring.datasource.url=jdbc:mysql://127.0.0.1/employees?createDatabaseIfNotExist=true spring.datasource.username=dev spring.datasource.password=sEcReT spring.datasource.driver-class-name=com.mysql.jdbc.Driver```
Now the basic database setup is done. We can then start adding model classes.
Getting some Employees on Board
MySQL offers a rather small but well documented sample database called employees, which is hosted on Github. Obtain and import the data as follows:
git clone https://github.com/datacharmer/test_db.git
cd test_db
mysql -u dev -p sEcReT < employees.sql
The script creates a new schema called employees and you will end up with a schema like this:
[][5]
In the course of this article, we are going to model this schema with Java POJOs by annotating the entities and the a appropriate fields with JPA.
Dependencies
Before we can start modelling the entities in Java, have a look at the Gradle build file. We include additional dependencies for the MySQL connector and Apache commons.
buildscript {
ext {
springBootVersion = '1.5.1.RELEASE'
}
repositories {
mavenCentral()
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
}
}
apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'org.springframework.boot'
jar {
baseName = 'SearchaRoo'
version = '0.0.1-SNAPSHOT'
}
sourceCompatibility = 1.8
repositories {
mavenCentral()
}
dependencies {
compile('org.springframework.boot:spring-boot-starter-data-jpa')
compile('org.springframework.boot:spring-boot-starter-web')
testCompile('org.springframework.boot:spring-boot-starter-test')
compile("mysql:mysql-connector-java")
compile('org.apache.commons:commons-lang3:3.5')
}
Modelling Reality
The next step covers modelling the data which we imported with Java POJOs. Obviously this is not the most natural way, because in general you would create the model first and then add data to it, but as we already had the data we decided to go in this direction. In the application.properties file, set the database to the imported employees database and set the Hibernate create property to validate. With this setting, we can confirm that we modelled the Java classed in accordance with the database model defined by the MySQL employees database.
An example of such a class is shown below, the other classes can be found in the Github repository.
package at.stefanproell.model;
import java.util.Date;
import java.util.List;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.Id;
import javax.persistence.OneToMany;
@Entity(name="employees")
public class Employee {
@Id
@Column(name="emp_no")
private int employee_id;
@Column(name="birth_date")
private Date birthdate;
@Column(name="first_name")
private String firstname;
@Column(name="last_name")
private String lastname;
@Column(name="gender",columnDefinition = "ENUM('M', 'F', 'UNKNOWN') DEFAULT 'UNKNOWN'")
@Enumerated(EnumType.STRING)
private Gender gender;
@Column(name="hire_date")
private Date hireDate;
@OneToMany(mappedBy="employee")
List<Title> titles;
@OneToMany(mappedBy="employee")
List<Department_Employee> employee_department;
// Setters and getters
}```
Now that we have prepared the data model, our schema is now fixed and does not change any more. We can deactivate the Hibernate based dynamic generation of the database tables and use the Spring database initialization instead.To see if we modelled the data correctly, we import MySQL employee data dump we obtained before and import it into our newly created schema, which maps the Java POJOs.
## Importing the Initial Data
In the next step, we import the data from the MySQL employee database into our schema spring_hibernate. This schema contains the tables that Hibernate created for us. The following script copies the data between the two schemata. If you see an error, then there is an issue with your model.
<pre class="theme:github lang:mysql decode:true">-- The original data is stored in the database called employees
-- Spring created the new schema called spring_employees
USE `spring_employees`;
-- Departments
INSERT INTO `spring_employees`.`departments`
(`dept_no`,
`dept_name`)
SELECT `departments`.`dept_no`,
`departments`.`dept_name`
FROM `employees`.`departments`;
-- Employees
INSERT INTO `spring_employees`.`employees`
(`emp_no`,
`birth_date`,
`first_name`,
`gender`,
`hire_date`,
`last_name`)
SELECT `employees`.`emp_no`,
`employees`.`birth_date`,
`employees`.`first_name`,
`employees`.`gender`,
`employees`.`hire_date`,
`employees`.`last_name`
FROM `employees`.`employees`;
-- Join table
INSERT INTO `spring_employees`.`dept_emp`
(`emp_no`,
`dept_no`,
`from_date`,
`to_date`)
SELECT
`dept_emp`.`emp_no`,
`dept_emp`.`dept_no`,
`dept_emp`.`from_date`,
`dept_emp`.`to_date`
FROM `employees`.`dept_emp`;
-- Join table
INSERT INTO `spring_employees`.`dept_manager`
(
`emp_no`,
`dept_no`,
`from_date`,
`to_date`)
SELECT `dept_manager`.`emp_no`,
`dept_manager`.`dept_no`,
`dept_manager`.`from_date`,
`dept_manager`.`to_date`
FROM `employees`.`dept_manager`;
-- Titles
INSERT INTO `spring_employees`.`titles`
(`emp_no`,
`title`,
`from_date`,
`to_date`)
SELECT `titles`.`emp_no`,
`titles`.`title`,
`titles`.`from_date`,
`titles`.`to_date`
FROM `employees`.`titles`;
-- Salaries
INSERT INTO `spring_employees`.`salaries`
(`emp_no`,
`salary`,
`from_date`,
`to_date`)
SELECT `salaries`.`emp_no`,
`salaries`.`salary`,
`salaries`.`from_date`,
`salaries`.`to_date`
FROM `employees`.`salaries`;```
We now imported the data in the database schema that we defined for our project. Spring can load schema and initial data during start-up. So we provide two files, one containing the schema and the other one containing the data. To do that, we create two dumps of the database. One containing the schema only, the other one containing the data only.
mysqldump -u dev -psEcReT –no-data –databases spring_employees > src/main/resources/schema.sql mysqldump -u dev -psEcReT –no-create-info –databases employees > src/main/resources/data.sql```
By deactivating the Hibernate data generation and activating the Spring way, the database gets initialized every time the application starts. Change and edit the following lines in the application.properties
spring.jpa.hibernate.ddl-auto=none
spring.datasource.initialize=true
spring.datasource.schema=classpath:/schema.sql
spring.datasource.data=classpath:/data.sql```
Before we can import the data with the scripts, make sure to drop the schema and disable foreign key checks in the schema file and enable them again at the end. Spring ignores the actionable MySQL comments. So your schema file should contain this
<pre class="theme:github lang:default decode:true ">DROP DATABASE IF EXISTS `spring_employees`;
SET foreign_key_checks = 0;
// rest of the code
SET foreign_key_checks = 1;```
And also insert the two foreign key statements to the data file. Note that the import can take a while. If you are happy with the initialized data, you can deactivate the initialization by setting the variable to false: <span class="lang:default decode:true crayon-inline">spring.datasource.initialize=false</span>
The application.properties file meanwhile looks like this:
<pre class="theme:github lang:default decode:true "># ===============================
# = JPA / HIBERNATE
# ===============================
# Specify the DBMS
spring.jpa.database = MYSQL
# Show or not log for each sql query
spring.jpa.show-sql = true
spring.datasource.url=jdbc:mysql://127.0.0.1/spring_employees?createDatabaseIfNotExist=true
spring.datasource.username=dev
spring.datasource.password=sEcReT
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.connectionProperties=useUnicode=true;characterEncoding=utf-8;
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect
# Do not initialize anything
spring.jpa.hibernate.ddl-auto=none
spring.datasource.initialize=false
spring.datasource.schema=classpath:/schema.sql
spring.datasource.data=classpath:/data.sql
spring.datasource.platform=mysql```
# Adding Hibernate Search
Hibernate search offers full-text search capabilities by using a dedicated index. We need to add the dependencies to the build file.
<pre class="theme:github lang:default decode:true ">dependencies {
compile('org.springframework.boot:spring-boot-starter-data-jpa')
compile('org.springframework.boot:spring-boot-starter-web')
testCompile('org.springframework.boot:spring-boot-starter-test')
compile("mysql:mysql-connector-java")
compile('org.apache.commons:commons-lang3:3.5')
compile("org.hibernate:hibernate-search-orm:5.5.6.Final")
compile('org.springframework.boot:spring-boot-starter-test')
compile('org.springframework.boot:spring-boot-starter-logging')
compile('org.springframework.boot:spring-boot-starter-freemarker')
}```
Refresh the gradle file after including the search dependencies.
## Adding Hibernate Search Dependencies
In this step, we annotate the model POJO classes and introduce the full-text search index. Hibernate search utilises just a few basic settings to get started. Add the following variables to tne application properties file.
===============================
= HIBERNATE SEARCH
===============================
Spring Data JPA will take any properties under spring.jpa.properties.* and
pass them along (with the prefix stripped) once the EntityManagerFactory is
created.
Specify the DirectoryProvider to use (the Lucene Directory)
spring.jpa.properties.hibernate.search.default.directory_provider = filesystem
Using the filesystem DirectoryProvider you also have to specify the default
base directory for all indexes (make sure that the application have write
permissions on such directory)
spring.jpa.properties.hibernate.search.default.indexBase = /tmp/SearchRroo/```
Please not that storing the Lucene index in the tmp directory is not the best idea, but for testing we can use this rather futile location. We also use the filesystem to store the index, as this is the simplest approach.
Create a Service
In order to facilitate Hibernate Search on our data, we add a service class, which offers methods for searching. The service uses a configuration, which is injected by Spring during run time. The configuration is very simple.
package at.stefanproell.service;
import javax.persistence.EntityManager;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class HibernateSearchConfiguration {
private final Logger logger = LoggerFactory.getLogger(HibernateSearchConfiguration.class);
@Autowired
private EntityManager entityManager;
@Bean
HibernateSearchService hibernateSearchService() {
HibernateSearchService hibernateSearchService = new HibernateSearchService(entityManager);
hibernateSearchService.initializeHibernateSearch();
return hibernateSearchService;
}
}```
The @Configuration is loaded when Spring builds the application context. It provides a bean of our service, which can then be injected into the application. The service itself provides methods for creating and searching the index. In this example, the search method is very simple: it only searches on the first and the last name of an employee and it allows users to make one mistake (distance 1).
<pre class="theme:github lang:default decode:true">package at.stefanproell.service;
import java.util.List;
import javax.persistence.EntityManager;
import javax.persistence.NoResultException;
import javax.persistence.PersistenceContext;
import org.apache.lucene.search.Query;
import org.hibernate.search.jpa.FullTextEntityManager;
import org.hibernate.search.jpa.Search;
import org.hibernate.search.query.dsl.QueryBuilder;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import at.stefanproell.model.Employee;
import ch.qos.logback.classic.Logger;
@Service
public class HibernateSearchService {
private final Logger logger = (Logger) LoggerFactory.getLogger(HibernateSearchService.class);
private final EntityManager entityManager;
@Autowired
public HibernateSearchService(EntityManager entityManager) {
super();
this.entityManager = entityManager;
}
public void initializeHibernateSearch() {
try {
FullTextEntityManager fullTextEntityManager = Search.getFullTextEntityManager(entityManager);
fullTextEntityManager.createIndexer().startAndWait();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
@Transactional
public List<Employee> fuzzySearch(String searchTerm){
FullTextEntityManager fullTextEntityManager = Search.getFullTextEntityManager(entityManager);
QueryBuilder qb = fullTextEntityManager.getSearchFactory().buildQueryBuilder().forEntity(Employee.class).get();
Query luceneQuery = qb.keyword().fuzzy().withEditDistanceUpTo(1).withPrefixLength(1).onFields("firstname", "lastname")
.matching(searchTerm).createQuery();
javax.persistence.Query jpaQuery = fullTextEntityManager.createFullTextQuery(luceneQuery, Employee.class);
// execute search
List<Employee> employeeList = null;
try {
employeeList = jpaQuery.getResultList();
} catch (NoResultException nre) {
logger.warn("No result found");
}
return employeeList;
}
}
The service implementation currently only contains an initialization method, which used for creating the Lucene index on the filesystem. Before we can test the index, we need to have at least one indexed entity. This can be achieved by simply adding the annotation @Indexed to the POJO.
