Java EE (Java Enterprise Edition) General Discussion

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Interested in getting your voice heard by members of the Developer Marketing team at Oracle? Check out this post for AppDev or this post for AI focus group information.

Develop Non-Blocking Web Applications in Java

Yolande Poirier-OracleJul 13 2015 — edited Aug 8 2016

by Re Lai

Explore the asynchronous support provided in several popular web frameworks for implementing non-blocking server-side request processing.

Web applications traditionally process requests synchronously on the server side. Asynchronous programming, such as Ajax, is generally used on the client side. With the advent of social networking, mobile devices, and the Internet of Things, non-blocking request processing on the server side has taken off as an important technique for meeting ever-more-daunting performance demands. Event-driven, non-blocking, reactive programming has attracted much interest. The rapid fame garnered by Node.js is reflective of the new era that has dawned.

Fortunately for Java developers, Java proves incredibly adaptable to the changing world. As a language, it has made steady progress in asynchronous support: non-blocking I/O (NIO) in Java 1.4, the executor service in Java 6, the fork/join framework in Java 7, and lambda expressions in Java 8. Support for asynchronous request processing in web applications began with Servlet 3.0 as part of Java EE 6. Because servlets are the underpinning of many Java web applications, this support marks a significant milestone that opened the door for others in the family. JAX-RS 2.0 followed suit in Java EE 7. High-level concurrency facilities are incorporated into Java EE 7 as managed executor services.

Much progress has also been made in the Java open source community. Akka, Spring Reactor, and RxJava all aim to provide foundational APIs for event-driven reactive programming. Many open source frameworks, such as Spring MVC and Grails, have been retrofitted with asynchronous support. In addition, a new generation of lightweight Java Virtual Machine (JVM) web frameworks, such as the Play Framework and Vert.x, has been built from inception to be non-blocking.

This article explores asynchronous support in several popular web frameworks: Servlet, JAX-RS, Spring MVC, Vert.x, and the Play Framework. These frameworks represent a large spectrum of choices in the JVM ecosystem, from the old and faithful servlets to recently minted Vert.x. A sample application, todosapp, is also presented to showcase how to implement non-blocking web applications in these frameworks.

Why Use Non-Blocking Processing?

In conventional web applications, each request is processed synchronously with a thread. This is a straightforward and simple thread model, having served many traditional web applications well.

The breakneck advancement of web applications has pushed the boundary on multiple fronts. The rise of Ajax, long-polling, and streaming increases the number—as well as the length—of requests to the server side. Meanwhile, so-called internet web applications incur an unprecedentedly large number of connections, which is dubbed the C10k problem. If we continue to maintain a blocking thread for each request, the total number of threads can be so large that the memory footprint and the cost of context switching between threads become prohibitively expensive.

Non-blocking request processing emerges as an effective way to address the problem. Instead of one blocking thread to handle one request, a small number of threads is used to handle a large number of requests asynchronously. This results in more efficient use of CPUs and better scalability, albeit at the cost of increased complexity.

Another benefit of asynchronous processing is that it allows workloads to be distributed across multicores or even across machines. For the first time in the history of the computer industry, Moore's law has run its course and CPU speed is reaching its physical limit. Distributed computing will become increasingly important to achieve further performance improvements in the future.

Servlets

The Servlet specification is the very foundation of Java web applications. Many higher-level frameworks—including JAX-RS, JavaServer Faces (JSF), and Spring MVC—are built on top of servlets. Traditionally, the specification followed the conventional blocking thread model. Asynchronous support was eventually added starting in Servlet 3, paving the way for non-blocking capacities in other higher-level frameworks.

An asynchronous servlet must be marked as asyncSupported=true. More importantly, at runtime, an instance of AsyncContext should be created to represent the asynchronous context. Listing 1 is a sample of an asynchronous servlet that displays a to-do list at /todos:

@WebServlet(urlPatterns = {"/todos"}, asyncSupported = true) public class TodoServlet extends HttpServlet {  /** The application service for CRUD operations on Todo entities */ @Inject private TodoService todoService;  /** The managed executor service for concurrency processing */ @Resource private ManagedExecutorService executorService;  /** Serves the to-do list upon HTTP GET. */ @Override protected void doGet(HttpServletRequest request,     HttpServletResponse response) throws ServletException, IOException {         AsyncContext ctx = request.startAsync();         executorService.submit(() -> {             List<Todo> todos = todoService.findAll();              request.setAttribute("todos", todos);             ctx.dispatch("/WEB-INF/jsp/todo-list.jsp");         });           } }

Listing 1. Example of asynchronous servlet

The asynchronous processing starts with request.startAsync. The AsyncContext object is created to indicate to the servlet container to relinquish control when the enclosing method returns. The workload is submitted to executorService, which accepts work as either a Runnable or Callable.

In our example, the workload is a Runnable defined in a lambda expression. While the Servlet specification does not concern itself about how asynchronous work is processed, the use of high-level utilities such as executor services is a best practice for managing concurrency. Once the executor service completes the workload, ctx.dispatch returns control back to the servlet container to render the .jsp page.

Servlet asynchronous support is a good example of how Java EE manages to evolve with time. The retrofit is accomplished with minimum changes to the existing programming model but moves the specification forward to respond to the ever-changing landscape.

JAX-RS

JAX-RS enables rapid development of web services in the REST architectural style. As a relatively new addition to Java EE, it embodies a more declarative and concise programming style. Chosen to be the foundation for the upcoming Java EE MVC specification, JAX-RS is poised to be a critical component of the Java EE offering.

Listing 2 is a JAX-RS RESTful web service for getting a to-do item synchronously by its ID:

@Path("todos") @Produces("application/json") public class TodoResource {  /** The application service for CRUD operations on Todo entities */ @Inject private TodoService todoService;  /**  * Serves the HTTP GET of a to-do item by its ID.  *  * @param id the ID of the to-do item  * @return the to-do item, which will be converted to JSON by JAX-RS  */   @GET  @Path("{id}")     public Todo findOne(@PathParam("id") Long id) {     Todo item = todoService.findOne(id);     Return item; } }

Listing 2. Web service for getting a to-do item synchronously

The code is fairly easy to read. The method findOne returns a to-do item by its ID. JAX-RS 2.0 offers asynchronous support in a manner similar to the Servlet specification. An AsyncResponse object is used to represent the asynchronous processing, following the same suspending-resuming paradigm. The change in Listing 3 makes the service asynchronous:

/** The managed executor service for concurrency processing */ @Resource private ManagedExecutorService executorService;  @GET @Path("{id}")  public void findOne(@PathParam("id") Long id, @Suspended AsyncResponse asyncResponse) {  exectuorService.submit(()-> {        Todo item = todoService.findOne(id);        asyncResponse.resume(item);     });        }

Listing 3. Asynchronous service

An asynchronous method must be injected with an AsyncResponse parameter with the @Suspended annotation, which serves to dissociate the request thread from the connection. Once the workload is processed asynchronously in another thread managed by the executor service, the connection and response are resumed by calling the resume method on the injected AsyncResponse instance.

Spring MVC

Spring MVC is a popular servlet-based web application framework outside of Java EE. The framework features a prominent model-view-controller (MVC) structure. Listing 4 is the code snippet of a Spring MVC web controller that synchronously serves the form to edit a to-do item:

@Controller @RequestMapping("/todos") public class TodoController {   /** The application service of CRUD operations on Todo entities */  @Autowired TodoRepository todoRepository;   /** * Serves the form to edit a to-do item. * * @param id    the ID of the to-do item.      * @param model the Spring MVC holder of attributes to render a view * @return      the target view name  */    @RequestMapping(value="/{id}", method=GET)    public String getTodo(@PathVariable("id") Long id, Model model) {        return doGetTodo(id, model);    }     private String doGetTodo(Long id, Model model) {        Todo item = todoRepository.findOne(id);        model.addAttribute(item);        return "todo-form";    }      }

Listing 4. Synchronous Spring MVC web controller

To change this synchronous handler to an asynchronous one, instead of returning the view name as a string, return a Callable<String>, as shown in Listing 5:

   @RequestMapping(value="/{id}", method=GET)    public Callable<String> getTodo(@PathVariable("id") Long id, Model model) {        return () -> doGetTodo(id, model);    }

Listing 5. Asynchronous Spring MVC web controller

At runtime, the Callable object will be evaluated in an executor service managed by Spring MVC. Afterward, Spring MVC dispatches the processing back to the servlet container with the view name.

RESTful web services are similarly supported by Spring MVC via RestController, as shown in Listing 6.

@RestController @RequestMapping("/api/todos") public class TodoRestController {  /** The application service of CRUD operations on Todo entities */ @Autowired TodoRepository todoRepository;  /**  * Serves HTTP GET of a to-do item by its ID.  *  * @param id the ID of the to-do item  * @return a Callable that returns the to-do item  */     @RequestMapping(value="/{id}", method=GET)     public Callable<Todo> get(@PathVariable("id") Long id) {        return () -> todoRepository.findOne(id);       } }

Listing 6. Spring MVC support for RESTful web services

Spring MVC, like JAX-RS, works at a level higher than servlets. As a result, the code is more declarative and succinct. It is also interesting to compare the design choices. Spring MVC completely encapsulates how asynchronous workloads are handled, whereas you manage and fully control the asynchronous execution yourself with JAX-RS and servlets.

Vert.x

Vert.x represents a new breed of web frameworks on the JVM. It is lightweight and designed from the beginning to be asynchronous in nature. Vert.x was inspired by Node.js, and it is sometimes called the Node.js in Java, though the framework is actually polyglot and supports a number of JVM languages including Java, Groovy, Ruby, JavaScript, and Python. Vert.x was recognized by JAX Innovation Awards as the most innovative Java technology in 2014.

At the heart of its non-blocking concurrency is the event bus. A Vert.x application consists of verticles, a package of code deployed together to run, potentially on multiple cores of a CPU or even on different machines. They communicate with each other through the event bus by passing messages. The result is a simple, actor-like concurrency model.

The snippet in Listing 7 shows how to serve the to-do item by ID in a RESTful web service from the main verticle:

JsonObject command = new JsonObject(); command.putString("action", "findOne")        .putNumber("id", Long.valueOf(request.params().get("id")));  String address = "todoService";              vertx.eventBus().send(address, command, (Message<JsonObject> message)-> {    JsonObject item = message.body();    String payload = item.encode();    request.response()           .putHeader("content-type", "application/json")           .end(item); });

Listing 7. Code for serving a to-do item by ID in a RESTful web service

A JSON command is constructed to find the to-do item by its ID. The command is sent over the event bus to the address at "todoService". Once the event has replied, the response message body contains the to-do item, which is then returned as the payload to the HTTP GET request.

At the other side of the event bus, the address "todoService" is serviced by TodoServiceVerticle, a worker verticle that provides data service to fetch to-do items, as shown in Listing 8.

public class TodoServiceVerticle extends Verticle{          /** Initializes the verticle at the start-up */     @Override public void start() {            // Initialize the verticle         ...          vertx.eventBus().registerHandler("todoService", this::onMessage);     }          private void onMessage(Message<JsonObject> message) {         JsonObject command = message.body();                    // Only "findOne" is supported in this example          assert ("findOne".equals(command.getString("action")));                  Long id = command.getLong("id");         Todo item = findOneById(id);         JsonObject payload = toJson(item);         message.reply(payload); }  ... }

Listing 8. Code for fetching to-do items

Upon its initialization, TodoServiceVerticle registers the event handler onMessage at the address "todoService". Whenever a message arrives at the address, it is pumped to the handler to fetch the to-do item from the database.

The Play Framework

The Play Framework is another well-known lightweight JVM asynchronous web framework. The Play Framework team has been a strong proponent of reactive programming. Designed to be non-blocking from the bottom up, the Play Framework bases its asynchronous infrastructure on Akka, an event-driven actor system. While rooted in Scala, the Play Framework supports both Java and Scala. Compared to Vert.x, the Play Framework is a highly opinionated MVC framework. In this regard, it was inspired by and very much resembles Ruby on Rails.

Listing 9 is a code snippet for a controller action that displays the form for editing a to-do item:

@Transactional(readOnly = true) public static Result showForm(Long id) {     TodoItem item = TodoService.findOne(id);     return ok(todoform.render(item)); }

Listing 9. Form for editing a to-do item

Actions in the Play Framework are executed asynchronously by default; thus, they use far fewer threads than traditional web applications. As a result of this design, actions must finish as soon as possible; otherwise, the thread pool would be quickly exhausted. In light of this, we can improve our code by moving the database query out of the action thread. Instead of a Result object, we can return a promise of the Result object, as shown in Listing 10:

@Transactional(readOnly = true) public static Promise<Result> form(Long id) {       return Promise.promise(()-> TodoService.findOne(id))                     .map(todo -> ok(todoform.render(todo))); }

Listing 10. Code for moving the database query out of the action thread

This change helps our action to run quickly without blocking, because the slow I/O operation is offloaded to another thread pool managed by Akka. You might wonder where Akka actors come into play here. Because executing workloads asynchronously is such a common use case, the Play Framework offers a higher-level helper API, as we just saw above, making the code easier to write and read. Akka actors are actually running behind the scene to make the magic happen.

Todosapp

A sample to-do management application, todosapp, has been built to showcase the server-side asynchronous processing in the frameworks discussed above. Table 1 shows an implementation matrix.

Table 1. Implementation matrix

The home page, which is shown in Figure 1, can be accessed at [http://localhost:8080/](http://localhost:8080/).

Figure 1. Home page of todosapp

There are actually three "applications" packaged in each project:

A "classic" server-side web application that renders the UI on the server
A RESTful web service API
A rich-client single-page web application (SPA) that is built on backbone.js at the browser side, which fetches data from the server via the RESTful web service API

The web application displays to-do lists. You can create, update, or delete to-do lists. You can also filter the items to show all to-do items, open to-do items, or completed to-do items.

Figure 2. To-do list

Food for Thought

While asynchronous request processing has to date been widely supported across JVM web frameworks, it is worthwhile to point out that events versus threads has long been a subject of debate. It has been argued that the performance issue faced by the blocking concurrency model does not necessarily come from the presence of a large number of threads themselves, but instead is due to the quality of the thread library, which normally defaults to the operating system threads.

With the preponderance of non-blocking support available, today's pendulum seems to swing toward the asynchronous threading model. In the meantime, interesting progress has been made to use fibers, which are lightweight threads, to improve scalability. We definitely should keep an eye on that.

Summary

Non-blocking request processing has enjoyed wide support in Java web frameworks for achieving high throughput. This article surveyed the support in a wide spectrum of popular frameworks. In addition, a sample application, todosapp, showcased the asynchronous capacity of these frameworks.

About the Author

Re Lai is a software development director at Oracle. He is also a Sun Certified Enterprise Architect (SCEA) and is interested in web application development using Java and other JVM languages.

Join the Conversation

Join the Java community conversation on Facebook, Twitter, and the Oracle Java Blog!

Added on Jul 13 2015

#java-ee-java-enterprise-edition-discussion, #legacy-documents

8 comments

48,099 views