Friday, February 10, 2017

How to Consume JSON from RESTful Web Service and Convert to Java Object - Spring RestTemplate Example

So far, I have not written much about REST and RESTful web service barring some interview questions e.g. REST vs SOAP, which is thankfully very much appreciated by my readers and some general suggestions about best books to learn REST in past, but today I am going to write something about RESTTemplate class from Spring MVC framework. Like its predecessors JdbcTemplate and JmsTemplate, the RestTemplate is another useful utility class which allows you to interact with RESTful web services from a Java application built using Spring framework. It's a feature rich and supports almost all REST methods e.g. GET, POST, HEAD, PUT or DELETE, though we'll only use the GET method in this article to consume a RESTful Web Service and convert the JSON response to Java objects. It's one of the basic but interesting examples, given you will often find scenarios to consume a RESTful web service from Java program.

I am also using Spring Boot to run my program as a main() method instead of building a WAR file and deploying in tomcat and then writing Servlet and JSP to demonstrate the example. I really find the convenience offered by Spring boot great as it embeds Tomcat servlet container as the HTTP runtime, which is enough to run this program.



Free RESTful Web Services on the Internet for Testing

In order to create a RESTful client, you need to have a RESTful web service which can provide JSON content you want to consume. Though you can develop a RESTful client in Spring framework itself, for testing purpose its better to use the existing free RESTful web service on the internet e.g. http://jsonplaceholder.typicode.com has several useful RESTful API to return comments and posts related data e.g. http://jsonplaceholder.typicode.com/posts/1 will return post data with id= 1, which looks something like this:

{
"userId": 1,
"Id": 1,
"Title": "a title "
"Body": "the body of the content"
}

Though, I have used another free REStful web service which returns Country's name and their two and three letter ISO codes as shown below:


{
  "RestResponse" : {
    "messages" : [ "More webservices are available at http://www.groupkt.com/post/f2129b88/services.htm", 
    "Country found matching code [IN]." ],
    "result" : {
      "name" : "India",
      "alpha2_code" : "IN",
      "alpha3_code" : "IND"
    }
  }
}

You can use any of these web services to build a RESTful client for testing. Though your domain class will change depending upon which RESTful web service you are consuming.


Here is the screenshot of this RESTful web service response in my browser:

Free RESTFul Web Services for Testing on Internet




Java Program to consume JSON from RESTful WebService using Spring RestTemplate

Here is our complete Java program to consume a RESTful Web Service using Spring framework and RestTemplate class. This program has four Java files :  App.java, Response.java, RestResponse.java, and Result.java. The first class is the main class which drives this application and others are classes corresponding to JSON response we get from this RESTful API.

App.java
This is our main class which drives this RESTful Spring application. It uses Spring boot to setup and runs the program. The class App implements CommandLineRunner and calls the SpringApplication.run() method by passing this instance of class i.e. App.class. This will, in turn, call the run method, where we have code to call a RESTful Web Service and consume JSON response using RestTemplate class of Spring framework.

Just like it's predecessor or close cousins e.g. JmsTemplate and JdbcTemplate, the RestTemplate class also does everything for you.  All you need to tell is the name of the class you want to map the incoming JSON response.

We have first created an instance of RestTemplate class and then called the getForObject() method. This accepts two parameters, first, a String URL to call the RESTful Web Service and second the name of the class it should return with the response.  So, in just one line of code, it calls the RESTful web service, parse the JSON response and return the Java object to you.

If you are curious to learn more about RestTemplate class then you can also read Spring REST book. It explains all nitty gritty of developing RESTful web services in Spring framework.



Here are the Java classes required to call a RESTful Web Service from Java Program using Spring and RestTemplate utility:

package rest;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.web.client.RestTemplate;

public class App implements CommandLineRunner {

  private static final Logger log = LoggerFactory.getLogger(App.class);

  public static void main(String args[]) {
    SpringApplication.run(App.class);
  }


  public void run(String... args) throws Exception {
    RestTemplate restTemplate = new RestTemplate();
    Response response = restTemplate.getForObject(
                           "http://services.groupkt.com/country/get/iso2code/IN",
                            Response.class);
    log.info("==== RESTful API Response using Spring RESTTemplate START =======");
    log.info(response.toString());
    log.info("==== RESTful API Response using Spring RESTTemplate END =======");
  }
}

Response.java
This is the top-level class to convert JSON response to Java object you receive by consuming the RESTful web service response. I have use @JSonProperty to annotate the RestResponse field to tell the Jackson that this is the key field in the JSON document.

package rest;

import com.fasterxml.jackson.annotation.JsonProperty;

public class Response {

  @JsonProperty
  private RestResponse RestResponse;
  
  public RestResponse getRestResponse() {
    return RestResponse;
  }

  public void setRestResponse(RestResponse restResponse) {
    RestResponse = restResponse;
  }

  public Response(){
    
  }

  @Override
  public String toString() {
    return "Response [RestResponse=" + RestResponse + "]";
  }

}

RestResponse.java
This class contains two fields corresponding to RestResponse section of JSON response we received from RESTful Web service. The first field is a String array, messages and Jackson will parse the JSON array to String array and store the output for that property in this field. The second field, the result is again a custom type Java object to store the data we need i.e. name and ISO code of the country.

package rest;

import java.util.Arrays;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;


public class RestResponse {
 
  private String[] messages;
  private Result result;
  
  public RestResponse(){    
  }
  
  public String[] getMessages() {
    return messages;
  }
  public void setMessages(String[] messages) {
    this.messages = messages;
  }
  public Result getResult() {
    return result;
  }
  public void setResult(Result result) {
    this.result = result;
  }

  @Override
  public String toString() {
    return "RestResponse [messages=" + Arrays.toString(messages) + ", result=" + result + "]";
  }
 
}

Result.java
This class has 3 fields, corresponding to 3 properties for result section in JSON response. All three fields are String, first, the name is the name of the country, second, alpha2_code is the two-digit ISO code for the country and third, alpha3_code is the three-digit ISO code of the country.

package rest;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;

@JsonIgnoreProperties(ignoreUnknown = true)
public class Result {

  private String name;
  private String alpha2_code;
  private String alpah3_code;

  public Result() {

  }

  public String getName() {
    return name;
  }

  public void setName(String name) {
    this.name = name;
  }

  public String getAlpha2_code() {
    return alpha2_code;
  }

  public void setAlpha2_code(String alpha2_code) {
    this.alpha2_code = alpha2_code;
  }

  public String getAlpah3_code() {
    return alpah3_code;
  }

  public void setAlpah3_code(String alpah3_code) {
    this.alpah3_code = alpah3_code;
  }

  @Override
  public String toString() {
    return "Result [name=" + name + ", alpha2_code=" + alpha2_code
        + ", alpah3_code=" + alpah3_code + "]";
  }

}

Here is how our project setup looks like in Eclipse IDE, you can see all the classes along with Maven dependencies and JARs:

How to Consume JSON from RESTful Web Service and Convert to Java Object


Maven dependencies:

Here are the Maven dependencies used by this Spring boot application to call the RESTful Web Services:
    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
            <version>1.2.7.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.2.3</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
            <version>4.0.0.RELEASE</version>
        </dependency>

    </dependencies>


JARs

Here is the list of JAR files used by this application, I have directly copied it from Maven Dependencies referenced in Eclipse.

C:\.m2\org\springframework\boot\spring-boot-starter\1.2.7.RELEASE\spring-boot-starter-1.2.7.RELEASE.jar
C:\.m2\org\springframework\boot\spring-boot\1.2.7.RELEASE\spring-boot-1.2.7.RELEASE.jar
C:\.m2\org\springframework\boot\spring-boot-autoconfigure\1.2.7.RELEASE\spring-boot-autoconfigure-1.2.7.RELEASE.jar
C:\.m2\org\springframework\boot\spring-boot-starter-logging\1.2.7.RELEASE\spring-boot-starter-logging-1.2.7.RELEASE.jar
C:\.m2\org\slf4j\jcl-over-slf4j\1.7.12\jcl-over-slf4j-1.7.12.jar
C:\.m2\org\slf4j\slf4j-api\1.7.12\slf4j-api-1.7.12.jar
C:\.m2\org\slf4j\jul-to-slf4j\1.7.12\jul-to-slf4j-1.7.12.jar
C:\.m2\org\slf4j\log4j-over-slf4j\1.7.12\log4j-over-slf4j-1.7.12.jar
C:\.m2\ch\qos\logback\logback-classic\1.1.3\logback-classic-1.1.3.jar
C:\.m2\ch\qos\logback\logback-core\1.1.3\logback-core-1.1.3.jar
C:\.m2\org\springframework\spring-core\4.1.8.RELEASE\spring-core-4.1.8.RELEASE.jar
C:\.m2\org\yaml\snakeyaml\1.14\snakeyaml-1.14.jar
C:\.m2\com\fasterxml\jackson\core\jackson-databind\2.2.3\jackson-databind-2.2.3.jar
C:\.m2\com\fasterxml\jackson\core\jackson-annotations\2.2.3\jackson-annotations-2.2.3.jar
C:\.m2\com\fasterxml\jackson\core\jackson-core\2.2.3\jackson-core-2.2.3.jar
C:\.m2\org\springframework\spring-web\4.0.0.RELEASE\spring-web-4.0.0.RELEASE.jar
C:\.m2\aopalliance\aopalliance\1.0\aopalliance-1.0.jar
C:\.m2\org\springframework\spring-aop\4.0.0.RELEASE\spring-aop-4.0.0.RELEASE.jar
C:\.m2\org\springframework\spring-beans\4.0.0.RELEASE\spring-beans-4.0.0.RELEASE.jar
C:\.m2\org\springframework\spring-context\4.0.0.RELEASE\spring-context-4.0.0.RELEASE.jar
C:\.m2\org\springframework\spring-expression\4.0.0.RELEASE\spring-expression-4.0.0.RELEASE.jar

Here is the response, you will see in Eclipse's console when you run this program by right clicking on App class and choosing "Run as Java Application"


You can see that response is correctly received and parsed from JSON to Java object by Jackson API automatically.



Important points

Here are a couple of important points you should remember or learn by writing and running this Java program to call a RESTful Web service using Spring Boot and RestTemplate

1. Since we are using Jackson library in our CLASSPATH, RestTemplate class will use it (via a message converter e.g. HttpMessageConverter to convert the incoming JSON response into a Result object.

2. We have used @JsonIgnoreProperties from the Jackson,  a JSON processing library to indicate that any properties not bound in any of model classes e.g. RestResponse or Result should be ignored.

3. In order to directly bind your data from JSON response to your custom Java class, you need to specify the field name in the Java class exactly same as the key in the JSON response returned from the RESTful API.

4. If field name in the Java class and key in JSON response are not matching, then you need to use the @JsonProperty annotation to specify the exact key of JSON document. You can see we have used this annotation to map RestResponse key to RestResponse field in our Response class.

4. Sometimes, when you run the application and you see only nulls in the response, you can remove the @JsonIgnoreProperties to check what's happening behind the scene. In my case, the RestResponse field was not mapping correctly due to name mismatch and that was revealed by the following exception when I removed the @JsonIgnoreProperties annotation. This is quite useful for debugging and troubleshooting.

Caused by: org.springframework.http.converter.HttpMessageNotReadableException: Could not read JSON: Unrecognized field "RestResponse" (class rest.example.SpringJSONRestTest.Response), not marked as ignorable (one known property: "restResponse"])
 at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@45af76e7; line: 2, column: 21] (through reference chain: rest.example.SpringJSONRestTest.Response["RestResponse"]); nested exception is com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "RestResponse" (class rest.example.SpringJSONRestTest.Response), not marked as ignorable (one known property: "restResponse"])
 at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@45af76e7; line: 2, column: 21] (through reference chain: rest.example.SpringJSONRestTest.Response["RestResponse"])
at org.springframework.http.converter.json.MappingJackson2HttpMessageConverter.readJavaType(MappingJackson2HttpMessageConverter.java:185)
at org.springframework.http.converter.json.MappingJackson2HttpMessageConverter.read(MappingJackson2HttpMessageConverter.java:177)
at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:95)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:535)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:489)
at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:226)
at rest.example.SpringJSONRestTest.App.run(App.java:20)
at org.springframework.boot.SpringApplication.runCommandLineRunners(SpringApplication.java:674)


5.  When you use the Spring, you get the embedded Tomcat to run your Java application as plain old Java application by running the main class. It uses Tomcat internally to make HTTP class and parse HTTP response.

That's all about how to consume JSON data from a RESTful web service in Java using Spring's RestTemplate class. This class is super useful and allows you to perform any REST operations. In this example, we have only used RestTemplate to make an HTTP GET request, but you can also use RestTemplate to execute HTTP POST, PUT or DELETE method.

Further Learning
REST Fundamentals By Howard Dierking
RESTFul Services in Java using Jersey
Introduction to Spring MVC

Btw, If you are someone who prefers training courses and coaching classes than books, then you can also check out Eugen's REST with Spring course, it's currently one of the best courses to learn RESTful web services development using Spring framework. The course expects candidates to know Java and Spring, hence, it's ideal for intermediate and experienced Java and Web developers.

How to create RESTful Web Service in Spring Java

Eugen has severals options on his courses suited for different experience level and needs e.g. REST with Spring: The Intermediate class is good for essential knowledge while REST with Spring: The Masterclass is more detail oriented. You can check out all his course options here.

2 comments :

parivartan said...

Hi,

Please tell me how can i debug restTemplate request using logger or how can i see request url using logger. or how to use restTemplate with logger to print request url or request method.

Sahoo Soumya Ranjan said...

In the above example look at getForObject. It have two parameters. First one is the requested url.

Post a Comment