Monday, January 25, 2016 0 comments

Integrating JQGrid with Spring MVC and Gson

I was working on a single page application where i wanted to use the grid functionality in one part of the application which was using Spring MVC. It has been some time since i last used JQGrid and finding the information required to get me up and about was bit of a challenge. On this post, I wanted to collate all the information and put it into a tutorial to follow so anyone who might be using the same functionality might find it helpful to setup JQGrid.

First of all, we will setup a sample web project on Eclipse and define the web.xml as follows;


 

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" id="WebApp_ID" version="2.5">
  <display-name>JQGridExample</display-name>
  <welcome-file-list>
    <welcome-file>index.html</welcome-file>
    <welcome-file>index.htm</welcome-file>
    <welcome-file>index.jsp</welcome-file>
    <welcome-file>default.html</welcome-file>
    <welcome-file>default.htm</welcome-file>
    <welcome-file>default.jsp</welcome-file>
  </welcome-file-list>
  
  <servlet>
        <servlet-name>JQGridExample</servlet-name>
        <servlet-class>
            org.springframework.web.servlet.DispatcherServlet
        </servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>
    <servlet-mapping>
        <servlet-name>JQGridExample</servlet-name>
        <url-pattern>/</url-pattern>
    </servlet-mapping>
    
 
</web-app>

To wire up Spring MVC, I have registered the DispatcherServlet to be loaded on start-up. This is basically how you will register any Spring MVC application. Next up, we need to create the spring configuration to register the required components/elements of our spring MVC application.

In this instance, I have kept the spring context file name to the "servlet-name" given on my web.xml because by default when the spring container loads up, it will look for a file with the format <servletname>-servlet.xml

If you want to use any other name you want for your spring context configuration file, you can do so. You just need to register the context loader on your web.xml.

So let us see how our spring context configuration file looks like;


 
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 xmlns:context="http://www.springframework.org/schema/context"
 xmlns:mvc="http://www.springframework.org/schema/mvc"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
 http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
 http://www.springframework.org/schema/mvc
 http://www.springframework.org/schema/mvc/spring-mvc-3.2.xsd
 http://www.springframework.org/schema/context
 http://www.springframework.org/schema/context/spring-context-3.0.xsd">
 
    <context:component-scan base-package="com.example.jqgrid.controller" />
     <bean id="multipartResolver" class="org.springframework.web.multipart.commons.CommonsMultipartResolver"/>
    <bean id="viewResolver" class="org.springframework.web.servlet.view.InternalResourceViewResolver">
    <property name="prefix" value="/WEB-INF/jsp/" />
    <property name="suffix" value=".jsp" />
    </bean>
    
    <mvc:resources mapping="/resources/**" location="/resources/"/>
    
    
    <mvc:annotation-driven/>
 
    
</beans>

We first register the package that contains all our controller classes. In this instance it will be just one controller. With the component-scan element, it will scan all classes under the "controller" package.

Next up, we tell the Spring container how to resolve our JSP files. In this instance the internal view resolver is being used and we provide the location of where our JSP files reside on the application.

The next interesting part on this configuration is the <mvc:resources> element. The reason to define this is to let the Spring container know about our static resources such as the javascript files, images, stylesheets. If we do not define them as resources, whenever you refer a javascript file for example in your application, spring mvc will try to match an existing controller by looking at the defined URL patterns. In this case, all my css, javascript,image files reside under the resources folder.

I then define the index.jsp which is the entry point into our application. Now I do not want to do anything on this page and i simply re-direct it to a different page which is resolved via spring-mvc. Our index.jsp file is as follows;


 
<script type="text/javascript">
 window.location.replace("jqGridExample");
</script>

I am simply re-directing the URL to jqGridExample. Now to understand how this is resolved from spring-mvc, we will need look at our controller class. Our controller class is as follows;


 
package com.example.jqgrid.controller;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;

import com.example.jqgrid.common.util.JsonUtil;
import com.example.jqgrid.dto.JQGridDTO;
import com.example.jqgrid.dto.SuperHeroDTO;
import com.example.jqgrid.handler.JQGridHandler;

/**
 * This class acts as the controller for JQGrid related functionality.
 * 
 * @author Dinuka Arseculeratne
 * 
 */
@Controller
public class JQGridController {

 /**
  * This method will display the page used to display the grid.
  * 
  * @param req
  * @param res
  * @return
  */
 @RequestMapping(method = { RequestMethod.POST, RequestMethod.GET }, path = "/jqGridExample")
 public String jqGrid(HttpServletRequest req, HttpServletResponse res) {
  String forward = "jqgrid/jqGridData";

  return forward;
 }

 /**
  * This method will handle fetching data required for the JQGrid.
  * 
  * @param req
  * @param res
  * @return
  */
 @RequestMapping(method = { RequestMethod.POST, RequestMethod.GET }, path = "/loadData")
 public String loadData(HttpServletRequest req, HttpServletResponse res) {
  String forward = "common/formData";

  JQGridDTO<SuperHeroDTO> gridData = new JQGridHandler().loadSuperHeroes(req);
  req.setAttribute("formData", JsonUtil.toJsonObj(gridData));
  return forward;
 }

}



So if we look at the first method, you can see we are simply return a text called "jqgrid/jqGridData". Now to understand what this does, we need to go back and look at our spring context configuration file. In that we specified that all our JSP files reside in the "WEB-INF/jsp" folder and the suffix is ".jsp". So in this instance the path we return from this method tells the spring container that the JSP to be returned is in fact in "WEB-INF/jsp/jqgrid/jqGridData.jsp". Note that we did not need to specify the suffix as ".jsp" because we already configured that on our spring context configuration.

We will come back to the second method after we look at our page where we have defined the JQGrid. The jqGridData.jsp is as follows;


 
<!DOCTYPE html>
<html>
 <head>
  <title>JQGrid Example</title>

   
    <link href="resources/css/jquery-ui.css" rel="stylesheet">
    <link href="resources/css/jquery-ui.theme.css" rel="stylesheet">
    <link href="resources/css/jquery-ui.structure.min.css" rel="stylesheet">
    <link rel="stylesheet" href="resources/css/ui.jqgrid.css">
    
 </head>
 
<body>

<div>
 
   <table id="list">
                <tr>
                        <td />
                </tr>
        </table>
        <div id="pager"></div>
        <div style="margin-top:10px;">
        <input type="button" id="showSelected" value="Show Selected"/>
        </div>
 
 
</div>

  <script src="resources/js/jquery-1.11.1.min.js"></script>
  <script src="resources/js/jquery-ui.min.js"></script>
  <script src="resources/js/i18n/grid.locale-en.js"></script>
  <script src="resources/js/jquery.jqGrid.min.js"></script>
  
  
<script type="text/javascript">

 $(document).ready(function(){
  $("#list").jqGrid({
                url : "loadData",
                datatype : "json",
                mtype : 'POST',
                colNames : [ 'Name','Alias','Super Power'],
                colModel : [ {
                        name : 'name',
                        index : 'name',
                        width : 150
                }, {
                        name : 'alias',
                        index : 'alias',
                        width : 150,
                        editable : false
                }, {
                        name : 'power',
                        index : 'power',
                        width : 550,
                        editable : false
                }],
                pager : '#pager',
                rowNum : 10,
                height: 'auto',
                rowList : [ 10 ],
                sortname : 'invid',
                sortorder : 'desc',
                viewrecords : true,
                gridview : true,
                multiselect: true,
            multiboxonly: false,
                caption : 'Super Heroes',
                jsonReader : {
                        repeatitems : false,
                }
        });
        jQuery("#list").jqGrid('navGrid', '#pager', {
                edit : false,
                add : false,
                del : false,
                search : false
        });
        
        
        $('#showSelected').on('click',function(){
         
         var selRowArr = jQuery("#list").getGridParam('selarrrow');
         var selectedAppIds = [];
         for(var i=0;i<selRowArr.length;i++){
          var celValue =  $('#list').jqGrid('getCell', selRowArr[i], 'alias');
          selectedAppIds.push(celValue);
         }
         alert(selectedAppIds);
         $('#list').trigger( 'reloadGrid' );
         
   
        });
 
 });
</script>

</body>
</html>

First of all, we need to define the element on which the JQGrid will be loaded. In this instance that is the HTML table element with the id of  "list". And since we want the pagination ability, we define our pagination section below the grid. In this instance, the pagination section is defined with the div with the id of "pager".

We then look at the java script code as the bottom. Here we load the JQGrid by calling the method jqGrid() passing in the required attributes. I will not be explaining all the attributes defined here as there are many more which i have not used in this instance. The most relevant attributes for this tutorial will be explained. So first off, the URL. This is defined as "loadData". We need to go back our controller class to understand how this is mapped.

On the controller, we have defined the second method as "loadData" which fetches the data required for the grid. Now the interesting part is, JQGrid expects the data sent across in a particular format. To adhere to this format, i have defined a class to hold this structure which is defined as JQGridDTO. Let us see how that class looks like;


 
package com.example.jqgrid.dto;

import java.io.Serializable;
import java.util.List;

/**
 * This class acts as common template for all pages that use the JQGrid.
 * 
 * @author Dinuka Arseculeratne
 * 
 * @param <T>
 */
public class JQGridDTO < T extends Serializable > {

    private int page;

    private String total;

    private String records;

    private List<T> rows;

    public int getPage() {
        return page;
    }

    public void setPage(int page) {
        this.page = page;
    }

    public String getTotal() {
        return total;
    }

    public void setTotal(String total) {
        this.total = total;
    }

    public String getRecords() {
        return records;
    }

    public void setRecords(String records) {
        this.records = records;
    }

    public List<T> getRows() {
        return rows;
    }

    public void setRows(List<T> rows) {
        this.rows = rows;
    }

}


This is the structure of the data required by JQGrid. I have kept the rows data structure generic in order to be able to use this same class to pass different types of data to the grid as required. It can be any type of object as long as it implements the Serializable interface.

So I am a big time super heroes fan, and hence in this instance I will be displaying some information on some of the super heroes. I have included super heroes from both the DC and Marvel universe to keep everyone happy.

So let us look at our data object and the handler class which will load our data;

 
package com.example.jqgrid.dto;

import java.io.Serializable;

/**
 * 
 * @author Dinuka Arseculeratne
 *
 */
public class SuperHeroDTO implements Serializable {

 /**
  * 
  */
 private static final long serialVersionUID = 1420635747715993129L;

 private String name;
 private String alias;
 private String power;

 public SuperHeroDTO(String name, String alias, String power) {
  this.name = name;
  this.alias = alias;
  this.power = power;
 }

 public String getName() {
  return name;
 }

 public void setName(String name) {
  this.name = name;
 }

 public String getAlias() {
  return alias;
 }

 public void setAlias(String alias) {
  this.alias = alias;
 }

 public String getPower() {
  return power;
 }

 public void setPower(String power) {
  this.power = power;
 }

}



 
package com.example.jqgrid.handler;

import java.util.LinkedList;
import java.util.List;

import javax.servlet.http.HttpServletRequest;

import com.example.jqgrid.dto.JQGridDTO;
import com.example.jqgrid.dto.SuperHeroDTO;

/**
 * The handler class used to fetch the data required.
 * 
 * @author Dinuka Arseculeratne
 *
 */
public class JQGridHandler {

 /**
  * This method will fetch the super hero list. Of course i have mixed and
  * matched DC and Marvel in order to keep peace on the universe.
  * 
  * @return
  */
 public JQGridDTO<SuperHeroDTO> loadSuperHeroes(final HttpServletRequest req) {
  /**
   * The page and rows are sent from the JQGrid component with the Ajax
   * query.
   * 
   */
  int page = Integer.valueOf(req.getParameter("page")).intValue();
  int pageSize = Integer.valueOf(req.getParameter("rows")).intValue();

  /**
   * I am not using the star index and end index in this case, but in an
   * ideal situation, you will be passing the start and end index to your
   * pagination SQL query.
   * 
   */
  int startIndex = page == 1 ? 0 : (pageSize * (page - 1));
  int endIndex = page == 1 ? pageSize : pageSize * page;
  int total = -1;

  JQGridDTO<SuperHeroDTO> jqGridData = new JQGridDTO<SuperHeroDTO>();
  List<SuperHeroDTO> superHeroList = new LinkedList<SuperHeroDTO>();
  SuperHeroDTO flash = new SuperHeroDTO("Barry Allen", "Flash", "Super speed, Taping into the speed force");
  superHeroList.add(flash);

  SuperHeroDTO superMan = new SuperHeroDTO("Clark Kent", "Superman", "Flying, super speed");
  superHeroList.add(superMan);

  SuperHeroDTO batman = new SuperHeroDTO("Bruce Wayne", "Batman", "Cool toys, Intelligence");
  superHeroList.add(batman);

  SuperHeroDTO professorX = new SuperHeroDTO("Professor Xavier", "Professor X", "Mind control");
  superHeroList.add(professorX);

  /**
   * The total in the ideal situation would be the count of the records of
   * your SQL query from the table you want to fetch data from.
   * 
   */
  total = superHeroList.size();

  jqGridData.setPage(page);
  jqGridData.setTotal(String.valueOf(Math.ceil((double) total / pageSize)));
  jqGridData.setRecords(String.valueOf(total));
  jqGridData.setRows(superHeroList);
  return jqGridData;
 }
}



Typically you will be using a database to fetch your data. To maintain the brevity of this tutorial i have just loaded up static data. On the code comments I have mentioned how you would be passing the data when using an actual database.

In this instance, the JQGrid is setup to receive the data in JSON format. So to convert our super hero object to its JSON equivalent, i have used Google's GSON library. I wrote a helper class to convert JSON objects to Java objects and Java objects to JSON objects which I have shared in one of my previous articles which you can find here.

I have not used the spring-mvc default functionality to send a JSON response. In this example what I do is set the JSON output in a request attribute and then forward the page to a common page where it just prints out that attribute and the response is sent back on the Ajax request made by the JQGrid component. This common page is defined as follows;


 
<%=request.getAttribute("formData")%>


Going back to our JSP file which defined the JQGrid, the next important attribute I want to focus on is the "colModel". This maps the data sent on your JSON output to the grid columns that are displayed. In this instance you can see the names mentioned here are the instance variable names defined on our super hero data object. The rest of the attributes are self-explanatory so I will not delve into the details on those attributes.

Another important use case I required was to be able to send the selected rows to the back-end. To do this, you can use the in-built JQGrid functions. The following code shows the code which retrieves the name of the super hero on all selected rows (in this case as the multi select feature is enabled on the grid) and puts it into a Java script array.


 
 $('#showSelected').on('click',function(){
         
         var selRowArr = jQuery("#list").getGridParam('selarrrow');
         var selectedAppIds = [];
         for(var i=0;i<selRowArr.length;i++){
          var celValue =  $('#list').jqGrid('getCell', selRowArr[i], 'alias');
          selectedAppIds.push(celValue);
         }
         alert(selectedAppIds);
         $('#list').trigger( 'reloadGrid' );
         
   
        });

And that ends by tutorial on how to setup JQGrid with Spring MVC and Gson. The working code is checked into my GIT repository and can be found here. You can clone the repository if required and run the application.
Saturday, January 16, 2016 0 comments

Everyone deserves a second chance

I was watching one of the episodes on a TV series which I loved called Outsourced last week. In that episode there is a scenario where the manager (Tod) finds out that one of his employees is struggling to meet the sales target and the top management instructs the manager to get rid of one employee to keep up with the revenues. But what does “Tod” the manager do. He did confront the employee on his lack of performance and asked him what the cause of it was. Then along with the employee, he helps this guy to reach his targets by helping him resolve his personal issue. Now that I believe is the quintessence of a great manager.

This got me thinking, often times, even in the software industry, I have seen people who struggle being cornered and ignored. Most of the times it is not due to the lack of competence on their part. If you give them a chance to express themselves, I have often seen that it was due to a personal matter, financial crisis, insufficient information during on-boarding and many other reasons for the employee to under-perform. The saying goes “Put yourself in their shoes”. It would be appropriate to put yourself in their shoes and walk in it as well to get a feel of what the employee is going through. In my opinion, employees should not only be measured by their personal excellence, but should also be measured by how they support their team mates to succeed as well. A team will only be successful if they work as one cohesive unit. In daily stand-ups if I see someone struggling on a task for more than two consecutive days I would approach them offline and ask how I can help them out. The fact that you ask them and are there for them in their difficult time brings out a sense of belonging to them.

Johanna Rothman in her book “Behind closed doors” gives an overview of how we can deal with such employees. She calls it the “Get well plan” where the manager along with the employee comes up with a plan for the employee to succeed. You cannot simply fire a person without giving him/her a second chance to redeem themselves. I believe this approach is a fabulous idea where you can set a fixed set of goals within a fixed time frame to help out an employee who is struggling in your team.


So the next time you see someone in your team struggling, talk to them, get to know what their problem is and help them out. You will have pride (humble of course) of knowing that you were able to make a significant impact on the life of someone.
Monday, January 11, 2016 2 comments

Being an architect and handling some of the managerial aspects

From mid last year, I have been playing a junior architect role, transitioning out of the technical lead role I was playing. It has been an interesting ride so far with many obstacles and many more to come I’m sure which makes it even more interesting. Taking the plunge to the new role required a change in the mindset. As a technical lead, I was predominantly focused on the technical aspects related to the product/project I was working on whereby the quality aspects of the code was the primary concern. With the switch to an architect, though most of those responsibilities still remain, there was an additional aspect which I had overlooked during the initial period.  In the capricious environment we usually work in, things do get overlooked which if left unattended will hinder the performance of your team and in turn affect the organization you work for.

Though I considered it to be a project management aspect, I came to figure out that this aspect was a responsibility of mine with this new role. So what is this overlooked aspect you ask? The management of the people aspect. Although you as an architect will not be involved with other project management aspects such as budgeting, schedule planning (though you need to assist on this one), goal setting, emitting CO2 with managerial meetings with no agenda (no pun intended of course), you do have the most important aspect of it all to manage, which is the people aspect. Though technical leadership and direction is your primary focus, you cannot overlook the fact that your team and the team morale is the determinant success factor of your team.

As an architect, you are now directly responsible of the career progression of your team. If you have a few under performers in your team, it does not mean you should corner them out and report them to management. On the contrary, you have to understand the individual’s issues, the reasons for under-performing and setting up a plan for that individual to focus on making the wrongs right. Most of the time, the under performers have a reason for their current predicament and understanding that will enable you to help that person out of the difficult situation. This is the time the team as a whole should drive the required productivity of the team until the under-performers do come up to the required standard. Constant feedback will help the people to understand their weakness and improve on it.

The catch here is that sometimes you figure out some people just do not take constructive feedback well. You cannot take this personally as this is the idiosyncratic behavior of some people. You can help someone as much as they want to be helped.

Learning to give feedback was another skill I had to obtain with this new role. Giving feedback is not the hardest part, but giving it in a way where the receiver does not go into a defensive mode is the harder part. Again, though I see this as a project manager’s responsibility, I believe as an architect, you are still responsible for the growth of your team and should give them feedback in order for them to grow. Feedback will include both the positive and the negative. “Good job with the XYZ module” is not a positive feedback. “Good job” works only if you are Hancock (you have to watch the movie to get that one). You have to be very specific regarding the accomplishment of the individual. “The new automation build script you setup for the team helped us to give regular internal releases whereby we were able to get feedback early on the features the team was working on”. Now that is a very specific positive feedback attributing what the person did and aligning that to the company’s goals of reducing operations costs and turnaround time.

Even though technical leadership still remains as the primary focus for you as an architect, you should not forget the most important people factor which will help you succeed as a team and move forward adding value to the organization you work for.
In the next post I will focus on the changes I had to make especially in the thought pattern transitioning from being a glorified geek.


Thank you for reading people, and as always your feedback is much appreciated and do please leave by a comment if you have anything to say, the good, the bad and the ugly.
Wednesday, October 14, 2015 0 comments

Cassandra anyone?

Been some time after my last post. Blogging has not been an option for the past few weeks with other commitments going on. But you cannot stop doing what you love the most. So here I am back again fervently nudging away on the keyboard. I was analyzing the use of Cassandra for one of the projects I am working on. This post will focus on the getting started aspect with related to Cassandra and mainly in terms of Windows because fortunately or unfortunately that is the operating system I am stuck with at the moment (no pun intended).

The latest Cassandra distribution can be downloaded at here

I wanted to have the ability to run Cassandra as a Windows service. Here are the steps on how to get that done;

  • Download Apache commons daemon from here
  • Extract the Cassandra distribution to a location of your choice
  • Go into the “bin” directory of the Cassandra distribution you just extracted
  • Create a folder called “daemon” inside the “bin’ directory
  • Extract the Apache commons daemon distribution to a location of your choice
  • Copy the “prunsrv.exe” related to your architecture (32bit/64bit) to the “daemon” directory created inside the “bin” directory in your Cassandra distribution extract
  • Go into your “bin” directory via the command line and execute the following command;

 cassandra.bat install

 That’s it. Now you have Cassandra installed as a windows service which you can set to start automatically if that is what you prefer.
By default, authentication and authorization is disabled on Cassandra. Enabling this is a breeze with the Cassandra YAML configuration file. Go to the “conf” directory of your Cassandra distribution where you will find a file named as “cassandra.yaml”. Open up this with your favourite text editor (Notepad++ is awesome. Just saying). Make the following changes to the already existing configuration;

  • authenticator: org.apache.cassandra.auth.PasswordAuthenticator
  • authorizer: org.apache.cassandra.auth.CassandraAuthorizer

Now that you have enabled authentication and authorization, you can go in and login to Cassandra. The super user in Cassandra has the following credentials which you must use to log in;

  • User name : cassandra
  • Password : cassandra

To log in, let us go back to the “bin” directory of the Cassandra distribution where you will find a batch file called “cqlsh”. Bring up your command line again and execute the following command;

cqlsh -u cassandra -p cassandra


Now that you are in, if required you can go and change your super user password. One of the advantages of Cassandra for me is how similar the syntax is to relational databases. You can get up and running in a very short amount of time as the commands are very similar to traditional SQL. Cassandra has its own query language called CQL.
Next up, I was looking for a convenient GUI to interact with Cassandra. Browsing around, the most user friendly interface I found was DBeaver which you can download from here.

On the next article, I will focus on functionality with respect to Cassandra and how your thinking pattern should change when working with Cassandra as it will not be the same as how you design a traditional database schema.


Sunday, February 22, 2015 0 comments

Performance monitoring and profiling – Part 1

Performance monitoring and profiling are two different aspects. The former takes a more of a proactive measure whereas the latter is a reactive approach. In my experience, performance is an afterthought in most cases rather than being built into the software development life cycle. I see the same thing in most cases when it comes to security as well but let us not go there today as that lends itself to a different post.

Performance monitoring


In order to monitor your application, you first need to understand what performance aspects the end users of your application expects. This would include throughput, response time, uptime etc. After you collate these information, the next step would be to create scripts for load testing your application. Coming primarily from a Java background, I normally use JMeter which comes with a user friends UI to create load testing depending on your use case be it database, Web Service, JMS etc.

So you have your load testing scripts and it is running fine. What next?

Well now that you have the load scripts in place, the next step would be note down what parameters you should monitor. Generally you would start out monitoring the CPU and Memory usage while your load test is in progress.

When you monitor CPU, you would be looking at the CPU usage as the application is being load tested. If there is high CPU usage, you need to further drill down to find out what issue might cause this. If there is low CPU usage, then probably you can try and increase the load to see what the optimum load that uses efficient CPU.


If you are on a Windows machine, by default you will open the task manager and go into the Performance tab in order to find out what your CPU is up to. A snippet of mine is as follows;

I’m running on a Core i5 with 8GB of RAM. In the upper right hand corner you can see the CPU usage history. The four boxes here represent the four CPUs. You will get the same value if you run Runtime.getRuntime().availableProcessors() within a Java program. 


By default it only shows the total CPU time. But you can easily add the kernel CPU time by clicking onn View-> Show Kernal Times. You will then be presented with the following graphs with a red line;


The space between the red line and the green line is the user CPU time. So what is the difference between the user and the kernel CPU time you might ask.

User CPU Time : The amount of time the CPU spends on running your application code.

Kernel CPU Time : The amount of time the CPU spends on operating system related activities. For example if your application does a lot of reads/writes to the disk, you will see high kernel CPU as it requires the operating system related functionality to be invoked.

But what if you needed further information on what your CPU is doing at any given moment of time? PerfMon to the rescue. Open up a Run tab and type in perfmon and you will be presented with the following screen;

You can add any counter which you are interested in monitoring by right clicking on the graph area and selecting the Add Counters menu option. If you want to monitor the user CPU and kernel CPU, select the Performance object and under that you will find the User time and Privileged time (kernel CPU time) counters.

Another key indicator to watch out for when monitoring CPU is the CPU schedule run queue. This is where all light weight processes needed to run are queued up waiting for CPU. As a rule of thumb, if your run queue depth is more than four times the available virtual processors on your machine, then you need to investigate your application further more to see what could be done. In the era of high memory machines, people often overlook the fact of using and writing efficient algorithms and data structures as there is plenty of memory available on high end servers. But the issue here is that many a times CPU is limited. If your algorithms/data structures do not scale well with added load, you will end up over loading your CPU with the only alternative being is to scale your application into different servers. So if you are having issues with the scheduler run queue depth, it is always best to see the possibility of writing certain sections of your application code in a more efficient manner so as to utilize your CPU better. The run queue depth can also be monitored using Perfmon by selecting Add Counters->System-> Processor Queue Length.

Now that we have covered some of the important factors to look into in terms of CPU when monitoring your application, in the next post we will look at what aspects we need to consider when monitoring the memory usage.

Please do leave by your comments and suggestions which is as always much appreciated as I love learning from the experience of others which I deem as invaluable.

Sunday, June 22, 2014 2 comments

A little bit on the JVM and JIT

As you might be aware, the JVM(Java Virtusal Machine) is what makes it possible for Java to adhere to the write-once-run-anywhere paradigm. At its core, the JVM consists of the following components;

Heap
Stack
PermGen and Method Area
JIT Compiler
Code cache

The heap is where memory is allocated for every new operator you use during the application code development stage. Stack will store the local variables that you will assign within the scope of a method. One thing to note is that the variables defined within the scope of a method will be removed after the completion of the method. If for example, a String is assigned within the scope of a method, and its scope is guaranteed to be of local scope, then this will be stored in the stack which would otherwise be assigned within the heap.
The PermGen space will store class and method level data as well as static variables that are defined in your application. The method area is actual an area within the PermGen space where it will store all method,field, constant pool level details of your application.

The JIT compiler and the code cache go hand in hand. The JVM at its core interprets the Java Byte Code into assembly code at runtime. Interpreting can be a slow process because the code needs to be converted from byte code to machine code at runtime every time a portion of your application code is executed. This is where the JIT compiler comes into action, with its super awesome compilation of methods which it then stores in the code cache.

The JIT compiler analyzes the application code at runtime to understand which methods can be categorized as hot methods. Hot in this context meaning code fragments that are accessed more frequently. At a very high level, what the JIT compiler does is that it will have a counter for each method executed in order to understand the frequency of its usage. When the counter reaches a defined threshold value, the method then becomes eligible to be compiled by the JIT compiler to its respective assemble code which will then be stored within the code cache. What happens is that now, whenever the JIT compiler comes across calls to those methods which were compiled and stored within the code cache, it will not try to interpret them yet again but will use the already compiled assembly code available within the code cache. This gives your application a performance boost because using the compiled code is much faster than interpreting it during runtime.

When talking about the JIT compiler, there are mainly two flavors of it which we are mostly oblivious of due to the fact of the lack of documentation around them. The two types are;

Client
Server

The default compiler used will defer according to the machine architecture and the JVM version (32bit or 64bit) that you are running on. Let us briefly see what each one does.

The client compiler starts compiling your byte code to assembly code at the application startup time. What this indirectly means is that your application will have a much improved startup time. But the main disadvantage this brings along with it is that your code cache will run out of memory faster. Most optimizations can be made only after your application has run for a brief period of time. But since the client compiler already took up the code cache space, you will not have space to store the assembly code for these optimizations. This is where the server cache excels.

Unlike the client compiler, the server compiler will not start compiling at the start of your application. It will allow the application code to run for some time (which is often referred to as the warm-up period) after which it will start compiling the byte code to assembly code which it will then store within the code cache.
In my next post I will discuss how we can actually mix and match the client and server compilation and also introduce you to a few more JVM flags that we seldom come across but are vital for increasing the performance of your application.

Friday, June 6, 2014 1 comments

Finding the Equilibrium index of an array

I wanted to do a brain teaser today so i took up an algorithmic question to give a shot at. Now i know this question already has many answer on the internet if you do a quick search. But i wanted to try it out with the solution that came to my mind. The problem statement is as follows;

Equilibrium index of an array is an index such that the sum of elements at lower indexes is equal to the sum of elements at higher indexes. For example, in an array A:

A[0] = -7, A[1] = 1, A[2] = 5, A[3] = 2, A[4] = -4, A[5] = 3, A[6]=0

3 is an equilibrium index, because:
A[0] + A[1] + A[2] = A[4] + A[5] + A[6]

6 is also an equilibrium index, because sum of zero elements is zero, i.e., A[0] + A[1] + A[2] + A[3] + A[4] + A[5]=0

7 is not an equilibrium index, because it is not a valid index of array A.

Write a function int equilibrium(int[] arr); that given a sequence arr[] of size n, returns an equilibrium index (if any) or -1 if no equilibrium indexes exist.

The solution i came up with is as follows;


 
public static int solution(int[] A) {

  if (A == null || A.length < 3)
   throw new RuntimeException("Cannot find equilirbium");
  int pointer = 1;
  int lowerIndCount = A[0];
  int upperIndCount = 0;

  for (int i = 2; i < A.length; i++) {
   upperIndCount += A[i];
   if (lowerIndCount < 0) {
    if (upperIndCount > lowerIndCount && i != A.length - 1)
     continue;
    if (upperIndCount == lowerIndCount && i == A.length - 1)
     break;
    lowerIndCount += A[pointer];
    upperIndCount = 0;
    ++pointer;
    i = pointer;
   } else {
    if (upperIndCount > lowerIndCount) {
     lowerIndCount += A[pointer];
     upperIndCount = 0;
     ++pointer;
     i = pointer;
    }
   }
  }

  if (upperIndCount == lowerIndCount)
   return pointer;

  return -1;
 }

If this is the optimum or not im not sure. Im also not sure if this will work on very large data sets. But for the data sets i tried(both negative and positive) it worked fine. And also this has O(N) complexity as I am iterating through the array only once.


 
;