A Customized User Interface for Mobile Phones Blog


    The StringItem class in Java ME is used for non-editable display of text. While it is very convenient and easy to use, it has certain disadvantages too. Implementations ofStringItem are highly device-and theme-dependent. This means the look-and-feel of an application can differ significantly from one device to another. Also, some of these implementations leave much to be desired. On one of my handsets, for example, aStringItem based text display flickers so much during scrolling that reading becomes extremely uncomfortable.

    A text display based on the Canvas class overcomes these drawbacks and offers advantages such as:

    • A highly uniform, device-independent and customizable user interface.
    • Text and display styles that can be more varied and can be changed dynamically.
    • Multilayer displays.

    An example
    Figure 1. An example.

    Figure 1 shows such a display with different font styles and with grouping for entries. Not only do such displays look virtually the same on all phones, they are also very easy to implement.

    In this article we shall examine a demo application showing the step-by-step implementation of such a text display. The application will use the following basic classes:

    • TextUI -- this is an abstract class that defines the main functionalities required for displaying text on a canvas.
    • MenuLayer -- this class is for creating a menu in the form of a Sprite.

    In addition to the above, there will have to be a MIDlet --TextUIDemo -- that will serve as the entry point. However, the real work will be done by the classes listed above and a subclass of TextUI.

    In order to run the demo application you'll need to download the Sun Java Wireless Toolkit and install it on your computer.

    The Display Parameters

    Before we get into the programming details, let us establish the broad visual aspects of what the user interface will look like. The sketch in Figure 2 shows the overall structure and the associated variables that have been used in the code to establish the dimensions needed to draw the complete interface.

    The basic layout
    Figure 2. The basic layout.

    The dimensions of the area for rendering the text we want to show can be easily visualized now. The height of this area, represented by the variable height, will be:

    height = totalheight - (2*topborder+titleheight+topmargin+ 2*bottomborder+bottomheight); 


    totalheight = height of available screen area titleheight = height of title bar topborder = thickness of borders above and below title bar bottomheight = height of bottom bar bottomborder = thickness of borders above and below bottom bar topmargin = height of a margin above the text 


    width = totalwidth - (leftborder+leftmargin+rightborder); 


    totalwidth = width of available screen area leftborder = width of the border at left rightborder = width of the border at right leftmargin = width of a margin at the left of the text 

    The TextUI Class

    The base class for displaying text will subclassCanvas and I've named it -- rather unimaginatively --TextUI. This is an abstract class and must be subclassed. The main function of TextUI is to break a string into lines that can fit into the available display area. In other words, width of a line must be less than or equal to the variable width, whose value we calculated above. The method used in this demo to perform this splitting is a very simple and intuitive one. Also it looks only for line-breaks (CRLF) and spaces. However, it is adequate for a demo and, if desired, a more sophisticated and complete algorithm can be easily substituted.

    The work of breaking down a string into lines is done by the following three methods working together:

    • getFormatted -- makes sure that the entireString has been split into lines and returns an array of these lines.
    • stringSplitter -- generates one line at a time.
    • wordSplitter -- if there are words that are too long to be displayed in one line, this method breaks them into manageable strings.

    Additionally, TextUI also provides support for scrolling, stipulates the abstract methods that have to be implemented by subclasses, and defines the variables required to implement its functionalities. Some of these variables have to be initialized by a subclass of TextUI.

    As this article is about creating a customized text display, we will not dwell on how to parse and tokenize a string. Instead we will go straight on to our main topic.

    The MenuLayer Class

    The implementation of a menu, too, is highly platform-dependent. On some devices, the menu always covers the entire screen. On others, only a part of the screen is obscured. On some devices, there are no system-enforced shortcut keys, while some have shortcut keys which may conflict with those defined by the application. So it is desirable to design one's own menu to ensure a uniform look-and-feel. In this section, we shall create a simple menu which will look the same and behave in the same way on all Java ME MIDP 2.0 compliant devices.

    In our application the menu will extend Sprite. Our menu will be designed to hold a number of options and the user will be able to select one of them either through a cursor or by clicking the designated shortcut key. Figure 3 shows what the menu is going to look like.

    The Menu
    Figure 3. The Menu.

    The main functions of the MenuLayer class are described below.

    Creating the Image forMenuLayer

    The image is made up of the frames for theSprite. The only thing that changes in a menu is the position of the cursor. So the number of frames will be equal to the number of options and each frame will show the cursor on a different option. The image is created by thecreateMenuImage method:

    //creates image for menu public void createMenuImage() { //this font is used to calculate menu dimensions Font font = Font.getFont(Font.FACE_SYSTEM, Font.STYLE_PLAIN, Font.SIZE_SMALL); //number of options int length = options.length; int fontheight = font.getHeight(); //index of the longest option string int longestindex = 0; //find the index of the longest option string for(int i = 0; i < length; i++) { longestindex = font.stringWidth(options[longestindex]) > font.stringWidth(options[i]) ? longestindex : i; } //width of longest option string int maxstringwidth = font.stringWidth(options[longestindex]); //calculate menu size menuwidth = maxstringwidth + 20; menuheight = length*fontheight + 16; //get reference to the image to be drawn menuimage = Image.createImage(menuwidth * 3, menuheight); //draw the image //context for drawing image Graphics gm = menuimage.getGraphics(); gm.setColor(backcolor); //draw the body of menu gm.fillRect(0, 0, menuwidth * 3, menuheight); gm.setColor(0x000000);//black //set the font that was used for size calculation gm.setFont(font); //draw the frames to be used for rendering the sprite for(int j = 0; j < length; j++) { //draw options for each frame for(int i = 0; i < length; i++) { //at the cursor if(i == j) { gm.setColor(cursorcolor); gm.fillRect(j*menuwidth, 8+fontheight*i, menuwidth, fontheight); gm.setColor(whitecolor); gm.drawString(options[i], j*menuwidth+10, 8+fontheight*i, Graphics.TOP|Graphics.LEFT); gm.setColor(0x000000); //black again } //at other positions else { gm.drawString(options[i], j*menuwidth+10, 8+fontheight*i, Graphics.TOP|Graphics.LEFT); } } } } 
    Creating the Sprite

    Now that the image is ready, we can set up the menu as aSprite. The Constructor ofMenuLayer first calls createMenuImage and then instantiates a Sprite:

    public MenuLayer(String[] options) { this.options = options; //create the image for sprite that will be used as a menu createMenuImage(); //make sprite with image created sprite = new Sprite(menuimage, menuwidth, menuheight); } 

    Creating the Display

    The subclass of TextUI that manages our display isNewTextUI. This class converts text (the textual matter that we want to display) into aTiledLayer so that we can use aLayerManager to show the text as well as the menu in a co-ordinated manner. A LayerManager, as the Java ME documentation describes it, "simplifies the process of rendering the Layers that have been added to it by automatically rendering the correct regions of each Layer in the appropriate order." Since both the text display and the menu are subclasses ofLayer, they can be added to aLayerManager. We can then show only the text or the text with the menu superimposed on it by calling the relevant methods of LayerManager without getting involved in the details of rendering them.

    Let us see how the NewTextUI class performs the tasks related to setting up the text display.

    Creating the Image for the Text Layer

    The image for a TiledLayer is composed of individual tiles. We shall convert the entire text into a single image so that there will be just one tile to consider. The rendering of the image is handled by thecreateTextImage method:

    //create image for text layer public void createTextImage() { //adjust bottomindex for short text if(textsize <= numoflines) { bottomindex = topindex + textsize - 1; } //image for creating textlayer textimage = Image.createImage(width, textsize*fontht+2); //graphics context for rendering textimage Graphics gt = textimage.getGraphics(); gt.setColor(0x000000); //black for writing text //for writing text set the font //that has been used in line size calculation gt.setFont(f); //draw all the lines for(int line = 0;line < textsize;line++) { gt.drawString(contents[topindex+line], leftmargin, topmargin+line*fontht, Graphics.TOP|Graphics.LEFT); } } 
    Creating the TiledLayer

    Now that the image is available, we can get theTiledLayer by calling getTextLayer:

    //creates and returns a TiledLayer with one cell //and with text image as the only tile public TiledLayer getTextLayer() { //new Tiled layer with text image TiledLayer tl = new TiledLayer(1, 1, textimage, width, textsize*fontht+2); //fill the only cell with the only tile tl.setCell(0, 0, 1); //make the layer visible tl.setVisible(true); return tl; } 

    Note that our TiledLayer needs is only onecell as we have just the one tile to display.

    The LayerManager

    The next step is to create a LayerManager and insert the newly obtained TiledLayer into it. All the tasks related to readying the TiledLayer and theLayerManager are performed within the constructor ofNewTextUI:

    public NewTextUI(String text, Display d, TextUIDemo tuid) { . . . createTextImage(); textlayer = getTextLayer(); manager = new LayerManager(); //initialise the view window manager.setViewWindow(0, 0, width, height+2*topborder+topmargin); //insert textlayer at the topmost position manager.insert(textlayer, 0); . . . } 

    After the LayerManager has been instantiated, theview window has to be initialized to display the text, starting from the first line. The dimensions of the view window are set to fit into the area designated for text.

    Displaying the Text Within Its Frame

    The paint method first paints the 'frame' around the text display (Figure 2 above) and then calls paint on manager (that is theLayerManager) to render the layers:

    //paint the frame, that is, the borders and the bars //and, finally, ask manager to paint all the layers public void paint(Graphics g) { g.setColor(0xffffff);//white g.fillRect(0, 0, totalwidth, totalheight); //draw the borders, titlebar and bottombar g.setColor(0x30e030);//dark green g.fillRect(0, 0, totalwidth, titleheight);//titlebar g.fillRect(0, totalheight-bottomheight, totalwidth, bottomheight);//bottombar g.setColor(0xffaaaa);//light red g.fillRect(0, 0, totalwidth, topborder);//top border g.fillRect(0, titleheight, totalwidth, topborder);//border below titlebar g.fillRect(0, totalheight-bottomborder, totalwidth, bottomborder);//bottom border g.fillRect(0, totalheight-bottomheight, totalwidth, bottomborder);//border above bottombar g.fillRect(0, 0, leftborder, totalheight);//left border g.fillRect(totalwidth-rightborder, 0, rightborder, totalheight);//rigt border g.setFont(tf);//set font for writing on the bars g.setColor(0x000000);//black for title g.drawString(title, width, 3, Graphics.TOP|Graphics.RIGHT);//write title //if first line of display is first line of text //then set white if(topindex==0) { g.setColor(0xffffff);//white } g.drawString(String.valueOf(topindex+1), orgx,3, Graphics.TOP|Graphics.LEFT);//write top line index //if last line of display is last line of text //then set white otherwise black g.setColor(bottomindex == textsize-1 ? 0xffffff : 0x000000); //write bottom line index g.drawString(String.valueOf(bottomindex+1), orgx, totalheight-bottomheight+3, Graphics.TOP|Graphics.LEFT); //paint all the layers manager.paint(g, leftborder, titleheight+topborder); flushGraphics(); } 

    When the application is launched the layer manager has only the text layer. So the text is displayed within its frame as shown in Figure 4.

    Text display
    Figure 4. Text display.

    The line number of the first line being shown is displayed on the title bar, and the last line's line number is on the bottom bar. When further scrolling in a given direction is not possible, the corresponding line number turns white. A right-justified title of the display is also written on the title bar. Conventionally the numbers should have been written on the right and the title on the left. This minor departure from convention is only to emphasize the flexibility available to us.

    Now that we have taken care of the visual aspects, let's see how things work together by implementing the functionalities of the text and the menu described below.

    Scrolling the Text

    The 'view window' of the LayerManager defines the location and size its visible portion. Scrolling the text can be achieved simply by moving the view window up or down. The unit distance for movement is determined by the height of the font used to render the text. The following methods in NewTextUIperform this task:

    //scroll up the text protected void scrollTextUp() { if(topindex > 0) { topindex--; . . . //topindex is decremented //so view window moves up manager.setViewWindow(0, topindex*fontht, width, height+2*topborder+topmargin); updateTextScreen(); } } //scroll down the text protected void scrollTextDown() { if(bottomindex < textsize - 1) { topindex++; . . . //topindex is incremented //so view window moves down manager.setViewWindow(0, topindex*fontht, width, height+2*topborder+topmargin); updateTextScreen(); } } 
    Showing the Menu

    When the Options command on the text display screen is selected, the showMenu method inNewTextUI is called, which performs the necessary housekeeping tasks and then shows the menu:

    //setup and show the menu protected void showMenu() { //remove commands from text canvas //because all user inputs now are for menu only tuid.removeCommands(); //set flag so that key events can be routed properly menushown = true; //initialise cursor position menu.setCursorIndex(0); //remove existing second layer if any if(manager.getSize() == 2) { manager.remove(manager.getLayerAt(0)); } //if menusprite hasn't already been obtained //then get it if(menusprite == null) { menusprite = menu.getSprite(); } //select the first frame so that //cursor is shown on first option menusprite.setFrame(0); //set initial position of menu menusprite.setPosition(posx, posy); //insert sprite into layer manager as topmost layer manager.insert(menusprite, 0); //ask layer manager to paint the layers //as menusprite has been added as topmost layer //it is shown superimposed on text manager.paint(g, leftborder, titleheight+topborder); //flush to the screen flushGraphics(); } 

    Figure 5 shows the menu superimposed on the text display.

    The menu is shown
    Figure 5. The menu is shown.

    Scrolling the Cursor on the Menu

    When the menu is visible on the screen and the Up or the Down key is pressed, the keyPressedmethod of NewTextUI catches the key event and calls the appropriate method in MenuLayer. Since the sprite we use for the menu has a frame for each position of the cursor, all we need to do to 'animate' the cursor is move from one frame to the next or to the previous one depending upon the direction of scrolling. The code snippet below shows the method for scrolling up:

    //scroll the cursor up on the menu public void scrollUp() { //cursor index decremented and //goes to highest value from zero cursorindex = cursorindex == 0? options.length - 1 : --cursorindex; //show the previous frame (frame is circular) sprite.prevFrame(); } 

    Note that our job has been made very easy as theSprite class has two methods -- prevFrameand nextFrame -- for traversing the sequence of frames. In addition to changing the frame, scrollUpupdates the value of cursorindex to keep track of where the cursor is positioned at any given time.

    The method for scrolling down is scrollDown and it operates in a similar fashion calling nextFrame to show the next frame in sequence.

    Selecting a Menu Option

    Menu options, as we know, can be selected in two different ways. The first is by positioning the cursor on the desired option and clicking the selection key. This can be the middle button in a 4+1 navigation scheme and/or, in keeping with gaming convention, the number key 5. In terms of the terminology used byCanvas, we are interested in theCanvas.FIRE key. The keyPressed method ofNewTextUI handles this key event when the menu is on screen:

    //when a key is pressed once protected void keyPressed(int keycode) { . . . //menu cursor control switch(getGameAction(keycode)) { . . . case Canvas.FIRE : menuAction(menu.getSelectedIndex()); break; . . . } . . . } 

    The MenuLayer keeps track of the cursor position and this value is returned when getSelectedIndex is called. To handle menu item selection, the keyPressedcalls menuAction with the cursor index as parameter and, depending on the value of cursor position, appropriate action is taken.

    The second method of selecting an option from the menu is to use the respective shortcuts. The numbers in square brackets shown against the options are the applicable shortcuts. To select Option 1, for instance, the number key 1 can be pressed. This key event also is handled by keyPressed inNewTextUI and menuAction is called with the proper value of parameter:

    //when a key is pressed once protected void keyPressed(int keycode) { . . . switch(keycode) { //menu action cases case Canvas.KEY_NUM1 : menuAction(0); break; case Canvas.KEY_NUM3 : menuAction(1); break; case Canvas.KEY_NUM0 : menuAction(2); break; . . . } . . . } 

    Within the menuAction method, the required action is taken and the menu is removed from the layer manager. Also the commands that were removed when the menu was popped up are added back and the menushown flag is cleared. This is shown below:

    //actions as per menu option selected private void menuAction(int actioncode) { Alert optionalert; switch(actioncode) { case 0 : menushown = false;//clear flag //remove menu from layer manager manager.remove(menusprite); tuid.addCommands();//add back commands //dummy action optionalert = new Alert("Text Canvas", "Option 1 selected", null, AlertType.INFO); optionalert.setTimeout(2000); display.setCurrent(optionalert, this); updateTextScreen(); break; case 1 : menushown = false; manager.remove(menusprite); tuid.addCommands(); optionalert = new Alert("Text Canvas", "Option 2 selected", null, AlertType.INFO); optionalert.setTimeout(2000); display.setCurrent(optionalert, this); updateTextScreen(); break; case 2 : menushown = false; manager.remove(menusprite); tuid.addCommands(); updateTextScreen(); } } 

    A Floating Menu

    Sometimes, when I open a menu, I want to take a final look at the original document before choosing an action. On mobile phones this can be a problem as some menu implementations completely cover the screen. Even on those that don't, the small display area means that a menu is likely to obscure most of the screen. At such times a menu that can be moved around would be very useful. In this section we make our menu movable.

    Before we look at the code, we need to choose the user action required. The most obvious choice would be the navigation keys. But we have already decided to use the Up and Downkeys to move the cursor on the menu. A popular practice for games is to use the numeral keys 2, 4, 6 and 8 for movement -- 'up', 'left', 'right' and 'down' respectively. So our application will also use these keys for menu movement.

    The first thing that we have to do is listen for the events corresponding to the keys that move the menu around. So the following code is added to the keyPressed method inNewTextUI:

    . . . //menu movement cases case Canvas.KEY_NUM2 : menuUp(); break; case Canvas.KEY_NUM8 : menuDown(); break; case Canvas.KEY_NUM4 : menuLeft(); break; case Canvas.KEY_NUM6 : menuRight(); break; . . . 

    We also add a keyRepeated method toNewTextUI so that holding one of the 'movement' keys down will make the menu move continuously:

    //when a key is held down protected void keyRepeated(int keycode) { //if menu is being shown then 2/8/4/6 keys //refer to movement of menu if(menushown) { switch(keycode) { //menu movement cases case Canvas.KEY_NUM2 : menuUp(); break; case Canvas.KEY_NUM8 : menuDown(); break; case Canvas.KEY_NUM4 : menuLeft(); break; case Canvas.KEY_NUM6 : menuRight(); } } //otherwise they refer to the text //let super class handle it else { super.keyPressed(keycode); } } 

    We now need to decide how much to move the menu for each key press. This value corresponds to variables posdelta andnegdelta as defined in NewTextUI and, in this example, both have been set to 2 pixels. Upward and leftward movements will use negdelta while downward and rightward movements will use posdelta. Changing the values of these variables will change the granularity of movement.

    The Layer class has a method which makes it very convenient to control the menu's movement. Since a sprite is also a layer,our menu movement methods in NewTextUI call this method. Note that the move method ofLayer class takes two parameters - dx anddy - that define, respectively, the horizontal and vertical distances for movement. So, for moving the menu updy is negative and, for moving it down, dy is positive. The motion of the menu is controlled by the following methods:

    //move menu up private void menuUp() { menusprite.move(0, negdelta); //repaint to show menu at new location updateTextScreen(); } //move menu down private void menuDown() { menusprite.move(0, posdelta); //repaint to show menu at new location updateTextScreen(); } //move menu left private void menuLeft() { menusprite.move(negdelta, 0); //repaint to show menu at new location updateTextScreen(); } //move menu right private void menuRight() { menusprite.move(posdelta, 0); //repaint to show menu at new location updateTextScreen(); } 

    That's it. Now the menu will move around on the screen if one of the "movement" keys (2/8/4/6) is pressed. Note that the menu will keep moving until it goes out of the screen as there is no limit check. If desired the movement can be easily stopped at a screen edge by limiting the bounding co-ordinates of the sprite appropriately. For example, to stop the menu at the upper edge of the screen, we have to ensure that the vertical position of its upper-left corner (as returned by getY method ofLayer) does not become negative.

    Figure 6 shows the display with the menu having been moved to a new position.

    Menu moved to a new location
    Figure 6. Menu moved to a new location.


    We've seen one implementation of a custom UI on a mobile phone. There are other ways to develop similar UIs and I hope you will experiment and find the approach that best suits your requirements. Remember that the technique shown here can be used for non-text displays and UIs too. You can implement an 'arrow' cursor as follows:

    • Make an image.
    • Create a sprite using the image.
    • Define a reference pixel. The tip would be the natural choice for an arrow.
    • Write methods to move the cursor.
    • To determine the position of the cursor usegetRefPixelX and getRefPixelY ofSprite class. This will tell you what the cursor is pointing at.
    • Take desired action when the Canvas.FIRE key is pressed.

    This year, at JavaOne, an ME equivalent of Swing was announced, the Lightweight UI Toolkit. No doubt LWUIT will provide the platform for sophisticated and consistent user interfaces. However, if you need a simple lightweight UI that needs to be different from the standard lcdui screens, look and act the same on all Java ME (MIDP 2.0) compatible phones and is easy to integrate into your application, you will find the approach shown here worth going with.


    Beginning J2ME: From Novice to Professional by Sing Li and Jonathan Knudsen has an excellent chapter on custom UIs. The design described in this article is an extension of the basic scheme outlined by Li and Knudsen.