Writing Reproducible Test Cases: Why and How

Version 5

    By Timo Hahn ace_director_gold.png

    We all have been in situations where the application we were developing or maintaining did not do what we expect it to do. This happens all the time. "No big deal," you think. But after hours of trying to figure out the problem, you still are stuck.

    In my experience it helps to stand back a bit and try to look at the problem from a different angle. There are different ways to do this. One is to try to explain the problem to someone. I  sometimes see the problem better while trying to explain what I have tried to do to resolve the problem. The interesting part is the the person you talk to does not have to be a programmer.

    In How to Ask Questions in Developer Community Spaces I offer a few guidelines for posting forum questions that will help you to get answers. But even the most carefully composed question may not produce a usable response.  This is the point at which you should think about writing a reproducible test case. This article offers guidelines on how write  a test case that will  make it easy for others to help you solve the problem.

    Requirements

    A test case should fulfill these basic requirements:

     

    1. It must reproduce the problem
    2. It must be as easy as possible in doing this
    3. It should be easy to use by other users which don't work in your environment
    4. It should use a data model which is known by other users (without the need to study table definitions, triggers, …)
    5. If a known data model can't be used, the model that is used must be described in great detail
    6. If possible, it should not be necessary to change  or insert data
    7. If data has to be changed or inserted you should include scripts or steps to get the original data back
    8. It should take no more than 15-20 minutes to set everything up

    If you think that it will cost a lot of your time to create such a test case, you're right.But it is time well spent:

    1. Chances are good that you'll find the solution to your problem while creating the test case. Then you are done. Be proud!
    2. Creating the test case forces you to see a clear picture of the problem. This helps to describe the problem to others.
    3. Providing a test case to the community dramatically increases the chances that somebody will find the fix.
    4. If nobody can help you and you ask support.oracle.com for help, they will ask for a reproducible test case. Normally they work faster if you can provide one and have a good description of the problem.

    Creating a Test Case

    Now that you understand the reasons and requirements for a test case, let's talk about how to create one. I always use the HE DB schema provided with every Oracle DB (or you can get it from Oracle Sample Model and Scripts). Almost every developer has access to a running HR DB instance. Depending on the problem and the relation to the original data I choose only the minimum number of tables needed to create the test case.

    Lets say the problem in your app occurs in a data entry form which consists of master-detail-detail data. In such a case I look at the HR model and use Regions, Countries, Locations as data model for the test case. Depending on the data, you can choose different tables, e.g. Departments, Employees, Jobs.

    Once the data model (or the tables) are known, I create a simple ADF Fusion Web Applicatio, which will create a workspace with two projects.  I create the model project by using the "Business Components from Table…" option, available by right-clicking on the model project and then selecting "New." As the whole process is to long to describe here, I made screenshots from every dialog . The sample application is created with JDev version 12.2.1.2.0, so the images below may not match what you see in your environment.

    Creating the workspace:

    Creating the model project:

    Checking the model project:

    After this the basic workspace for the test case is finished and can be used to add the problem-specific parts. Here you start with the business logic and add whatever you need to show the problem in the UI.

    If your problem is related to cascading selectOneChoice in a form, it’s a good idea to add the configuration to the model project and test this using the application module tester. Once the business logic is running you continue with setting up the UI. You don't have to design a pretty looking UI. Make it as simple as you can. Concentrate on showing the problem.

    Here are some specific things you should take into account:

    1. If your problem needs a fancy design, e.g. you have a layout problem or a skin problem, then make sure to add all resources necessary to reproduce the problem.
    2. If your problem needs specific data in the DB, you should create DDL scripts to create the tables and to insert data into the tables. Avoid adding table space names and other storage information in the scripts, as other users likely don't have those table spaces. A description of the data model is mandatory!
    3. If your case needs third party libraries you must provide the exact version of the libraries and the location where they can be downloaded. Smaller libraries should be added to the test case zip file.
    4. Write a detailed description of how to start the application (which page to run) and what to do to get to the problem.
    5. Format your code! I hate reading unformatted code. It costs time to interpret things that is better spent on solving the problem, not by trying to figure out which part of the code I'm looking at.
    6. Comment your code! The better the code is commented the easier it is to understand what you have tried to do. In a perfect world I would like to understand the code by just reading the comments of the methods without having to go into the details of the implementation. Once the problem area is clear, I start looking into the implementation. One thing I do is to read the comment of a method and check it against its implementation.

    Finally,  zip everything together and make the zip file available to the public. Make sure that you delete unnecessary files and folders from the workspace (like .data and .classes). These folders are huge and are recreated automatically when you compile the application. Pay special attention to library files (jars), which are sometimes huge. They can be omitted because you are providing the version and download location (and instructions on where to put them!). You can use Google Drive or Dropbox, which are well known to other developers. If you use an unknown file host service you risk that nobody will download the test case.

    A Final Word of Advice

    There is no service level agreement (SLA) for community responses to forum questions. Community members will try to help. But we all have our normal work to do too. Creating a test case, even after you are asked to provide one, is no guarantee that you will get help.

    If you need urgent support ask Oracle Support or help. The test case will help you there, too, as support won't need weeks to understand and reproduce the problem.

    About the Author

    Oracle ACE Director Timo Hahn is a principal consultant for virtual7 GmbH, an Oracle Platinum partner based in Germany. He is a developer and software architect with extensive expertise in Oracle Fusion Middleware and deep specialization in the Oracle Application Development Framework.  Timo is a blogger, a frequent presenter at conferences and special events, and is well known in the OTN Community for his extensive help in the JDeveloper and ADF space.