5 Replies Latest reply on Jul 3, 2014 9:57 AM by Iniyavan

    Execution order in Test Suite

    Iniyavan

      Hi friends,

       

      I need to run 5 Tests through a Suite. I have created a Suite with the 5 Tests. The Tests are appearing in alphabetical order. When I execute it, it is executing in random (not understood) order.

       

      I want to execute it in the specified order (for example, Test_3, Test_2, Test_4, Test_1, Test_5). How can I achive this?

       

      Thanks in advance!

        • 1. Re: Execution order in Test Suite

          Your question is confusing as it mentions three different orders: alphabetical, random and specified'.

           

          Post the order the five tests appear in the nav tree. Is it alphabetical? Did you add the tests in alphabetical order?

           

          Post the order the tests are executing.

          • 2. Re: Execution order in Test Suite
            Gary Graham-Oracle

            Agreed, your question is confusing.  I read a complaint along these same lines recently on some blog, however I cannot find it at the moment.

             

            If I recall correctly, possibly renaming the tests so that the alphabetic order matches the desired execution order might work.  On the other hand, if it works, it may just be a matter of luck that (an unsorted) result set of rows gets retrieved from the unit test repository in the correct order.

             

            Regards,

            Gary

            SQL Developer Team

            • 3. Re: Execution order in Test Suite
              Iniyavan

              Hi rp and Gary,

               

              I make it clear.

               

              1. I have created 5 Tests. The names of the Tests are: Test_1, Test_2, Test_3, Test_4 and Test_5. They are working fine individually.

               

              2. I have created a Suite by combining the 5 Tests. The name of the Suite is Suite_1.

               

              3. When I open the Suite, the Tests are listed in the same order. i.e., Test_1, Test_2, Test_3, Test_4 and Test_5.

               

              4. But, when I run the Suite, the Tests are executed in the following order:

                   Test_3, Test_1, Test_4, Test_5 and Test_2

               

              5. But I want the Tests to be executed in the following order:

                   Test_1, Test_2, Test_3, Test_4 and Test_5

               

              6. I am unable to rename the Tests under the Suite node.

               

              Even if I rename the Tests under the Tests node, and recreate the Suite, the order of execution is not proper. It is executing randomly.

              • 4. Re: Execution order in Test Suite
                Gary Graham-Oracle

                Hi Iniyavan,

                 

                Thanks for the clarification.  I could be wrong, but I don't believe there have been complaints of suite test items executing out of sequence -- that is, there exists a SEQUENCE column in the SUITE_ITEM table for the purpose of controlling the order of test execution in the suite.  If you run a query similar to the following in a worksheet (or as a user defined report) against your unit test repository, does the execution order match the sequence column order?

                SELECT UT_SUITE.NAME AS SUITE_NAME,

                  UT_TEST.NAME AS TEST_NAME,

                  UT_SUITE_ITEMS.SEQUENCE,

                  UT_SUITE_ITEMS.RUN_START,

                  UT_SUITE_ITEMS.RUN_TEAR

                FROM UT_SUITE_ITEMS

                INNER JOIN UT_SUITE

                ON UT_SUITE_ITEMS.UT_SID = UT_SUITE.UT_SID

                INNER JOIN UT_TEST

                ON UT_TEST.UT_ID = UT_SUITE_ITEMS.UT_ID

                ORDER BY SUITE_NAME, UT_SUITE_ITEMS.SEQUENCE;

                Anyway, perhaps the point of confusion is that the alphabetic list of tests under a suite does not expose the order of creation (sequence).  So, if I ...

                1. Create 10 tests:  test_0, test_2, ..., test_9

                2 Add them all to suite_1 in ascending order.

                3. Run the query above.  It shows the sequence number matches the test name's suffix

                4. Remove tests with an even numbered suffix.

                5. Add back in tests with even numbered suffixes, but in descending order.

                6. Run the query above.  It then shows the tests, in sequence 0, ..., 9, as tests_ 1, 3, 5, 7, 9, 8, 6, 4, 2, 0

                Does that help?   I suppose listing the tests under a suite in creation (sequence) order and allowing drag & drop reordering might be more intuitive.

                 

                Regards,

                Gary

                • 5. Re: Execution order in Test Suite
                  Iniyavan

                  Hi Gary, thanks for the clarification.

                   

                  The confusion is due to the alphabetical list in the Suite node. I thought that the execution will be in the order it displays. But it is not so. The order is shown in the Suite window.