11 Replies Latest reply: Jul 22, 2014 3:12 PM by LosLobo RSS

    How do you build test cases so when a new class is added it automatically gets tested?

    LosLobo

      I'm trying to develop a system for test cases so that whenever a test case is added to a particular package it will automatically be included in testing without having to manually add that particular test case.  What is the best way to achieve this?  Should I use java reflection?

      Please excuse me if theres some obvious solution to this.  I'm just getting started with Jenkins and trying to configure Selenium test cases.

        • 1. Re: How do you build test cases so when a new class is added it automatically gets tested?
          rp0428
          I'm trying to develop a system for test cases so that whenever a test case is added to a particular package it will automatically be included in testing without having to manually add that particular test case.  What is the best way to achieve this?  Should I use java reflection?

          Please excuse me if theres some obvious solution to this.  I'm just getting started with Jenkins and trying to configure Selenium test cases.

          What does any of that have to do with Java and this forum?

           

          You should post your question in a forum for Jenkins or Selenium.

           

          Generally speaking you should NOT hard-code lists of ANYTHING or embed those lists in code. That method is not scaleable, flexible or easily maintained. Just because a test case exists does NOT mean that is should be automatically included in every test run that is performed.

          • 2. Re: How do you build test cases so when a new class is added it automatically gets tested?
            LosLobo

            Hoping someone might have a suggestion.  Right now reflection is the only option thats coming to me.  Seems more like a java question than selenium, Jenkins, thats why I submitted it to this forum.  Hoping someone has some useful thoughts.

            • 3. Re: How do you build test cases so when a new class is added it automatically gets tested?
              rp0428
              Hoping someone might have a suggestion

              And you got my suggestion: DON"T DO THAT!

               

              Configuration should be done the way it has ALWAYS been done. Create a configuration file. For your use case create a list (XML, name=value pairs, etc) of test cases that you want executed. The framework reads that configuration file and executes ONLY the test cases you specify.

               

              You should NEVER hard-code configuration information or lists of test cases the way you plan to.

              • 4. Re: How do you build test cases so when a new class is added it automatically gets tested?
                TPD-Opitz

                The best thing I can come up with is no technical solution but TDD.

                 

                You want your classes beeing automatically covered by some tests but this cannot work. The reason is that a test must specify what the expected results are.

                But how should any kind of artificial intelligence know what results of your new class should deliver compared to what it currenty does?

                 

                If computers were able to do this decision they could do the whole programming job in the first place...

                 

                bye

                TPD

                • 5. Re: How do you build test cases so when a new class is added it automatically gets tested?
                  LosLobo

                  With JUnit/TestNG/Selenium (automated testing tools) you just define a test and they return a success or fail.

                  I'm just trying to determine when a test is created and say added to a package, if I should use java reflection and run all the test cases sitting in said package.

                  So when someone creates a new test all they have to do is put it in the specified path to know it will be on the test stack.

                  I'm sure someone has confronted this so thats what I'm seeking insight for.

                  Thanks!

                  • 6. Re: How do you build test cases so when a new class is added it automatically gets tested?
                    TPD-Opitz

                    Whats wrong with JUnit and/or TestNG? Why do you need your own test framework?

                     

                    And why dont't you look into JUnits or TestNGs source code how they do it (both are open source IIRC...)

                     

                    bye

                    TPD

                    • 7. Re: How do you build test cases so when a new class is added it automatically gets tested?
                      LosLobo

                      I have been the last couple days.  Looks like there is an add on to Jenkins called Selenium Grid that provides much in the way of configuration.

                      I think this is one of those questions that its best to be an expert on Jenkins, Maven, Ant, JUnit, TestNG, and Selenium then you can speak intelligently as to the best approach, i.e. where/how to do the configuration.  Still not sure if the answer lies in one of those tools or within a Java Design pattern but I am learning more.

                      • 8. Re: How do you build test cases so when a new class is added it automatically gets tested?
                        rp0428
                        With JUnit/TestNG/Selenium (automated testing tools) you just define a test and they return a success or fail.

                         

                        Yes - but with JUnit you create a test class and then define test methods within that class. JUnit then reflects the test class you specify and uses the method name to identify test methods. If you want to add additional tests you add methods to the SAME class.

                         

                        You are talking about adding new classes - not adding a new test to an existing test class. If you are adding new classes those might have some methods that need tested and others that don't. That is why you would need some test setup/teardown and define the actual tests.

                        I'm just trying to determine when a test is created and say added to a package, if I should use java reflection and run all the test cases sitting in said package.

                        So when someone creates a new test all they have to do is put it in the specified path to know it will be on the test stack.

                        I'm sure someone has confronted this so thats what I'm seeking insight for.

                        You can certainly 'roll your own' methodology and use reflection to look for appropriately named methods or classes.

                         

                        But why are you so adverse to using configuration files like everyone else has used for years?

                        So when someone creates a new test all they have to do is put it in the specified path to know it will be on the test stack.

                        How is that any easier that what I have been suggesting to you?

                        So when someone creates a new test all they have to do is add it to a configuration file or to a list of configuration files.

                        It makes no sense to me to hard-code the list of test classes/methods that you want to execute for a particular test run.

                         

                        It makes a lot more sense to provide that infomation in one or more configuration files so you have MAXIMUM control over the tests that are run and the test process itself.

                         

                        Using a configuration file allows you to specify, FOR EVERY TEST, what to do if a test fails or something unforeseen occurs. Do you want to continue? Do you want to abort? Do you want to execute an entirely new chain of tests?

                         

                        Configuration files allow you to create 'test chains'; sets of tests for a specific purpose. They also let you then specify dependencies between tests and 'test chains'. For example so you emulate activity for multiple days: execute certain tests on 'test day1' and others on 'test day 2'. Or execute 'test chain 4' if 'test 13 fails.

                         

                        When you hard-code such information you just make the entire process more rigid and unscalable. I don't understand you desire to do that at all.

                         

                        If you entire chain of tests includes 10 tests your test harness should be able to execute ANY one of those tests or execute ANY combination of those tests based on the configuration information you provide. It is far too restrictive to just automatically execute all 10 tests every time you make a test run.

                        • 9. Re: How do you build test cases so when a new class is added it automatically gets tested?
                          jschellSomeoneStoleMyAlias

                          I use maven to build.  I basically just follow the docs and set it up as specified.  Once I do that the test classes go in the test directories.  Once there they are run without me doing anything else.

                          • 10. Re: How do you build test cases so when a new class is added it automatically gets tested?
                            LosLobo

                            What is the testing mechanism?  Do the new Classes follow contain JUnit style test cases?

                            • 11. Re: How do you build test cases so when a new class is added it automatically gets tested?
                              TPD-Opitz

                              maven knows about JUnit by default. Just add a dependency to JUnit to your pom.xml.

                              maven expects Tests in PROJECTROOT/main/test/java and executes all tests That either are subclasses of (JUnit3) TestCase or have methods anontated with @Test.

                               

                              bye

                              TPD