1 Reply Latest reply on Nov 1, 2007 2:47 PM by 807603

    Automated Regression Testing

      Apologies for putting a more advanced question in a general forum but there is no Topic for QA/Testing (that I know of).

      Okay, I've been tasked with finding a solution for fully automated unit, component, system, and regression testing. I realize that sounds like a tall order and can't truly be done. However, I'd like to get as close as possible to being able to test every aspect of our system and thought I'd post to the community for ideas. We have a typical multi-tiered web app. We have a web interface, business logic on the server-side, Web Services, DAO, etc. Naturally, some combination of testing needs to be performed. So I suppose that first I need to make a list of all the (general) types of testing that can be done. So far I have:
      - syntax validation
      - link checking
      - unit testing
      - record and playback (robot user for web UI)
      - custom scripting (for anything I can't buy a tool for)
      - performance monitoring & stress testing
      - security testing
      - etc.

      So I guess my first question is: What other types of testing am I missing here? I honestly think I'll end up with 2-3 sets of tests that hopefully give me good testing coverage. The end goal is to have a nightly build that includes a series of tests which generate reports. So in the morning someone can open the reports and see if anything's really busted. So basically, after the build finishes, the tests should be kicked off. I'd like to use Ant for this but I suppose a cron job could be written or whatever. I'm not worried about that right now.

      Anyway, I don't have very specific requirements yet but I'm thinking of breaking my work up into unit, component, and system level testing. My initial thoughts are that I'd need:

      1. Unit Testing - Ant would simply kick off all the JUnit tests we have available. This would do basic class level testing and validation.
      2. Component Testing - Custom Java code/scripts that would tests some specific functional area. This is open ended in my mind as I'm not that familir with the codebase yet. But an example would be "register a user" or "make a reservation".
      3. System Testing - Needs to simuilate a user hitting the web interface and would invoke all tiers. So I'm guessing I need to buy some software product to fill out forms automatically and whatever else.

      Can you all let me know if there's anything major I'm missing? What methodologies or tools do you recommend? Any links on good tech articles regarding this? Anything comments or feedback welcome.

      Thanks in advance!