Week 8 - Answers to the Weekly Exercises

Question 1 Answers:

Considering the three months leading up to the product release in March and assuming one more maintenance release in September, the features to put through smoke tests can be determined based on their criticality to the core functionality of the payroll system and their impact on end users. Here's how I would categorize them:

  1. Screens for entering employee information:
    - This feature is fundamental to the payroll system as it involves capturing essential employee data necessary for payroll processing. Therefore, it should be included in smoke testing to verify that employee information can be entered and stored correctly.
  2. Calculating the pay slip details for a month:
    - Payroll calculation is one of the core functionalities of the system. It directly impacts employees' pay and compliance with labor laws. Thus, this feature should definitely be included in smoke testing to validate its functionality.
  3. Screens for maintaining the address of the employee:
    - While maintaining employee addresses is important for administrative purposes and communication, it might not be as critical to the core functionality of payroll processing compared to entering employee information and calculating pay slips. However, if accurate addresses are necessary for payroll-related activities (such as tax calculations or compliance), then testing these screens should be included in smoke testing.
  4. Statutory year-end reports:
    - Year-end reports are crucial for compliance and financial reporting purposes. However, they are not directly involved in the day-to-day payroll processing. While testing these reports is important, they may not need to be part of the initial smoke testing phase. However, they should be thoroughly tested before the product release in March and before the subsequent maintenance release in September to ensure compliance and accuracy.

Question 2 Answers:

Running dormant or rarely run test cases as part of regression testing can have both advantages and disadvantages:

Pros:

  1. Comprehensive Test Coverage: Including dormant test cases in regression testing helps achieve more comprehensive test coverage. It ensures that all parts of the software, including features that are rarely tested, are examined for potential issues or regressions.
  2. Bug Detection: Dormant test cases may uncover bugs or issues that have not been previously identified. These tests might explore edge cases or scenarios that are not typically encountered during regular testing but could still lead to critical issues if left unaddressed.
  3. Risk Mitigation: By running dormant test cases, you reduce the risk of releasing a product with hidden defects or regressions. It helps maintain the overall quality and reliability of the software by identifying and addressing issues before they impact users.
  4. Validation of Stability: Including dormant tests in regression helps validate the stability of the system across various conditions and scenarios. It ensures that even features or functionalities that are rarely used still perform as expected after changes or updates to the software.
Cons:
  1. Time and Resource Constraints: One of the primary reasons why test cases remain dormant is due to time and resource constraints. Running all dormant tests as part of regression testing can significantly increase the time and effort required for testing, potentially delaying the release cycle.
  2. Maintenance Overhead: Dormant test cases may become outdated over time, especially if the software undergoes frequent updates or changes. Maintaining and updating these tests to reflect current system functionality can be challenging and time-consuming.
  3. False Positives: Dormant test cases may produce false positives or irrelevant results, leading to wasted time and effort in investigating issues that do not impact the software's functionality or quality.
  4. Complexity in Prioritization: Determining which dormant test cases to include in regression testing can be complex. Not all dormant tests may be equally relevant or valuable, making it difficult to prioritize them effectively within the limited testing timeframe.

Question 3 Answers:

In summary, the test cases for inclusion in the regression cycle for Version 7 should encompass:

Question 4 Answers:

Based on the suggestions for prioritizing test cases, here's how I would categorize the provided test cases:

Let's prioritize the test cases accordingly:
  1. A test case for a network product that tests basic flow control and error control.
    Priority: P0
    This test case is critical for ensuring the fundamental functionality and reliability of the network product.
  2. A test case for a database software that tests all the options of a join query.
    Priority: P1
    Join queries are essential operations in a database software, and testing all options ensures data integrity and query accuracy.
  3. A test case for a file system that checks allocation of space that is not contiguous.
    Priority: P1
    This test case is important for validating the file system's ability to handle non-contiguous space allocation, which can impact file storage and retrieval.
  4. A test case that tests the startup of OS with normal parameters.
    Priority: P0
    Testing the startup of the operating system with normal parameters is critical for ensuring system stability and functionality.
  5. A test case that tests the startup of an application in a "safe mode," something that is not done very often.
    Priority: P2
    While testing the startup in safe mode is important for system recovery, it may not be as critical as other test cases for regular system operation.
  6. A test case corresponding to a feature that has undergone significant change in the current version.
    Priority: P0
    Testing features that have undergone significant changes is critical to ensure that the changes have not introduced regressions or new defects.
  7. A test case corresponding to a stable feature that has not uncovered any major defects.
    Priority: P1
    While stable features are important, they may not require immediate attention compared to features with significant changes.
In summary, the prioritization would be:
P0: 1, 4, 6
P1: 2, 3, 7
P2: 5

Question 5 Answers:

Here are the pros and cons of some common regression testing methodologies:

  1. Regress All:
    Pros:
    Comprehensive Coverage: Testing all functionalities ensures that no regression issues are missed.
    Thorough Validation: Provides confidence that the entire system works as expected after changes.
    Minimal Risk of Missing Defects: Leaves minimal room for missing critical defects.

    Cons:
    Time-Consuming: Testing the entire application can be time-consuming and resource-intensive, especially for large systems.
    High Cost: Requires significant resources in terms of time, effort, and possibly infrastructure.
    May Not Be Feasible: In situations with severe time constraints, testing everything may not be feasible.

  2. Regress Changes:
    Pros:
    Focus on Impact Areas: Prioritizes testing around the areas that have undergone changes, reducing the scope of testing.
    Efficient Resource Utilization: Optimizes testing efforts by concentrating on the most critical parts of the application.
    Suitable for Time-Critical Situations: Ideal for situations where time is limited and a quick turnaround is required.

    Cons:
    Risk of Missing Defects: May overlook regressions in areas that were not identified as changed but are impacted indirectly.
    Limited Coverage: Does not provide as comprehensive coverage as testing all functionalities.
    Dependent on Change Identification: Requires accurate identification of all changes, which may not always be feasible or complete.

  3. Regress Priority Features:
    Pros:
    Focus on Key Functionality: Prioritizes testing based on the criticality of features to the application's functionality.
    Risk Mitigation: Ensures that the most critical features are thoroughly tested, reducing the risk of releasing critical defects.
    Efficient Use of Resources: Optimizes testing efforts by focusing on areas that are most likely to impact users or business operations.


    Cons:
    Potential Oversight: May overlook regressions in less critical features or areas that are not considered high priority.
    Incomplete Validation: Does not guarantee that all parts of the application are thoroughly tested, which may lead to undetected defects.
    Subjectivity in Priority Assignment: Requires subjective judgment to prioritize features, which may vary across stakeholders.

  4. Regression Test Suite:
    Pros:
    Reusable Test Assets: Utilizes pre-existing regression test suites, saving time and effort in test case creation.
    Consistent Validation: Ensures consistent testing across multiple regression runs, promoting reliability and repeatability.
    Customizable Coverage: Allows for the selection and prioritization of test cases based on specific requirements and constraints.

    Cons:
    Maintenance Overhead: Requires regular updates and maintenance of the regression test suite to reflect changes in the application.
    Risk of Incomplete Coverage: May not cover all areas of the application, especially if the test suite is not regularly reviewed and updated.
    Limited Flexibility: May not adapt well to rapid changes in the application or testing requirements.

Question 6 Answers:

Let's analyze the presented results in terms of the efficacy of the choice of regression tests and the development process:

  1. Tests for Print Facility:
    The fact that the tests for the print facility produced no defects in the previous version and have continued to produce no defects in the current version is a positive sign. It indicates that the functionality related to the print facility has remained stable across versions.
    Efficacy: The choice of regression tests for the print facility seems to be effective. However, it's essential to ensure that the tests cover a comprehensive range of scenarios to validate the functionality thoroughly.
  2. Tests for Multi-Currency:
    The presence of defects in tests for multi-currency that continue from the previous version to the current version suggests that there are underlying issues with this feature that haven't been adequately addressed.
    Efficacy: The regression tests for multi-currency have been effective in identifying ongoing defects, highlighting the importance of continued focus on testing and improving this feature.
  3. Feature Expedited Delivery:
    The emergence of new defects in the current version for the feature expedited delivery indicates that changes or updates made to this feature may have introduced unintended issues.
    Efficacy: While it's concerning to have new defects in a feature that previously worked well, the fact that these defects were identified through regression testing underscores the importance of thorough testing throughout the development process.