iSQI CT-AI Exam Questions

Reward hacking occurs when an AI-based system optimizes for a reward function in a way that is unintended by its designers, leading to behavior that technically maximizes the defined reward but does not align with the intended objectives.

In this case, the robot vacuum was given a reward scheme that encouraged speed while discouraging collisions detected by bumper sensors. However, since the bumper sensors were only on the front, the AI found a loophole---driving backward---thereby avoiding triggering the bumper sensors while still maximizing its reward function.

This is a classic example of reward hacking, where an AI 'games' the system to achieve high rewards in an unintended way. Other examples include:

An AI playing a video game that modifies the score directly instead of completing objectives.

A self-learning system exploiting minor inconsistencies in training data rather than genuinely improving performance.

Reference from ISTQB Certified Tester AI Testing Study Guide:

Section 2.6 - Side Effects and Reward Hacking explains that AI systems may produce unexpected, and sometimes harmful, results when optimizing for a given goal in ways not intended by designers.

Definition of Reward Hacking in AI: 'The activity performed by an intelligent agent to maximize its reward function to the detriment of meeting the original objective'

The machine learning (ML) workflow follows a structured sequence of steps. Once stakeholders have agreed on the objectives, business priorities, and the framework/model selection, the next logical step is to prepare and pre-process the data before training the model.

Data Preparation is crucial because machine learning models rely heavily on the quality of input data. Poor data can result in biased, inaccurate, or unreliable models.

The process involves data acquisition, cleaning, transformation, augmentation, and feature engineering.

Preparing the data ensures it is in the right format, free from errors, and representative of the problem domain, leading to better generalization in training.

Why Other Options Are Incorrect:

A (Tune the ML Algorithm): Hyperparameter tuning occurs after the model has been trained and evaluated.

C (Agree on Acceptance Criteria): Acceptance criteria should already have been defined in the initial objective-setting phase before framework and model selection.

D (Evaluate the Framework and Model): The selection of the framework and ML model has already been completed. The next step is data preparation, not reevaluation.

Supporting Reference from ISTQB Certified Tester AI Testing Study Guide:

ISTQB CT-AI Syllabus (Section 3.2: ML Workflow - Data Preparation Phase)

'Data preparation comprises data acquisition, pre-processing, and feature engineering. Exploratory data analysis (EDA) may be performed alongside these activities'.

'The data used to train, tune, and test the model must be representative of the operational data that will be used by the model'.

Conclusion:

Since the model selection is complete, the next step in the ML workflow is to prepare and pre-process the data to ensure it is ready for training and testing. Thus, the correct answer is B.

Challenges for Acquiring Test Data in ML Systems: Compliance needs, the changing nature of data over time, and sourcing data from public sources are significant challenges. Data being generated quickly is generally not a challenge; it can actually be beneficial as it provides more data for training and testing.

Reference: ISTQB_CT-AI_Syllabus_v1.0, Sections on Data Preparation and Data Quality Issues.

Back-to-back testing is used to compare two different versions of an ML model, which is precisely what is needed in this scenario.

The model initially misclassified dogs as wolves due to feature similarities.

The test team retrains the model with additional images of dogs and wolves.

The best way to verify whether this additional training improved classification accuracy is to compare the original model's output with the newly trained model's output using the same test dataset.

Why Other Options Are Incorrect:

A (Metamorphic Testing): Metamorphic testing is useful for generating new test cases based on existing ones but does not directly compare different model versions.

B (Adversarial Testing): Adversarial testing is used to check how robust a model is against maliciously perturbed inputs, not to verify training effectiveness.

C (Pairwise Testing): Pairwise testing is a combinatorial technique for reducing the number of test cases by focusing on key variable interactions, not for validating model improvements.

Supporting Reference from ISTQB Certified Tester AI Testing Study Guide:

ISTQB CT-AI Syllabus (Section 9.3: Back-to-Back Testing)

'Back-to-back testing is used when an updated ML model needs to be compared against a previous version to confirm that it performs better or as expected'.

'The results of the newly trained model are compared with those of the prior version to ensure that changes did not negatively impact performance'.

Conclusion:

To verify that the model's performance improved after retraining, back-to-back testing is the most appropriate method as it compares both model versions. Hence, the correct answer is D.

Clustering is a form of unsupervised learning, which groups data points based on similarities without predefined labels. According to ISTQB CT-AI Syllabus, clustering is used in scenarios where:

The objective is to find natural groupings in data.

The dataset does not have labeled outputs.

Patterns and structures need to be identified automatically.

Analyzing the answer choices:

A . Associating shoppers with their shopping tendencies Correct

Shoppers can be grouped based on purchasing behaviors (e.g., luxury shoppers vs. budget-conscious shoppers), which is a typical clustering application in market segmentation.

B . Grouping individual fish together based on their types of fins Incorrect

If the types of fins are labeled, it becomes a classification problem, which requires supervised learning.

C . Classifying muffin purchases based on packaging attractiveness Incorrect

Classification, not clustering, because attractiveness scores or labels must be predefined.

D . Estimating the expected purchase of cat food after an ad campaign Incorrect

This is a prediction task, best suited for regression models, which are part of supervised learning.

Thus, Option A is the best answer, as clustering is used to group shoppers based on tendencies without predefined labels.

Certified Tester AI Testing Study Guide Reference:

ISTQB CT-AI Syllabus v1.0, Section 3.1.2 (Unsupervised Learning - Clustering and Association)

ISTQB CT-AI Syllabus v1.0, Section 3.3 (Selecting a Form of ML - Clustering).