Amazon Personalize – Evaluating recommendation effectiveness

AWS ML Business Metrics

The correct answer is C – Amazon Kendra, AWS’s enterprise search service designed for organizations with large, diverse document repositories. Kendra uses machine learning and natural language understanding (NLU) to provide semantic search, meaning it retrieves results based on meaning rather than keyword matching. According to AWS documentation, Kendra is ideal for educational, enterprise, and knowledge-management scenarios where users need fast, accurate retrieval across PDFs, HTML, Office documents, FAQs, and multimedia transcripts. Kendra connectors can index content from S3, SharePoint, LMS platforms, and internal databases, making it perfect for large volumes of training materials. Amazon Comprehend (A) is for NLP tasks like entity extraction, not enterprise search. Amazon Textract (B) extracts text from PDFs and scanned materials but does not provide search capabilities. Amazon Personalize (D) is for personalized recommendations, not document retrieval. Kendra is purpose-built for enterprise search and aligns directly with the company ' s needs.

Referenced AWS Documentation:

Amazon Kendra Developer Guide – Enterprise Search

AWS ML Specialty Guide – Intelligent Search Systems

Comprehensive and Detailed Explanation From Exact AWS AI documents:

Data residency refers to requirements that dictate where data is physically stored and processed, often to meet regulatory, legal, or organizational policies.

AWS governance guidance emphasizes that:

PII data may be legally required to remain within a specific geographic region

Customers can control data location by selecting AWS Regions

Data residency is a core component of data governance and compliance

Why the other options are incorrect:

Data mining (A) refers to extracting insights from data.

Pre-training bias (C) relates to model training data, not storage location.

Geolocation routing (D) concerns request routing, not data governance.

AWS AI document references:

AWS Data Governance and Residency

Responsible AI and Data Management on AWS

Security and Compliance in AWS Regions

To build an AI model responsibly and minimize bias, it is essential to ensure fairness and transparency throughout the model development and deployment process. This involves detecting and mitigating data imbalances and thoroughly evaluating the model ' s behavior to understand its impact on different groups.

Option A (Correct): " Detect imbalances or disparities in the data " : This is correct because identifying and addressing data imbalances or disparities is a critical step in reducing bias. AWS provides tools like Amazon SageMaker Clarify to detect bias during data preprocessing and model training.

Option C (Correct): " Evaluate the model ' s behavior so that the company can provide transparency to stakeholders " : This is correct because evaluating the model ' s behavior for fairness and accuracy is key to ensuring that stakeholders understand how the model makes decisions. Transparency is a crucial aspect of responsible AI.

Option B: " Ensure that the model runs frequently " is incorrect because the frequency of model runs does not address bias.

Option D: " Use the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) technique to ensure that the model is 100% accurate " is incorrect because ROUGE is a metric for evaluating the quality of text summarization models, not for minimizing bias.

Option E: " Ensure that the model ' s inference time is within the accepted limits " is incorrect as it relates to performance, not bias reduction.

AWS AI Practitioner References:

Amazon SageMaker Clarify: AWS offers tools such as SageMaker Clarify for detecting bias in datasets and models, and for understanding model behavior to ensure fairness and transparency.

Responsible AI Practices: AWS promotes responsible AI by advocating for fairness, transparency, and inclusivity in model development and deployment.

Comprehensive and Detailed Explanation From Exact AWS AI documents:

Amazon SageMaker Model Monitor detects:

Data drift

Concept drift

Model quality degradation

AWS MLOps guidance recommends Model Monitor to:

Continuously evaluate production models

Trigger alerts when performance deviates from baselines

Maintain long-term model reliability

Why the other options are incorrect:

Amazon A2I (A) adds human review workflows.

Rekognition (C) is an image analysis service.

Trusted Advisor (D) provides cost and security recommendations.

AWS AI document references:

Amazon SageMaker Model Monitor Overview

Detecting Model Drift on AWS

Production ML Monitoring Best Practices

In generative AI models, such as those built on Amazon Bedrock, inference costs are driven by the number of tokens processed. A token can be as short as one character or as long as one word, and the more tokens consumed during the inference process, the higher the cost.

Option A (Correct): " Number of tokens consumed " : This is the correct answer because the inference cost is directly related to the number of tokens processed by the model.

Option B: " Temperature value " is incorrect as it affects the randomness of the model ' s output but not the cost directly.

Option C: " Amount of data used to train the LLM " is incorrect because training data size affects training costs, not inference costs.

Option D: " Total training time " is incorrect because it relates to the cost of training the model, not the cost of inference.

AWS AI Practitioner References:

Understanding Inference Costs on AWS: AWS documentation highlights that inference costs for generative models are largely based on the number of tokens processed.

Amazon SageMaker JumpStart is the correct service for quickly deploying and consuming a foundation model (FM) within a team ' s VPC.

Amazon SageMaker JumpStart:

Provides access to a wide range of pre-trained models and solutions that can be easily deployed and consumed within a VPC.

Designed to simplify and accelerate the deployment of machine learning models, including foundation models.

Why Option B is Correct:

Rapid Deployment: JumpStart allows for quick deployment of models with minimal configuration, directly within a secure VPC environment.

Ease of Use: Provides a user-friendly interface to select and deploy models, reducing the time to value.

Why Other Options are Incorrect:

A. Amazon Personalize: Focuses on creating personalized recommendations, not deploying foundation models.

C. PartyRock: Not a recognized AWS service.

D. Amazon SageMaker endpoints: Endpoints are for deploying specific models, not a feature for quickly starting with pre-trained foundation models.

Chain-of-thought prompting is a prompt engineering technique where you instruct the model to explain its reasoning step by step, which is particularly useful for tasks involving logic, math, or reasoning.

A is correct: Asking the model to " explain its reasoning step by step " directly invokes chain-of-thought prompting, as documented in AWS and generative AI literature.

B is unrelated (prompt injection is a security concern).

C (few-shot) provides examples, but doesn’t specifically require step-by-step reasoning.

D (templating) is about structuring the prompt format.

" Chain-of-thought prompting elicits step-by-step explanations from LLMs, which improves performance on complex reasoning tasks. "

(Reference: Amazon Bedrock Prompt Engineering Guide, AWS Certified AI Practitioner Study Guide)

Comprehensive and Detailed Explanation From Exact Extract:

In a RAG (Retrieval Augmented Generation) architecture, there are steps that can be optimized using offline batch processing, particularly for operations that do not require real-time updates:

A. Generation of content embeddings:When new content is published, it can be processed in batches to generate embeddings (vector representations) offline. These embeddings are then used at query time for similarity search. As new documents come in daily, batch processing is ideal for generating embeddings for all new content together.

“Content/document embeddings are typically generated offline, as this operation can be computationally expensive and does not need to happen in real-time.”

(Reference: AWS GenAI RAG Blog, Amazon Bedrock RAG Pattern)

C. Creation of the search index:After generating the content embeddings, these are indexed in a vector database or search service. This indexing is also typically performed in batch as part of the offline pipeline.

“Building or updating the vector index is often performed as a batch operation, reflecting the latest state of the content repository.”

(Reference: AWS RAG Pattern Whitepaper)

B, D, and E are real-time steps. Embeddings for user queries (B), retrieval of relevant content (D), and response generation (E) must be processed in real-time to provide an interactive experience.

The company needs to address the degradation in model inference quality after 4 months in production and prevent future occurrences by receiving notifications. Retraining the model can address the current degradation, likely caused by data drift (changes in the data distribution over time). Amazon SageMaker Model Monitor is designed to detect and monitor model drift, alerting the company when inference quality degrades, thus meeting both requirements.

Exact Extract from AWS AI Documents:

From the Amazon SageMaker Developer Guide:

" Amazon SageMaker Model Monitor enables you to monitor machine learning models in production for data drift, model performance degradation, and other quality issues. It can detect drift in feature distributions and inference quality, sending notifications when deviations are detected, allowing you to take corrective actions such as retraining the model. "

(Source: Amazon SageMaker Developer Guide, Monitoring Models with SageMaker Model Monitor)

Detailed Explanation:

Option A: Retrain the model. Monitor model drift by using Amazon SageMaker Clarify.SageMaker Clarify is used for bias detection and explainability, not for monitoring model drift or inference quality in production. This option does not fully meet the requirements.

Option B: Retrain the model. Monitor model drift by using Amazon SageMaker Model Monitor.This is the correct answer. Retraining addresses the current degradation, and SageMaker Model Monitor can detect future drift in inference quality, sending notifications to prevent recurrence, as required.

Option C: Build a new model. Monitor model drift by using Amazon SageMaker Feature Store.SageMaker Feature Store is for managing and sharing features, not for monitoring model drift or inference quality. Building a new model may not be necessary if retraining can address the issue.

Option D: Build a new model. Monitor model drift by using Amazon SageMaker JumpStart.SageMaker JumpStart provides pre-trained models and solutions for quick deployment, but it does not offer specific tools for monitoring model drift or inference quality in production.