AI Platform supports hyperparameter tuning for PyTorch models using custom containers. This allows you to use any Python dependencies and libraries that are not included in the pre-built AI Platform Training runtime versions. You can also use a pre-trained model such as ResNet as a base for your custom model. To run a hyperparameter tuning job on AI Platform using custom containers, you need to do the following steps:

Create a Dockerfile that defines the container image for your training application. The Dockerfile should install PyTorch and any other dependencies, copy your training code and configuration files, and set the entrypoint for the container.

Build the container image and push it to Container Registry or another accessible registry.

Create a YAML file that defines the configuration for your hyperparameter tuning job. The YAML file should specify the container image URI, the training input and output paths, the hyperparameters to tune, the metric to optimize, and the tuning algorithm and budget.

Submit the hyperparameter tuning job to AI Platform using the gcloud command-line tool or the AI Platform Training API.

References:

Hyperparameter tuning overview

Using custom containers

PyTorch on AI Platform Training

Sampled Shapley is a fast and scalable approximation of the Shapley value, which is a game-theoretic concept that measures the contribution of each feature to the model prediction. Sampled Shapley is suitable for online prediction requests, as it can return feature attributions with minimal latency. The path count parameter controls the number of samples used to estimate the Shapley value, and a lower value means faster computation. Integrated Gradients is another explanation method that computes the average gradient along the path from a baseline input to the actual input. Integrated Gradients is more accurate than Sampled Shapley, but also more computationally intensive. Therefore, it is not recommended for online prediction requests, especially with a high path count. Prediction drift is the change in the distribution of feature values or labels over time. It can affect the performance and accuracy of the model, and may require retraining or redeploying the model. Vertex AI Model Monitoring allows you to monitor prediction drift on your deployed models and endpoints, and set up alerts and notifications when the drift exceeds a certain threshold. You can specify an email address to receive the notifications, and use the information to retrigger the training pipeline and deploy an updated version of your model. This is the most direct and convenient way to achieve your goal. Training-serving skew is the difference between the data used for training the model and the data used for serving the model. It can also affect the performance and accuracy of the model, and may indicate data quality issues or model staleness. Vertex AI Model Monitoring allows you to monitor training-serving skew on your deployed models and endpoints, and set up alerts and notifications when the skew exceeds a certain threshold. However, this is not relevant to the question, as the question is about the feature attributions of the model, not the data distribution. References:

Vertex AI: Explanation methods

Vertex AI: Configuring explanations

Vertex AI: Monitoring prediction drift

Vertex AI: Monitoring training-serving skew

To organize the artifacts for dozens of pipelines, you should store the parameters in Vertex ML Metadata, store the models’ source code in GitHub, and store the models’ binaries in Cloud Storage. This option has the following advantages:

Vertex ML Metadata is a service that helps you track and manage the metadata of your ML workflows, such as datasets, models, metrics, and parameters1. It can also help you with data lineage, model versioning, and model performance monitoring2.

GitHub is a popular platform for hosting and collaborating on code repositories. It can help you manage the source code of your models, as well as the configuration files, scripts, and notebooks that are part of your ML pipelines3.

Cloud Storage is a scalable and durable object storage service that can store any type of data, including model binaries4. It can also integrate with other services, such as Vertex AI, Cloud Functions, and Cloud Run, to enable easy deployment and serving of your models5.

References:

1: Introduction to Vertex ML Metadata | Vertex AI | Google Cloud

2: Manage metadata for ML workflows | Vertex AI | Google Cloud

3: GitHub - Where the world builds software

4: Cloud Storage | Google Cloud

5: Deploying models | Vertex AI | Google Cloud

 According to the official exam guide1, one of the skills assessed in the exam is to “explain the predictions of a trained model”. TensorFlow2 is an open source framework for developing and deploying machine learning and deep learning models. TensorFlow supports various model explainability methods, such as Integrated Gradients3, which is a technique that assigns an importance score to each input feature by approximating the integral of the gradients along the path from a baseline input to the actual input. Integrated Gradients can help explain the output of a deep learning-based model by highlighting the most influential features in the input images. Therefore, option C is the best way to build the model for the given use case. The other options are not relevant or optimal for this scenario. References:

Professional ML Engineer Exam Guide

TensorFlow

Integrated Gradients

Google Professional Machine Learning Certification Exam 2023

Latest Google Professional Machine Learning Engineer Actual Free Exam Questions

To develop a credit risk model that meets the regulatory requirements and can be monitored over time, you should follow these steps:

Use Vertex Explainable AI with the sampled Shapley method. Vertex Explainable AI is a service that provides feature attributions for machine learning models, which can help you understand how each feature contributes to the prediction1. The sampled Shapley method is a technique that estimates the Shapley values for each feature, which are based on the marginal contribution of each feature to the prediction across all possible feature subsets2. The sampled Shapley method is suitable for neural networks and other complex models, as it can capture the non-linear and interaction effects of the features3.

Enable Vertex AI Model Monitoring to check for feature distribution drift. Vertex AI Model Monitoring is a service that helps you track and manage the performance and quality of your deployed models over time4. Feature distribution drift is a type of data drift that occurs when the distribution of the input features changes significantly from the training data, which can affect the model accuracy and reliability. By checking for feature distribution drift, you can detect when your model needs to be retrained or updated with new data.

References:

1: Introduction to Vertex Explainable AI | Vertex AI | Google Cloud

2: Shapley value - Wikipedia

3: Explainable AI: Interpreting, Explaining and Visualizing Deep Learning

4: Introduction to Vertex AI Model Monitoring | Vertex AI | Google Cloud

[5]: Monitor models for data drift | Vertex AI | Google Cloud

To create a Vertex AI Workbench environment for your team and limit access to other employees in your project, you should follow these steps:

Create a new service account and grant it the Vertex AI User role. This role grants full access to all resources in Vertex AI, including creating and managing notebook instances1.

Grant the Service Account User role to each team member on the service account. This role allows the team members to impersonate the service account and use its permissions2.

Grant the Notebook Viewer role to each team member. This role allows the team members to view and connect to the notebook instance, but not to modify or delete it3.

Provision a Vertex AI Workbench user-managed notebook instance that uses the new service account. This way, the notebook instance will run as the service account and only the team members who have the Service Account User and Notebook Viewer roles will be able to access it.

References:

1: Vertex AI access control with IAM | Google Cloud

2: Understanding service accounts | Cloud IAM Documentation

3: Manage access to a Vertex AI Workbench instance | Google Cloud

[4]: Create and manage Vertex AI Workbench instances | Google Cloud

Vertex AI Model Registry is a central repository where you can manage the lifecycle of your ML models1. You can import models from various sources, such as BigQuery ML, AutoML, or custom models, and assign them to different versions and aliases1. You can also deploy models to endpoints, which are resources that provide a service URL for online prediction2.

By importing the new model to the same Vertex AI Model Registry as a different version of the existing model, you can keep track of the model versions and compare their performance metrics1. You can also use aliases to label the model versions according to their readiness for production, such as default or staging1.

By deploying the new model to the same Vertex AI endpoint as the existing model, you can use traffic splitting to gradually shift the production traffic from the old model to the new model2. Traffic splitting is a feature that allows you to specify the percentage of prediction requests that each deployed model in an endpoint should handle2. This way, you can minimize the impact to the existing and future model users, and monitor the performance of the new model over time2.

The other options are not suitable for your scenario, because they either require creating a separate endpoint or a Cloud Run service, which would increase the complexity and maintenance of your deployment, or they do not allow you to use traffic splitting, which would create a sudden change in your prediction results. References:

Introduction to Vertex AI Model Registry | Google Cloud

Deploy a model to an endpoint | Vertex AI | Google Cloud