[FIXED] Return confidence score with custom model for Vertex AI batch predictions

Issue

I uploaded a pretrained scikit learn classification model to Vertex AI and ran a batch prediction on 5 samples. It just returned a list of false predictions with no confidence score. I don't see anywhere in the SDK documentation or Google console for how to get batch predictions to include the confidence scores. Is that something Vertex AI can do?

My intent is to automate a batch prediction pipeline using the following code.

# Predict
# "csv", ""bigquery", "tf-record", "tf-record-gzip", or "file-list"
batch_prediction_job = model.batch_predict(
    job_display_name = job_display_name,
    gcs_source = input_path,
    instances_format = "", # jsonl, csv, bigquery, 
    gcs_destination_prefix = output_path,
    starting_replica_count = 1,
    max_replica_count = 10,
    sync = True,
)

batch_prediction_job.wait()

return batch_prediction_job.resource_name

I tried it out in google console as a test to make sure my input data was properly formatted.

Solution

I don't think so; the stock sklearn container provided by vertex doesn't provide such a score I guess. You might need to write a custom container.

Answered By - Shawn