Vertex AI was announced at Google I/O 2021. More than just a rebranding of the Google AI Platform, this product starts to unify a lot of different APIs (including AutoML) under one product offering. Google states in a press release that this allows companies to start to implement MLOps easier.
In this blog, we are going to do the equivalent of "Hello World" for Data Science using the Vertex AI platform. In short, we are going to use a Vertex AI "Jupyter" Notebook to communicate with the logging service of Google Cloud. Think of a notebook as a way of running Python code in an iterative manner that allows you to capture the results along the way.
TLDR - Show me the code!
If you use a Vertex AI notebook, you can easily test out the python library for Cloud Logging within Google Cloud. The notebook that you need in order to test Cloud Logging can be downloaded here.
Prerequisites
In order to complete the steps in this blog, you need to have the following:
1) You need to have a Google Cloud Account. If you don't already have an account, take a look at this video which shows how to set up an account.
2) You need to enable the Cloud Logging API. After you have set up an account, you can find details about enabling this API at https://console.cloud.google.com/apis/api/logging.googleapis.com.
3) You need to go to https://console.cloud.google.com/vertex-ai/notebooks. Enable the notebooks API if you see a corresponding warning.
Images courtesy of https://www.whiteowleducation.com
Create a Vertex AI notebook
Within the Vertex AI console, the first step that we need to do is we need to create a notebook instance. This instance is this is going to be backed by a CPU, so the key is to run this exercise, and when you're done, be sure to delete the notebook instance so that you don't incur additional fees.
Make sure to delete the notebook instance after you get done using it. This is important to manage costs.
As seen above, I'm creating a notebook called test-logging in us-central1 (and you should create your instance in a location that is close to you). I create this notebook with libraries such as TensorFlow and Pandas that would typically be used in data science, and I do this by selecting the TensorFlow Enterprise 2.5 environment.
If we are just examining logging, I am minimizing CPU to manage costs.
After clicking create, you will see the test-logging notebook appear in the console, and click on "OPEN JUPTYERLAB" in order to continue.
Once you are in JupyterLab, click on Python [conda env:root] in order to open up a notebook for experimentation.
Now go ahead and enter the following Python code into the notebook.
import logging
import google.cloud.logging_v2 as logging_v2
from os import environ
client = logging_v2.client.Client()
google_log_format= logging.Formatter(
fmt='%(name)s | %(module)s | %(funcName)s | %(message)s',
datefmt='%Y-%m-$dT%H:%M:%S')
handler = client.get_default_handler()
handler.setFormatter(google_log_format)
cloud_logger = logging.getLogger("vertex-ai-notebook-logger")
cloud_logger.setLevel("INFO")
cloud_logger.addHandler(handler)
log = logging.getLogger("vertex-ai-notebook-logger")
log.info("This is a log from a Vertex AI Notebook!")
If any of the above code looks unfamiliar, or if you have not used the google-cloud-logging library before, I strongly encourage you to take a look at this video which discusses how to set up the format for logging and how to get a python logger to output information to the cloud.
Verify Results
After running this test code in the notebook, you can head over to the Logs Explorer to see your results.
Finally, remember to go back into the notebook console within Vertex AI to:
1) Select the "instance name" that you created (such as "test-logging")
2) Click the delete icon at the top of the console in order to delete the instance
When successful, you should see something on the console that states " You don't have any notebook instances in this project yet."
Summary
In this blog post, we briefly reviewed Vertex AI notebooks, and we looked at how those notebooks can communicate with the centralized logging in Google Cloud.
It would be great to hear your thoughts about this blog, you can reach me through my company (White Owl Education) which is on Twitter at @whiteowled.
Top comments (0)