Using Release Checklists

Documentation and How To

For every model version that will be released, whether to create a new service or replace another version already in production, the release carries some level of risk. To minimize any risk during deployment and speed up a well-governed process, we have created Release Checklists (also known as Governance Checklists) within the Verta platform.
Platform admins can customize the Verta Release Checklist template, create their own templates, and set them up for models. This makes it easier for everyone involved in launching and releasing a model, from data scientists and engineers to governance team members.
Here is a step-by-step guide on how to create a release readiness checklist:
Note: This version of release checklist is only available for customers using our new permissions system.

Step 1: Define the Release Checklist

To create a release readiness checklist, the first step is to define the release criteria. These should include the minimum requirements that must be met before any model can be released. Examples of release criteria include:
  • Conducting a vulnerability scan and ensuring the model has no critical vulnerabilities.
  • Creating model documentation and updating the README.
  • Filing any engineering tickets and recording the JIRA Ticket ID in Verta.
  • Coordinating a release timeline with Product Managers.
  • And so on
Keep in mind that these checklists apply to all model versions, so consider the repeat activities that must be confirmed for each release.

Step 2: Admins, build the Release Checklist into Verta

On the Organization Administration page, you will find a new tab called “Governance”. The ability to edit the default checklist or create your own is available on this page.
To create a new custom checklist, select the “Create Checklist” button to bring up this configuration:
Each checklist must include:
  • A visible name for all organization team members
  • At least one item to check off or include in the checklist
  • For now, all items are checkboxes. Other types of automations and governance rules are on our roadmap. Reach out if you would like early access.
Using the UI, give each checklist item a name that is visible to everyone in your organization.
Optionally, you may also:
  1. 1.
    Provide a description of what must be completed for your organization members to reference.
  2. 2.
    Mark the item as a hard requirement, which will enforce a more stringent progress bar on the checklist for each RMV.
  3. 3.
    Ask your team members to provide evidence, which gives them a text field to record any documents or notes that need to be referenced. For example, a Service Now ticket ID that corresponds to a request.
Create a checklist with 5 to 8 items to make it easy for your team to complete governance tasks. Add new items to the checklist by clicking "Add Checklist Item".

Step 3: Team members complete checklists to release models

You can see your organization's release checklists on the "Release Checklist" widget. To find it, go to a model version and select the "Release" tab. The widget will show how ready you are to release, as a percentage of progress on the checklists.
The way we calculate the progress bar will vary depending on how many checklists and hard requirements you have. But no matter what, if your team checks off everything on the list, it means the model is ready for release.

Step 4: Reviewing and Auditing Release Checklists

At any time, you might have to get ready for an audit or give an account of how closely you're following your model governance processes. The Verta platform will keep track of all the items on your checklist that you've checked off, and it will show you who did it and when in the user interface.
For a more detailed record, platform logs are also available. If you need assistance accessing this information, please contact us.

We want to hear from you

We're happy to let you know that Verta platform now has governance/release checklists. We're also planning to add more features to these checklists like automation, user access controls, setting model risk levels, and creating different checklists based on specific RMV needs.
For a sneak peek of these upcoming features, early access, and a chance to give us feedback, please contact your Verta point of contact.