Skip to main content

git integration best practices

When you connect Omni to git, you get the power of version control, collaboration, and change tracking, but you may also encounter some quirks and pitfalls. A little planning up front can save you from frustration and unexpected behavior.

This guide walks through best practices for working with Omni’s git integration so you can keep your models (and your sanity) intact.

Edit models in the model IDE​

While models can be edited outside of the model IDE, we don't recommend it. Developing in the model IDE will prevent issues from invalid YAML, such as unexpected commits or, in some cases, even reverts.

We recommend using Omni's IDE to develop the model using Omni YAML. The built-in IDE will makes validating the YAML straightforward, which will allow you to quickly identify syntax issues - something that isn't currently possible in external IDEs today. This can help cut down on issues surrounding syntax on merges, which can cause Omni to reject the change due to invalid syntax.

For example:

  1. A commit containing invalid YAML is merged to the repository's default branch
  2. Omni attempts to perform a schema refresh
  3. Schema refresh fails due to the invalid YAML
  4. Omni authors a commit that resets git to Omni's current state

In this example, the changes that Omni commits to reset git will not only remove the problematic YAML, but it will also overwrite any other changes.

One model per repository​

The best practice for the integration’s current state is to have each model in its own repository. While multiple models can exist in a single repository - also called a monorepo - we don’t recommend it due to the integration’s reliance on webhooks.

Each connection to git - which is currently on a model by model basis - requires setting up a webhook in the git repository. When the connected git repository is updated, a webhook is triggered that notifies Omni of the event.

If a repository contains multiple models, it’s possible for multiple webhooks to fire at once if more than one model is updated at a time. This can result in a race condition and lead to unexpected results in Omni.

Keep Omni & dbt separate in monorepos​

While you can use a monorepo to connect git and dbt to Omni, there are a few things you should keep in mind:

Use gitignore​

In your dbt project, add your Omni files to gitignore to prevent pulling your Omni files into where you edit dbt files. It’s possible that the Omni model YAML could be autolinted, resulting in a number of formatting changes that could cause issues in Omni. For this reason, we strongly recommend editing your models in Omni's model IDE.

Use separate branches for Omni & dbt​

Make Omni model and dbt changes in separate branches. Currently, Omni’s git integration promotes the Omni branch when a pull request is merged. If dbt and Omni changes are merged at the same time, the dbt changes won’t have a chance to build first.

To avoid this, you could use a workflow like the following:

  1. Create a new Omni branch to develop against dbt
  2. Use a dbt virtual environment to develop against your Omni model
  3. Merge the dbt branch
  4. Refresh the Omni schema model
  5. Switch the Omni branch to point to the production dbt environment; verify changes
  6. Promote the Omni model

Use separate deploy keys​

Each connection to the repo will need a separate deploy key. Even though your Omni and dbt models are in the same repository, you should think of them as independent of each other.

Options for protected branches​

Protecting the main branch of a repository from direct commits is a basic best practice in git. If you have branch protection rules enabled, you have a few options for working with Omni’s git integration:

  • Allow deploy keys to bypass branch protection rules. This approach is the most permissive as it allows Omni to directly commit to the default branch. However, other users with access to the repository would still not be allowed to directly push to the default branch.
  • Enable Branch schema refresh & Require system syncs. When these Omni settings are enabled, Omni will open a pull request with schema refresh changes instead of directly committing them to the default branch. These changes would have to be approved to be merged into the repository’s default branch.

Note: Some branch protection rules - for example, branch names cannot contain specific characters - are not enforced by Omni.