Learning › Transformer › Branching IModels
Edit this page

iModel Transformation and iModel Branching

What is branching

It can be useful in some applications to take iModels and their linear concept of history, and introduce the concept of branching, where an iModel diverges into two separate iModels after a changeset, allowing each to have new distinct changes and a distinct history. Typically, one of them is referred to as the master iModel, and the other as the branch iModel, with the master iModel being the one that existed before the branch was created from it. Importantly, these branches can often be used to make updates to one branch iModel while the master iModel can receive updates from other contexts. Usually the intention is to move the changes in the branch's history back into the master. It can also be useful to update a branch with changes that the master received since their divergence, to keep them synchronized.

Provenance

Provenance is the tracking of where an entity originated from in an iModel. All connectors store provenance as they connect entities from some source format into an iModel. The IModelTransformer will also store provenance as it makes changes to a target; provenance will be used during synchronizations to identify which elements in the branch correspond to those in the master, in order to find conflicts in changes.

The IModelTransformer has several options defined in the IModelTransformOptions type that configure what provenance is stored, and particular configurations are required for performing each branching operation with the transformer.

Provenance is stored for entities by adding an ExternalSourceAspect to them, which contains properties referencing the original repository they came from. The ExternalSourceAspects must reference an element that represents the real-world external source from which their corresponding entity was derived; in the case of transformation where the external source is an external iModel, the iModel is best conventionally represented with an ExternalSource element in a RepositoryLink element.

Additional notes on provenance from the connector application viewpoint can be found here.

TargetScopeElement

The Target Scope Element is the element in the target iModel of a transformation that represents the source repository as a whole, and is therefore used as the scope of ExternalSourceAspects. Typically creating a branch iModel includes inserting a new RepositoryLink and ExternalSource pair representing the master, and then all forward synchronization transformations will use the master-representing ExternalSource as the Target Scope Element by setting the targetScopeElementId option in the IModelTransformOptions option.

It is possible to not specify a targetScopeElementId for a branch transformation, which will default to using the target's root subject as the targetScopeElement. While this works for simple singular branches, it is not recommended because an explicit external source contains more information, and if you decide to branch your branch iModel, you will need a unique target scope element for both synchronization directions, one for the forward synchronizations from master and one for reverse synchronizations from the sub branch.

Provenance, element Ids, and federation guids

When correlating elements between branches, it is important to know how to tell which elements correspond to each other. Element Ids are local to an iModel, and transformations do not preserve them. However, federation guids (globally-unique-identifiers) are preserved. Neither are a substitute for provenance. Only provenance can correctly correlate all elements between branches and must be used for any comparisons between them unless restrictions are used.

Generally, it is not possible to map elements between iModels without provenance, because there is not always a one-to-one relationship of elements between iModels. It is possible, for example, that a transformation involves operations such as "splitting" an existing element into multiple different elements. In that case, federation guids and element Ids cannot be used to correlate the new multiple elements to the original one since they must be unique so it is impossible for the multiple elements to share an Id.

Synchronization (implementing branching)

The process of transferring change history between iModels is called synchronization, and is implemented by the IModelTransformer to support branching concepts.

  • First Synchronization is the initialization of a new iModel as a branch iModel for some existing master iModel. The initialization can filter the master iModel arbitrarily using other transformation techniques.
  • Synchronization, or Forward Synchronization, is the transfer of iModel changes from a master iModel to an existing branch iModel.
  • Reverse Synchronization is the transfer of iModel changes from a branch iModel back to the master iModel from which it was created.

synchronization diagram

In the above diagram you can see an example change history of a master and its branch. Each arrow is an application submitting ChangeSets to the iModel, and each vertical connection (notably an arrow with with two sources) is a transformation reading both iModels and submitting a ChangeSet to one of them, synchronizing them.

Synchronization logic is implemented by the IModelTransformer, which can be configured to perform each type of synchronization, as shown in the samples below.

Squashing changesets

In the above diagram, the synchronizations occur after several ChangeSets, each synchronization here squashes several changesets from the synchronization source into the synchronization target. Alternatively, one could run a synchronization per changeset in the source of the synchronization, so that the synchronization target has a changeset corresponding to each one in the synchronization source, keeping the changeset granularity intact in the history. A combination of both can be done, such as only squashing when reverse synchronizing (squashing branch changes) but keeping changeset data from the master intact during forward synchronizations.

Synchronization conflicts

Conflicts during a transformation are resolved in favor of the element which was modified most recently, as stored in the LastMod property of an element. Elements in transformations are considered in conflict when their code is the same.

You can override the method IModelTransformer.hasElementChanged in your transformer implementation to use more specific logic for determining if an element should be considered changed.

Some other data in the iModel follows more specific rules for conflicts:

  • ECSchemas will not be inserted into the target if the same version already exists in it
  • File properties are not carried over through transformations

Synchronization conflicts are not to be confused with concurrent edit conflicts which are handled by the ConcurrencyControl API.

Known risk: pushes by other users during a synchronization

A synchronization records the changesets it is about to push, so that a later synchronization that reads those changesets skips them instead of processing the synchronization's own changes. The transformer records these changesets before they are pushed, assuming they will be the next changesets in each iModel: the target, and for a reverse synchronization also the branch, which receives provenance updates.

If another user pushes to one of those iModels after the synchronization runs but before its changes are pushed, the synchronization's changeset lands at a later index than recorded. A later synchronization that reads that iModel can then skip the other user's changeset, so its changes are never synchronized, and process the synchronization's own changeset.

Until this is addressed, push a synchronization's changes as soon as it finishes, and avoid other pushes to either iModel in the meantime.

The schema lock reduces this risk. Acquire it with IModelDb.acquireSchemaLock in the target iModel, and for a reverse synchronization also in the branch, before the synchronization runs. Pass retainLocks: true to BriefcaseDb.pushChanges until both iModels are pushed, because a push releases the briefcase's locks by default. While the schema lock is held, other briefcases can't acquire any locks, so they can't make edits that require locks, such as inserting, updating, or deleting elements and their aspects. This isn't a guarantee: an iModel that doesn't use the lock server doesn't enforce locks, and some writes don't require them, such as inserting a CodeSpec.

A custom IModelTransformer.collectChanges receives changeset ranges that already exclude these recorded changesets, so it is subject to the same risk.

Synchronization examples

Creating a branch (First Synchronization)

// Download and open the master iModel. const masterDbProps = await BriefcaseManager.downloadBriefcase({ accessToken: myAccessToken, iTwinId: myITwinId, iModelId: masterIModelId, }); const masterDb = await BriefcaseDb.open({ fileName: masterDbProps.fileName }); // Create the branch from the master's current file as its initial version. // hubAccess is the BackendHubAccess that the application passes to IModelHost.startup. const branchIModelId = await hubAccess.createNewIModel({ accessToken: myAccessToken, iTwinId: myITwinId, iModelName: "my-branch-imodel", version0: masterDb.pathName, // Enable locks when the application requires briefcase lock coordination. noLocks: true, }); // Download and open the branch iModel. const branchDbProps = await BriefcaseManager.downloadBriefcase({ accessToken: myAccessToken, iTwinId: myITwinId, iModelId: branchIModelId, }); const branchDb = await BriefcaseDb.open({ fileName: branchDbProps.fileName }); const branchEditTxn = new EditTxn(branchDb, "initialize branch iModel"); branchEditTxn.start(); let initializeSucceeded = false; try { // Create the repository link and external source that identify the master as the // branch's Target Scope Element. const masterLinkRepoId = branchDb .constructEntity<RepositoryLink, RepositoryLinkProps>({ classFullName: RepositoryLink.classFullName, code: RepositoryLink.createCode( branchDb, IModelDb.repositoryModelId, "example-code-value" ), model: IModelDb.repositoryModelId, url: "https://wherever-you-got-your-imodel.net", format: "iModel", repositoryGuid: masterDb.iModelId, description: "master iModel repository", }) .insert(branchEditTxn); const masterExternalSourceId = branchDb .constructEntity<ExternalSource, ExternalSourceProps>({ classFullName: ExternalSource.classFullName, model: IModelDb.rootSubjectId, code: Code.createEmpty(), repository: new ExternalSourceIsInRepository(masterLinkRepoId), connectorName: "iModel Transformer", connectorVersion: transformerPackageJson.version, }) .insert(branchEditTxn); // Initialize provenance for the branch. const branchInitializer = new IModelTransformer( { source: masterDb, target: branchEditTxn }, { // The branch is a raw copy of the source, so record the provenance required for // future synchronization. wasSourceIModelCopiedToTarget: true, // Store synchronization provenance under the ExternalSource that represents the master. targetScopeElementId: masterExternalSourceId, } ); try { await branchInitializer.process(); } finally { branchInitializer.dispose(); } initializeSucceeded = true; } finally { // Save a successful initialization and abandon a failed one. branchEditTxn.end(initializeSucceeded ? "save" : "abandon"); } // Push the branch changes to the hub. const description = "initialized branch iModel"; await branchDb.pushChanges({ accessToken: myAccessToken, description, });

Update branch with master changes (Forward Synchronization)

const repositoryLinkId = branchDb.elements.queryElementIdByCode( RepositoryLink.createCode( branchDb, IModelDb.repositoryModelId, "example-code-value" ) ); let masterExternalSourceId: Id64String | undefined; if (repositoryLinkId !== undefined) { for await (const row of branchDb.createQueryReader( `SELECT ECInstanceId FROM ${ExternalSource.classFullName} WHERE Repository.Id=:id`, QueryBinder.from({ id: repositoryLinkId }) )) { masterExternalSourceId = row.ECInstanceId; } } // Without this ID, the transformer would use the root subject as the target scope. if (masterExternalSourceId === undefined) throw new Error("The branch has no ExternalSource for the master iModel."); const branchEditTxn = new EditTxn(branchDb, "synchronize master changes"); branchEditTxn.start(); const synchronizer = new IModelTransformer( { source: masterDb, target: branchEditTxn }, { // Read synchronization provenance for the ExternalSource that represents the master. targetScopeElementId: masterExternalSourceId, // Pass an empty argsForProcessChanges object to process the next unsynchronized changes. argsForProcessChanges: {}, } ); let syncSucceeded = false; try { await synchronizer.process(); syncSucceeded = true; } finally { synchronizer.dispose(); // Save a successful synchronization and abandon a failed one. branchEditTxn.end(syncSucceeded ? "save" : "abandon"); } // Push the synchronized branch changes. const description = "updated branch with recent master changes"; await branchDb.pushChanges({ accessToken: myAccessToken, description, });

Update master with branch changes (Reverse Synchronization)

// The master and branch databases are already open (see the initialization example). const repositoryLinkId = branchDb.elements.queryElementIdByCode( RepositoryLink.createCode( branchDb, IModelDb.repositoryModelId, "example-code-value" ) ); let masterExternalSourceId: Id64String | undefined; if (repositoryLinkId !== undefined) { for await (const row of branchDb.createQueryReader( `SELECT ECInstanceId FROM ${ExternalSource.classFullName} WHERE Repository.Id=:id`, QueryBinder.from({ id: repositoryLinkId }) )) { masterExternalSourceId = row.ECInstanceId; } } // Without this ID, the transformer would use the root subject as the target scope. if (masterExternalSourceId === undefined) throw new Error("The branch has no ExternalSource for the master iModel."); const masterEditTxn = new EditTxn(masterDb, "synchronize branch changes"); masterEditTxn.start(); const branchEditTxn = new EditTxn( branchDb, "record synchronization provenance" ); branchEditTxn.start(); const reverseSynchronizer = new IModelTransformer( { source: branchDb, target: masterEditTxn }, { // Identify the master's ExternalSource in the branch source for reverse synchronization. targetScopeElementId: masterExternalSourceId, // Pass an empty argsForProcessChanges object to process the next unsynchronized changes. argsForProcessChanges: {}, // Record updated provenance in the branch source transaction. sourceEditTxn: branchEditTxn, } ); let syncSucceeded = false; try { await reverseSynchronizer.process(); syncSucceeded = true; } finally { reverseSynchronizer.dispose(); // Save or abandon the master changes and branch provenance together. const endMode = syncSucceeded ? "save" : "abandon"; masterEditTxn.end(endMode); branchEditTxn.end(endMode); } // Push the synchronized master changes and branch provenance. const description = "merged changes from branch into master"; await masterDb.pushChanges({ accessToken: myAccessToken, description, }); await branchDb.pushChanges({ accessToken: myAccessToken, description: `${description} provenance`, });

Synchronization workflow examples

More in depth samples exist in the tests for the @itwin/imodel-transformer package.

Last Updated: 29 October, 2026