Synchronise/clone web applications configuration and avoid ID collision

[Viachaslau]

Dynatrace has property id and key as unique values and what's worse is it's cashed for 30 days, and what's even worse is that Dynatrace doesn't generate random IDs just user sequences 1,2,3, etc for each app.

If you have in the master application config property A and B, but a teammate will create property C in a secondary app, it is impossible to clone A and B because of id collision.

It is nasty but you can't just delete/clone configuration app1 to app2.

At this stage I tried to use constant Ids and as Ops support to clear cache for clone.

But it is ineffective and de-facto banned UI approach for properties creation.

The same situation is indeed is with the meta objects.

Do we already have any elegant solution to resolve collisions between different configurations?

[Ana_Kuzmenchuk]

Hi @Viachaslau, have you maybe managed to find a solution for this on your own? If you did, we'd really appreciate you sharing it with the Community in case anyone else has faced a similar issue.

And if you didn't, let's hope more people see this question now and share their ideas 😊

[Viachaslau · Accepted solution]

Yes, id did workaround with python script:
The main idea use random ids in this case chance of collision if very rare 
And this allows me avoid collision with creation manually changes as well.

I don't know why dynatrace use sequence ids instead of generation random ids - it is really bad front end solution.

Just 1 gap which I can't solve - this is collision between user tag metadata.
This because metadata entity doesn't have unique keys.

So I did the next logic (attached) - this allows me to push app configuration from Source application to Target.
e.g PROD->DEV DEV->PROD without collisions.
The main logic:

# Usecases
# 0. Find tags and copy tags and meta without changes - don't apply changes and (no idea how handle the collisions)
# 1. Same key is not found on target - create new action and meta if needed
# 2. Same key is in target and no meta - update action, and meta if needed
# 3. New action and/or meta - generate random ids

# Objects which can generate meta: placeholders, tags, properties

[Viachaslau]

            for _source_prop in _source_web_app_config.get('userActionAndSessionProperties'):
                """Find source property and related meta in source"""
                _new_prop = _source_prop
                _new_meta = get_meta_prop_by_id(_source_web_app_config, metadata_id=_source_prop.get('metadataId'))
                _is_prop_found_in_target = False
                """Update meta or create new- Find source property and related meta in target if present"""
                for _target_prop in _target_web_app_config.get('userActionAndSessionProperties'):
                    if is_prop_key_matched_to_target(source_prop=_source_prop, target_prop=_target_prop):
                        _is_prop_found_in_target = True
                        """Replace id from target property to avoid errors"""
                        _new_prop = copy_target_id(prop=_target_prop, new_prop=_new_prop)
                        if _source_prop.get('metadataId'):
                            if is_source_prop_and_target_has_meta(_source_prop, _target_prop):
                                """Replace meta id if meta is not None"""
                                _new_prop['metadataId'] = _target_prop.get('metadataId')
                                _new_meta['uniqueId'] = _target_prop.get('metadataId')
                            else:
                                _new_meta_id = random.randint(10000, 2147483647)
                                _new_prop['metadataId'] = _new_meta_id
                                _new_meta['uniqueId'] = _new_meta_id
                if not _is_prop_found_in_target:
                    """Generate new ids for new properties in target app"""
                    _new_prop['uniqueId'] = random.randint(10000, 2147483647)
                    """Generate random meta id if meta is not None"""
                    if _new_prop.get('metadataId'):
                        _new_meta_id = random.randint(10000, 2147483647)
                        _new_prop['metadataId'] = _new_meta_id
                        _new_meta['uniqueId'] = _new_meta_id

                if _new_prop:
                    new_properties.append(_new_prop)
                if _new_meta:
                    new_metas.append(_new_meta)

            print(f'new_properties \n{new_properties}')
            print(f'new_metas \n{new_metas}')
            pprint(new_metas)

            """Copy tags and metas from source to target"""
            """We can't use random approach, and this mean it should be configured with the same IDs across all apps to avoid collisions"""
            new_tags = _source_web_app_config['userTags']
            for _tag in new_tags:
                for _meta in _source_web_app_config['metaDataCaptureSettings']:
                    if _tag['metadataId'] == _meta['uniqueId']:
                        new_metas.append(_meta)
            print(f'new_tags \n{new_tags}')

            """Copy placeholders and metas from source to target"""
            new_placeholders = _source_web_app_config.get('userActionNamingSettings').get('placeholders')
            for _placeholder in new_placeholders:
                for _meta in _source_web_app_config['metaDataCaptureSettings']:
                    if _placeholder['metadataId'] == _meta['uniqueId']:
                        new_metas.append(_meta)
            print(f'new_placeholders \n{new_placeholders}')

            """Record new values into target application config"""
            new_config['userTags'] = new_tags
            new_config['userActionNamingSettings']['placeholders'] = new_placeholders
            new_config['userActionAndSessionProperties'] = new_properties
            new_config['metaDataCaptureSettings'] = new_metas

            app_config_enriched_file_name = f'WEB_APP_CONFIG__{_target_web_app_config["name"]}__{app_id}__enriched.json'
            save_json(file_name_or_path=_temp_path(file_name=app_config_enriched_file_name),
                      data=_target_web_app_config)

offset 1000 was created to avoid collision with dynatrace OOB sequence IDs