Keeping track of the deleted records
Our applications replicate the complete data from the source. Some source systems include a "State" attribute to track the asset's health or lifecycle. For example:
Source | Object Type | Attribute | Possible Values |
|---|---|---|---|
Azure AD | Users | Account Enabled | True / False |
Intune | Managed Devices | Device Registration State | notRegistered, registered, revoked, keyConflict, approvalPending, certificateReset, notRegisteredPendingEnrollment, unknown |
Okta | People | Status | ACTIVE / STAGED / DEPROVISIONED |
Datadog | Hosts | Is muted | True / False |
On the other hand, if a resource is deleted from the source, then there is no information coming from the source for that record.
There may be several reasons for a record being deleted from the source. For example:
When a device is wiped and reinstalled, Microsoft Intune deletes the old record and creates a new one for the same laptop.
When a virtual machine is terminated and launched again in a cloud environment, even though it may have the same server name, it would have a new ID assigned by the cloud platform (AWS, or vmware etc…)
We recommend using Assets' features for this purpose.
You can either use the existing attribute or create a new one for this use case. The section below explains the steps to create a new attribute and configure it.
Instructions
Go to Object Schema Configuration.
Select the “Statuses” tab.
Click “Create a status”
Call it “Deleted” and select the red category (inactive).
 |
|---|
Go back to Object Schema.
Select the object type you would like to modify.
Select the Attributes tab.
Add a new attribute for “Source State” with a type of Status.
|
|---|
Go to Object Schema Configuration again.
Select the import tab.
Select “Edit Mapping” for the import configuration.
Select edit object type mapping for the Object Type that you would like to update.
|
|---|
Change the setting for “Missing objects” to “Update”. Select the “Source State” attribute. And write “Deleted” for the new value. Set the Threshold Number to 3.
What does the Threshold parameter mean?
If you set it to 3, this means that if a record is not found three times, it will be marked as “Deleted” during the fourth import.
|
|---|
It is important to note that if you set the “Missing objects” to “Remove”, then Assets will remove the objects if they are not found in the source data. This is risky, and we don’t recommend it for the following reasons:
1- Assets doesn’t have a trashcan/rollback/recover feature. When an object is deleted, you can not recover it. It will be gone, including the references and the linked issues.
2- You will also lose the history of the object. If you have audits, you won’t be able to view an asset's activity history.
Another critical configuration parameter is the threshold number. If you set it to 0, this means Assets will delete the missing objects immediately. This is also not recommended. When the import process doesn’t work properly due to a service outage at the source, or Assets, then you may lose all your objects. We recommend keeping this value at least 3 if you have a daily schedule, or 36 if you have an hourly schedule set. This will give you time to check the import status and stop the data flow if necessary before all data is deleted.
If you really need to delete records, we recommend doing so manually once a month (or at another frequency you prefer).
Another important note is that this configuration of Assets covers only one scenario. It runs when the record is missing. But it doesn’t update the attribute if the record reappears at the source. You need to use the default attribute if this is a possible scenario in your environment. Because the default attributes that are created by our apps would be updated when the data is received by Assets.
From now on, you can follow the imports and check if the records are being marked as Deleted.
Example AQL:
"Source State" = "Deleted"
|
|---|
We recommend testing the new configuration in your sandbox first and confirming that it works as expected.