Skip to main content

Smart Data Audit

Description​

The Smart Data Audit API can be used to get detailed information about the data returned as part of the data inquiry response for a submission. The audit data can be retrieved for a data package or at a more granular level, i.e., for a single data point. The audit includes references from the different data sources for the data point in question.

Prerequisites​

  • The file is submitted for processing via the Universal Submit API.
  • Once the transaction reaches the Review_required status, the data is inquired for that transaction via the Data Inquiry API.
  • To retrieve the audit, a Data Package ID, or a Data Element ID / Data Element Key for the data element, is required.
    • The Data Element ID (Data Point ID) is defined as part of the data package shared with you.
    • The Data Element Key can be obtained from the Data Inquiry response.

Authentication​

Step 1: Obtain a Bearer Token

The Smart Data Audit API uses the Insurance Intelligence Authenticate API to obtain a bearer token using your client_id, client_secret, and API key. The bearer token obtained from the Authentication API can be used to authenticate requests to the Smart Data Audit API.

Your Bold Penguin project team will provide you with a unique client_id and client_secret for each environment as well as an API key and Data Package ID (dp_id).

Step 2: Access the Smart Data Audit API

Data Package Audit​

Get the audit for all the data points in a data package.

Endpoint​

Staging​

POST https://plb.di-beta.boldpenguin.com/universal-audit/v2/datapackage

Production​

POST https://plb.ii.boldpenguin.com/universal-audit/v2/datapackage

Pass the following variables as part of the request

Request ParametersComments
Authentication TokenThe token obtained via the Authentication API
API Key(Provided by Bold Penguin)
data_package_id(Provided by Bold Penguin)
doc_reference_idThe tx_id returned as part of the Universal Submit response
tenant_id(Provided by Bold Penguin)

The response from the audit will contain the audit_data list, which has the audit details for every data point in the data package. The following are the audit attributes for every data point.

Response ParametersComments
data_element_idUnique ID for the data element
data_element_nameThe business name for the data element
destination_pathJSON path of the element in the data inquiry response
value_typeThe type of the value (for example, STRING)
final_scoreFinal triangulated confidence score (BP score)
final_valueFinal triangulated value
data_sources.sourceThe source name
data_sources.source_idIdentifier of the source configuration for the data point
data_sources.valueValue retrieved from the source
data_sources.value_typeValue type retrieved from the source
data_sources.trust_scoreTrust score for the source
data_sources.confidence_scoreExtraction confidence score

When the data is extracted from multiple documents of the same classified document type, the dependency node further refers to every sub document type.

Response ParametersComments
data_sources.dependency.sourceSource name.
Examples: ACORD_DATA, APP_FORMS_DATA, SMART_FORMS_DATA, POLICY_FORMS_DATA, SOV_DATA, PDF_SOV_DATA, LOSS_RUN_DATA, DNB, CORTERA
data_sources.dependency.source_idIdentifier of the source configuration for the data point
data_sources.dependency.sub_doc_typeSub document type.
Examples: ACORD 125, ACORD 130, EML, MKTP
data_sources.dependency.trust_scoreTrust score for the source
data_sources.dependency.valueValue retrieved from the source
data_sources.dependency.value_typeValue type retrieved from the source
data_sources.dependency.confidence_scoreExtraction confidence score
data_sources.dependency.meta_data.key_lookupThe key to uniquely identify an item in the value, depending on the type of value:
string: value
list: items in the list
object: keys in the object
list of objects: keys in the objects inside an array
list of lists: keys in the object inside an array
data_sources.dependency.meta_data.classified_file_idReference to the classified document from which the data was extracted
data_sources.dependency.meta_data.file_idReference to the file from which the data was extracted
data_sources.dependency.meta_data.file_nameReference to the name of the file from which the data was extracted
data_sources.dependency.meta_data.page_numReference to the page number from which the data was extracted
data_sources.dependency.meta_data.coordinatesReference to the coordinates from which the data was extracted (x1, y1 and x2, y2)

Sample Smart Data Audit API Request​

Request audit data using the Data Package ID

POST /universal-audit/v2/datapackage HTTP/1.1
Host: plb.di-beta.boldpenguin.com
x-api-key: xxxxxxxxxxx
Content-Type: application/json
Authorization: Bearer xxxxxxxxxx
Content-Length: 139
{
"doc_reference_id": "236fb681-c533-4b6c-ba65-05fafbbebcdb",
"data_package_id": "bookroll-us-c0001",
"tenant_id": "kowalski"
}

Sample Smart Data Audit Response​

The response below is truncated to show the company_name and website data points.

{
"doc_reference_id": "236fb681-c533-4b6c-ba65-05fafbbebcdb",
"tenant_id": "kowalski",
"data":
{
"data_package_id": "bookroll-us-c0001",
"audit_data":
[
[
{
"data_element_id": "S-C000003",
"data_element_name": "Company Name",
"destination_path": "facts.company_name",
"value_type": "STRING",
"final_value": "Tesco Express",
"final_score": "93.27",
"data_sources":
[
{
"source": "SMD_SEED_DATA",
"source_id": "smd_company_name",
"trust_score": "7",
"value_type": "STRING",
"value": "Tesco Express"
},
{
"source": "CLEARBIT",
"source_id": "clearbit_company_name",
"trust_score": "6",
"value_type": "STRING",
"value": "Tesco PLC"
},
{
"source": "DNB",
"source_id": "dnb_company_name",
"trust_score": "10",
"value_type": "STRING",
"value": "TESCO STORES LIMITED"
},
{
"source": "BP_JACKIQ",
"source_id": "bp_jackiq_company_name",
"trust_score": "5",
"value_type": "STRING",
"value": "Tesco Express"
},
{
"source": "EML",
"source_id": "depn_sfd_company_name",
"trust_score": "7",
"value_type": "STRING",
"value": "TESCO",
"confidence_score": "96.50",
"dependency":
[
{
"source": "SMART_FORMS_DATA",
"source_id": "sfd_company_name_llm",
"sub_doc_type": "EML(AI)",
"trust_score": "10",
"value_type": "STRING",
"value": "TESCO",
"confidence_score": 96.5,
"meta_data":
[
{
"key_lookup": "TESCO",
"classified_file_id": "7a26a2a0-2bed-4747-8ad1-2b1127e13dc2",
"file_id": "63b26594-a176-4083-80e0-62c1e2141bbe",
"page_num": "1",
"file_name": "EML + MKPT + SOV.eml",
"coordinates":
{
"x1": 0.2396078431372549,
"y1": 0.03727272727272727,
"x2": 0.28823529411764703,
"y2": 0.04818181818181818
}
}
]
},
{
"source": "SMART_FORMS_DATA",
"source_id": "sfd_company_name_llm",
"sub_doc_type": "EML(AI)",
"trust_score": "10",
"value_type": "STRING",
"value": "TESCO",
"confidence_score": 96.5,
"meta_data":
[
{
"key_lookup": "TESCO",
"classified_file_id": "7a26a2a0-2bed-4747-8ad1-2b1127e13dc2",
"file_id": "63b26594-a176-4083-80e0-62c1e2141bbe",
"page_num": "1",
"file_name": "EML + MKPT + SOV.eml",
"coordinates":
{
"x1": 0.2396078431372549,
"y1": 0.03727272727272727,
"x2": 0.28823529411764703,
"y2": 0.04818181818181818
}
}
]
}
]
},
{
"source": "MKTP",
"source_id": "depn_mkpt_company_name",
"trust_score": "6",
"value_type": "STRING",
"value": "TESCO PLC",
"confidence_score": "96.00",
"dependency":
[
{
"source": "MKPT_DATA",
"source_id": "mkpt_company_name_llm",
"sub_doc_type": "MKTP(AI)",
"trust_score": "10",
"value_type": "STRING",
"value": "TESCO PLC",
"confidence_score": 96.0,
"meta_data":
[
{
"key_lookup": "TESCO PLC",
"classified_file_id": "a07ccecb-14fe-436e-b5d9-f02f1093b41f",
"file_id": "752fcb90-50e0-48da-9af4-37ef4e52dd0e",
"page_num": "1",
"file_name": "Tesco.pdf",
"coordinates":
{
"x1": 0.14862745098039215,
"y1": 0.8296969696969697,
"x2": 0.37333333333333335,
"y2": 0.8578787878787879
}
}
]
}
]
}
]
},
{
"data_element_id": "S-C001044",
"data_element_name": "Broker Email",
"destination_path": "options.broker_email",
"value_type": "STRING",
"final_value": "laura.razo@geninsbrokers.com",
"final_score": "79.11",
"data_sources":
[
{
"source": "EML",
"source_id": "depn_sfd_broker_email",
"trust_score": "9",
"value_type": "STRING",
"value": "laura.razo@geninsbrokers.com",
"confidence_score": "96.50",
"dependency":
[
{
"source": "SMART_FORMS_DATA",
"source_id": "eml_broker_email_llm",
"sub_doc_type": "EML(AI)",
"trust_score": "10",
"value_type": "STRING",
"value": "laura.razo@geninsbrokers.com",
"confidence_score": 96.5,
"meta_data":
[
{
"key_lookup": "laura.razo@geninsbrokers.com",
"classified_file_id": "7a26a2a0-2bed-4747-8ad1-2b1127e13dc2",
"file_id": "63b26594-a176-4083-80e0-62c1e2141bbe",
"page_num": "1",
"file_name": "EML + MKPT + SOV.eml",
"coordinates":
{
"x1": 0.16549019607843138,
"y1": 0.22272727272727272,
"x2": 0.41411764705882353,
"y2": 0.23363636363636364
}
}
]
}
]
}
]
},
{
"data_element_id": "S-C001043",
"data_element_name": "Broker State",
"destination_path": "options.broker_state",
"value_type": "STRING",
"final_value": "",
"final_score": "",
"data_sources":
[]
},
{
"data_element_id": "S-C000011",
"data_element_name": "State",
"destination_path": "facts.state",
"value_type": "STRING",
"final_value": "ENG",
"final_score": "93.61",
"data_sources":
[
{
"source": "DNB",
"source_id": "dnb_address",
"trust_score": "8",
"value_type": "STRING",
"value": "ENG"
},
{
"source": "SMD_SEED_DATA",
"source_id": "sd_address",
"trust_score": "7",
"value_type": "STRING",
"value": "Greater Manchester"
}
]
},
{
"data_element_id": "S-C000010",
"data_element_name": "City",
"destination_path": "facts.city",
"value_type": "STRING",
"final_value": "MANCHESTER",
"final_score": "93.61",
"data_sources":
[
{
"source": "DNB",
"source_id": "dnb_address",
"trust_score": "8",
"value_type": "STRING",
"value": "MANCHESTER"
},
{
"source": "SMD_SEED_DATA",
"source_id": "sd_address",
"trust_score": "7",
"value_type": "STRING",
"value": "Manchester"
},
{
"source": "CLEARBIT",
"source_id": "clearbit_address",
"trust_score": "6",
"value_type": "STRING",
"value": "Welwyn Garden City"
},
{
"source": "BP_JACKIQ",
"source_id": "bp_jackiq_address",
"trust_score": "7",
"value_type": "STRING",
"value": "Manchester"
},
{
"source": "EML",
"source_id": "depn_sfd_address",
"trust_score": "7",
"value_type": "STRING",
"value": "Manchester",
"confidence_score": "93.70",
"dependency":
[
{
"source": "SMART_FORMS_DATA",
"source_id": "sfd_company_address_llm",
"sub_doc_type": "EML(AI)",
"trust_score": "10",
"value_type": "STRING",
"value": "Manchester",
"confidence_score": 97.2,
"meta_data":
[
{
"key_lookup": "Manchester",
"classified_file_id": "7a26a2a0-2bed-4747-8ad1-2b1127e13dc2",
"file_id": "63b26594-a176-4083-80e0-62c1e2141bbe",
"page_num": "1",
"file_name": "EML + MKPT + SOV.eml",
"coordinates":
{
"x1": 0.08666666666666667,
"y1": 0.37393939393939396,
"x2": 0.15450980392156863,
"y2": 0.38212121212121214
}
}
]
}
]
},
{
"source": "MKTP",
"source_id": "depn_mkpt_address",
"trust_score": "6",
"value_type": "STRING",
"value": "Manchester",
"confidence_score": "90.90",
"dependency":
[
{
"source": "MKPT_DATA",
"source_id": "mkpt_company_address_llm",
"sub_doc_type": "MKTP(AI)",
"trust_score": "10",
"value_type": "STRING",
"value": "Manchester",
"confidence_score": 91.6,
"meta_data":
[
{
"key_lookup": "Manchester",
"classified_file_id": "a07ccecb-14fe-436e-b5d9-f02f1093b41f",
"file_id": "752fcb90-50e0-48da-9af4-37ef4e52dd0e",
"page_num": "5",
"file_name": "Tesco.pdf",
"coordinates":
{
"x1": 0.3819607843137255,
"y1": 0.2087878787878788,
"x2": 0.4807843137254902,
"y2": 0.22181818181818183
}
}
]
}
]
}
]
},
{
"data_element_id": "S-C000012",
"data_element_name": "Postal Code",
"destination_path": "facts.postal_code",
"value_type": "STRING",
"final_value": "M1 7ED",
"final_score": "93.61",
"data_sources":
[
{
"source": "DNB",
"source_id": "dnb_address",
"trust_score": "8",
"value_type": "STRING",
"value": "M1 7ED"
},
{
"source": "SMD_SEED_DATA",
"source_id": "sd_address",
"trust_score": "7",
"value_type": "STRING",
"value": "M1 7ED"
},
{
"source": "CLEARBIT",
"source_id": "clearbit_address",
"trust_score": "6",
"value_type": "STRING",
"value": "AL7 1TW"
},
{
"source": "BP_JACKIQ",
"source_id": "bp_jackiq_address",
"trust_score": "7",
"value_type": "STRING",
"value": "M1 7ED"
},
{
"source": "EML",
"source_id": "depn_sfd_address",
"trust_score": "7",
"value_type": "STRING",
"value": "M1 7ED",
"confidence_score": "93.70",
"dependency":
[
{
"source": "SMART_FORMS_DATA",
"source_id": "sfd_company_address_llm",
"sub_doc_type": "EML(AI)",
"trust_score": "10",
"value_type": "STRING",
"value": "M1 7ED",
"confidence_score": 95.1,
"meta_data":
[
{
"key_lookup": "M1 7ED",
"classified_file_id": "7a26a2a0-2bed-4747-8ad1-2b1127e13dc2",
"file_id": "63b26594-a176-4083-80e0-62c1e2141bbe",
"page_num": "1",
"file_name": "EML + MKPT + SOV.eml",
"coordinates":
{
"x1": 0.15764705882352942,
"y1": 0.37393939393939396,
"x2": 0.20313725490196077,
"y2": 0.38212121212121214
}
}
]
}
]
},
{
"source": "MKTP",
"source_id": "depn_mkpt_address",
"trust_score": "6",
"value_type": "STRING",
"value": "M1 7ED",
"confidence_score": "90.90",
"dependency":
[
{
"source": "MKPT_DATA",
"source_id": "mkpt_company_address_llm",
"sub_doc_type": "MKTP(AI)",
"trust_score": "10",
"value_type": "STRING",
"value": "M1 7ED",
"confidence_score": 93.9,
"meta_data":
[
{
"key_lookup": "M1 7ED",
"classified_file_id": "a07ccecb-14fe-436e-b5d9-f02f1093b41f",
"file_id": "752fcb90-50e0-48da-9af4-37ef4e52dd0e",
"page_num": "5",
"file_name": "Tesco.pdf",
"coordinates":
{
"x1": 0.48705882352941177,
"y1": 0.20909090909090908,
"x2": 0.548235294117647,
"y2": 0.2196969696969697
}
}
]
}
]
}
]
}
]
]
}
}