Skip to main content

Universal Submit

Description

Authentication and Credentials

  • Your Bold Penguin project team will provide you with a unique Client ID and Client Secret per Bold Penguin environment.

  • Additional credentials will be provided once the connection in the UAT environment is validated.

  • This endpoint requires an access_token from the authentication endpoint in the same environment. Include it as a bearer token in an Authorization header for the request. Access tokens are valid for 12 hours.

    Client ID: (Shared with the customer)

    Client Secret: (Shared with the customer)

    API Key: this will be shared separately via email

    Data Package ID: (Shared with the customer)

Authentication URL:

POST <https://boldpenguin-auth-uat.beta.boldpenguin.com/auth/token?grant_type=client_credentials>

Step 1: Get the Auth Token

  • You must submit the shared credentials as a basic authentication header.
  • Combine your Client ID and Client Secret separated by a colon, (client_id:client_secret), then apply Base64 encoding.

Sample Authentication Request

POST /auth/token?grant_type=client_credentials HTTP/1.1
Host: boldpenguin-auth-uat.beta.boldpenguin.com
Authorization: Basic OXlXUHY5NTlqV1laWEsxSzVncmRKaW9MbnlYSll4U3Y6TkRVNFFwZ1BaUmFTS0F0elFuNkNOVDd5UkIyakd0ZkNINFVRRUNvblJtaGhKM0t6OE5IMkJBTkVCMkw5a2FOZg==

Sample Authentication Response

{
"access_token": "5ZhAzSrQCuyYct367GqfSnwE",
"token_type": "bearer",
"expires_in": 43200,
"refresh_token": "7EWVgH9bVdwBQTmeXH1HXJmV",
"scope": "",
"uid": "ee9e3e03-7261-4ec7-8988-1f88ba1ec2df",
"info": {
"name": "SmartData Integrations Service Client",
"email": null,
"first_name": "SmartData Integrations",
"last_name": "Service Client"
},
"extra": {
"raw_info": {
"user_id": "0715308c-db1c-4e63-b854-779bc93bc77c",
"tenant_id": "96a5f17b-ebc8-4459-aca7-b49ba6fbc752",
"tenant_name": "Kowalski",
"tenant_subdomain": "kowalski",
"tenant_theme": "default",
"tenant_type": null,
"primary_user_group_id": "0d5aef45-9b88-436c-a0ef-344b674bb6d3",
"user_group_ids": [
"0d5aef45-9b88-436c-a0ef-344b674bb6d3"
],
"name": "SmartData Integrations Service Client",
"email": null,
"first_name": "SmartData Integrations",
"last_name": "Service Client",
"phone": null,
"permissions": [
"::data/v2:get",
"::universal/v3/universal-submit:get",
"::universal/v3/universal-submit:post",
"::universal/v2/universal-submit:get",
"::universal/v2/universal-submit:post",
"::universal/v1/universal-submit:post",
"::universal/v1/universal-submit:get",
"::audit/v1/report:get",
"::location/v1/location_submit:post",
"::data/v1:get",
"::company/v1/quick_submit:post",
"::company/v1/company_submit:post"
],
"auth_uid": null,
"created_at": "2021-05-06T15:24:53.870Z",
"tenant_created_at": "2021-05-06T15:24:18.941Z",
"completed_steps": [],
"accepted_agreements": [],
"plan_sku": null,
"sub_status": null
}
}
}

Tokens expire after 12 hours, after which point you can use the refresh_token to refresh the token, or request a new access_token using the steps above.


Accessing the Universal Submit API

  • The Universal Submit API requires the file (eml/doc/pdf/xls etc.) to be passed along with the request for processing.
  • To accommodate handling of files with sizes 10 MB or more the process of triggering the Universal Submit API is broken down into multiple steps as below.

Step 2: Get the File Upload URL

  • The call to the below endpoint will return a URL where the file can be uploaded.
  • The URL is only valid for 5 minutes.
  • The tx_id returned as part of the response is required to communicate with submit, status check and inquiry api’s in the context of this submission.
POST https://api-smartdata.di-beta.boldpenguin.com/universal/v2/universal-submit/file-upload-url

Pass the following variables as part of the request

Request ParametersComments
Authentication TokenThe token obtained in STEP 1
API Key(Shared with the Customer)
FilenameThe full name of the file/eml being uploaded

Sample Request

POST /universal/v2/universal-submit/file-upload-url HTTP/1.1
Host: api-smartdata.di-beta.boldpenguin.com
x-api-key: F4jHKvQcVDaY0sDivJgEe6QIuqArhDin8XldfxoK
Authorization: Bearer 5ZhAzSrQCuyYct367GqfSnwE
Content-Type: application/json
Content-Length: 70

{
"filename":"Somerville Aluminum Inc. - Combined Liability.eml"
}

Sample Response

{
"tx_id": "9fb6b3fe-5e90-4610-a2ab-dbea6656dd0e",
"upload_url": "https://plb.di-beta.boldpenguin.com/file-transfer/upload-file?data=AAAAuAECAwB4KBTA570fKFbYe1Mk3uskjyVUYNCRanx2YpT7T1sYaigBlA1trG8UX%2BFBHhxdQ3IoDgAAAH4wfAYJKoZIhvcNAQcGoG8wbQIBADBoBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDJgGaPjP3tnH9pTYAAIBEIA7LwAtmiX4MtCIyxhTNHeodgNHM2drgOYnvRLhqGhG8sKRa2Kc6YTePGfW6YSq7KNYVDqNDLkFMbVAKN5nQUFBQUFCaE1qQ2dvQ0ZjVnpiSTFCeDdVT2tIMk1Pd01lbWhtOVNWclF0TmxpekFsazRoeThEZng1X1ZmeTZjVHhNeUp0ckFLLUpxUGpMeUhUcjVIUU1WR21seHJ6cE9tQnBGZjA1WFpleFdoRDZFVVRqMDhydnhhQWZSMzNDWlJEM1FMdW5jSVRMdkJJeHM5TDNWMlVvbGpGbUFncGNzTWRfUTJUVFRHQXY0ZmtyVjMzNXJRRzY1dW5nSFhQTEhmU29KUWVsWmRvelJtM1NMcTdmU3JteWxmTHZMeDNNWTJkTHdVMEtoeC03V3ZCV08xNG9xaG9zOU9GWWUwc1d2QU1aOHVFTG45T3dXQ0hlZEFGN2hsRXFjbmxlUTJhdG44SWRvY0wwT0JEdm14dTBVQWg5c21yQnZIYXZoczE3MTk4VUI5M3owMVhmRkJFMkNkSXo3MmZvUWpYUmJsb254NFl5LTVDWHpVVFdpQzV3NE10ZW9qdVE9"
}

Step 3: Upload the File to AWS S3

  • Use the “upload_url” returned as part of Step 2 and upload the file

Sample Request

PUT /file-transfer/upload-file?data=AAAAuAECAwB4KBTA570fKFbYe1Mk3uskjyVUYNCRanx2YpT7T1sYaigBlA1trG8UX%2BFBHhxdQ3IoDgAAAH4wfAYJKoZIhvcNAQcGoG8wbQIBADBoBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDJgGaPjP3tnH9pTYAAIBEIA7LwAtmiX4MtCIyxhTNHeodgNHM2drgOYnvRLhqGhG8sKRa2Kc6YTePGfW6YSq7KNYVDqNDLkFMbVAKN5nQUFBQUFCaE1qQ2dvQ0ZjVnpiSTFCeDdVT2tIMk1Pd01lbWhtOVNWclF0TmxpekFsazRoeThEZng1X1ZmeTZjVHhNeUp0ckFLLUpxUGpMeUhUcjVIUU1WR21seHJ6cE9tQnBGZjA1WFpleFdoRDZFVVRqMDhydnhhQWZSMzNDWlJEM1FMdW5jSVRMdkJJeHM5TDNWMlVvbGpGbUFncGNzTWRfUTJUVFRHQXY0ZmtyVjMzNXJRRzY1dW5nSFhQTEhmU29KUWVsWmRvelJtM1NMcTdmU3JteWxmTHZMeDNNWTJkTHdVMEtoeC03V3ZCV08xNG9xaG9zOU9GWWUwc1d2QU1aOHVFTG45T3dXQ0hlZEFGN2hsRXFjbmxlUTJhdG44SWRvY0wwT0JEdm14dTBVQWg5c21yQnZIYXZoczE3MTk4VUI5M3owMVhmRkJFMkNkSXo3MmZvUWpYUmJsb254NFl5LTVDWHpVVFdpQzV3NE10ZW9qdVE9 HTTP/1.1
Host: plb.di-beta.boldpenguin.com
Content-Length: 231
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW

----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename="Somerville Aluminum Inc. - Combined Liability.eml"
Content-Type: <Content-Type header here>

(data)
----WebKitFormBoundary7MA4YWxkTrZu0gW

Sample Response

{
"data": "s3://riskgenius-riskgenius-beta-eu-uploadfiless3bucket-a7aqf1omt7i/003d21a5-1cc6-4a7f-bacf-90be517d2209"
}

Step 4: Trigger the File Processing Flow

Universal Submit URL:

POST https://api-smartdata.di-beta.boldpenguin.com/universal/v2/universal-submit/file/tx_id

Pass the following variables as part of the request

Request ParametersComments
Authentication TokenThe token obtained in STEP 1
API Key(Shared with the Customer)
tx_idThe txid returned as part of Step 2: _Get the file upload URL

Sample Universal Submit Request

POST /universal/v2/universal-submit/file/9fb6b3fe-5e90-4610-a2ab-dbea6656dd0e HTTP/1.1
Host: api-smartdata.di-beta.boldpenguin.com
x-api-key: F4jHKvQcVDaY0sDivJgEe6QIuqArhDin8XldfxoK
Authorization: Bearer 5ZhAzSrQCuyYct367GqfSnwE

Sample Universal Submit Response

{
"tx_id": "01853abd-4f76-4796-a667-6a217245bf74",
"tx_start_time": "2022-03-08T13:12:49.631128+00:00",
"tx_end_time": "2022-03-08T13:12:49.675991+00:00",
"response_time_secs": 0.044863,
"tenant_id": "kowalski",
"status": "Submission_in_progress"
}

Upon submission completion, data extraction is a long running process. Each stage of the flow will transition the status of the transaction as below:

Submission_in_progress -> Scheduled -> Started -> Completed or Failed


Accessing the Universal Submit Status Check API

  • Once the file is submitted for processing, we can check the status by calling the Universal Submit Status Check API

Step 5: Access the Universal Submit Status Check API

Universal Submit Status Check URL:

GET https://api-smartdata.di-beta.boldpenguin.com/universal/v2/universal-submit/status/{tx_id}

Pass the following variables as part of the request

Request ParametersComments
Authentication TokenThe token obtained in STEP 2
API Key(Shared with the Customer)
tx_idThis Id is returned as part of the Universal Submit response

Sample Universal Submit Status Check Request

GET /universal/v2/universal-submit/status/9fb6b3fe-5e90-4610-a2ab-dbea6656dd0e HTTP/1.1
Host: api-smartdata.di-beta.boldpenguin.com
x-api-key: F4jHKvQcVDaY0sDivJgEe6QIuqArhDin8XldfxoK
Authorization: Bearer 5ZhAzSrQCuyYct367GqfSnwE

Sample Universal Submit Status Check Response

{
"tx_id": "8343ae00-ae0b-47c7-9abd-6ff05a75f982",
"tenant_id": "kowalski",
"user_id": "test",
"tx_start_time": "2021-06-28T16:18:35.139714+00:00",
"tx_end_time": "2021-06-28T16:18:35.505500+00:00",
"response_time_secs": 0.365786,
"status": "Pending"
}

The status will begin as “Pending”, move to “Started”, and will then complete with a status of either “Succeeded” or “Failed”.

NOTE: Additionally, there will also be a c_report_id and l_report_id generated upon successful completion of data for enrichment

{
"tx_id": "8343ae00-ae0b-47c7-9abd-6ff05a75f982",
"tenant_id": "kowalski",
"user_id": "test",
"tx_start_time": "2021-06-28T16:18:35.139714+00:00",
"tx_end_time": "2021-06-28T16:18:35.505500+00:00",
"response_time_secs": 0.365786,
"status": "Completed",
"c_report_id": "CUS1187A9346D1C83129",
"l_report_id": "LUS1187A9346D1C8F76E"
}

Universal Submit Status Details URL:

GET https://api-smartdata.di-beta.boldpenguin.com/universal/v2/universal-submit/status/{tx_id}/details

Pass the below variables as part of the request

Request ParametersComments
Authentication TokenThe token we get in STEP 2
API Key(Shared with the Customer)
tx_idThis Id is returned as part of the Universal Submit response

Sample Universal Submit Status Details Request

GET /universal/v2/universal-submit/status/9fb6b3fe-5e90-4610-a2ab-dbea6656dd0e/details HTTP/1.1
Host: api-smartdata.di-beta.boldpenguin.com
x-api-key: F4jHKvQcVDaY0sDivJgEe6QIuqArhDin8XldfxoK
Authorization: Bearer 5ZhAzSrQCuyYct367GqfSnwE

Sample Universal Submit Status Details Response

{
"tx_id": "9fb6b3fe-5e90-4610-a2ab-dbea6656dd0e",
"tenant_id": "kowalski",
"user_id": "0715308c-db1c-4e63-b854-779bc93bc77c",
"tx_start_time": "2022-05-03T13:56:23.892802+00:00",
"tx_end_time": "2022-05-03T13:56:25.189395+00:00",
"response_time_secs": 1.296593,
"status": "Completed",
"flows": [
{
"name": "EML",
"files": [
{
"file_name": "Somerville Aluminum Inc. - Combined Liability.eml",
"document_type": "EML",
"status": "Succeeded"
}
]
}
],
"c_report_id": "CUS1187AD689F0AA102D",
"l_report_id": "LUS1189CA5500E714151"
}

Accessing the Extracted/Enriched Data

  • Once the status of the submission is completed, we can access the Data Inquiry API to get the extracted and enriched data

Step 6: Access the Data Inquiry API

Company Data Inquiry URL:

GET <https://api-smartdata.di-beta.boldpenguin.com/data/v2/{dp_id}/{tx_id}?idType=transaction>

Pass the following variables as part of the request

Request ParametersComments
Authentication TokenThe token obtained in STEP 2
API Key(Shared with the Customer)
dp_id(Data Package Id shared with the Customer)
tx_idThis Id is returned as part of the Universal Submit response

Sample Company Data Inquiry Request

GET /data/v2/kowalski-us-c0006/9fb6b3fe-5e90-4610-a2ab-dbea6656dd0e?idType=transaction HTTP/1.1
Host: api-smartdata.di-beta.boldpenguin.com
x-api-key: F4jHKvQcVDaY0sDivJgEe6QIuqArhDin8XldfxoK
Authorization: Bearer 5ZhAzSrQCuyYct367GqfSnwE

Sample Company Data Inquiry Response

{
"tx_id": "67611507-0687-4aec-aca1-0a9d3976a69c",
"tx_start_time": "2022-03-08T18:53:57.242Z",
"tx_end_time": "2022-03-08T18:54:11.075Z",
"response_time_secs": "13",
"data_package_id": "kowalski-us-c0006",
"tenant_id": "kowalski",
"user_id": "test",
"c_report_id": "CUS1187A9346D1C83129",
"l_report_id": "LUS1187A9346D1C8F76E",
"doc_reference_id": "9fb6b3fe-5e90-4610-a2ab-dbea6656dd0e ",
"seed_input": {
"company_name": "Somerville",
"address": "20 County Line Rd Branchburg NJ 08876-3498 UK"
},
"cleansed_input": {
"companyName": "SOMERVILLE ALUMINUM INC.",
"website": "www.somervillealuminum.com",
"phoneNumber": "8003463693",
"confidence_score": 0.9966666666666667,
"address": {
"streetAddr": "20 COUNTY LINE RD",
"city": "BRANCHBURG",
"state": "NJ",
"zipOrPostalCode": "08876",
"country": "US",
"countryISOCode": "",
"geoPosition": {
"latitude": 40.6076817,
"longitude": -74.7155544
}
},
"seedInput": {
"companyName": "SOMERVILLE ALUMINUM INC",
"address": {
"streetAddr": "20 County Line Rd",
"city": "Branchburg",
"state": "NJ",
"zipOrPostalCode": "08876",
"country": "US"
}
},
"dunsNumber": "011739927"
},
"data": {
"facts": {
"company_name": "SOMERVILLE ALUMINUM INC.",
"website": "www.somervillealuminum.com",
"address": "20 County Line Rd",
"city": "BRANCHBURG",
"state": "NJ",
"postal_code": "08876",
"duns_number": "011739927",
"legal_entity_type": "Corporation",
"year_founded": "1956",
"annual_revenue": "4947336.0",
"bankruptcy_indicator": "N",
"smart_company_credit_grade": "A",
"primary_naics_2012": [
{
"code": "444110",
"desc": "Home Centers"
}
],
"secondary_naics_2012": [
{
"code": "423310",
"desc": "Lumber, Plywood, Millwork, and Wood Panel Merchant Wholesalers"
},
{
"code": "238170",
"desc": "Siding Contractors"
},
{
"code": "321911",
"desc": "wood window and door manufacturing"
},
{
"code": "722310",
"desc": "food service contractors"
},
{
"code": "331313",
"desc": "alumina refining and primary aluminum production"
},
{
"code": "23611",
"desc": "residential building construction"
}
],
"company_number": "",
"has_company_been_liquidated": "",
"company_jurisdiction": "",
"insolvency_history": "",
"country": "US",
"global_ultimate_duns": "",
"company_description": [
"Retails lumber or other building materials, specializing in door or window products and siding. Wholesales lumber, plywood and millwork, specializing in doors or windows and exterior building materials. Contractor of siding work."
],
"vat_number": "",
"ern_number": "",
"subsidiary_companies": [],
"does_company_has_office365_exposure": "Y",
"does_any_employee_works_at_airports": "N",
"does_any_employee_works_at_blast_furnaces": "N",
"does_any_employee_works_at_boat_yard": "N",
"does_any_employee_works_at_bridges": "N",
"does_any_employee_works_at_chimney_shafts": "N",
"does_any_employee_works_at_collieries": "N",
"does_any_employee_works_at_computer_rooms": "N",
"does_any_employee_works_at_docks": "N",
"does_any_employee_works_at_fences_on_motorways": "N",
"does_any_employee_do_gas_exploration_work": "N",
"does_any_employee_works_at_hangars": "N",
"does_any_employee_works_at_harbours": "N",
"does_any_employee_works_at_marine_watercraft_hovercraft": "N",
"does_any_employee_works_at_mines": "N",
"does_any_employee_works_at_motor_vehicle_cycles": "N",
"does_any_employee_works_at_nuclear_establishments": "N",
"does_any_employee_works_at_offshore_structures": "N",
"does_any_employee_works_at_oil_gas_chemical_bulk_storage": "N",
"does_any_employee_works_at_oil_gas_chemical_production_premises": "N",
"does_any_employee_works_at_petrochemical_works": "N",
"does_any_employee_works_at_pharmaceuticals": "N",
"does_any_employee_works_at_piers": "N",
"does_any_employee_works_at_power_stations": "N",
"does_any_employee_works_at_quarries": "N",
"does_any_employee_works_at_railways": "N",
"does_any_employee_works_at_refineries": "N",
"does_any_employee_works_at_steel_structures": "N",
"does_any_employee_works_at_steeples": "N",
"does_any_employee_works_at_tanks": "N",
"does_any_employee_works_at_towers": "N",
"does_any_employee_works_at_underground": "N",
"does_any_employee_works_at_underwater": "N",
"is_non_profit_company": "",
"has_business_in_operation_for_more_than_3yrs": "Y",
"does_any_employee_undertakes_demolition_work": "N",
"does_any_employee_undertakes_work_involving_roofing_scafolding_abseiling_cradles": "Y",
"is_any_work_carried_out_other_than_collection_and_delivery": "N",
"does_any_employee_works_at_dams": "N",
"does_any_employee_works_with_heat_fire": "N",
"parent_company_revenue": "",
"registered_in_countries": [
"US"
],
"assets_in_countries": [],
"product_sold_in_countries": [
"US"
],
"parent_company_revenue_currency_code": "USD",
"total_legal_fillings": "",
"dnb_rating": "68BI",
"dnb_rating_description": "Moderate",
"dnb_review_date": "2021-07-18",
"parent_company_global_sales": "",
"has_there_been_health_and_safety_breach": "N"
},
"options": {
"programme_type": "",
"product": [
"Casualty",
"Liability",
"Combined Liability"
],
"policy_inception_date": "2021-04-01",
"field_office": "",
"exclusive_flag": "N",
"end_date": "",
"document_date": "2021-03-12",
"broker_contact_points": "Laura Razo",
"coverage": [],
"100_pct_limit": "",
"broker_name": "",
"underwriter": "",
"broker_address": "",
"broker_city": "",
"broker_postal_code": "",
"broker_state": "",
"broker_email": "laura.razo@boldpenguin.com",
"estimated_premium": "28,000",
"risk_country": "",
"business_interruption_period": "",
"decline_reason": "",
"form_submission_type": [
"submission",
"quote"
],
"broker_contact_phone": "",
"policy_inception_effective_time": "",
"policy_end_time": ""
}
},
"scores": {
"company_name": "84.34",
"website": "78.19",
"address": "80.64",
"city": "80.64",
"state": "80.64",
"postal_code": "80.64",
"legal_entity_type": "84.34",
"year_founded": "82.42",
"annual_revenue": "75.59",
"smart_company_credit_grade": "82.96",
"primary_naics_2012": "84.34",
"policy_inception_date": "90.00",
"exclusive_flag": "80.00",
"document_date": "90.00",
"broker_contact_points": "90.00",
"broker_email": "90.00",
"estimated_premium": "90.00"
}
}

Location Data Inquiry URL:

GET <https://api-smartdata.di-beta.boldpenguin.com/data/v2/{dp_id}/{tx_id}?idType=sov>

Pass the below variables as part of the request

Request ParametersComments
Authentication TokenThe token we get in STEP 2
API Key(Shared with the Customer)
dp_id(Data Package Id shared with the Customer)
tx_idThis Id is returned as part of the Universal Submit response

Sample Location Data Inquiry Request

GET /data/v2/kowalski-us-c0006/9fb6b3fe-5e90-4610-a2ab-dbea6656dd0e?idType=sov HTTP/1.1
Host: api-smartdata.di-beta.boldpenguin.com
x-api-key: F4jHKvQcVDaY0sDivJgEe6QIuqArhDin8XldfxoK
Authorization: Bearer 5ZhAzSrQCuyYct367GqfSnwE

Sample Location Data Inquiry Response

{
"tx_id": "1708b37d-deb2-4d60-96df-aa29358f02fc",
"tx_start_time": "2022-03-08T18:54:11.463Z",
"tx_end_time": "2022-03-08T18:54:11.524Z",
"response_time_secs": "0",
"data_package_id": "kowalski-us-l0006",
"tenant_id": "kowalski",
"user_id": "test",
"c_report_id": "CUS1187A9346D1C83129",
"l_report_id": "LUS1187A9346D1C8F76E",
"doc_reference_id": "9fb6b3fe-5e90-4610-a2ab-dbea6656dd0e",
"data": [
{
"facts": {
"location_address": "20 COUNTY LINE RD",
"location_city": "BRANCHBURG",
"location_state": "NJ",
"location_postal_code": "08876",
"location_country": "US",
"latitude": "40.6076817",
"longitude": "-74.7155544",
"construction_type": "",
"location_id": "LUS1187A9346D1C8F76E",
"flood_score": "",
"earthquake_score": "",
"location_occupancy_code": "",
"location_have_intrusion_detectors": "",
"location_have_smoke_detectors": "",
"location_have_sprinklers": "",
"wind_score": ""
},
"options": {
"building_value": "",
"business_interruption_value": "",
"contents_tiv": "",
"stock_tiv": "",
"location_application_limit": "",
"100_pct_coverage_limits": "",
"100_pct_limit": "",
"location_pd": "",
"location_charge_value": "",
"100_pct_bi_limit": "",
"100_pct_pd_limit": ""
},
"scores": {}
}
]
}