You should consider implementing these best practices when using webhooks in your integration.
You should configure webhook endpoints to only process the types of events required for your integration. It is not recommended to listen to extra events as this will place undue stress on your server.
It is strongly recommended to decouple the reception of webhook events from their processing to help ensure deliverability, order, and long-term replay-ability. We recommend having the webhook endpoint consume and enqueue events in a FIFO (first in, first out) queue for later processing by a separate service. This helps ensure timely responses to avoid timeouts and provides a mechanism for re-processing events on your end if needed.
Webhook responses are expected to be fast. You should ensure that your system can respond to each event in a timely manner. If a connection to your service cannot be opened in ~5 seconds, or if the event takes longer than 60 seconds to respond, webhooks will timeout and fail.
In the event that a webhook notification fails to deliver to a subscribed endpoint, we will continue to retry the delivery attempt with an exponential back-off before eventually giving up. Currently, this exponential back-off results in 25 retries over roughly 3 weeks.
Note: Bold Penguin retry logic is not tied solely to TCP/endpoint down errors. Any non-2XX response will result in a retry as well.
A single record may be included in multiple payloads, such as when an agent completes a Question Set, goes back to edit it, and completes it again. As a result, you should ensure your event processing is idempotent - that is, processing the same event twice should still only result in one record on your end. As these events can have new information, it is not advised to ignore all subsequent events for a given record.