Use the Dead Letter Strategy

To use the deadletter strategy, you must:

Add two properties to the application.conf file.
Create an index layer for archiving messages that encounter an error.

Dead Letter Properties in application.conf

The two properties you must add to the application.conf file are as follows:

deadletter.hrn
deadletter.layer

Both properties go under error settings. For more information, see Configure Your Application.

Dead Letter Index Layer

The deadletter error handling strategy archives messages that encounter an error to an index layer. This special index layer must have the following settings:

Content Type must be application/x-avro-binary.
Content Encoding must be uncompressed.
There must be four indexing attributes with the following names and settings:
- A timewindow type attribute with the name **indexUploadTime**. You can select the desired duration. This attribute stores the timestamp of the index upload, truncated by the duration value.
- A string type attribute with the name **realm**. This attribute stores the realm of the index layer where messages that were successfully processed are indexed.
- A string type attribute with the name **catalogId**. This attribute stores the catalogId of the index layer where messages that were successfully processed are indexed.
- A string type attribute with name **layerId**. This attribute stores the layerId of the index layer where messages that were successfully processed are indexed.

You can select any TTL setting.

If the special index layer does not follow these settings, the pipeline will fail to initialize.

Custom Dead Letter Transformation

For deadletter error strategy, you can provide a custom implementation of the method transformForDeadLetter(message) in the UDF interface. This allows you to modify the contents of the message field in the deadletter message schema, shown below, before it is archived to the deadletter index layer.

Example

@Override
public byte[] transformForDeadLetter(byte[] message) {
    return (new String(message) + "this message has been modified").getBytes();
}

Note

transformForDeadLetter is an optional API. If you don't provide any implementation, the original message will be archived in deadletter index layer.
If your implementation of transformForDeadLetter API causes an error, the original message will be archived in deadletter index layer.

Dead Letter Messages

For deadletter error strategy, the Data Archiving Library aggregates and archives the messages that encounter an error scenario in a User-Defined Function's implementation.

Messages are archived with following metadata:

messageId
ingestionTime
processingTime

The Data Archiving Library archives the deadletter messages in the Avro format. The Avro Schema DeadLetterMessage is:

{
  "namespace": "com.here.platform.dal",
  "type": "record",
  "doc": "deadletter message format",
  "name": "DeadLetterMessage",
  "fields": [
    { "name": "message", "type": "bytes"},
    { "name": "messageId", "type": "string"},
    { "name": "ingestionTime", "type": "string"},
    { "name": "processingTime", "type": "string"}
  ]
}

You can use this Avro schema to reprocess or deserialize the deadletter messages stored in the deadletter index layer.

Use the Dead Letter Strategy