[Bug]: SQSEvent parsing error

[sukumarh]

Expected Behavior

When a durable function is subscribed to an SQS queue, then the incoming SQS event should directly convert to SQSEvent model of AWS Lambda Java Events library.

Actual Behavior

Durable execution fails with the following error.

Cannot invoke \"java.util.List.iterator()\" because the return value of \"com.amazonaws.services.lambda.runtime.events.SQSEvent.getRecords()\" is null

Steps to Reproduce

1. Set up SNS -> SQS -> Durable Lambda (Java)
2. Use SQSEvent model of AWS Lambda Java Events library.

public SQSBatchResponse handleRequest(final SQSEvent event, final DurableContext ctx) {

}

3. Publish an SNS message.

SDK Version

1.0.1

Java Version

17

Is this a regression?

No

Last Working Version

No response

Additional Context

No response

[SilanHe]

Workaround

In your handler code, you can add the following:

import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.MapperFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import software.amazon.lambda.durable.DurableConfig;
import software.amazon.lambda.durable.serde.JacksonSerDes;

@Override
       protected DurableConfig createConfiguration() {
           return DurableConfig.builder()
               .withSerDes(new JacksonSerDes(new ObjectMapper()
                   .registerModule(new JavaTimeModule())
                   .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
                   .disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
                   .enable(MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES))) // this is the crux of the workaround
               .build();
       }

aws-durable-execution-sdk-java/sdk/src/main/java/software/amazon/lambda/durable/serde/JacksonSerDes.java

Line 37 in b9d143f

public JacksonSerDes() {

We are effectively overriding the default JacksonSerDes provided by the SDK.

Potential long term fix suggestions

1. Add the workaround feature enable(MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES))) to the JacksonSerDes initialization in the SDK.

We aren't doing this yet because it can break customers who are relying on the case sensitive behaviour. We also aren't sure if this would cover all the cases from aws-lambda-java-serialization.

2. Add dependency on aws-lambda-java-serialization.

This introduces some 3rd party dependency bloat that we're not ready to introduce permanently to our SDK.

Path forward

We want to use this issue to get some data on customers who might be encountering this problem. We will revisit this in a few months time. Feel free to comment your experience and use case when encountering this issue. It would be really helpful.

[sukumarh]

Thanks, @SilanHe .

The workaround worked.

I had a quick query on implementing custom serializer. Are there other parts of the durable execution (other than the initial message parsing) that would be impacted?

For example, when using waitForCallback, the callback model (provided by the resumer) is deserialized into a Java class. Will the durable lambda resort to using the custom serializer provided or the default one in the SDK?

[zhongkechen]

Are there other parts of the durable execution (other than the initial message parsing) that would be impacted?

The other parts of the durable execution like steps, contexts, callbacks have their own serDes in their config like StepConfig, CallbackConfig, etc. If it's not specified, the serDes from DurableConfig will be used.