Consuming Messages with Metadata for Multiple Consumers

Answer - D.

The AWS Documentation mentions the following.

Amazon SNS supports the delivery of message attributes which let you provide structured metadata items (such as timestamps, geospatial data, signatures, and identifiers) about the message.

Each message can have up to 10 attributes.

https://docs.aws.amazon.com/sns/latest/dg/sns-message-attributes.html

Options A is not correct because you are asked to send metadata information along with the message.

Here it's referring to work with many custom endpoints (which is much more overhead).

Option B is invalid since you cannot mimic the functionality of SNS Topics through many SQS queues.

This would be a bad practice.

Option C is not the intended use for FIFO SQS.

While it does maintain the order of items in the queue and their underlying message attributes, it does not fan output to endpoints.

For more information specifically on SQS message attributes, please refer to the below URL-

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-message-attributes.html

The requirement is to consume a message by multiple consumers in an application while ensuring that metadata can be sent along with the messages. AWS provides multiple messaging services to address this requirement, and among them, the most suitable option would depend on the specific needs and use case.

Option A suggests implementing an SNS topic and using different endpoints for different types of metadata. Amazon SNS (Simple Notification Service) is a fully managed messaging service that enables message delivery to multiple subscribers or endpoints, such as email, SMS, and HTTP/S. It supports filtering of messages based on message attributes, which can be used to route messages to specific endpoints. This option could work if there is a limited number of metadata types and specific endpoints can be associated with each metadata type. However, this may not be scalable or flexible enough for complex applications.

Option B suggests using SQS (Simple Queue Service) queues and creating different queues for different types of metadata. Amazon SQS is a fully managed message queuing service that allows decoupling of message producers and consumers, enabling asynchronous and distributed processing. Each queue can have its own set of message attributes, which can be used to identify the metadata type. This option is more scalable and flexible compared to Option A, as it allows multiple consumers to consume messages from the same queue, and the number of queues can be dynamically adjusted based on the workload. However, it may require additional logic to ensure that messages are processed in the correct order.

Option C suggests using a FIFO (First-In-First-Out) SQS queue to maintain the order of messages and fan them out. A FIFO queue guarantees that messages are processed in the order they are received, and it also provides deduplication of messages to ensure that each message is processed only once. This option could work if the order of messages is critical and multiple consumers need to consume messages in the same order. However, it may not be suitable if the workload is highly variable or if multiple metadata types need to be processed concurrently.

Option D suggests using an SNS topic and adding message attributes to the messages. This option is similar to Option A, but it allows adding metadata to individual messages rather than routing messages to specific endpoints. Message attributes are key-value pairs that can be added to a message, and they can be used for filtering and routing purposes. This option could work if the metadata needs to be attached to individual messages rather than the entire topic, and if the number of metadata types is relatively small.

In summary, the most suitable option depends on the specific needs and use case. Option B using SQS queues and creating different queues for different types of metadata is likely the most scalable and flexible option.