Your team is working on an event-driven application.
The frontend needs to collect the customers' requests and publish the events to an AWS service.
This AWS service should then deliver the events to several SQS queues in parallel.
Each queue and the following pipeline have their own purposes such as event storage, event search or event replay.
Each pipeline should process only a subset of events so a filter mechanism is required.
How would you design the AWS service in the easiest way?
Click on the arrows to vote for the correct answer
A. B. C. D.Correct Answer: B.
Option A is incorrect because for an SNS subscription, you can set up a subscription filter policy to filter the events.
You do not need to create Lambda functions in the pipelines to filter the events.
Option B is CORRECT because an SNS topic can deliver messages to all subscriptions in parallel.
And you can use the subscription filter policy to only deliver the messages that you need.
Option C is incorrect because you have to maintain a Lambda function to filter the events and deliver them to SQS queues in parallel.
Option B is more straightforward.
Option D is incorrect because the Amazon EventBridge event bus is not a suitable service to receive events from the frontend of the application.
You also need to maintain a Lambda function to filter the events.
Option B is easier.
Reference:
https://aws.amazon.com/blogs/compute/enriching-event-driven-architectures-with-aws-event-fork-pipelines/, https://docs.aws.amazon.com/sns/latest/dg/sns-fork-pipeline-as-subscriber.html, https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.htmlSure, I'd be happy to explain the different options and their potential advantages and disadvantages.
Option A: Publish events to an SNS topic and subscribe SQS queues The first option is to publish the events to an SNS (Simple Notification Service) topic and then subscribe SQS (Simple Queue Service) queues to the SNS topic. This allows multiple queues to receive the same events in parallel. In each pipeline, you would create a Lambda function to pull the messages from the queue and filter them based on the requirements of that specific pipeline. This option is relatively simple and straightforward to implement, but it does require more Lambdas and setup time.
Option B: Publish events to an SNS topic with subscription filter policies The second option is to publish the events to an SNS topic and create subscriptions for each of the SQS queues. Using subscription filter policies, you can filter the messages before they are delivered to each queue, so that each pipeline only receives the subset of events it needs to process. This option is also relatively simple to implement and reduces the number of Lambdas required compared to Option A, but it does require additional configuration of subscription filter policies.
Option C: Publish events to an SQS queue and use a Lambda function to filter The third option is to publish the events directly to an SQS queue and then use a Lambda function to get the messages from the queue, filter them based on the requirements of each pipeline, and forward them to other SQS queues for processing. This option is similar to Option A, but with a single SQS queue instead of an SNS topic. It reduces the number of Lambdas required but does add more complexity to the filtering process since the Lambda needs to forward the messages to multiple queues.
Option D: Publish events to Amazon EventBridge and use a Lambda function to filter The fourth option is to publish the events to an Amazon EventBridge event bus, which is a fully-managed service for event routing and processing. You can then use a Lambda function to get the messages from the event bus, filter them based on the requirements of each pipeline, and forward them to multiple SQS queues in parallel. This option is similar to Option B but with Amazon EventBridge instead of SNS. It can be easier to manage and configure than the other options but does require some familiarity with EventBridge and may not be the best choice for simple applications.
In summary, each option has its own advantages and disadvantages. Option A is simple but requires more Lambdas, Option B reduces the number of Lambdas but requires more configuration, Option C reduces the number of Lambdas but adds more complexity, and Option D is fully-managed but may not be the best choice for simple applications. Ultimately, the best option depends on the specific needs of your application and your team's preferences and expertise.