A Brief Comparison of Messaging Services in AWS
What is messaging service? and where is it used in A Cloud-Native Solution? If you have been confused about these questions as well keep reading this article.
Messaging Services are simply ways of communicating information from one unit of the Application or System to the other. In order to decouple the Producer and Consumers of these information packets Messaging Services are used.
The reason behind this disassociation is fundamentally creating a solution that is high in cohesion and loosely coupled. Somewhat I believe it incorporates security in your solution as well since information is not crammed up in one place.
Messaging Services in AWS
AWS provides four Messaging services designed for different use-cases. Each messaging service has its own model of operation which translates its functional use. In order to find an appropriate solution for your architecture, it is important that you understand the characteristics and working model of each messaging service.
AWS Simple Queue Service (SQS)
- Model of Operation: Consumer pulls Queued Messages. Once a Message is consumed it is removed and can’t be consumed again.
- The retention period of SQS messages is 1 minute to 14 days (4 days as the default value). And guaranteed delivery is maintained by SQS.
- Cost: Billed as per the number of messages.
If your architecture demands Asynchronous Message Processing (zero wait) Simple Queue Service is your best option. Your Consumer can be a Lambda function waiting for the required information of EC2 instances ready to process your information.
AWS Simple Notification Service (SNS)
- Model of Operation: Runs on Publish-Subscribe Model. Messages are broadcasted to various subscribers/consumers.
- Consumers/Subscribers (those who subscribed to the topic), in this case, can be resources and entities as well. Entities can receive emails, app notifications and SMS in case of an event. Similarly, AWS Resources such as Lambda functions, Kinesis Firehose and SQS can receive messages.
- Cost: Billed as per the number of the messages.
A usual use case for SNS is publishing alerts from AWS resources (such as EC2 instances or S3 buckets). You can simply link events to your SNS topics so that your subscribers can receive notifications in case of a hit.
Fan-out Pattern Event Notification Architecture (SQS and SNS Combo)
A combination fan-out pattern of both SQS and SNS is used for Asynchronous message processing by multiple subscribers. For this architecture simply:
- Create one SNS topic.
- Create as many SQS Queues as required with Email or SMS endpoints.
- Subscribe queues to the topic.
Kinesis Data Streams
- Kinesis is a comprehensive toolkit to pull (Kinesis Data Streams), process (Kinesis Data Analytics) and store (Kinesis Data Firehose) data streams.
- Mode of Operation: Consumer pulls data streams and there can be multiple consumers at the same time.
- Cost: Billed as per shard hour and messages received.
Used in case of Data Processing and Real-time data processing scenarios where there can be multiple stand-alone consumers. Kinesis Data Streams is suitable in case you are dealing with sizeable amount of data that requires high throughput.
- Mode of Operation: is an AWS managed Event Bus that receives messages from AWS Services and third party units as well.
- Billed as per the number of messages.
Used in case of Event-driven architectures that require different loosely coupled units to share information/messages of various types. If your architecture demands API based communication specially external APIs then Event Bridge is your best option.