Cloud Pub/Sub

Pub/Sub, which stands for Publisher/Subscriber.Publish/subscribe messaging, or pub/sub messaging, is a form of asynchronous service-to-service communication used in serverless and microservices architectures. In a pub/sub model, any message published to a topic is immediately received by all of the subscribers to the topic. In this blog we will be discussing 4 major types of cloud Pub/Sub.

Types of Cloud Pub/Sub:

1]Azure Pub/Sub service:

Pub/Sub is an asynchronous communication mechanism for exchanging messages between applications, as well as the most reliable method for communicating between microservices.

The Azure Web Pub/Sub service is suitable for any application that requires real-time content changes. Here are some nice examples of how to utilise the Azure Web Pub/Sub service:

• High frequency data updates: gaming, voting, polling, auction.
• Live dashboards and monitoring: company dashboard, financial market data, instant sales update, multiplayer game leaderboard, and IoT monitoring.
• Cross-platform live chat: live chat room, chat bot, on-line customer support, real-time shopping assistant, messenger, in-game chat, and so on.
• Real-time location on map: logistic tracking, delivery status tracking, transportation status updates, GPS apps.
• Real-time targeted ads: personalized real-time push ads and offers, interactive ads.
• Collaborative apps: co authoring, whiteboard apps and team meeting software.
• Push instant notifications: social network, email, game, travel alert.
• Real-time broadcasting: live audio/video broadcasting, live captioning, translating, events/news broadcasting.
• IoT and connected devices: real-time IoT metrics, remote control, real-time status, and location tracking.
• Automation: real-time trigger from upstream events.

Pub/Sub Design:

The above Pattern shows the components of Pub/Sub Design.

• An input messaging channel used by the sender.The sender uses a recognised message format to bundle events into messages, which are then sent via the input channel. In this design, the sender is also known as the publisher.
• Each consumer has one output message channel. Subscribers are the people who use the service.
• A system for replicating each message from the input channel to all subscribers interested in that message’s output channels. An intermediary, such as a message broker or an event bus, is usually in charge of this activity.

When to use the Pub/Sub design:

• An application must disseminate information to a large number of people.
• Consumers can get information from an application without having to respond in real time.
• The systems that are being integrated are intended to provide a data consistency model in the future.
• An application must provide information to several consumers, each of whom may have different availability or uptime requirements than the sender.

2] Amazon Web Services

PubSub architecture:

The PubSub architecture is a common way to create isolated inter-service communication between multiple services. This allows you to dynamically add, remove, modify, and notify multiple services at the same time. In the cloud, this pattern has additional benefits. With multiple dynamically allocated resources, all services need to know to function properly, with a single intermediary address, it’s much easier to handle them all.

Instead of implementing your own PubSub solutions, AWS has a number of managed services that can be used as a backbone of PubSub.

AWS provides three main services to help you implement PubSub in your account.

Kinesis

Kinesis is the AWS version of Kafka. It is a very efficient and highly scalable system for collecting, processing and analyzing real-time streaming data. The Kinesis feature can be used as a PubSub mechanism, but because the service is designed for a completely different purpose (such as streaming video), the infrastructure is much more complex than the PubSub mechanism required. Also, there is no seamless connection with other AWS services (SQS queues, mail) that other services have. Therefore, it is not recommended to be used as a PubSub backbone.

SNS- Amazon Simple Notification Service

(Amazon SNS) is a web service that allows applications, end users, and devices to send and receive notifications instantly from the cloud. SNS is a “push” service that immediately triggers all topic subscribers when a message is sent to a topic (average delay 25ms). In addition, SNS supports message attributes and subscription filtering. So, for example, if you add a type attribute to every message you publish on social media, you can use it to subscribe to specific subscribers (Lambda functions, SQS queues, email services) only to specific message types. It limits the number of unnecessary calls and automates the process.

SNS workflow:

Event bridge:

EventBridge is a new version of CloudWatch Events on AWS. In contrast to social networks created as notification services for some protocols (HTTP, SMS, push notifications, email, etc.), EventBridge is pre-designed as an event bus, providing more event-oriented options. EventBridge does not support direct publishing to other protocols and is limited to 5 Destinations per Rule due to its nature as an Event Bus. On the plus side, it supports publishing directly to 17 (and rising) AWS destinations, as well as receiving messages from external software services (such as Zendesk) and defining message schemas and performing basic message transformations.

Benefits of AWS Pub/Sub Messaging:

Eliminate Polling: Pub / Sub messaging offers great benefits to developers building applications that rely on real-time events. Message topics enable immediate push-based delivery, eliminating the need for message consumers to regularly check for new information or updates.

Dynamic Targeting:Pub / Sub makes service discovery easier, more natural, and less error-prone. Instead of maintaining a list of peers to which your application can send messages, publishers simply post messages to topics.Subscribers can change, upgrade, multiply, or disappear, and the system adapts dynamically.

Decouple and Scale Independently- Pub/Sub makes the software more flexible. Publishers and Subscribers are separate and operate independently of each other, allowing you to grow and scale them independently.

Simplify Communication:Communication and integration code is one of the hardest to write. Publishing Subscriptions model reduces complexity by eliminating all point-to-point connections with a single connection to the message subject, which manages subscriptions to decide which notifications to send to which endpoints.

How to use Pub/sub messaging:

Parallel Asynchronous Processing

• Multiple workers can be triggered to do necessary but unrelated activities at the same time by events published to a message topic.

Application and System Alerts

• Instantly supply important updates and asynchronous occasion notifications to affected software additives and end-users.

Manage Workflows

• Events may be relayed across applications, data can be moved between data stores, distributed caches can be refreshed, and business records can be updated.

Balance Workloads

• Using message queuing, you can break up a larger task into many smaller ones and divide the work among numerous employees, or you can batch up tasks for bulk processing.

Log to Multiple Systems

• Record events for subsequent analysis, collect logs for operations and security, or archive for backup or compliance purposes.

Use Fanout for Replication

• Data can be replicated between production and development environments, or live data can be used for development and testing.

Coordinate Serverless Applications

• To coordinate the components of your serverless application, fan out asynchronous event notifications to distributed functions and message queues.

Stream IoT Data

• Devices may simply broadcast data to backend systems or to each other using Pub/Sub messaging, which is a very powerful way for IoT devices to interact.

3] Google

Pub/Sub, which stands for Publisher/Subscriber, allows services to interact asynchronously with latencies of 100 milliseconds or less. Pub/Sub is used to ingest and disseminate data in streaming analytics and data integration pipelines. It works equally well as messaging-oriented middleware for service integration or as a queue for job parallelization.Publishers transmit events to the Pub/Sub service without knowing how or when they will be handled. Pub/Sub then distributes events to any services that must respond to them. When compared to systems interacting via RPCs, where publishers must wait for subscribers to get data, such asynchronous integration boosts the system’s overall flexibility and robustness.

Common use cases

• Data from IoT devices is being streamed. A home sensor, for example, can send data to backend servers stored in the cloud.
• Distributed caches are being refreshed. An application, for example, can publish invalidation events to update the IDs of objects that have changed.
• Load balancing is important for reliability. For example, instances of a service may be deployed across several zones on Compute Engine yet subscribe to the same topic. When a service fails in one zone, the others can instantly take over the burden.
• User interaction and server events are being ingested. You may employ user interaction events from end-user apps or server events from your system by sending them to Pub/Sub and then using a stream processing tool (such as Dataflow) to transport them to BigQuery, Bigtable, Cloud Storage, and other databases.
• Pub/Sub allows you to collect events from several clients at the same time.Event dispersal in real time. Events, whether raw or processed, may be made accessible for real-time processing to numerous apps within your team and company. This facilitates the use of an “enterprise event bus” and event-driven application design principles.
• Pub/Sub integrates with a variety of Google services that export events to Pub/Sub.Data replication between databases. Change events from databases are routinely sent via Pub/Sub. These occurrences can be

Pub/Sub or Pub/Sub Lite

Pub/Sub is made up of two services:

• Pub/Sub: Most users and apps should utilize the Pub/Sub service as the default. It has the best dependability and the most integrations, as well as automated capacity management.
• Pub/Sub Lite service: A distinct but comparable communications service designed to be inexpensive. It provides zonal storage and necessitates the pre-provisioning and management of storage and throughput capacity.

Comparing Pub/Sub to other messaging technologies

Pub/Sub combines the horizontal scalability of Apache Kafka and Pulsar with classic messaging middleware capabilities like dead-letter queues and filtering found in Apache ActiveMQ and RabbitMQ.

Per-message parallelism is another characteristic borrowed from messaging middleware by Pub/Sub (rather than partition-based). Pub/Sub “sends” individual messages to subscriber clients and then tracks whether or not each message is properly digested.

Other horizontally scalable messaging systems, on the other hand, employ partitions for horizontal scaling. This compels subscribers to process messages in each partition in the order they were received, and it restricts the number of concurrent clients to the number of partitions. Per-message processing increases the parallelism of subscriber programmes and contributes to publisher/subscriber independence.

Core concepts:

• Topic: Publishers send messages to a designated resource called a topic.
• Subscription: A named resource that represents a stream of messages from a single subject that will be provided to the subscribing application.
• Message: A collection of data and (optional) characteristics sent by a publisher to a topic and finally distributed to subscribers.
• Subscriber: An application that has subscribed to a topic(s) in order to receive messages from it.
• Acknowledgement (or “ack”): A signal given by a subscriber to Pub/Sub after it has successfully received a message. Messages that have been acknowledged are deleted from the subscription’s message queue.
• Push and pull: The two message delivery mechanisms are push and pull. Messages are sent to a subscriber either by Pub/Sub pushing them to the subscriber’s preferred endpoint or by the subscriber fetching them from the service.

The figure below depicts how a message travels from a publisher to a subscriber.

4]Salesforce

Introducing the Pub/Sub API

Salesforce introduced a new Pub/Sub API. This new API combines the power of publish & subscribe in an interface. It provides client-driven flow control for event delivery. The Pub/Sub API can be used with 11 different coding languages. Beyond making it easier to integrate Salesforce with external systems, we want to offer an API that’s tailored to pub/sub & builds on our experience with the Streaming API.

• Publish-subscribe patterns and event-driven architectures are a popular choice for Salesforce and customers to create dynamic and flexible solutions that perform at scale.
• Over the last some years, we’ve seen tremendous growth in these all patterns.
• The Salesforce Event Bus is a multi-tenant(one resource multiple tenants) eventing service built on Apache Kafka.

All this growth has led Salesforce to think about the runtime for platform events and Change Data Capture. These all products have been operating at scale for the last few years as a part of the core Salesforce platform. That means the Salesforce Event Bus has shared the same infrastructure as products like Sales Cloud as well as Service Cloud while powering pub/sub. This architecture has been extremely successful, but as Salesforce has grown and acquired new companies like Tableau,MuleSoft, the time has come for a new change.

Communicate Across the DOM

There are two ways to communicate between components in Salesforce

1. Lightning message service (lightning/messageService),
2. publish-subscribe pattern.

Containers that don’t support Lightning Messaging Service, we can use the pubsub module.

In a publish-subscribe pattern, one component publishes an event and other components subscribe to receive and handle the event. Every component that subscribes to the event receives the respective event. This pubsub module restricts events to a single page.

In the above image, one component is acting as publisher component .There are three different component acting as subscriber components. Between Publisher & Subscribers component there is a channel known as Pub-Sub model.

• Pub-Sub is a JavaScript file containing multiple methods
• Pub-Sub is like Application event in Component like Aura

Example

Publish–Subscribe Pattern in Lightning Web Components (pubsub)

• How do components communicate with each other

To communicate between components, we use a singleton library that follows the publish-subscribe pattern.

In a publish-subscribe pattern, one component publishes an event and then the other components subscribe to receive and handle the event. Every component that subscribes to the event receives the respective event. Let’s say, you have two components to a Lightning page in App Builder, then use this pubsub module .So the you can send events between them. You have to add PubSub to by yourself.

1. Fire an event

To fire an even using Publish–Subscribe Pattern you have to do following things:.

import the two lines shown below in the js file of the component from where you want to fire the event, import{CurrentPageReference}

from ‘lightning/navigation’;

import { fireEvent }

from ‘c/pubsub’;

When the required condition is met then use the below code to fire the event.

fireEvent({pageReference}, {eventName}, {data})

• Subscribe and Receive the Event

In the component import the following, where you want to subscribe and receive event,

import{CurrentPageReference}

from ‘lightning/navigation’;

import { registerListener,

unregisterAllListeners }

from ‘c/pubsub’;

connectedCallback() {

}

disconnectedCallback() {

}

• In connectedCallback we subscribe to the event.
• In disconnectedCallback we unsubscribe.

Authors:

Riya Kaktikar ,Ankita Karwankar, Shraddha Kshirsagar, Rujuta Kumbhare