This document assumes that you are already familiar with the process of subscribing to a Pub/Sub topic and receiving messages in your subscriber client.
If you're new to Pub/Sub, see one of the Quickstart guides and learn how to run Pub/Sub using the console, gCloud CLI, or the client libraries.
Choose the right subscription
Pub/Sub offers standard subscriptions such as push and pull subscriptions. In addition to the standard subscriptions, Pub/Sub also offers export subscriptions that lets you store messages directly to a Google Cloud resource, without requiring Dataflow as an intermediary. For example, BigQuery subscriptions store messages in a BigQuery table.
Push subscriptions are recommended for the following scenarios:
You cannot include any code in your subscriber application that imports the client library as a dependency.
The subscriber client cannot make any outgoing requests.
You want to use the same instance to process messages from different topics and subscriptions where the subscriber client doesn't know the list of subscriptions.
For general cases, we recommend using the
high-level client library. If you're
instead using unary pull, don't set
returnImmediately
to true. Setting it to true adversely impacts pull performance.
The field returnImmediatelyis now deprecated.
To compare all the subscription types and choose the one that best fits your business needs, see the Pub/Sub subscription comparison table.
To learn about the benefits of an export subscription, see When to use an export subscription.
Process messages before acknowledging them
By default, Pub/Sub discards a message from a subscription after the message is acknowledged. If you don't process a message before sending an acknowledgment and the processing fails, the service does not redeliver the message. The exception is when you have configured retaining acknowledged messages or topic retention and you perform a seek operation.
If you have high-latency subscribers, you might need to set custom values for flow control and lease management.
Configure subscriber flow control for transient traffic spikes
Flow control on the subscriber side lets you prevent subscribers from being overloaded by traffic spikes. It can allow time for autoscaling mechanisms to respond to an increased load or it can spread the processing of the load over a longer period of time. The former method saves latency, while the latter saves costs.
To configure flow control, you must set
appropriate values for maximum outstanding messages and
total outstanding message bytes. The default values for these flow control
variables and the names of the variables might differ across client libraries.
Maximum outstanding messages defines the maximum number of messages delivered to the client for which Pub/Sub has not received acknowledgments or negative acknowledgments.
Total outstanding message bytes defines the maximum total size of messages delivered to the client for which Pub/Sub has not received acknowledgments or negative acknowledgments.
If the limit for one of these options is crossed, the subscriber client does not pull more messages. This behavior continues until the messages that are already pulled get acknowledged or negatively acknowledged. In this way, you can tradeoff throughput with the cost associated with running more subscribers.
Best practices for ordered messaging in subscribing
If you use message ordering, ensure the following:
Choose either StreamingPull or Pull subscriptions. For a push subscription, Pub/Sub supports only one outstanding message for each ordering key at a time. Sending parallel push requests in such a scenario would be similar to sending multiple batches of messages for the same ordering key to pull subscribers simultaneously. Therefore, push subscriptions are not recommended for topics where multiple messages are frequently published with the same ordering key or where latency is extremely important.
Enable message ordering in the subscription. On the publisher side, if you send messages with an ordering key and in the same region, you can configure the subscribers to receive those messages in order. On the subscriber side, enable the message ordering property for only those subscriptions for which you want to receive ordered messages. Depending on the property status, each subscription attached to the topic can determine if they need ordered delivery without impacting each other.
Acknowledge messages in order. When using ordered delivery, acknowledgments for later messages are not processed until acknowledgments for earlier messages are processed per ordering key. For example, if you have messages 1, 2, and 3 with the same ordering key and you receive them all and only acknowledge message 3, the service does not consider message 3 as acknowledged until messages 1 and 2 are also acknowledged. If the acknowledgements for messages 1 and 2 are never received, then messages 1, 2, and 3 are all redelivered.
Summary of best practices
The following table summarizes the best practices recommended in this document:
| Topic | Task |
|---|---|
| Choose a subscription type | Choose the right subscription type for your business needs. If supported by your subscription, also use the high-level client library. |
| Replay an acknowledged message | Process a message before you acknowledge it. Or, configure for a seek operation so that you don't lose acknowledged messages. |
| Flow control | Configure flow control in your subscriber settings to ensure that subscribers don't get overloaded until autoscaling kicks in or time passes. |
| Ordering messages | When using ordered messaging, choose StreamingPull or Pull, enable message ordering in the subscription, and acknowledge messages in order. |