PubSubMessage memory leak on MessageDispatcher#processOutstandingMessage

[labianchin]

Note: we have an ongoing case with GCP PubSub engineering. Creating this issue so we can reference on the PR.

Please run down the following list and make sure you've tried the usual "quick fixes":

• Search the issues already opened: https://github.com/googleapis/java-pubsub/issues
• Check for answers on StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform

If you are still having issues, please include as much information as possible:

Environment details

1. Specify the API at the beginning of the title. For example, "BigQuery: ...").
   General, Core, and Other are also allowed as types
2. OS type and version: Linux
3. Java version: 11
4. version(s): 1.116.0 onwards

Steps to reproduce

See https://gist.github.com/labianchin/368194bae8c3fe09f9a2a3a0e75291c6

Code example

See https://gist.github.com/labianchin/368194bae8c3fe09f9a2a3a0e75291c6

Stack trace

Any relevant stacktrace here.

External references such as API reference guides

• ?

Any additional information below

On June 28th we made some library upgrades to our service.
The upgrade contained a bump of google-cloud-pubsub from 1.115.5 to 1.119.0.
The change caused some of the workloads to crash with OutOfMemoryError.
Not all workloads were affected, the ones affected the most consume from topics with average message size ranging from 10 KiB to 100 KiB.

Investigation:
When taking a heap histogram (with jmap -histio) we noticed significant more com.google.pubsub.v1.PubsubMessage and com.google.protobuf.ByteString$LiteralByteString object when using newer google-cloud-pubsub library version.

Hypothesis: a) Could these long changes have introduced some issue #1022? b) Perhaps a memory leak was introduced?

[labianchin]

@mmicatka @maheshgattani , for visibility.