本文整理汇总了Python中kafka.consumer.subscription_state.SubscriptionState.group_subscription方法的典型用法代码示例。如果您正苦于以下问题：Python SubscriptionState.group_subscription方法的具体用法？Python SubscriptionState.group_subscription怎么用？Python SubscriptionState.group_subscription使用的例子？那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类kafka.consumer.subscription_state.SubscriptionState的用法示例。
在下文中一共展示了SubscriptionState.group_subscription方法的3个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Python代码示例。
示例1: KafkaConsumer
# 需要导入模块: from kafka.consumer.subscription_state import SubscriptionState [as 别名]
# 或者: from kafka.consumer.subscription_state.SubscriptionState import group_subscription [as 别名]
#.........这里部分代码省略.........
pattern (str): Pattern to match available topics. You must provide
either topics or pattern, but not both.
listener (ConsumerRebalanceListener): Optionally include listener
callback, which will be called before and after each rebalance
operation.
As part of group management, the consumer will keep track of the
list of consumers that belong to a particular group and will
trigger a rebalance operation if one of the following events
trigger:
* Number of partitions change for any of the subscribed topics
* Topic is created or deleted
* An existing member of the consumer group dies
* A new member is added to the consumer group
When any of these events are triggered, the provided listener
will be invoked first to indicate that the consumer's assignment
has been revoked, and then again when the new assignment has
been received. Note that this listener will immediately override
any listener set in a previous call to subscribe. It is
guaranteed, however, that the partitions revoked/assigned
through this interface are from topics subscribed in this call.
"""
if not topics:
self.unsubscribe()
else:
self._subscription.subscribe(topics=topics, pattern=pattern, listener=listener)
# regex will need all topic metadata
if pattern is not None:
self._client.cluster.need_metadata_for_all = True
log.debug("Subscribed to topic pattern: %s", topics)
else:
self._client.set_topics(self._subscription.group_subscription())
log.debug("Subscribed to topic(s): %s", topics)
def subscription(self):
"""Get the current topic subscription.
Returns:
set: {topic, ...}
"""
return self._subscription.subscription
def unsubscribe(self):
"""Unsubscribe from all topics and clear all assigned partitions."""
self._subscription.unsubscribe()
self._coordinator.close()
self._client.cluster.need_metadata_for_all_topics = False
log.debug("Unsubscribed all topics or patterns and assigned partitions")
def _update_fetch_positions(self, partitions):
"""
Set the fetch position to the committed position (if there is one)
or reset it using the offset reset policy the user has configured.
Arguments:
partitions (List[TopicPartition]): The partitions that need
updating fetch positions
Raises:
NoOffsetForPartitionError: If no offset is stored for a given
partition and no offset reset policy is defined
"""
if self.config["api_version"] >= (0, 8, 1):
# refresh commits for all assigned partitions示例2: KafkaConsumer
# 需要导入模块: from kafka.consumer.subscription_state import SubscriptionState [as 别名]
# 或者: from kafka.consumer.subscription_state.SubscriptionState import group_subscription [as 别名]
#.........这里部分代码省略.........
trigger a rebalance operation if one of the following events
trigger:
* Number of partitions change for any of the subscribed topics
* Topic is created or deleted
* An existing member of the consumer group dies
* A new member is added to the consumer group
When any of these events are triggered, the provided listener
will be invoked first to indicate that the consumer's assignment
has been revoked, and then again when the new assignment has
been received. Note that this listener will immediately override
any listener set in a previous call to subscribe. It is
guaranteed, however, that the partitions revoked/assigned
through this interface are from topics subscribed in this call.
Raises:
IllegalStateError: if called after previously calling assign()
AssertionError: if neither topics or pattern is provided
TypeError: if listener is not a ConsumerRebalanceListener
"""
# SubscriptionState handles error checking
self._subscription.subscribe(topics=topics,
pattern=pattern,
listener=listener)
# regex will need all topic metadata
if pattern is not None:
self._client.cluster.need_all_topic_metadata = True
self._client.set_topics([])
log.debug("Subscribed to topic pattern: %s", pattern)
else:
self._client.cluster.need_all_topic_metadata = False
self._client.set_topics(self._subscription.group_subscription())
log.debug("Subscribed to topic(s): %s", topics)
def subscription(self):
"""Get the current topic subscription.
Returns:
set: {topic, ...}
"""
return self._subscription.subscription
def unsubscribe(self):
"""Unsubscribe from all topics and clear all assigned partitions."""
self._subscription.unsubscribe()
self._coordinator.close()
self._client.cluster.need_all_topic_metadata = False
self._client.set_topics([])
log.debug("Unsubscribed all topics or patterns and assigned partitions")
def metrics(self, raw=False):
"""Warning: this is an unstable interface.
It may change in future releases without warning"""
if raw:
return self._metrics.metrics
metrics = {}
for k, v in self._metrics.metrics.items():
if k.group not in metrics:
metrics[k.group] = {}
if k.name not in metrics[k.group]:
metrics[k.group][k.name] = {}
metrics[k.group][k.name] = v.value()
return metrics示例3: AIOKafkaConsumer
# 需要导入模块: from kafka.consumer.subscription_state import SubscriptionState [as 别名]
# 或者: from kafka.consumer.subscription_state.SubscriptionState import group_subscription [as 别名]
#.........这里部分代码省略.........
As part of group management, the consumer will keep track of
the list of consumers that belong to a particular group and
will trigger a rebalance operation if one of the following
events trigger:
* Number of partitions change for any of the subscribed topics
* Topic is created or deleted
* An existing member of the consumer group dies
* A new member is added to the consumer group
When any of these events are triggered, the provided listener
will be invoked first to indicate that the consumer's
assignment has been revoked, and then again when the new
assignment has been received. Note that this listener will
immediately override any listener set in a previous call
to subscribe. It is guaranteed, however, that the partitions
revoked/assigned
through this interface are from topics subscribed in this call.
Raises:
IllegalStateError: if called after previously calling assign()
AssertionError: if neither topics or pattern is provided
TypeError: if listener is not a ConsumerRebalanceListener
"""
# SubscriptionState handles error checking
self._subscription.subscribe(topics=topics,
pattern=pattern,
listener=listener)
# regex will need all topic metadata
if pattern is not None:
self._client.set_topics([])
log.debug("Subscribed to topic pattern: %s", pattern)
else:
self._client.set_topics(self._subscription.group_subscription())
log.debug("Subscribed to topic(s): %s", topics)
def subscription(self):
"""Get the current topic subscription.
Returns:
set: {topic, ...}
"""
return self._subscription.subscription
def unsubscribe(self):
"""Unsubscribe from all topics and clear all assigned partitions."""
self._subscription.unsubscribe()
self._client.set_topics([])
log.debug(
"Unsubscribed all topics or patterns and assigned partitions")
@asyncio.coroutine
def _update_fetch_positions(self, partitions):
"""
Set the fetch position to the committed position (if there is one)
or reset it using the offset reset policy the user has configured.
Arguments:
partitions (List[TopicPartition]): The partitions that need
updating fetch positions
Raises:
NoOffsetForPartitionError: If no offset is stored for a given
partition and no offset reset policy is defined
"""
if self._group_id is not None: