本文整理汇总了Python中airflow.hooks.hive_hooks.HiveMetastoreHook.get_partitions方法的典型用法代码示例。如果您正苦于以下问题：Python HiveMetastoreHook.get_partitions方法的具体用法？Python HiveMetastoreHook.get_partitions怎么用？Python HiveMetastoreHook.get_partitions使用的例子？那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类airflow.hooks.hive_hooks.HiveMetastoreHook的用法示例。
在下文中一共展示了HiveMetastoreHook.get_partitions方法的1个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Python代码示例。
示例1: closest_ds_partition
# 需要导入模块: from airflow.hooks.hive_hooks import HiveMetastoreHook [as 别名]
# 或者: from airflow.hooks.hive_hooks.HiveMetastoreHook import get_partitions [as 别名]
def closest_ds_partition(
table, ds, before=True, schema="default",
metastore_conn_id='metastore_default'):
"""
This function finds the date in a list closest to the target date.
An optional parameter can be given to get the closest before or after.
:param table: A hive table name
:type table: str
:param ds: A datestamp ``%Y-%m-%d`` e.g. ``yyyy-mm-dd``
:type ds: list[datetime.date]
:param before: closest before (True), after (False) or either side of ds
:type before: bool or None
:returns: The closest date
:rtype: str or None
>>> tbl = 'airflow.static_babynames_partitioned'
>>> closest_ds_partition(tbl, '2015-01-02')
'2015-01-01'
"""
from airflow.hooks.hive_hooks import HiveMetastoreHook
if '.' in table:
schema, table = table.split('.')
hh = HiveMetastoreHook(metastore_conn_id=metastore_conn_id)
partitions = hh.get_partitions(schema=schema, table_name=table)
if not partitions:
return None
part_vals = [list(p.values())[0] for p in partitions]
if ds in part_vals:
return ds
else:
parts = [datetime.datetime.strptime(pv, '%Y-%m-%d')
for pv in part_vals]
target_dt = datetime.datetime.strptime(ds, '%Y-%m-%d')
closest_ds = _closest_date(target_dt, parts, before_target=before)
return closest_ds.isoformat()