本文整理汇总了Python中allennlp.data.dataset_readers.dataset_reader.DatasetReader方法的典型用法代码示例。如果您正苦于以下问题：Python dataset_reader.DatasetReader方法的具体用法？Python dataset_reader.DatasetReader怎么用？Python dataset_reader.DatasetReader使用的例子？那么恭喜您, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类allennlp.data.dataset_readers.dataset_reader的用法示例。
在下文中一共展示了dataset_reader.DatasetReader方法的3个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Python代码示例。
示例1: __init__
# 需要导入模块: from allennlp.data.dataset_readers import dataset_reader [as 别名]
# 或者: from allennlp.data.dataset_readers.dataset_reader import DatasetReader [as 别名]
def __init__(self, model: Model, dataset_reader: DatasetReader) -> None:
super().__init__(model, dataset_reader)
# Due to a formatting issue, the word overlap model includes the newline character in the
# label string. So we first create a map from the stripped label to the model label and
# then use the model label to get the index of entailment class. It is designed in this
# manner to also automatically handle any new models that do not have this bug.
labels = {label.strip(): label
for label in self._model.vocab.get_index_to_token_vocabulary("labels").values()}
if "entails" in labels:
self._entailment_idx = self._model.vocab.get_token_index(labels["entails"], "labels")
elif "entailment" in labels:
self._entailment_idx = self._model.vocab.get_token_index(labels["entailment"], "labels")
else:
raise Exception("No label for entailment found in the label space: {}".format(
",".join(labels))) 示例2: __init__
# 需要导入模块: from allennlp.data.dataset_readers import dataset_reader [as 别名]
# 或者: from allennlp.data.dataset_readers.dataset_reader import DatasetReader [as 别名]
def __init__(self, model: Model, dataset_reader: DatasetReader) -> None:
super().__init__(model, dataset_reader)
labels = self._model.vocab.get_index_to_token_vocabulary("labels").values()
#
if "entails" in labels:
self._entailment_idx = self._model.vocab.get_token_index("entails", "labels")
elif "entailment" in labels:
self._entailment_idx = self._model.vocab.get_token_index("entailment", "labels")
else:
raise Exception("No label for entailment found in the label space: {}".format(
",".join(labels))) 示例3: __init__
# 需要导入模块: from allennlp.data.dataset_readers import dataset_reader [as 别名]
# 或者: from allennlp.data.dataset_readers.dataset_reader import DatasetReader [as 别名]
def __init__(self, model: Model, dataset_reader: DatasetReader) -> None:
super().__init__(model, dataset_reader)
self._stemmer = PorterStemmer()
self._stop_words = set(stopwords.words('english'))