本文整理汇总了Python中data.Corpus方法的典型用法代码示例。如果您正苦于以下问题：Python data.Corpus方法的具体用法？Python data.Corpus怎么用？Python data.Corpus使用的例子？那么, 这里精选的方法代码示例或许可以为您提供帮助。您也可以进一步了解该方法所在类data的用法示例。
在下文中一共展示了data.Corpus方法的3个代码示例，这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞，您的评价将有助于系统推荐出更棒的Python代码示例。
示例1: main
# 需要导入模块: import data [as 别名]
# 或者: from data import Corpus [as 别名]
def main(args):
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print('=> load data...')
corpus = data.Corpus(args.data)
eval_batch_size = 10
train_loader = batchify(corpus.train, args.batch_size, device)
val_loader = batchify(corpus.valid, eval_batch_size, device)
ntokens = len(corpus.dictionary)
main_worker(train_loader, val_loader, ntokens, args, device) 示例2: update
# 需要导入模块: import data [as 别名]
# 或者: from data import Corpus [as 别名]
def update(self, val, n=1):
self.val = val
self.sum += val * n
self.count += n
self.avg = self.sum / self.count
# corpus = data.Corpus(args.data) 开发者ID:nadavbh12，项目名称:Character-Level-Language-Modeling-with-Deeper-Self-Attention-pytorch，代码行数:10，代码来源:main.py
示例3: load_corpus
# 需要导入模块: import data [as 别名]
# 或者: from data import Corpus [as 别名]
def load_corpus(self):
self.corpus = Corpus(self.vocab, self.config.debug)
self.encoded_train = np.array(self.corpus.encoded_train)
self.encoded_dev = np.array(self.corpus.encoded_dev)
self.encoded_test = np.array(self.corpus.encoded_test)