本文整理匯總了Python中data.Corpus方法的典型用法代碼示例。如果您正苦於以下問題：Python data.Corpus方法的具體用法？Python data.Corpus怎麽用？Python data.Corpus使用的例子？那麽, 這裏精選的方法代碼示例或許可以為您提供幫助。您也可以進一步了解該方法所在類data的用法示例。
在下文中一共展示了data.Corpus方法的3個代碼示例，這些例子默認根據受歡迎程度排序。您可以為喜歡或者感覺有用的代碼點讚，您的評價將有助於係統推薦出更棒的Python代碼示例。
示例1: main
# 需要導入模塊: import data [as 別名]
# 或者: from data import Corpus [as 別名]
def main(args):
if args.seed is not None:
random.seed(args.seed)
torch.manual_seed(args.seed)
cudnn.deterministic = True
warnings.warn('You have chosen to seed training. '
'This will turn on the CUDNN deterministic setting, '
'which can slow down your training considerably! '
'You may see unexpected behavior when restarting '
'from checkpoints.')
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print('=> load data...')
corpus = data.Corpus(args.data)
eval_batch_size = 10
train_loader = batchify(corpus.train, args.batch_size, device)
val_loader = batchify(corpus.valid, eval_batch_size, device)
ntokens = len(corpus.dictionary)
main_worker(train_loader, val_loader, ntokens, args, device) 示例2: update
# 需要導入模塊: import data [as 別名]
# 或者: from data import Corpus [as 別名]
def update(self, val, n=1):
self.val = val
self.sum += val * n
self.count += n
self.avg = self.sum / self.count
# corpus = data.Corpus(args.data) 開發者ID:nadavbh12，項目名稱:Character-Level-Language-Modeling-with-Deeper-Self-Attention-pytorch，代碼行數:10，代碼來源:main.py
示例3: load_corpus
# 需要導入模塊: import data [as 別名]
# 或者: from data import Corpus [as 別名]
def load_corpus(self):
self.corpus = Corpus(self.vocab, self.config.debug)
self.encoded_train = np.array(self.corpus.encoded_train)
self.encoded_dev = np.array(self.corpus.encoded_dev)
self.encoded_test = np.array(self.corpus.encoded_test)