5fe8d7b7f5770d82e4f3bfb489a67999d9442ce6,scripts/language_model/run_glue.py,,preprocess_data,#,285

Before Change


    test_tsv_list = test_tsv if isinstance(test_tsv, list) else [test_tsv]
    loader_test_list = []
    for segment, data in test_tsv_list:
        data_test = mx.gluon.data.SimpleDataset(pool.map(test_trans, data))
        loader_test = mx.gluon.data.DataLoader(data_test,
                                               batch_size=_dev_batch_size,
                                               num_workers=num_workers,
                                               shuffle=False,

After Change


    test_tsv_list = test_tsv if isinstance(test_tsv, list) else [test_tsv]
    loader_test_list = []
    for segment, data in test_tsv_list:
        data_test = mx.gluon.data.SimpleDataset(list(map(test_trans, data)))
        loader_test = mx.gluon.data.DataLoader(data_test,
                                               batch_size=dev_batch_size,
                                               num_workers=4,
                                               shuffle=False,
Italian Trulli
In pattern: SUPERPATTERN

Frequency: 3

Non-data size: 3

Instances


Project Name: dmlc/gluon-nlp
Commit Name: 5fe8d7b7f5770d82e4f3bfb489a67999d9442ce6
Time: 2020-01-22
Author: 798672141@qq.com
File Name: scripts/language_model/run_glue.py
Class Name:
Method Name: preprocess_data