e284fe0449413d11e7ade24582c673882714477d,scripts/bert/dataset.py,BERTTransform,__call__,#BERTTransform#Any#,90

Before Change


            padding_length = self._max_seq_length - valid_length
            // use padding tokens for the rest
            input_ids.extend([self._tokenizer.vocab["[PAD]"]] * padding_length)
            segment_ids.extend([self._tokenizer.vocab["[PAD]"]] * padding_length)

        return np.array(input_ids, dtype="int32"), np.array(valid_length, dtype="int32"),\
               np.array(segment_ids, dtype="int32")

After Change


            // use padding tokens for the input_ids and 0 for segment_ids
            input_ids.extend(
                [self._tokenizer.vocab[self._tokenizer.vocab.padding_token]] * padding_length)
            segment_ids.extend([0] * padding_length)

        return np.array(input_ids, dtype="int32"), np.array(valid_length, dtype="int32"),\
            np.array(segment_ids, dtype="int32")
Italian Trulli
In pattern: SUPERPATTERN

Frequency: 3

Non-data size: 3

Instances


Project Name: dmlc/gluon-nlp
Commit Name: e284fe0449413d11e7ade24582c673882714477d
Time: 2019-01-14
Author: 1026639175@qq.com
File Name: scripts/bert/dataset.py
Class Name: BERTTransform
Method Name: __call__


Project Name: chainer/chainercv
Commit Name: a7706fbde22887909db42f96a696437c084c05db
Time: 2017-05-31
Author: Hakuyume@users.noreply.github.com
File Name: chainercv/evaluations/eval_detection_voc.py
Class Name:
Method Name: eval_detection_voc


Project Name: dmlc/gluon-nlp
Commit Name: c84c7a0f0e2657f9772c2301dfcdc014b909fa9c
Time: 2019-01-22
Author: 1026639175@qq.com
File Name: scripts/bert/dataset.py
Class Name: BERTTransform
Method Name: __call__