【Paddle打比赛】全球人工智能技术创新大赛-商品标题实体识别竞赛_副本

比赛地址：https://www.heywhale.com/org/gaiic2022/competition/area/620b34ed28270b0017b823ad/content/2

京东商品标题包含大量关键信息，商品标题的实体识别是NLP应用程序中的一项核心基本任务可以种下游场景中重复使用。从标题文本中准确提取与商品相关的实体，可以提高用户体验和平台效率，如搜索和推荐。本主题要求玩家使用该模型从商品标题文本中提取实体。
与传统的实体提取不同，JD.COM商品标题文本的实体密度高，实体粒度细，点。

在2017年之前，工业界和学术界抗NLP文本处理依赖于序列模型Recurrent Neural Network (RNN).

近年来，随着深度学习的发展，模型参数的数量迅速增加。为了训练这些参数，需要更大的数据集来避免过拟合。然而，对大多数人来说NLP任务来说，构建大规模的标注数据集非常困难（成本过高），特别是对于句法和语义相关的任务。相比之下，大规模未标注语料库的建设相对容易。为了利用这些数据，我们可以先从中学习一个好的表达，然后将这些表达应用到其他任务中。最近的研究表明，基于大规模未标记语料库的预训练模型（Pretrained Models, PTM) 在NLP任务表现良好。

近年来，大量研究表明，基于大型语料库的预训练模型（Pretrained Models, PTM）学习一般语言表达有利于下游NLP同时，可以避免从零开始训练模型。随着计算能力的发展，深度模型(即 Transformer）训练技巧的增强使得 PTM 从浅到深不断发展。

在此，使用ERNIE完成序列标注任务的预训练模型。

# 训练集查看 !head -n20 data/data129945/train_500.txt 

手 B-40 机 I-40 三 B-4 脚 I-4 架 I-4 网 B-14 红 I-14 直 B-5 播 I-5 支 B-4 架 I-4 桌 B-7 面 I-7 自 B-4 拍 I-4 杆 I-4 蓝 B-11 牙 I-11 遥 B-11 控 I-11 

PaddleNLP格式是记录一个记录的样式，需要调整原始数据的格式。

import os   def format_data(source_filename, target_filename):     datalist = []     with open(source_filename, 'r', encoding='utf-8') as f:         lines = f.readlines()     words = ''     labels = ''     word=''     label=''     flag = 0     
       
        for line 
        in lines
        : 
        if line 
        == 
        '\n'
        : item 
        = words 
        + 
        '\t' 
        + labels 
        + 
        '\n' 
        # print(item) datalist
        .append
        (item
        ) words 
        = 
        '' labels 
        = 
        '' flag 
        = 
        0 
        continue 
        elif line 
        == 
        ' O\n'
        : word 
        = 
        ' ' label 
        = 
        'O' 
        else
        : word
        , label 
        = line
        .strip
        (
        '\n'
        )
        .split
        (
        ' '
        ) 
        # 使用\002是为了区分空格 
        if flag 
        == 
        1
        : words 
        = words 
        + 
        '\002' 
        + word labels 
        = labels 
        + 
        '\002' 
        + label 
        else
        : words 
        = words 
        + word labels 
        = labels 
        + label flag 
        = 
        1 
        # 添加最后一行数据 item 
        = words 
        + 
        '\t' 
        + labels 
        + 
        '\n' datalist
        .append
        (item
        ) 
        with 
        open
        (target_filename
        , 
        'w'
        , encoding
        =
        'utf-8'
        ) 
        as f
        : lines 
        = f
        .writelines
        (datalist
        ) 
        print
        (f
        '{source_filename}文件格式转换完毕，保存为{target_filename}'
        ) 
       

format_data('data/data129945/train_500.txt', 'train_500_converted.txt')

data/data129945/train_500.txt文件格式转换完毕，保存为train_500_converted.txt

!head train_500_converted.txt

按8:2进行分割，具体如下：

# 前400条设为训练集
!head -n400 train_500_converted.txt >train.txt

# 后100条设为测试集
!tail -n100 train_500_converted.txt >val.txt

!pip install -U pip --user
!pip install --upgrade paddlenlp

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: pip in ./.data/webide/pip/lib/python3.7/site-packages (22.0.3)
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: paddlenlp in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (2.1.1)
Collecting paddlenlp
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/2b/ec/927e81ad9c349980b1076b2721adcc3c1bbb8f0f432af21995692350c05a/paddlenlp-2.2.4-py3-none-any.whl (1.1 MB)
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m1.1/1.1 MB[0m [31m7.3 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0mm
[?25hRequirement already satisfied: seqeval in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (1.2.2)
Requirement already satisfied: h5py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (2.9.0)
Requirement already satisfied: multiprocess in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.70.11.1)
Requirement already satisfied: colorama in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.4.4)
Requirement already satisfied: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (4.1.0)
Requirement already satisfied: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.42.1)
Requirement already satisfied: six in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from h5py->paddlenlp) (1.16.0)
Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from h5py->paddlenlp) (1.19.5)
Requirement already satisfied: dill>=0.3.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from multiprocess->paddlenlp) (0.3.3)
Requirement already satisfied: scikit-learn>=0.21.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from seqeval->paddlenlp) (0.24.2)
Requirement already satisfied: scipy>=0.19.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp) (1.6.3)
Requirement already satisfied: joblib>=0.11 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp) (0.14.1)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp) (2.1.0)
Installing collected packages: paddlenlp
  Attempting uninstall: paddlenlp
    Found existing installation: paddlenlp 2.1.1
    Uninstalling paddlenlp-2.1.1:
      Successfully uninstalled paddlenlp-2.1.1
Successfully installed paddlenlp-2.2.4

from functools import partial

import paddle
from paddlenlp.datasets import MapDataset
from paddlenlp.data import Stack, Tuple, Pad
from paddlenlp.transformers import ErnieTokenizer, ErnieForTokenClassification
from paddlenlp.metrics import ChunkEvaluator
from utils import convert_example, evaluate, predict, load_dict

/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddlenlp/transformers/funnel/modeling.py:30: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Iterable

def load_dataset(datafiles):
    def read(data_path):
        with open(data_path, 'r', encoding='utf-8') as fp:
            next(fp)  # Skip header
            for line in fp.readlines():
                words, labels = line.strip('\n').split('\t')
                words = words.split('\002')
                labels = labels.split('\002')
                yield words, labels

    if isinstance(datafiles, str):
        return MapDataset(list(read(datafiles)))
    elif isinstance(datafiles, list) or isinstance(datafiles, tuple):
        return [MapDataset(list(read(datafile))) for datafile in datafiles]   

# 创建dataset
train_ds, dev_ds = load_dataset(datafiles=(
        './train.txt', './val.txt'))

for i in range(5):
    print(train_ds[i])

(['牛', '皮', '纸', '袋', '手', '提', '袋', '定', '制', 'l', 'o', 'g', 'o', '烘', '焙', '购', '物', '服', '装', '包', '装', '外', '卖', '打', '包', '袋', '子', '礼', '品', '袋', '纸', '质', ' ', '黑', '色', ' ', '3', '2', '*', '1', '1', '*', '2', '5', ' ', '大', '横', '1', '0', '0', '个'], ['B-4', 'I-4', 'I-4', 'I-4', 'B-4', 'I-4', 'I-4', 'B-29', 'I-29', 'I-29', 'I-29', 'I-29', 'I-29', 'B-9', 'I-9', 'B-5', 'I-5', 'B-40', 'I-40', 'B-4', 'I-4', 'B-40', 'I-40', 'B-5', 'I-5', 'B-4', 'I-4', 'B-4', 'I-4', 'I-4', 'B-12', 'I-12', 'O', 'B-16', 'I-16', 'O', 'B-18', 'I-18', 'I-18', 'I-18', 'I-18', 'I-18', 'I-18', 'I-18', 'O', 'B-13', 'I-13', 'B-18', 'I-18', 'I-18', 'I-18'])
(['彩', '色', '金', '属', '镂', '空', '鱼', '尾', '夹', '长', '尾', '夹', ' ', '手', '帐', '设', '计', '绘', '图', '文', '具', '收', '纳', ' ', '夹', '子', ' ', '鱼', '尾', '夹', '炫', '彩', '大', '号'], ['B-16', 'I-16', 'B-12', 'I-12', 'B-13', 'I-13', 'B-4', 'I-4', 'I-4', 'B-4', 'I-4', 'I-4', 'O', 'B-13', 'I-13', 'B-5', 'I-5', 'B-5', 'I-5', 'B-4', 'I-4', 'B-11', 'I-11', 'O', 'B-4', 'I-4', 'O', 'B-4', 'I-4', 'I-4', 'B-13', 'I-13', 'B-13', 'I-13'])
(['B', 'o', 's', 'e', ' ', 'S', 'o', 'u', 'n', 'd', 'S', 'p', 'o', 'r', 't', '', 'F', 'r', 'e', 'e', ' ', '真', '无', '线', '蓝', '牙', '耳', '机', ' ', '运', '动', '耳', '机', ' ', '博', '士', '防', '掉', '落', '耳', '塞', ' ', '黑', '色'], ['B-1', 'I-1', 'I-1', 'I-1', 'O', 'B-3', 'I-3', 'I-3', 'I-3', 'I-3', 'I-3', 'I-3', 'I-3', 'I-3', 'I-3', 'I-3', 'I-3', 'I-3', 'I-3', 'I-3', 'O', 'O', 'B-11', 'I-11', 'B-4', 'I-4', 'I-4', 'I-4', 'O', 'B-5', 'I-5', 'B-4', 'I-4', 'O', 'O', 'O', 'B-11', 'I-11', 'I-11', 'B-4', 'I-4', 'O', 'B-16', 'I-16'])
(['壁', '挂', '炉', '专', '用', '水', '空', '调', '散', '热', '器', '带', '风', '扇', '暖', '气', '片', '水', '暖', '空', '调', '明', '装', '吹', '风', '机', '盘', '管', '家', '用', ' ', '流', '线', '8', '5', '#', '6', '进', '6', '出', '2', '4', '铜', '管', '(', '左', ')', '带', '软', '管'], ['B-4', 'I-4', 'I-4', 'O', 'O', 'B-4', 'I-4', 'I-4', 'B-4', 'I-4', 'I-4', 'B-22', 'I-22', 'I-22', 'B-4', 'I-4', 'I-4', 'B-4', 'I-4', 'I-4', 'I-4', 'B-13', 'I-13', 'B-4', 'I-4', 'I-4', 'B-13', 'I-13', 'B-7', 'I-7', 'O', 'O', 'O', 'O', 'O', 'O', 'B-11', 'I-11', 'B-11', 'I-11', 'O', 'O', 'B-40', 'I-40', 'O', 'O', 'O', 'O', 'B-40', 'I-40'])
(['爱', '好', '油', '画', '棒', '儿', '童', '可', '水', '洗', '水', '溶', '性', '画', '画', '笔', '小', '学', '生', '2', '4', '色', '3', '6', '色', '4', '8', '色', '手', '绘', '彩', '绘', '易', '擦', '幼', '儿', '园', '宝', '宝', '旋', '转', '蜡', '笔', ' ', '6', '8', '0', '8', '1', '旋', '转', '焕', '彩', '棒', '2', '4', '色', '（', '水', '溶', '性', '）'], ['O', 'O', 'B-4', 'I-4', 'I-4', 'B-8', 'I-8', 'B-11', 'I-11', 'I-11', 'B-12', 'I-12', 'I-12', 'B-4', 'I-4', 'I-4', 'B-8', 'I-8', 'I-8', 'B-18', 'I-18', 'I-18', 'B-18', 'I-18', 'I-18', 'B-18', 'I-18', 'I-18', 'B-5', 'I-5', 'B-5', 'I-5', 'B-11', 'I-11', 'B-9', 'I-9', 'I-9', 'B-8', 'I-8', 'O', 'O', 'B-4', 'I-4', 'O', 'O', 'O', 'O', 'O', 'O', 'B-5', 'I-5', 'B-4', 'I-4', 'I-4', 'B-18', 'I-18', 'I-18', 'O', 'B-12', 'I-12', 'I-12', 'O'])

每条数据包含一句文本和这个文本中每个汉字以及数字对应的label标签

# 生成标签
def gernate_dic(source_filename1,  target_filename):
    data_list=[]

    with open(source_filename1, 'r', encoding='utf-8') as f:
        lines=f.readlines()

    for line in lines:
        if line != '\n':
            dic=line.strip('\n').split(' ')[-1]
            if dic+'\n' not in data_list:
                data_list.append(dic+'\n')    


    with open(target_filename, 'w', encoding='utf-8') as f:
        lines=f.writelines(data_list)    

# 生成dic
gernate_dic('data/data129945/train_500.txt', 'mytag.dic')

# 查看生成的dic文件
!cat mytag.dic

B-40
I-40
B-4
I-4
B-14
I-14
B-5
I-5
B-7
I-7
B-11
I-11
B-13
I-13
B-8
I-8
O
B-16
I-16
B-29
I-29
B-9
I-9
B-12
I-12
B-18
I-18
B-1
I-1
B-3
I-3
B-22
I-22
B-37
I-37
B-39
I-39
B-10
I-10
B-36
I-36
B-34
I-34
B-31
I-31
B-38
I-38
B-54
I-54
B-6
I-6
B-30
I-30
B-15
I-15
B-2
I-2
B-49
I-49
B-21
I-21
B-47
I-47
B-23
I-23
B-20
I-20
B-50
I-50
B-46
I-46
B-41
I-41
B-43
I-43
B-48
I-48
B-19
I-19
B-52
I-52

由于是放出来的测试用数据集，不够全面，所以只有81个。

预训练模型ERNIE对中文数据的处理是以字为单位。PaddleNLP对于各种预训练模型已经内置了相应的tokenizer。指定想要使用的模型名字即可加载对应的tokenizer。

tokenizer作用为将原始输入文本转化成模型model可以接受的输入数据形式。

label_vocab = load_dict('mytag.dic')
tokenizer = ErnieTokenizer.from_pretrained('ernie-1.0')

trans_func = partial(convert_example, tokenizer=tokenizer, label_vocab=label_vocab)

train_ds.map(trans_func)
dev_ds.map(trans_func)
print (train_ds[0])

[2022-03-01 23:50:35,030] [    INFO] - Already cached /home/aistudio/.paddlenlp/models/ernie-1.0/vocab.txt


([1, 961, 760, 1067, 1930, 247, 160, 1930, 91, 108, 2310, 2744, 2715, 2744, 2782, 3718, 817, 122, 231, 371, 392, 371, 137, 1225, 445, 392, 1930, 85, 953, 100, 1930, 1067, 207, 17963, 669, 199, 17963, 284, 249, 17963, 208, 208, 17963, 249, 317, 17963, 19, 1151, 208, 540, 540, 27, 2], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 53, [16, 2, 3, 3, 3, 2, 3, 3, 19, 20, 20, 20, 20, 20, 21, 22, 6, 7, 0, 1, 2, 3, 0, 1, 6, 7, 2, 3, 2, 3, 3, 23, 24, 16, 17, 18, 16, 25, 26, 26, 26, 26, 26, 26, 26, 16, 12, 13, 25, 26, 26, 26, 16])

数据读入

使用paddle.io.DataLoader接口多线程异步加载数据。

ignore_label = -1
batchify_fn = lambda samples, fn=Tuple(
    Pad(axis=0, pad_val=tokenizer.pad_token_id),  # input_ids
    Pad(axis=0, pad_val=tokenizer.pad_token_type_id),  # token_type_ids
    Stack(),  # seq_len
    Pad(axis=0, pad_val=ignore_label)  # labels
): fn(samples)

train_loader = paddle.io.DataLoader(
    dataset=train_ds,
    batch_size=200,
    return_list=True,
    collate_fn=batchify_fn)
dev_loader = paddle.io.DataLoader(
    dataset=dev_ds,
    batch_size=64,
    return_list=True,
    collate_fn=batchify_fn)

快递单信息抽取本质是一个序列标注任务，PaddleNLP对于各种预训练模型已经内置了对于下游任务文本分类Fine-tune网络。以下教程以ERNIE为预训练模型完成序列标注任务。

paddlenlp.transformers.ErnieForTokenClassification()一行代码即可加载预训练模型ERNIE用于序列标注任务的fine-tune网络。其在ERNIE模型后拼接上一个全连接网络进行分类。

paddlenlp.transformers.ErnieForTokenClassification.from_pretrained()方法只需指定想要使用的模型名称和文本分类的类别数即可完成定义模型网络。

# Define the model netword and its loss
model = ErnieForTokenClassification.from_pretrained("ernie-1.0", num_classes=len(label_vocab))

[2022-03-01 23:50:40,125] [    INFO] - Already cached /home/aistudio/.paddlenlp/models/ernie-1.0/ernie_v1_chn_base.pdparams
W0301 23:50:40.128793  4266 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1
W0301 23:50:40.132701  4266 device_context.cc:465] device: 0, cuDNN Version: 7.6.

PaddleNLP不仅支持ERNIE预训练模型，还支持BERT、RoBERTa、Electra等预训练模型。
下表汇总了目前PaddleNLP支持的各类预训练模型。您可以使用PaddleNLP提供的模型，完成文本分类、序列标注、问答等任务。同时我们提供了众多预训练模型的参数权重供用户使用，其中包含了二十多种中文语言模型的预训练权重。中文的预训练模型有bert-base-chinese, bert-wwm-chinese, bert-wwm-ext-chinese, ernie-1.0, ernie-tiny, gpt2-base-cn, roberta-wwm-ext, roberta-wwm-ext-large, rbt3, rbtl3, chinese-electra-base, chinese-electra-small, chinese-xlnet-base, chinese-xlnet-mid, chinese-xlnet-large, unified_transformer-12L-cn, unified_transformer-12L-cn-luge等。

更多预训练模型参考：PaddleNLP Transformer API。

更多预训练模型fine-tune下游任务使用方法，请参考：examples。

metric = ChunkEvaluator(label_list=label_vocab.keys(), suffix=True)
loss_fn = paddle.nn.loss.CrossEntropyLoss(ignore_index=ignore_label)
optimizer = paddle.optimizer.AdamW(learning_rate=2e-5, parameters=model.parameters())

模型训练的过程通常有以下步骤：

1. 从dataloader中取出一个batch data
2. 将batch data喂给model，做前向计算
3. 将前向计算结果传给损失函数，计算loss。将前向计算结果传给评价方法，计算评价指标。
4. loss反向回传，更新梯度。重复以上步骤。

每训练一个epoch时，程序将会评估一次，评估当前模型训练的效果。

step = 0
for epoch in range(50):
    for idx, (input_ids, token_type_ids, length, labels) in enumerate(train_loader):
        logits = model(input_ids, token_type_ids)
        loss = paddle.mean(loss_fn(logits, labels))
        loss.backward()
        optimizer.step()
        optimizer.clear_grad()
        step += 1
        print("epoch:%d - step:%d - loss: %f" % (epoch, step, loss))
    evaluate(model, metric, dev_loader)

    paddle.save(model.state_dict(),
                './checkpoint/model_%d.pdparams' % step)

训练日志

由日志可见，F1为0.838540，因为数据只有500条，还不是正式数据，所以准确率较低。

epoch:45 - step:91 - loss: 1.138152
epoch:45 - step:92 - loss: 1.139734
eval precision: 0.821243 - recall: 0.839224 - f1: 0.830136
epoch:46 - step:93 - loss: 1.118141
epoch:46 - step:94 - loss: 1.119747
eval precision: 0.821822 - recall: 0.841101 - f1: 0.831350
epoch:47 - step:95 - loss: 1.089805
epoch:47 - step:96 - loss: 1.086849
eval precision: 0.820427 - recall: 0.841727 - f1: 0.830940
epoch:48 - step:97 - loss: 1.051158
epoch:48 - step:98 - loss: 1.054089
eval precision: 0.827618 - recall: 0.845480 - f1: 0.836454
epoch:49 - step:99 - loss: 1.032197
epoch:49 - step:100 - loss: 1.033427
eval precision: 0.829306 - recall: 0.847982 - f1: 0.838540

!mkdir ernie_result
model.save_pretrained('./ernie_result')
tokenizer.save_pretrained('./ernie_result')

重启notebook，释放显存，开始训练。

训练保存好的训练，即可用于预测。如以下示例代码自定义预测数据，调用predict()函数即可一键预测。

import numpy as np
import paddle
from paddle.io import DataLoader
import paddlenlp as ppnlp
from paddlenlp.datasets import load_dataset
from paddlenlp.data import Stack, Tuple, Pad, Dict
from paddlenlp.datasets import MapDataset
from paddlenlp.transformers import ErnieTokenizer, ErnieForTokenClassification
from paddlenlp.metrics import ChunkEvaluator
from utils import convert_example, evaluate, predict, load_dict
from functools import partial

由于预测用数据未放出，此处使用val数据集替代

!head -n20 val.txt

def load_dataset(datafiles):
    def read(data_path):
        with open(data_path, 'r', encoding='utf-8') as fp:
            next(fp)  # Skip header
            for line in fp.readlines():
                words, labels = line.strip('\n').split('\t')
                words = words.split('\002')
                labels = labels.split('\002')
                yield words, labels

    if isinstance(datafiles, str):
        return MapDataset(list(read(datafiles)))
    elif isinstance(datafiles, list) or isinstance(datafiles, tuple):
        return [MapDataset(list(read(datafile))) for datafile in datafiles]   

test_ds = load_dataset(datafiles=('./val.txt'))

for i in range(20):
    print(test_ds[i])

(['尼', '凡', ' ', '1', '7', '.', '3', '英', '寸', '雷', '神', '9', '1', '1', '', 'P', 'l', 'u', 's', ' ', '召', '唤', '师', '笔', '记', '本', '电', '脑', '纯', '色', '碳', '纤', '维', '贴', '纸', '个', '性', '图', '案', '外', '壳', '保', '护', '贴', '膜', ' ', '图', '案', '款', '式', '二', ' ', 'A', 'C', 'D', '面'], ['B-1', 'I-1', 'O', 'B-54', 'I-54', 'I-54', 'I-54', 'I-54', 'I-54', 'B-37', 'I-37', 'B-38', 'I-38', 'I-38', 'I-38', 'I-38', 'I-38', 'I-38', 'I-38', 'O', 'B-38', 'I-38', 'I-38', 'B-40', 'I-40', 'I-40', 'I-40', 'I-40', 'B-16', 'I-16', 'B-12', 'I-12', 'I-12', 'B-4', 'I-4', 'B-14', 'I-14', 'O', 'O', 'B-7', 'I-7', 'B-4', 'I-4', 'I-4', 'I-4', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B-13', 'I-13', 'I-13', 'I-13'])
(['日', '本', 'u', 'n', 'i', '-', 'b', 'a', 'l', 'l', '三', '菱', 'p', 'i', 'n', ' ', '针', '管', '笔', '/', '针', '笔', ' ', '漫', '画', '设', '计', '勾', '线', '笔', ' ', '专', '业', '绘', '图', '制', '图', '笔', '防', '水', ' ', '0', '.', '3', 'm', 'm', ' ', '1