Biomedical text mining is becoming increasingly important. Recently, biomedical pre-trained language models such as BioBERT and SciBERT, which can capture biomedical knowledge from text, have achieved promising results in biomedical NLP tasks. However, most biomedical pre-trained language models rely on the traditional masked language model (MLM) pre-training strategy, which cannot fully capture the semantic relations of context. It is challenging to learn biomedical knowledge via language models in the Chinese biomedical fields due to the lack...
