PatentTransformer-1.5: Measuring Patent Claim Generation by Span Relevancy

Jieh-Sheng Lee*, Jieh Hsiang


研究成果: Conference contribution同行評審

1 引文 斯高帕斯(Scopus)


PatentTransformer is our codename for patent text generation based on Transformer-based models. Our long-term goal of patent claim generation is to realize “augmented inventing” for inventors by leveraging new Deep Learning techniques. We envision the possibility of building an “auto-complete” function for inventors to conceive better inventions in the era of artificial intelligence. In order to generate patent claims with reasonable quality, a fundamental question is how to measure the quality. In PatentTransformer-1.5, we tackle the problem from the perspective of claim span relevancy as a proof of concept. Patent claim language was rarely explored in the NLP field. In this work, we propose a span-based approach and a generic framework to measure patent claim generation quantitatively. In order to study the effectiveness of patent claim generation, we define a metric to measure whether two consecutive spans in a generated patent claims are relevant. We treat such relevancy measurement as a span-pair classification problem, following the concept of natural language inference. Technically, the span-pair classifier is implemented by fine-tuning a pre-trained language model. The patent claim generation is implemented by fine-tuning the other pre-trained model. Specifically, we fine-tune a pre-trained Google BERT model to measure the patent claim spans generated by a fine-tuned OpenAI GPT-2 model. In this way, we re-use two of the state-of-the-art pre-trained models in the NLP field. Our result shows the effectiveness of the span-pair classifier after fine-tuning the pre-trained model. It further validates the quantitative metric of span relevancy in patent claim generation. Particularly, we found that the span relevancy ratio measured by BERT becomes lower when the diversity in GPT-2 text generation becomes higher.

主出版物標題New Frontiers in Artificial Intelligence - JSAI-isAI International Workshops, JURISIN, AI-Biz, LENLS, Kansei-AI, 2019, Revised Selected Papers
編輯Maki Sakamoto, Naoaki Okazaki, Koji Mineshima, Ken Satoh
發行者Springer Science and Business Media Deutschland GmbH
出版狀態Published - 11 9月 2020
事件11th JSAI International Symposium on Artificial Intelligence, JSAI-isAI 2019 - Yokohama, Japan
持續時間: 10 11月 201912 11月 2019


名字Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
12331 LNAI


Conference11th JSAI International Symposium on Artificial Intelligence, JSAI-isAI 2019


深入研究「PatentTransformer-1.5: Measuring Patent Claim Generation by Span Relevancy」主題。共同形成了獨特的指紋。