[Python] kobert 패키지 오류: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...

 

Python 패키지인 kobert 사용시 아래와 같은 경고가 뜬다면

huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)

 

아래 코드를 소스코드에 넣어 해결 가능

import os
os.environ["TOKENIZERS_PARALLELISM"] = "false"