TextTransformerAI
Automated Text Processing Tool for Historical Data
Overview
Client: A NYU Prof
Date: 2024 April
Category: AI
Automate text processing and OpenAI API interactions in Python. Transform text files into valuable insights, perfect for developers and content creators. Streamline data cleaning, content generation, and insight extraction with ease.
I have a raw txt file consists of roughly 8500 figures from previous dynasty, but the txt is only separated by "【", "】", and new line. We want to extract their name, position, tribute, hometown, etc and store all the information into a xlsx file with those columns.
The Details
Firstly split the txt by new line, and in every new line we split by "】". We have 2 for loops, and we only insert each person into the array to avoid further splitting. Also to notice eliminating the empty one by filtering the lentgh. Next I picked GPT-turbo-3.5 as our model. So I would take the array we have from last step to GPT, and get the target output that we want.
