Full Contents of Word Extraction Tool Description , Download

I wrote an explanation of the word extraction tool in six parts and posted them all on my blog. Write the entire table of contents of the word extraction tool description and post it as a separate article.

• 1. Word Extraction Tool Overview
  • 1.1. Why We Developed Word Extraction Tool
  • 1.2. word extraction tool concept
  • 1.3. How the word extractor works
    • 1.3.1. word extraction tool input material
    • 1.3.2. Word extraction tool processing logic
    • 1.3.3. Word extraction tool output data

• 2. Configuration of the word extraction tool execution environment
  • 2.1. Environment configuration overview
    • 2.1.1. Recommendations
    • 2.1.2. Choose a stemmer Mecab
    • 2.1.3. Overall order of environment configuration
  • 2.2. Install Miniconda
  • 2.3. Creating and activating a virtual environment
  • 2.4. Install Python in virtual environment
  • 2.5. Install required packages

• 3. Run the word extraction tool
  • 3.1. Download word extraction tool
  • 3.2. How to Run the Word Extraction Tool
    • 3.2.1. Unzip the downloaded file and activate the Python virtual environment
    • 3.2.2. Check Help
    • 3.2.3. Method 1 Extract words only from document files
    • 3.2.4. Method 2 Extract words only from DB Table, Column comments
    • 3.2.5. Execution Method 3 Extract words from all document files, DB Table, and Column comments
    • 3.2.6. How to check execution results
    • 3.2.7. Precautions/Notes on Execution

• 4. Word extraction tool source code
  • 4.1. summary
    • 4.1.1. Note the source code
    • 4.1.2.Word extraction tool function call relationship
  • 4.2. main function
    • 4.2.1. argument parsing
    • 4.2.2. Extract list of files to process
    • 4.2.3. Execute get_file_text with multi processing
    • 4.2.4. Execute get_word_list with multi processing
    • 4.2.5. Get word frequency and run make_word_cloud
    • 4.2.6. Save the extracted word list and word frequency as an Excel file, print the execution time, and exit
  • 4.3. get_file_text function
    • 4.3.1. get_doc_text function
    • 4.3.2. get_ppt_text function
    • 4.3.3. get_txt_text function
    • 4.3.4. get_db_comment_text function
    • 4.3.5. get_hwp_text function
    • 4.3.6. get_pdf_text function
  • 4.4. get_word_list function
  • 4.5. make_word_cloud function

• 5. Additional explanation of word extraction tool
  • 5.1. Why use OLE Automation?
  • 5.2. Text file encoding related (only UTF-8 is supported)
  • 5.3. Why use multi processing?
  • 5.4. Notes on compound word extraction
  • 5.5. morpheme analyzer part-of-speech type

---

The word extraction tool can be downloaded from the github repository below.

https://github.com/DAToolset/ToolsForDataStandard/tree/main/WordExtractor

Source codes, fonts, table/column list example files, and output example files necessary for execution are bundled into a compressed file for distribution, so you can download this file.

https://github.com/DAToolset/ToolsForDataStandard/raw/main/WordExtractor/word_extractor.7z

I hope this will be of some help in data standardization.