Automate Markdown and Image Translations Using Co-op Translator: Phi-3 Cookbook Case Study

 

Overview

 

In today's global environment, it's important to make technical content accessible to a wider audience. This is especially true for open source projects and technical documentation. In this blog post, I'll introduce you to Co-op Translator, an open source tool designed to automate multilingual translations for your projects.

I'll show you how I used the translation tool to effortlessly translate the Phi-3CookBook which is an official open source guide for Phi-3 / 3.5 small language model. You'll be able to follow the same process for your own projects, streamlining the translation of even large technical resources.

 

What is Co-op Translator?

 

Co-op Translator is an open source tool designed to automate the translation of Markdown files and images containing embedded text into multiple languages. Powered by Azure AI Services, it streamlines the traditionally time-consuming translation process, allowing you to make your projects globally accessible with minimal manual effort.

 

Below is the architecture of this project: 

 

 

 

 

The process begins with markdown and image files from your project folder, which are then processed by Azure AI Services. Azure OpenAI handles text translations from the markdown files, while Azure Computer Vision extracts text from the images. Once the text is extracted, Azure OpenAI translates the text from the images. The final translated files - both markdown and images - are saved in the designated translation folder, ready for use in multiple languages.

 

Key features

 

• Translates both Markdown files and text within images.
• Supports multiple languages simultaneously.
• Leverages Azure Computer Vision and Azure OpenAI for high-quality translations.
• Can be easily integrated into your existing workflows.

 

Table of contents

 

1. Phi-3 Cookbook translation: A case study
2. Set up Azure resources
3. Create the .env file in the root directory
4. Install the Co-op Translator package
5. Translate your project using Co-op Translator
6. Conclusion: Making your projects global

 

Phi-3 Cookbook translation: A case study

 

The Phi-3 Cookbook is an official open-source guide that provides detailed instructions on Phi-3 and Phi-3.5 small language models. Given its technical nature and importance to the global AI community, translating it into multiple languages was a crucial step toward making this valuable resource accessible to non-English-speaking developers and researchers.

By using Co-op Translator, I was able to streamline the translation process, automating the conversion of both Markdown files and images containing embedded text into several languages. This case study explains how the tool was applied to the Phi-3 Cookbook, the challenges encountered, and the solutions implemented.

 

The translation process

1. Preparation: First, I organized the markdown files and image assets from the Phi-3 Cookbook. I removed any manually translated files from previous versions to ensure the automation process started from a clean slate. Leaving those files would have resulted in unnecessary duplicates. Additionally, I created an .env file in the root directory to securely store the Azure API keys and configuration settings required for the translation.
   
   

2. Azure setup: I configured Azure OpenAI to handle the translation of the Markdown content and Azure Computer Vision to extract and translate text from images. This setup enabled Co-op Translator to automatically detect and process both text-based and image-based content seamlessly.
   
   

3. Installing Co-op Translator: I installed the Co-op Translator package using Poetry to manage dependencies.
   
   

4. Multi-language support setup: Before starting the translation process, I created a table in the README linking to the translated versions of the document. During translation, Co-op Translator automatically adjusted the links, allowing users to switch between different language versions seamlessly. For example, if a user navigates to the Korean README, they can easily switch to other translations like Spanish or Japanese without leaving the translated page.
   
   

   ## 🌐 Multi-Language Support
   
   > **Note:**
   > These translations were automatically generated using the open-source [co-op-translator](https://github.com/Azure/co-op-translator) and may contain errors or inaccuracies. For critical information, it is recommended to refer to the original or consult a professional human translation. If you'd like to add or update a translation, please refer to the [co-op-translator](https://github.com/Azure/co-op-translator) repository, where you can easily contribute using simple commands.
   
   | Language             | Code | Link to Translated README                               | Last Updated |
   |----------------------|------|---------------------------------------------------------|--------------|
   | Chinese (Simplified) | zh   | [Chinese Translation](./translations/zh/README.md)      | 2024-10-04   |
   | Chinese (Traditional)| tw   | [Chinese Translation](./translations/tw/README.md)      | 2024-10-04   |
   | French               | fr   | [French Translation](./translations/fr/README.md)       | 2024-10-04   |
   | Japanese             | ja   | [Japanese Translation](./translations/ja/README.md)     | 2024-10-04   |
   | Korean               | ko   | [Korean Translation](./translations/ko/README.md)       | 2024-10-04   |
   | Spanish              | es   | [Spanish Translation](./translations/es/README.md)      | 2024-10-04   |
   

   
   
   

    

5. Execution with Co-op Translator: Using Co-op Translator, I initiated the translation process by running the translate command with the appropriate language codes, such as Spanish (es), French (fr), Korean (ko), and others.

   • The Markdown files were translated using Azure OpenAI.
   • Azure Computer Vision was utilized to extract text from images, and Azure OpenAI handled the translation of the extracted text.
   • The translated Markdown and image files were saved in dedicated, language-specific folders.
   • Below is an example of how the Korean translation was generated for the Phi-3 Cookbook:
     
     

   (.venv) C:\Users\sms79\dev\Phi-3CookBook>translate -l "ko"
   Translating images: 100%|███████████████████████████████████████████████████| 276/276 [1:09:56<00:00, 15.37s/it]
   Translating markdown files: 100%|███████████████████████████████████████████| 153/153 [1:43:07<00:00, 241.31s/it]
   

   • In this example, Co-op Translator processed 276 images and 153 Markdown files for the Korean translation, automatically saving the results in the appropriate language folder.
     
     

6. Review: After the translations were completed, I reviewed the output for accuracy. The automated process produced high-quality translations, significantly reducing the need for manual adjustments.

 

 

Challenges and Solutions

 

While the process was mostly smooth, a few challenges arose:

• Handling files with many code blocks: Some markdown files contained many code blocks. During translation, the tool splits the content into chunks, and if a split occurred within a code block, the translated output would sometimes break. To resolve this, we implemented a solution where code blocks are temporarily replaced with placeholders and skipped during translation, ensuring the integrity of both the code and the translation. In future versions, we plan to enhance this feature by translating comments within code blocks. This will involve separating code blocks from the rest of the markdown, accurately translating each part, and then reintegrating them into the final translated file.

• Text from images: Extracting and translating text from complex images was challenging, especially when the images had a lot of text in small areas. This often led to the translated text in image being either overly stretched or compressed, affecting readability. While we managed to mitigate some of these issues, further improvements are needed in this area to ensure higher accuracy.

 

Results

 

The translation of the Phi-3 Cookbook has been successful. You can now view the translated version in multiple languages. If you're interested in viewing the translated Phi-3 Cookbook, you can visit the Phi-3CookBook. This link will navigate you to the multilingual versions of the Phi-3 Cookbook. 

Below is a markdown example from Phi-3 Cookbook, translated into Korean using the Co-op Translator:

 

 

Set up Azure resources

 

To translate markdown files and extract text from images, we need to set up two key Azure services: Azure OpenAI and Azure Computer Vision.

 

Create an Azure account

 

If you don't already have an Azure account, you'll need to create one.

1. Navigate to the Azure Sign Up page.
2. Select Try Azure for free or Pay as you go.
3. Follow the on-screen instructions to create your account.
   • Provide your personal details and contact information.
   • Verification: You'll need to verify your identity using a credit card or phone number.

 

Create an Azure Computer Vision resource

 

1. Sign in to the Azure Portal.

2. Type computer vision in the search bar at the top of the portal page and select Computer vision from the options that appear.
   
   

   

    

3. Select + Create from the navigation menu.
   
   

   
   
   

4. Perform the following tasks:

   • Select your Azure Subscription.
   • Select the Resource group to use (create a new one if needed).
   • Select the Region you'd like to use.
   • Enter Name. It must be a unique value.
   • Select the Pricing tier you'd like to use.

    

   

    

5. Select Review + Create.

6. Select Create.

 

Create an Azure OpenAI resource

 

1. Type azure openai in the search bar at the top of the portal page and select Azure OpenAI from the options that appear.
   
   

   

    

2. Select + Create from the navigation menu. 
   
   

   

3. Perform the following tasks:

   • Select your Azure Subscription.
   • Select the Resource group to use (create a new one if needed).
   • Select the Region you'd like to use.
   • Enter Name. It must be a unique value.
   • Select the Pricing tier you'd like to use.
     
     
   

4. Select Next to move to the Network page.

5. Select a network security Type you'd like to use.
   
   

   

    

6. Select Next to move to the Tags page.

7. Select Next to move to the Review + submit page.

8. Select Create.
   
   

   

    

Deploy Azure OpenAI models

 

1. Navigate to the Azure OpenAI resource that you created.

2. Select Go to Azure OpenAI Studio from the navigation menu.
   
   

   

    

3. Inside Azure OpenAI Studio, select Deployments from the left side tab.

4. Select + Deploy model from the navigation menu.

5. Select Deploy base model from the navigation menu to create a new gpt-4o deployment.
   
   

   

    

6. Perform the following tasks:

   • Inside Select a model page, select gpt-4o.
   • Select Confirm to navigate to the Deploy model gpt-4o page.
   • Inside Deploy model gpt-4o page, enter Deployment name. It must be a unique value. For example, gpt-4o.
   • Inside Deploy model gpt-4o page, select the Deployment type you'd like to use.
     
     
   

    

7. Select Deploy.

 

Create an .env file in the root directory

 

In this tutorial, we will guide you through setting up your environment variables for Azure services using an .env file. Environment variables allow you to securely manage sensitive credentials, such as API keys, without hard-coding them into your codebase.

 

Create the .env File

In the root directory of your project, create a file named .env. This file will store all your environment variables in a simple format.

 

 

1. Navigate to the root directory of your project.

2. Create an .env file in the root directory of your project.
   
   

   

    

3. Open the .env file and paste the following template:
   
   

    

   # Azure Credentials
   AZURE_SUBSCRIPTION_KEY="your_azure_subscription_key"
   AZURE_AI_SERVICE_ENDPOINT="https://your_azure_ai_service_endpoint"
   
   # Azure OpenAI Credentials
   AZURE_OPENAI_API_KEY="your_azure_openai_api_key"
   AZURE_OPENAI_ENDPOINT="https://your_azure_openai_endpoint"
   AZURE_OPENAI_MODEL_NAME="your_model_name"
   AZURE_OPENAI_CHAT_DEPLOYMENT_NAME="your_deployment_name"
   AZURE_OPENAI_API_VERSION="your_api_version"
   

    

 

Gather your Azure credentials

 

You will need the following Azure credentials on hand to configure the environment:

1. For Azure AI Service:

   • Azure Subscription Key: Your Azure subscription key, which allows you to access the Azure AI services.
   • Azure AI Service Endpoint: The endpoint URL for your specific Azure AI service.

2. For Azure OpenAI Service:

   • Azure OpenAI API Key: The API key for accessing Azure OpenAI services.
   • Azure OpenAI Endpoint: The endpoint URL for your Azure OpenAI service.
   • Azure OpenAI Model Name: The name of the model you will be interacting with.
   • Azure OpenAI Deployment Name: The name of your deployment for Azure OpenAI models.
   • Azure OpenAI API Version: The version of the Azure OpenAI API you are using.

 

Add Azure environment variables

 

1. Perform the following tasks to add the Azure Subscription key and Azure AI Services Endpoint:

   • Type computer vision in the search bar at the top of the portal page and select Computer vision from the options that appear. 

      

     

      

   • Navigate the Azure Computer Vision resource that you are currently using.
   • Copy and paste your Subscription key and Endpoint into the .env file.
      

     
     
     

2. Perform the following tasks to add the Azure OpenAI API Key and Endpoint:

   • Type azure openai in the search bar at the top of the portal page and select Azure OpenAI from the options that appear. 
     
     

      

   • Navigate the Azure OpenAI resource that you are currently using.
   • Select Keys and Endpoint from the left side tab.
   • Copy and paste your Azure OpenAI API Key and Endpoint into the .env file.
      

      

3. Perform the following tasks to add the Azure OpenAI Deployment Name and Version:

   • Navigate to the Azure OpenAI resource that you created.

   • Select Go to Azure OpenAI Studio from the navigation menu. 
     
     

     

      

   • Inside Azure OpenAI Studio, select Deployments from the left side tab.

   • Copy and paste your Azure OpenAI Name and model Version into the .env file.
     
     

     

      

4. Save the .env file.

5. Now, you can access these environment variables to use Co-op Translator with your Azure services.

 

Install the Co-op translator package

 

The Co-op Translator is a command-line interface (CLI) tool designed to help you translate all the markdown files and images in your project into multiple languages. This tutorial will guide you through configuring the translator and running it for various use cases.

 

Create a virtual environment

 

You can create a virtual environment using either pip or Poetry. Type one of the following commands inside your terminal.

 

Using pip

 

python -m venv .venv

 

Using Poetry

 

poetry init

 

Activate the virtual environment

 

After creating the virtual environment, you'll need to activate it. The steps differ based on your operating system. Type the following command inside your terminal.

 

For both pip and Poetry

 

• Windows:
  
  

  .venv\Scripts\activate.bat
  

• Mac/Linux:
  
  

   

  source .venv/bin/activate
  

   

 

Using Poetry

 

1. If you created the environment with Poetry, type the following command inside your terminal to activate it.

    

   poetry shell
   

 

Installing the Package and required Packages

 

Once your virtual environment is set up and activated, the next step is to install the necessary dependencies.

 

Using pip (from requirements.txt)

 

1. If you're using pip, type the following command inside your terminal. It will automatically install the required packages specified in the requirements.txt file:
   
   

   pip install -r requirements.txt
   

 

Using Poetry (from pyproject.toml)

 

1. If you're using Poetry, type the following command inside your terminal. It will automatically install the required packages specified in the pyproject.toml file:
   
   

   poetry install

 

Translate your project using Co-op Translator

 

The Co-op Translator is a command-line interface (CLI) tool that helps you translate markdown and image files in your project into multiple languages. This section explains how to use the tool, covers the various CLI options, and provides examples for different use cases.

 

CLI options overview

 

The Co-op Translator CLI offers several options to customize the translation process:

 

• -l (or --language-codes): Space-separated list of language codes for translation (e.g., "es fr de" for Spanish, French, and German). Use "all" to translate into all supported languages.
• -r (or --root-dir): Specifies the root directory of the project (default is the current directory).
• -a (or --add): Adds new translations without deleting existing ones (default behavior).
• -u (or --update): Updates translations by deleting existing ones and re-creating them. Warning: This will delete all current translations.
• -img (or --images): Translates only image files.
• -md (or --markdown): Translates only markdown files.
• -chk (or --check): Checks translated files for errors and retries translation if needed.
• -d (or --debug): Enables debug mode for detailed logging.

---

 

Example scenarios and commands

 

Here are a few common use cases for the Co-op Translator, along with the appropriate commands to run.

 

1. Basic translation (Single language)

 

To translate your entire project (markdown files and images) into a single language, like Korean, use the following command:

 

translate -l "ko"

This command will translate all markdown and image files into Korean, adding new translations without deleting any existing ones.

 

 

Example on Phi-3 CookBook

 

In the Phi-3 CookBook, I used the following method to add the Korean translation for the existing markdown files and images.

 

(.venv) C:\Users\sms79\dev\Phi-3CookBook>translate -l"ko"
Translating images: 100%|████████████████████████████████████████████| 150/150 [45:53<00:00, 15.55s/it]
Translating markdown files: 100%|███████████████████████████████████| 95/95 [1:39:27<00:00, 125.62s/it]

 

2. Translating multiple languages

 

To translate your project into multiple languages (e.g., Spanish, French, and German), use this command:

 

translate -l "es fr de"

 

This command will translate the project into Spanish, French, and German, adding new translations without overwriting existing ones.

Example on Phi-3 CookBook

 

In the Phi-3 CookBook, after pulling the latest changes to reflect the most recent commits, I used the following method to translate newly added markdown files and images.

 

(.venv) C:\Users\sms79\dev\Phi-3CookBook>translate -l"ko ja zh tw es fr" -a
Translating images: 100%|███████████████████████████████████████████████████| 273/273 [1:09:56<00:00, 15.37s/it]
Translating markdown files: 100%|████████████████████████████████████████████████| 6/6 [24:07<00:00, 241.31s/it]

 

3. Specifying the root directory

 

By default, the translator uses the current working directory. If your project is located elsewhere, specify the root directory with the -r option:

 

translate -l "es fr de" -r "./my_project"

 

This command translates the files in ./my_project into Spanish, French, and German.

 

4. Add new translations without deleting existing ones

 

The default behavior is to add new translations without deleting existing ones. You can explicitly specify this by using the -a option:

 

translate -l "ko" -a

 

This command will add new translations in Korean without affecting the existing translations.

 

Example on Phi-3 CookBook

 

In the Phi-3 CookBook, to update the README.md translations, I first deleted the existing README.md translations and then used the following method to translate the updated content.

 

(.venv) C:\Users\sms79\dev\Phi-3CookBook>translate -l"ko ja zh tw es fr" -a
Translating markdown files: 100%|████████████████████████████████████████████████| 6/6 [24:07<00:00, 241.31s/it]

 

5. Updating translations (Deletes existing translations)

 

To update existing translations (i.e., delete the current translations and replace them with new ones), use the -u option. This will delete all existing translations for the specified languages and re-translate them.

 

translate -l "ko" -u

 

Warning: This command will prompt you for confirmation before proceeding with deleting the existing translations.

 

Example on Phi-3 CookBook

 

In the Phi-3 CookBook, I used the following method to update all translated files in Spanish. I recommend using this method when there are significant changes to the original content across multiple markdown documents. If there are only a few translated markdown files to update, it’s more efficient to manually delete those specific files and then use the -a method to add the updated translations.

 

(.venv) C:\Users\sms79\dev\Phi-3CookBook>translate -l "es" -u
Warning: The update command will delete all existing translations for 'es' and re-translate everything.
Do you want to continue? Type 'yes' to proceed: yes
Proceeding with update...
Translating images: 100%|████████████████████████████████████████████| 150/150 [43:46<00:00, 15.55s/it]
Translating markdown files: 100%|███████████████████████████████████| 95/95 [1:40:27<00:00, 125.62s/it]

 

6. Translating only images

 

To translate only the image files in your project, use the -img option:

 

translate -l "ko" -img

 

This command will translate only the images into Korean, without affecting any markdown files.

 

7. Translating only markdown files

 

To translate only the markdown files in your project, use the -md option:

 

translate -l "ko" -md

 

8. Checking for errors in translated files

 

If you want to check translated files for errors and retry the translation if necessary, use the -chk option:

 

translate -l "ko" -chk

 

This command will scan the translated markdown files and retry translation for any files with errors.

 

Example on Phi-3 CookBook

 

In the Phi-3 CookBook, I used the following method to check for translation errors in the Korean files and automatically retry translation for any files with detected issues.

 

(.venv) C:\Users\sms79\dev\Phi-3CookBook>translate -l"ko" -chk 
Checking translated files for errors in ko...
Checking files for ko: 100%|██████████████████████████████████████████████████| 95/95 [00:01<00:00, 65.47file/s]
Retrying vsc-extension-quickstart.md for ko:   0%|                                     | 0/17 [00:00<?, ?file/s] 

 

This option checks for translation errors. Currently, if the difference in line breaks between the original and translated files is more than six, the file is flagged as having a translation error. I plan to improve this criterion for greater flexibility in the future.

For example, this method is useful for detecting missing chunks or corrupted translations, and it will automatically retry the translation for those files.

However, if you already know which files are problematic, it’s more efficient to manually delete those files and use the -a option to re-translate them.

 

9. Debug mode

 

To enable detailed logging for troubleshooting, use the -d option:

 

translate -l "ko" -d

 

This command will run the translation in debug mode, providing additional logging information that can help you identify issues during the translation process.

 

Example on Phi-3 CookBook

 

In the Phi-3 CookBook, I encountered an issue where translations with many links in markdown files caused formatting errors, such as broken translations and ignored line breaks. To diagnose this problem, I used the -d option to see how the translation process was works.

 

(.venv) C:\Users\sms79\dev\Phi-3CookBook>translate -l "ko" -d
DEBUG:openai._base_client:Request options: {'method': 'post', 'url': '/chat/completions', 'headers': {'api-key': 'af04e0bea45747d8a7b8c131c1971044'}, 'files': None, 'json_data': {'messages': [{'role': 'user', 'content': "Translate the following text to ko. NEVER ADD ANY EXTRA CONTENT OUTSIDE THE TRANSLATION. TRANSLATE ONLY WHAT IS GIVEN TO YOU.. MAINTAIN MARKDOWN FORMAT\n\n# Phi-3 Cookbook: Hands-On Examples with Microsoft's Phi-3 Models [![Open and use the samples in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/microsoft/phi-3cookbook) [![Open in Dev Containers](https://img.shields.io/static/v1?style=for-the-badge&label=Dev%
...

 

10. Translating all languages

 

If you want to translate the project into all supported languages, use the all keyword.

 

 

translate -l "all"

 

This command will translate the project into all available languages. If you proceed, the translation may take a significant amount of time depending on the size of the project.

 

Conclusion: Making your projects global

 

How can large-scale projects efficiently reach a global audience without getting bogged down in the complexities of translation? The Co-op Translator provides a compelling answer.

 

 

 

 

Based on my experience, here are a few key takeaways:

• Time Management: Translating a large project can take considerable time. For example, translating the Phi-3 Cookbook into Spanish took about 2 hours. It’s important to account for this when working on large-scale translations to avoid being caught off guard by the time required. However, since this translation process only adds translated markdown or images once the translation is complete, you can pause the process at any time. Simply press Ctrl + C (or Cmd + C on macOS) at any time to stop the process, and when you're ready to resume, use the -a option to continue the translation from where you left off.

• Team Collaboration: If you're part of a team, dividing the workload by assigning each member a specific language can help streamline the process. Co-op Translator's ability to easily add and update translations ensures smooth collaboration, allowing everyone to work without overwriting each other's progress. Or, if you're managing an open source project, I recommend encouraging contributors to use this tool to translate content into their respective languages, making the project more accessible to a global audience.

The Phi-3 Cookbook translation stands as a great example of how Co-op Translator can automate the localization process, making your project accessible to a global audience with minimal effort. Whether you're translating Markdown files or images, Co-op Translator simplifies the task, handling the complexities of translation seamlessly.

 

Clean Up Azure Resources

 

Cleanup your Azure resources to avoid additional charges to your account. Go to the Azure portal and delete the following resources:

• The Azure Computer Vision resource.
• The Azure OpenAI resource

If you're interested in trying out Co-op Translator or contributing to its development, check out the Co-op Translator's GitHub repository. I'd love to hear your thoughts and experiences - feel free to contact me with any questions or feedback!