Quiz

Let me start with a quick two‑question quiz to test your knowledge on Open‑Source LLMs vs Closed LLMs. The answers are provided at the end of this blog *.

1)  Which category do GPT‑4, Claude, and Gemini LLMs fall under?

1. Open‑source models
2. Research only models
3. Closed / proprietary models
4. Edge only models

2)  Which is a defining trait of open‑source LLMs?

1. Always hosted by hyperscalers
2. Full access to model weights and architecture
3. Higher accuracy than closed models
4. Built‑in enterprise support

Overview Of LLMs In Enterprise Context

LLMs are advanced AI models trained in vast data. They enable tasks such as summarization, translation, content creation, and data analysis.

When companies build applications that use AI, one of the most important decisions they face is choosing the right type of Large Language Model (LLM). There are two main choices: open‑source LLMs and closed or proprietary LLMs. Understanding the differences between them helps businesses decide which option fits their needs, goals, and security requirements.

Open‑source LLMs are models whose code and architecture are publicly available. This means companies can customize them, host them on Cloud or on-premises, and control how the data is handled. They offer flexibility and transparency, but they also require more technical skills and resources to manage.

• Typical Enterprise Use Cases: Enterprises (including our customers) utilize LLMs across multiple domains to drive innovation and efficiency.

Some examples of where customers might use LLMs include chatbots, virtual assistants, code generation, document processing, knowledge management, market research, sentiment analysis, sales enablement, resume screening, incident root cause analysis, and financial fraud detection using narrative pattern analysis.

• Key Considerations for LLM Adoption: Data privacy, security compliance, fine-tuning options for domain specific data, integration with existing enterprise systems, total cost of ownership, model accuracy & bias mitigation, resource requirements.

Types: Open-Source vs Closed LLMs

• Open-Source LLMs: Open‑source Large Language Models (LLMs) are AI models whose model weights, architecture, and often training code are publicly available, allowing organizations to inspect, modify, fine‑tune, and deploy the models on their own infrastructure (cloud, on‑premises, or hybrid). Unlike proprietary models, open‑source LLMs give enterprises full control over how the model is hosted, secured, customized, and governed, but also place greater responsibility on the organization for operations, compliance, and lifecycle management.

They are also called open-weights.

• Closed LLMs: Closed (Proprietary) Large Language Models (LLMs) are AI models whose architecture, training data, and model weights are not publicly available and are owned, hosted, and managed by a vendor. Enterprises consume these models via managed services or APIs, with the vendor responsible for infrastructure, scaling, security controls, and ongoing model updates. Organizations can use and configure these models but cannot inspect or modify the core model internals.

Comparative Analysis

Demo

Internals of the Llama 3 model on Google Colab using Hugging Face Transformers library & Python code: Later, I plan to try something similar on the Azure platform using Microsoft Foundry. For this demo, I chose to use completely open‑source platforms.

This was done for two main reasons:

• My goal was to show how easy it is to explore and interact with the open‑source Llama3 model without relying on any proprietary tools. By using openly available frameworks and environments, the entire workflow remains transparent, customizable, and accessible to anyone who wants to learn, experiment, or build with Llama 3 (or any open-source LLM) in a fully open ecosystem.

• My other reason was to highlight open‑source tools and processes especially since these topics may come as talking points in customer conversations.

AI Tools Used:

• Llama 3: Llama 3 is Meta's advanced, open-source family of Large Language Models (LLMs), offering reasoning, coding, and instruction-following capabilities for AI applications like Meta AI (on Facebook, Instagram, etc.), providing tools for developers. It comes in various sizes (8B, 70B, and larger) and versions (base and instruction-tuned).

• Hugging Face: Hugging Face is a open‑source AI and machine learning platform used by developers, researchers, and enterprises to build, share, and deploy AI models. It is often described as the “GitHub of Machine Learning”, because it hosts millions of models, datasets, and applications in a collaborative community environment. 

• Hugging Face Transformers: Hugging Face Transformers is an open-source Python library that provides APIs and tools to access and use pre-trained machine learning models. It simplifies the application of complex AI models for tasks across various domains, including natural language processing (NLP), computer vision, and audio processing.

• Google Colab: This is a free, cloud-based platform. It allows users to write and run Python code in a Jupyter Notebook environment through a web browser. 

• Jupyter Notebook: Jupyter Notebook is an open-source, web-based application for creating and sharing documents with live code, equations, visualizations, and narrative text. It is used for data cleaning, scientific computing, machine learning, and data exploration. It allows users to combine code execution (in Python, R, Julia, etc.) with rich text and output (like charts and images) in one interactive document, facilitating reproducible research and storytelling with data. 

• Python: Python is a high level‑, interpreted programming language known for its simple, readable syntax and wide range of uses in web development, data science, AI, automation, and more. 

The Python code will examine the internals of the Llama 3 model on Google Colab, using Hugging Face and Python code. The Hugging Face Transformers library is used to load the model and inspect its configuration and architecture:

• Step-1: Get access to the Llama 3 model on Hugging Face (e.g., meta-llama/Meta-Llama-3-8B).

• Step-2: Generate a Hugging Face API Token with "read" or "write" permissions. Configure Hugging Face Access Token in Google Colab Environment.

• Step-3: In Google Colab, create a Jupyter Notebook and write Python code for the following tasks (code in following section):

o   Install Libraries: Install the necessary Python packages.

o   Retrieve the token from Colab secrets.

o   The Python script first loads the model’s configuration, then loads the model itself to examine its architecture and inspect its layers.

The screenshot from Google Colab environment:

Jupyter Notebook Python Code

• Get access to the Llama 3 model on Hugging Face (e.g., meta-llama/Meta-Llama-3-8B). Install Libraries: Install the necessary Python packages.

!pip install transformers torch accelerate bitsandbytes

 

• Generate a Hugging Face API token with "read" or “write” permissions

#!huggingface-cli login

from google.colab import userdata

from huggingface_hub import login

# Retrieve the token from Colab secrets

TokenAllAccessWrite = userdata.get('TokenAllAccessWrite')

# Log in to Hugging Face

if TokenAllAccessWrite:

    login(TokenAllAccessWrite)

    print("Successfully logged in to Hugging Face!")

else:

    print("TokenAllAccessWrite not found in Colab secrets.")

 

• Python Code to Inspect Llama 3 Internals. The following Python script loads the model's configuration and then the model itself, allowing you to print the architecture and inspect its layers.

# 1. Inspect the model configuration

# The config object contains hyperparameters defining the architecture (e.g., number of layers, hidden size, attention heads)

print(f"--- Loading Configuration for {model_id} ---")

config = AutoConfig.from_pretrained(model_id, token=True) # Use token=True if HF_TOKEN env var is set

print(config)

 

print("\n--- Key Architectural Details from Config ---")

print(f"Vocab size: {config.vocab_size}")

print(f"Hidden size: {config.hidden_size}")

print(f"Number of attention heads: {config.num_attention_heads}")

print(f"Number of hidden layers (Transformer blocks): {config.num_hidden_layers}")

print(f"Max position embeddings (Context length): {config.max_position_embeddings}")

print(f"Grouped Query Attention (GQA) num_key_value_heads: {config.num_key_value_heads}") # Llama 3 uses GQA

 

# 2. Load the actual model and inspect its structure

# This will download the model weights (approx. 16GB for 8B model) and cache them

# device_map="auto" efficiently loads the model across available resources (GPU/CPU)

print(f"\n--- Loading Model {model_id} to Inspect Architecture ---")

try:

    model = AutoModelForCausalLM.from_pretrained(

        model_id,

        device_map="auto",

        torch_dtype=torch.bfloat16, # Llama 3 trained in bfloat16

        token=True

    )

    print("\n--- Model Architecture (pytorch modules) ---")

    print(model)

 

    # You can access specific layers, e.g., the first decoder layer

    first_decoder_layer = model.model.layers[0]

    print("\n--- Details of the First Decoder Layer ---")

    print(first_decoder_layer)

except Exception as e:

    print(f"\nAn error occurred: {e}")

    print("Ensure you have requested access on Hugging Face and your token is set correctly.")

 

Conclusion

• Decision-Making Considerations for Enterprises: Choosing the right LLM depends on enterprise goals, constraints and technical needs.

Careful evaluation of cost, scalability, security, and long-term sustainability will guide the decision towards Open-Source vs Closed LLMs.

• Enterprises must weigh the benefits of open-source LLMs: Control, Autonomy, Customizable, Strong community support.

• Enterprises must weigh the benefits of closed (proprietary) LLMs: Speed, High performance, Integrated services, Reliability, Governance.

Reference Links / Recommended Reading

• Azure OpenAI in Foundry Models: https://azure.microsoft.com/en-us/products/ai-foundry/models/openai/

• Hugging Face: https://huggingface.co/ 

• Hugging Face Transformers:

https://huggingface.co/docs/transformers/en/index

https://github.com/huggingface/transformers 

• Google Colab: https://colab.research.google.com/ 

• Llama 3: https://www.llama.com/models/llama-3/ 

• Jupyter Notebook: https://jupyter.org/

• Python:  https://www.python.org/

 

* Quiz: The correct answers are option C for question 1 and option B for question 2.