Embarking on a journey to redefine the landscape of e-commerce in India, Meesho stands as a beacon of innovation and empowerment. In the realm of technology, Meesho's commitment to revolutionizing retail is evident through its strategic utilization of GitHub Enterprise. In this article, we delve into the insights gained from Meesho's innovative employment of GitHub Enterprise, shedding light on how technology acts as the catalyst for their mission. With a focus on democratizing internet commerce, Meesho's distinct approach combines social media, data analytics, and seamless logistics, transforming the retail experience for small businesses and entrepreneurs across the nation.
How Meesho is using Github Enterprise
Meesho maximizes the potential of GitHub Enterprise, employing various features to streamline their software development lifecycle. Meesho uses Github for CI/CD using Actions, managing code repositories effectively, and enforcing security with SAML single sign-on, centralized repository rules and more. But their standout adoption is Github Copilot, our AI programming assistant. Notably, GitHub Copilot has emerged as a game-changer for their developers. With a 33% overall completion acceptance rate, it has significantly boosted productivity across Java (35.76%), Kotlin (30.22%), Go (35.19%), TypeScript (28.61%), Python (33.75%) and other programming languages.
At Meesho, GitHub Copilot isn't just an AI programming assistant; it's a catalyst for efficiency. Developers laud Copilot, which is powered by Azure OpenAI Service, for its ability to drastically reduce boilerplate code, freeing up valuable time for tackling more intricate development challenges. Its contextual understanding and accurate code generation enable engineers to maintain focus, fostering a state of uninterrupted productivity. This seamless integration into the developer's workflow ensures that the cognitive load is reduced, allowing them to invest their energy in innovation and nuanced problem-solving.
Moreover, Copilot serves as a guiding companion for navigating uncharted territories. Whether it's navigating a new codebase or adapting to a different programming language, Copilot's suggestions expedite the learning curve and empower developers to perform optimally.
The ease of integration is another cornerstone of Copilot's success at Meesho. Seamlessly embedding into the existing coding environment, it transforms natural language comments into functional code snippets, accelerating coding speed and minimizing errors.
The statistics from their internal developer surveys are a testament to the effectiveness of Github Copilot: the majority of their developers reported significant time savings and a high likelihood of recommending Copilot to their teams, reflecting an enhanced development experience across the board.
Enhancing Developer Insights with a Custom Telemetry Solution for Github Copilot
When Meesho incorporated GitHub Copilot into their development process, the immense potential in boosting coding efficiency became evident. However, they encountered an issue with Copilot's default telemetry data, which offered only superficial insights. These insights proved inadequate for accurately gauging Copilot's adoption across the organization and evaluating the effectiveness of its code suggestions.
To address this challenge, their team developed an innovative solution: an HTTP pass-through proxy specifically created to intercept Copilot's telemetry API calls. This proxy functions as a conduit, enabling them to feed data directly into their in-house analytics systems. This integration played a pivotal role in enhancing their understanding of Copilot's usage within the organization.
The results proved enlightening. They were able to monitor the Daily and Weekly Active Users (DAU and WAU) of Copilot in real time. More significantly, they tracked the Completion Acceptance Rate at a language level. This granular data empowered them to make more informed decisions about resource allocation and training, ensuring that their developers received optimal support in the most efficient way possible.
Digging into the shell script and proxy
The custom telemetry solution consists of two main parts:
-
A shell script:
This script is responsible for finding the Copilot installations available on an employee’s device and patching them to start hitting in-house proxy URLs instead of the default Github Telemetry URL when sending Telemetry data.
-
A transparent HTTP proxy:
This is a simple service that relays incoming requests to the Github Telemetry URL. In addition, it reads the events from the request body and ingests them into an in-house analytics system.
Meesho used their existing MDM (Mobile Device Management) workflow to run shell scripts on their developer machines in the background without any manual intervention.
When Meesho initially deployed this script with a one-off execution rule, they realized in a couple of days that the patch changes were being reverted, essentially resetting to the factory version when the Copilot plugin updates. Subsequently, they fixed this by running the shell script periodically in the background so that developers stay on the patched version.
Initially, the shell script retrieves the list of installed IDEs on the device. Since Github Copilot is available as a JetBrains IDE plugin, it can be installed on IntelliJ, Android Studio, WebStorm and other IDEs in the JetBrains family.
Once they have the list of installed IDEs, they check if they have Github Copilot installed by looking for the Copilot directory in the IDE plugins directory.
APPLICATION_SUPPORT_DIR="$HOME/Library/Application Support"
COPILOT_INTELLIJ_PLUGIN_DIR_NAME="github-copilot-intellij"
JETBRAINS_IDE_DIR_PATTERN=".*/(IntelliJIdea|IdeaIC|PyCharm|PyCharmCE|WebStorm).*"
find_compatible_jetbrains_based_ide_plugins_dirs() {
if [ -d "$APPLICATION_SUPPORT_DIR"/Google ]; then
echo "Google dir exists under Application Support"
find_result=$(find "$APPLICATION_SUPPORT_DIR"/Google -maxdepth 1 -type d || true)
if [ -n "$find_result" ]; then
STUDIO_DIRS=$(echo "$find_result" | grep -E ".*/AndroidStudio.*" || true)
fi
for dir in $STUDIO_DIRS; do
if [ -d "${dir}/plugins/${COPILOT_INTELLIJ_PLUGIN_DIR_NAME}" ]; then
JETBRAINS_BASED_IDE_PLUGINS_DIRS="${dir}/plugins $JETBRAINS_BASED_IDE_PLUGINS_DIRS"
fi
done
fi
if [ -d "$APPLICATION_SUPPORT_DIR/JetBrains" ]; then
echo "JetBrains dir exists under Application Support"
find_result=$(find "$APPLICATION_SUPPORT_DIR/JetBrains" -maxdepth 1 -type d || true)
echo "dirs under JetBrains: $find_result"
if [ -n "$find_result" ]; then
INTELLIJ_DIRS=$(echo "$find_result" | grep -E "${JETBRAINS_IDE_DIR_PATTERN}" || true)
fi
for dir in $INTELLIJ_DIRS; do
if [ -d "${dir}/plugins/${COPILOT_INTELLIJ_PLUGIN_DIR_NAME}" ]; then
JETBRAINS_BASED_IDE_PLUGINS_DIRS="${dir}/plugins $JETBRAINS_BASED_IDE_PLUGINS_DIRS"
fi
done
JETBRAINS_BASED_IDE_PLUGINS_DIRS=$(echo "$JETBRAINS_BASED_IDE_PLUGINS_DIRS" | xargs)
fi
}
The next step is to patch the list of available Copilot plugin directories by replacing the Telemetry URL strings in the JavaScript source code.
The Copilot JetBrains plugin actually has two modes of execution:
-
Using native binaries that are present under the `copilot-agent/bin/` directory in the plugin. This is the default mode of execution.
-
Using `copilot-agent/dist/agent.js`. This is the fallback mode that requires Node on the system. It’s only used when native binaries are not available.
Since they patch agent.js, they require Copilot plugin to use it for execution. Consequently, the binaries under `copilot-agent/bin/` are renamed to facilitate the plugin's fallback to the 2nd approach.
Following the renaming process, the next step involves replacing the copilot-proxy and copilot-telemetry URLs with the designated PROXY_URL.
PROXY_URL="https://copilot-proxy\.myhost\.com"
SEARCH_STR_COPILOT_PROXY="https://copilot-proxy\.githubusercontent\.com"
SEARCH_STR_COPILOT_TELEMETRY="https://copilot-telemetry\.githubusercontent\.com/telemetry"
patch_intellij_plugin() {
local plugin_dir=$1
echo "List of binaries in $plugin_dir/copilot-agent/bin:"
ls -l "$plugin_dir/copilot-agent/bin"
echo "Renaming macos agents..."
if [ -f "${plugin_dir}/copilot-agent/bin/copilot-agent-macos" ]; then
echo "Renaming copilot-agent-macos..."
mv "${plugin_dir}/copilot-agent/bin/copilot-agent-macos" "${plugin_dir}/copilot-agent/bin/_copilot-agent-macos"
else
echo "copilot-agent-macos not found. No changes made to file."
fi
if [ -f "${plugin_dir}/copilot-agent/bin/copilot-agent-macos-arm64" ]; then
echo "Renaming copilot-agent-macos-arm64..."
mv "${plugin_dir}/copilot-agent/bin/copilot-agent-macos-arm64" "${plugin_dir}/copilot-agent/bin/_copilot-agent-macos-arm64"
else
echo "copilot-agent-macos-arm64 not found. No changes made to file."
fi
echo "Listing binaries again:"
ls -l "$plugin_dir/copilot-agent/bin"
echo "Updating endpoints..."
str_regex_replace \
"$plugin_dir/copilot-agent/dist/agent.js" "${SEARCH_STR_COPILOT_TELEMETRY}" "${PROXY_URL}/telemetry"
str_regex_replace \
"$plugin_dir/copilot-agent/dist/agent.js" "${SEARCH_STR_COPILOT_PROXY}" "${PROXY_URL}"
echo "Patch apply complete for $plugin_dir. Please restart your IDE."
}
str_regex_replace() {
local filename=$1
local search_str=$2
local replace_str=$3
if grep -qE "$search_str" "$filename"; then
tmp_file=$(mktemp)
sed -E "s|${search_str}|${replace_str}|g" "$filename" >"$tmp_file"
mv "$tmp_file" "$filename"
echo "Replaced '$search_str' with '${replace_str}' in $filename."
else
echo "'$search_str' not found in $filename. No changes made in file."
fi
}
Link to full script: https://gist.github.com/vibin/01aed5d3a3c5e9595e84a46933b8d084
Conclusion
Meesho's strategic adoption of GitHub Enterprise, especially GitHub Copilot, which is powered by Azure OpenAI Service, has created a highly productive developer environment. The insights gained from the detailed telemetry data have enabled Meesho to tailor their tools and training to the specific needs of their developers, which results in improved efficiency.
Moreover, the feedback collected from Meesho's developers highlights the tangible benefits of integrating Copilot into their workflow, including significant time savings and a strong endorsement of the tool's continued use.