Today, we are excited to bring Insert Data from Picture to iPhone users, now supporting 21 languages for both versions of the Excel mobile app (Android version available since March). This tabular recognition feature for Excel combines advanced optical character recognition (OCR) technology and machine learning models to transform paper-based information into digital data. Insert Data from Picture brings printed, tabular data directly into Excel, where you can perform various kinds of analysis that are time-consuming or even impossible with pen and paper. Read on to learn how the feature works.
Going from analog to digital
Simply open the Excel app on your phone, snap a picture of your paper-based data table, crop and review the image, and you’re done—no technical skills required. The data table is automatically embedded and ready for analysis in Excel.
“If only I had this before, I’d have saved so much time avoiding manual data entry” is one very popular reaction of Excel users so far. Going from analog to digital gives you time for more important tasks and is more secure. In addition, once a printed copy is converted, digital records are easier to organize and search—eliminating paper waste and reducing physical storage space.
Here are a few examples of how users can benefit from Insert Data from Picture:
- Consolidate dozens, hundreds, or even thousands of rows of paper-based data in a flash—all without a single pencil mark.
- Create illustrative charts and graphs to summarize information that was extremely difficult to communicate before.
- Use Ideas in Excel to surface new trends and dependencies you might have missed when your data only existed on paper.
-
Easily archive data documents for future reference and compliance purposes.
Inside Excel: the technology behind the feature
To enable seamless data extraction from an image, Insert Data from Picture reuses many of the same OCR and Layout technologies previously released for Word (i.e., the PDF Reflow feature), Office Lens, and Seeing AI.
First, the image is analyzed to detect the main building blocks of the document, like text and graphical elements (e.g., table borders). For this, we further enhanced our Microsoft-built OCR engine to handle images with scattered text, which is often the case in tabular data and leveraged various image processing techniques to detect graphical elements.
Once the image is decomposed into main building blocks, Insert Data from Picture starts inferring the layout of the table. The most important part is detecting the grid of the table, which is done by generating grid candidates from horizontal and vertical lines (for bordered tables) and empty spaces between text (for borderless tables). After all the candidates are generated, the feature uses a combination of various heuristics and machine learning models to filter false positives and produce the final grid that will be reconstructed in the output. Producing that final grid relies on the analysis of each cell to build out other structures like paragraphs, font properties, and lists.
Artificial intelligence (AI) for more accurate results
For those who have used OCR-based features before, you know it doesn’t always get everything right. Insert Data from Picture takes special care to highlight potential errors, so you can focus on individual entries rather than on the whole thing. The good news is that this is an AI-powered feature designed for continues improvement, meaning that data accuracy will increase over time. To achieve this, we leveraged the collection of machine learning models where each model detects a specific case of misrepresented content (e.g., missed or added characters). Watch this Inside Excel episode to learn how the feature was built to address user needs.
Left: A smart preview is generated after you crop the image. You can select any cell in this view to zoom into the source data. Center: Review and correct (or ignore) low-confidence entries, which are highlighted in red. Right: Tap “Insert” after reviewing potential issues to insert the final data into Excel.
Download Excel for your mobile phone and start using the feature today: Excel app for iPhone and Excel app for Android. To learn more about Insert Data from Picture, see our support article. Also, stay connected with Excel by visiting the Excel Tech Community, and follow us on Facebook and Twitter. We encourage you to send us ideas for new features on UserVoice and send us a smile or frown to help improve the product.
Frequently asked questions
What are the plans to support this feature on other platforms like Windows or iOS?
Insert Data from Picture will be available on iOS for Office Insiders next month. We’re also working to bring this feature to other platforms soon.
Which languages does the feature support?
The feature currently supports 21 Latin languages (such as French, German, and Spanish) for both Android and iPhone Excel apps. You can find the list of all supported languages here. More languages will be added in the future.
Do you need to have an Office subscription to use this feature?
You don’t need an Office 365 subscription. Anyone can download the app on their phones and start using Insert Data from Picture.
