Receipt recognition with Azure

We are constantly looking for ways to help you get the most out of your data. Our customer ask us a POC to recognize information from receipts.

Expense reports can be a very cumbersome and time-consuming task. Between all the manual data entry, approval workflows, and auditing, there are many pain points across the end-to-end process. With the you can minimize those pain points and increase the productivity of your employees, delivering real value back to your business.

Receipt processing lets you read and save key information from common sales receipts, like those used in restaurants, gas stations, retail, and more. Using this information, you can automatically pre-populate expense reports simply by scanning photos of your receipts. And when you automate the process at a large scale, there is the potential to save you and your business valuable time and money.

The prebuilt model uses state-of-the-art optical character recognition (OCR) to extract both printed and handwritten text from receipts. You can retrieve valuable information such as the merchant details, transaction date and time, list of purchased items, tax, and totals.

No training or prior configuration is required to use this prebuilt model. Start processing receipts right away in your apps and flows using the new canvas app component and AI Builder flow action.

Text translation

You can now use AI Builder to easily translate text to more than 60 languages. This prebuilt model is powered by the latest innovations in machine translation. You can use Text translation to process text in real-time from different languages from your customers worldwide, for internal and external communications and to keep language consistency in the text data that you store. Now available in preview, no trial or subscription required to try this feature.

Building a Text Classifier using Azure Machine Learning

Recently a client came to us to see if we could help them automate their RFP distribution system.  Currently the client has an employee manually check several websites for RFPs and alert the appropriate business vertical when a relevant RFP is found.  The current system requires manual data scraping, meaning the process is slow and results in RFPs being missed.  For the proof of concept phase with the client, we decided to build a machine learning model to classify the RFPs correctly and provide a way to automate the routing of the RFPs.  The client wanted to break the project into stages so once the initial Proof of Concept was successful, other parts required to automate the whole process would receive the go-ahead. If you would like a proof of concept, visit our Business Analytics page for information.

Due to the abbreviated time-period, we decided to use Microsoft’s Azure Machine Learning Studio to build the model.  Azure Machine Learning Studio provided great visualizations of the model for the client.  When developing an end-to-end solution for the client, Azure Machine Learning Service will be implemented.  If you are curious about the differences between Azure ML Studio and Azure ML Service, this article provides an excellent explanation. 


First I looked through the Azure AI Gallery to see if there were any projects that would provide guidance in building our text classifier. I found the BBC News Classifier was a great fit. 


Model Evaluation – Confusion Matrix:  


If the model is built correctly, one should see distribution like what is shown above.  The model assigns a probability per category to reflect its confidence in how to categorize each story.  It is normal for a news story to be classified in one main class, but the model recognizes there is a probability that the story could belong to multiple classes. 

The metrics from the model also showed good accuracy on the model. 


Step 1: Receiving and cleaning the data. 

The client uploaded several RFPs into different folders in Teams that were labeled with the client’s verticals.  One of the challenges not solved in this POC is scraping the data from an RFP.  Our focus was on starting small with the classifier to keep things moving forward. Every municipality creates their own version of an RFP, so most RFPs are not uniform.  For this POC, the RFP summary data was scraped manually and added to a data file. 

Step 2: Create the Model 

To start, we followed the BBC News Classifier model outline.  The R-Script module and the found in the BBC News Classifier were switched with the pre-built Preprocess Text module.  When the initial model was run, it classified all the data into the one bucket that had the largest number of examples.  The model was run again including only data with labels with a high number of examples and a comparable amount in each bucket.  Again, poor results.  Time to re-think the model. 

Microsoft has a great reference library around the modules available in Machine Learning Studio.  While looking through the documentation around Text Analytics, two modules additional modules were found to test: “Extract Key Phrases from Text” and “Extract N-Gram Features from Text.”  The Extracting Key Phrases from Text module extracts one or more phrases deemed meaningful.  The Extract N-Gram Features from Text module creates a dictionary of n-grams from free text and identifies the n-grams that have the most information value.  The new model was run with a Multi-class Decision Forest algorithm instead of the Multi-Class Neural Network.  When the model was run with all the category labels, the results were closer to what was expected, but not yielding accurate results. 

Artboard 6-1
Artboard 7
Artboard 8

One drawback was the labels with minimal data were not classifying correctly.  The model was re-run with only category labels with higher and comparable amounts of data. 

Artboard 10
Artboard 11
Artboard 12

Whoops! That was a step in the wrong direction.  Maybe the n-gram feature wasn’t the best text analytics module to try.  What happens if Feature Hashing is used instead?  Feature Hashing transforms a stream of English text into a set of features represented as integers.  The hashed features can then be passed to the machine learning algorithm to train the text analysis model. 

Artboard 13
Artboard 14
Artboard 15

Well, that accuracy is much better but maybe a bit too good.  Even though the lowest number of decision trees, least amount of depth, and the least number of random splits were used the accuracy of the model was too good. We should expect to see some distribution or a small probability that the RFP could be classified in other categories. 

This could be due to the size of the dataset that is being used.  It was good to find out that Feature Hashing does a better job than Extracting Key Phrases from Text or Extracting N-Gram Features.  What happens if a different machine learning algorithm (like the Multi-Class Neural Network) is used? 

Artboard 16
Artboard 17
Artboard 18

This is the best model yet.  Distribution is across category labels as expected.  There is a good chance of overfitting, but that can be worked out with additional data added to the model. 

Since this was the best model yet, it was re-run will data from all category labels. 

Artboard 19
Artboard 20
Artboard 21

Results were encouraging, but clearly more data will be required to appropriately label all categories.  As more data is added, there will be more improvements to the model.  Two options worth considering would be applying an ensemble approach or trying NLP techniques like entity extraction, chunking, or isolating nouns and verbs. 

Step 3:  Automate the Model 

Azure Machine Learning Studio’s option to Set Up a Web Service was used to create a Predictive Experiment and deploy as a web service.  Then using the ML Studio add-in in Excel, a template was created where data can be added, the model can be run, and predictions bucketed into a scored probability column. 

Artboard 22

The next step was to create a table that reads the predicted data that can be picked up by a Flow.  The Flow is set up to send a notification to a channel on Microsoft Teams. 

Artboard 23

This is not a final solution.  Several additional steps in a further POC will be needed to be completed to set up a fully automated solution, but the initial results are promising.  What’s important to understand is how flexible this process can be. If the client scoped a different set of requirements, or was in a different industry, we could easily tailor a solution to fit their pain. 

How AI Is used in the Algorithmic Trading Sector

Artificial intelligence (AI) is rapidly transforming the global financial services industry, playing a key role in everything from fraud detection and compliance to banking chatbots and robo-advisory services. It’s also changing the ever-evolving world of algorithmic trading helping to eliminate human error and streamlining decision-making processes. But, how exactly is AI utilised in this sector and what are the overall benefits? Let’s take a closer look.

What exactly is AI?

Before you can really get to grips with how AI is used in the algorithmic trading sector, you must first understand what it is. Coined in 1955 by John McCarthy, AI is a term which describes the intelligence displayed by machines, in contrast to the natural intelligence displayed by humans. AI systems will typically demonstrate at least some of the following behaviors including planning, learning, reasoning, problem-solving, knowledge representation and perception.

Important AI applications

While AI is rather broad by definition, there are specific branches that play a prominent role within the algorithmic trading sector including ‘machine learning’ (ML). Named by Arthur Samuel of IBM in 1959, ML is an AI application that focusses on the idea that machines can learn for themselves by accessing Big Data. Such systems can automatically improve based on experience, without being explicitly programmed.

‘Deep learning’ (DL) is another AI concept and a branch of ML which revolved around problem-solving. Such networks do not necessarily need structure or labels to make sense of data. You may have also across ‘neural networks.’ These have AI roots and are inspired by the way humans think. They’re becoming increasingly integrated into today’s AI-related trading world.

Algorithmic trading uses powerful computers, running complex mathematical formulas, to generate returns. This is very different from days gone by where humans used to crowd busy exchanges or pick out the best assets to buy and sell from an office.

Sophisticated algorithms now play a significant role in market transactions and while algorithmic trading isn’t necessarily new, artificial intelligence is giving algorithmic traders extra tools to enhance their performance. Indeed, feeding AI predictions into algorithms can give you a more solid overview of the market including when to enter and exit positions and the best assets to long and short.

So, how exactly does AI tie in with today’s algorithmic trading sector?

Well, algorithmic trading is all about executing orders using automated and pre-programmed trading instructions, accounting for numerous variables such as volume, price and time. Algorithmic trading nowadays involves the use of complex AI systems with computers generating 50-70% of equity market trades, 60% of futures trades and 50% of treasuries. The benefits of AI in algorithmic trading.

Fast trading speeds and improved accuracy

When it comes to algorithmic trading, large numbers of orders are executed within seconds adding liquidity to the market. High-Frequency Trading (HFT) of this kind happens in a fraction of a second and simply can’t be done by humans alone – that’s why algorithms are needed to execute and place bids before the market changes.

Automation streamlines the entire process with AI and machine learning adding an extra clever twist. Essentially ML computer systems are trained to recognise market movements with impressive accuracy, helping algorithms to bid accordingly. By accessing and understanding large data sets, ML systems can predict future outcomes, enhance trading strategies and tweak portfolios accordingly.

AI-enhanced algorithmic trading therefore helps to improve the performance and meet the demands of target clientele including hedge funds, propriety trading houses, corporates, bank propriety trading desks and next-generation marketing makers.

Elimination of human error

Algorithmic trading also helps to reduce errors based on emotional and psychological factors. Often, traders let past trades, FOMO or market pressures affect their judgement and this can lead to poor decision making.

But with algorithmic trading, algorithms are used to ensure trader order placement is instance and accurate – based on pre-defined sets of instructions.

With the help of AI, it’s also possible for computer systems to check multiple market conditions and adjust trades instantly depending on the market environment. Of course, if this were to be done manually, it would take hours and hours of physical labour, research and fact-checking. And even then, errors might occur. Opportunities are likely to be missed too which is why AI is rapidly being integrated into financial institutions and shaping the sector significantly.

AI and algorithmic trading in the real world

AI is not just something that’s being talked about. It’s already here and changing the financial world significantly, especially when it comes to trading practices. Top financial institutions including UBS and JP Morgan have already introduced AI into their trading tools with the former using AI techniques to trade volatility (which is notoriously difficult to navigate) and the latter using AI algorithms to execute equity trades. Algorithms enhanced by AI are also being used to guide venture capitalist investments.

So, as you can see, AI is being increasingly utilised in the algorithmic trading sector and offers many benefits. As 80% of all data is completely unstructured, AI and its complexed applications including ML and DL aims to deliver a more structured, organised and data-fuelled approach to the trading world, helping to make the whole process efficient, while providing split-second insights.

Overview Of Azure Kinect

Pre-requisite Knowledge

 Before we start with the understanding of what is Azure Kinect, we should know,

Background I would like to explain the short information about ‘Artificial Intelligence and Kinect’ before jumping in to ‘Azure Kinect’. 

What is Artificial Intelligence

 In simple words ‘Artificial Intelligence (AI)’ is the artificial creation of the system like a human who can observe, react, learn, plan and process the instructions, virtual reality and provide intelligence on it. It is rapidly emerging technology and internet enable technology. Sometimes AI is also called as Machine Learning. 

What is Kinect and its background

 Kinect is the motion sensor device using in Xbox 360 gaming console. This device provides natural user interface to interact with it without any intermediate device. This device has capability of face detection as well as the voice recognition. This device has 3D camera which creates the virtual images and with the help of motion sensor it detects the movements of the images. The first-generation Kinect for Xbox 360 was introduced in November 2010. This device was originally created for gaming purpose, but now a days this technology is applying to real worlds applications in the virtual shopping, education, healthcare industries, digital signage etc. This product is developed by Microsoft. 

Introduction of Azure Kinect

 As I explained above Kinect is the motion sensor device. Azure Kinect device has,

  1. DK camera system
  2. 1MP depth camera
  3. 360-degree microphone
  4. 12MP RGB camera
  5. Orientation senor
  6. Size and weight – 103 x 39 x 126 mm and weighs only 440g
Overview Of Azure Kinect

Image Source – Microsoft Docs Azure Kinect has ability to create platform for developers with Artificial tools and plug this in to the Azure cloud for cloud-based service, computer vision and speech models. Azure Kinect has its own developer kit (DK) by Microsoft which is available in the portal site here. Microsoft Azure Kinect SDK has new sensor SDK, body tracking SDK, vision APIs, speech service SDK for Azure Kinect DK. This is the latest released feature by Microsoft for Azure cloud. Please note that Azure Kinect DK is not designed for use with Xbox. By using Azure Kinect, now we can build the applications like cashier less stores, manage inventory of the products, track the patient movements integrate these motions with the AI in hospital, enhance physical therapy, improve and monitor athletic performance, computer vision and speech models etc. We can enhance feature of Azure Kinect application with Azure cognitive services. Transcribe and translate speech in real time using Speech Services. Add object, scene, and activity detection or optical character recognition using Computer Vision or use Azure IoT Edge to manage PCs connected to your Azure Kinect DK device. 

Overview Of Azure Kinect

Image Source – Microsoft Docs Azure Kinect device price is $399.00 and can be purchased from Microsoft’s store here. As of now (12th August 2019) this product is only available in the US and China. 

Inside of Azure Kinect DK

Overview Of Azure Kinect

 Image Source – Microsoft Docs

  1. 1MP depth sensor with FOV option
  2. 7-microphone array for speech and sound capture
  3. 12-MP RGB video camera for an additional color stream
  4. Accelerometer and gyroscope (IMU) for sensor orientation and spatial tracking
  5. External sync pins to easily synchronize sensor streams from multiple Kinect devices
  6. Azure Kinect Developer Kit
  7. Purchase Azure Kinect from Microsoft Store

7 Benefits of Artificial Intelligence for Business

Artificial Intelligence can have a very positive impact on a business. Here’s how…

One of the newest benefits of cloud computing is that it enables businesses to take advantage of artificial intelligence (AI). This rapidly developing technology offers significant development opportunities that many companies have already been quick to seize upon. In this post, we’ll look at some of the ways your company can benefit from cloud-based AI.

1. Improving personalised shopping experiences

Providing customers with personalised marketing increases engagement, helps generate customer loyalty and improves sales. This is why companies are putting so much effort into it. One of the advantages of using AI is that it is able to identify patterns in customers’ browsing habits and purchasing behaviour. Using the millions of transactions stored and analysed in the cloud, AI is able to provide highly accurate offers to individual customers.

2. Automating customer interactions

Most customer interactions, such as emails, online chat, social media conversations and telephone calls, currently require human involvement. AI, however, is enabling companies to automate these communications. By analysing data collected from previous communications it is possible to program computers to respond accurately to customers and deal with their enquiries. What’s more, when AI is combined with machine learning, the more the AI platforms interact, the better they become.

One example of this is AI Chatbots which, unlike humans, can interact with unlimited customers at the same time and can both respond and initiate communication – whether on a website or an app.

It is estimated, that by 2020, 85 percent of all customer interactions will be taken care of by intelligent machines that are able to replicate human functions. The days of using a call centre look like they are coming to a close.

3. Real-time Assistance

AI is also useful for businesses that need to constantly communicate with high volumes of customers throughout each day. For example, in the transport industry, bus, train and airlines companies, which can have millions of passengers a day, can use AI to interact, in real-time, to send personalised travel information, such as notice of delays. Some bus companies, for example, are already tracking the location of their buses and using AI to provide travellers with real-time updates about where the bus is along its route and its estimated time of arrival. Customers receive this information on the bus company’s app.

4. Data mining

One of the biggest advantages of using cloud-based AI is that artificial intelligence apps are able to quickly discover important and relevant findings during the processing of big data. This can provide businesses with previously undiscovered insights that can help give it an advantage in the marketplace.

5.Operational automation

AI is able to operate other technologies that increase automation in business. For example, AI can be used to control robots in factories or maintain ideal temperatures through intelligent heating. In Japan, human-looking robots now serve as receptionists in some of the countries’ hotels automating check-ins, booking services and dealing (in four languages) with customer enquiries. In retail, AI is also being linked with RFID and cloud technology to track inventory. In China, police forces use AI to catch criminals. The country has a vast CCTV network and AI uses facial recognition to spot and track suspects so that they can be apprehended.

6.Predicting outcomes

Another advantage of AI is that it is able to predict outcomes based on data analysis. For example, it sees patterns in customer data that can show whether the products currently on sale are likely to sell and in what volumes. It will also predict when the demand will tail off. This can be very useful in helping a company purchase the correct stock and in the correct volumes. It is predicted that, within 10 years, the days of seasonal sales will be over as AI will mean there is too little leftover stock to sell off.

This ability to predict is not just useful in retail. AI is also being used in many other areas, for example, in banking where it can predict currency and stock price fluctuations or in healthcare where, remarkably, it can predict outbreaks of infections by analysing social media posts.

7.Improve the recruitment process

It may be bad news for recruitment companies, but AI is now helping businesses automate the recruitment of new employees. It is able to quickly sift through applications, automatically rejecting those which do not meet the company’s personal specification. This not only saves time (or money spent on a recruitment agency), but it also ensures that there is no discrimination or bias in the shortlisting process. The AI programs available can even take care of the many administrative tasks of recruitment.


As you can see, AI systems provide businesses with a wide range of benefits, including personalised marketing, customer service, operational automation, inventory management and recruitment. And these are just a few of the many ways AI can be used. What’s remarkable, however, is that many of the AI apps, which are designed specifically for cloud-based systems, are quickly and easily deployable. Companies whose systems are in the cloud can be benefitting from them in no time at all.

The Benefits of A.I. Powered Risk and Compliance Automation for Banks

Artificial intelligence-powered solutions are promising to deliver next-gen features to banks. A.I. will allow us to automate and streamline risk and compliance processes. When we look at the benefits these solutions can provide, it becomes clear why banks are implementing risk and compliance technology at an increasingly faster pace.

Faster and better risks assessments

With artificial intelligence in banking automation, it is possible to make a much more comprehensive and detailed risk analysis for all types of risks. When we talk about credit risks, these solutions can consider aspects such as consumption habits, banking history, behavioral patterns, and much more. In short, the ability to add variables to analysis, and to process them more quickly improves exponentially.

We can expect an in-depth analysis of all types of risks. For example, risk management solutions can simultaneously analyze more variables in relation to the country’s political and financial scenario, news, and other market movements to perform predictive analytics that helps the bank to anticipate potential risks and preemptively mitigate them.

Efficiency in combating compliance violations

Machine learning, in conjunction with AI, analyses real-time transactions and information to detect violations and give alerts in suspicious situations, mainly by assessing patterns in the company’s database. Banks can instantly detect if an account opening officer did not collect all the required documentation. The audit trail created by these solutions also provides a great countermeasure to fraud and compliance-related issues. Every action taken by employees when working with risk and compliance documentation is logged into the system, allowing banks to quickly detect anomalies and discover the source of problems.

This helps with regulatory inspections as well. Instead of having to dedicate resources to create extensive reports that show that the bank is taking the right steps to improve its risk and compliance framework, banks can now automatically generate reports and streamline the flow of risk assessments. They can provide extensive and exhaustive audit trails to regulatory bodies, which can reduce the time required for regulatory examinations.

Cost reductions

Manual risk and compliance require extensive labor hours. Risk and compliance are both highly specialized fields that require highly qualified and experienced personnel. Banks often have to limit the risk and compliance capabilities of their organization because the cost to expand risk and compliance teams is not sustainable for the bank’s bottom line. A.I. powered solutions can augment a bank’s risk and compliance teams, allowing them to deliver exponentially more value to the organization.

In this regard, in addition to reducing risks, artificial intelligence greatly reduces the time required for risk and compliance processes, which reduces the costs of operations. In fact, these same professionals can dedicate more time to managerial and decisive activities, while technology takes care of the heavy work and menial tasks that would otherwise take most of their time. This means that banks can increase profitability by reducing cost while still improving the way they manage risk and compliance.

Benefits for customers

A.I. powered solutions also promise to deliver many benefits to the customers of banks. By empowering banks to deliver better and faster compliance, these solutions enable banks to provide better and faster services to customers. Many banking processes can take multiple because of the compliance and risk-related tasks that need to be performed. Banks need to ensure that every account they open and every transaction that goes through the bank is compliant with rules and regulations. This requires a lot of cross-checking and information collection, most of which are handled manually in smaller banks.

Automation will allow banks to speed up banking processes to ensure that banking customers do not have to wait for their transactions to go through. Users will also gain access to more digital services when risk and compliance management are automated. There are many payment platforms available right now which enable instant payments and customers often wonder why their banking applications do not deliver the same functionality. The simplest answer is that the born-digital payment app does not have to worry about the rules and regulations that a bank does. Technology can help close this service gap and enable banks to deliver the functionality that banking customers want.

Want to see how an artificial intelligence-powered solution can help your bank automate risk and compliance processes? Get in touch with our experts to see a demonstration of our artificial intelligence-powered solution that has been endorsed by the American Bankers Association.

How to Create an Emotional Connection with Your Fans

There is a huge issue that many brands, sports teams, and anyone offering a service face today when it comes to building their fanbase. They do not put in the time or effort to make an emotional connection. Too many people want to skip the “real work” and go right to the part where they have engagement and a loyal fanbase. Without creating this emotional connection, this is virtually impossible. 

While it may seem like an impossible task, creating an emotional connection is not as hard as it may seem. 

Consider Your Fan’s Journey

The fan’s journey means how they find out about you and how they become a fan initially. What is the journey they have gone on to discover you and fall in love with your team? 

Usually, this is going to happen via television, online, or from personal experience. Once a potential fan gets a “whiff” of what you offer, what you stand for, and what you can do, they will want to learn more. Where will this journey go? This is what you have to consider. Take time to connect the dots to provide the best fan journey possible. 

Remember, a crucial part of any fan’s journey is likely going to be social media. They want to see what their friends say and what random people have to say. Be sure to monitor what is going on and being said to put your best foot forward. 

Build an Email List Then Use It

If you are a fan of something, you want to learn more about it, right? Your fans are going to want to hear from you. It is up to you to keep this line of communication open. While it may seem daunting at first, there are an array of methods you can use to make email marketing easier and more efficient, including automation. Take time to build an email list and then use it to your advantage. By connecting with your fans in this manner, you will be building that ever-important emotional connection. 

Take Your Fans Behind the Scenes 

One of the fastest and most effective ways to build an emotional connection with your fans is by taking them behind the scenes. Use features like Facebook Live and Instagram Live to give your fans a glimpse of something unique. When you provide an exclusive look behind the scenes, your fans will appreciate it and want to see more. Also, this helps them go on the journey with you, which is going to deepen their loyalty. 

If the process of building an emotional connection with your fan base seems daunting, do not worry. The professionals can help ensure you create the connections that mean the most and that you are able to build something that will last. By using the tips here, you will create loyal fans that stick with you for life.