Introduction:
The day is not too far when we talk to computers and they perform required action based on human voice instruction and respond interactively. Here we present to you, a new way of interaction with MII Applications. Imagine a world where shop floor user just says – “Post Confirmation”, “Quantity 200”, “Scrap 5”, “For Order 10001234” and the system responds automatically by posting confirmation to SAP based on this voice command. That’s what we are talking about and details of implementation are as below.
Demo:
Advantages:
- Shop Floor workers can now use MII application with simple voice instructions.
- Faster end user training.
- Many shop floor users are not used to computers as it’s not a part of their day to day job. Now, they don’t have additional burden to get trained on computers to just to use MII applications.
Limitations:
- This particular API works only with Chrome browser.
- Server should be SSL enabled if we don’t want to be prompted to allow or deny the use of microphone every time.
- End users may have to learn and voice out the instructions multiple times in case system is unable to understand as Voice APIs are still experimental.
Step by Step Implementation procedure:
STEP 1: To start with we will reuse the code from AJAX Dynamic Editable Data grid using servlet and XSL blog with slight modifications.
- loadXMLDoc() function is called onLoad of the page.
- Input box to enter servlet, populate div and read values buttons have been removed.
- Query output and table fields include – Order Number, Material Number, Material Description, Quantity and Description.
- Goods Issue, Goods Receipt and Confirmation Buttons have been added.
- Please note that the scope of the demo is limited to how technically we can drive the application with voice API only and doesn’t show or supply all required parameters for successful postings in SAP from MII.
-----------------------------------------
BEGIN OF CODE – Main IRPT
-----------------------------------------
<HTML>
<HEAD>
<TITLE>Your Title Here</TITLE>
<SCRIPT language="JavaScript">
function loadXMLDoc()
{
var xmlhttp;
if (window.XMLHttpRequest)
{
xmlhttp=new XMLHttpRequest();
}
Else {
Alert(“Please use Chrome Browser for Voice API to work”);
}
xmlhttp.onreadystatechange=function()
{
if (xmlhttp.readyState==4 && xmlhttp.status==200)
{
document.getElementById('servletDiv').innerHTML=xmlhttp.responseText;
}
}
xmlhttp.open("GET",url,true);
xmlhttp.send();
}
</SCRIPT>
</HEAD>
<BODY onLoad="loadXMLDoc()">
<table cellpadding="1" cellspacing="5" border="1" width="100%" valign="top">
<tr>
<td>
<div id="servletDiv"></div>
</td>
</tr>
<tr>
<td>
<input type="button" value="Goods Issue" onClick="showPopup('GoodsIssue')"> </input>
<input type="button" value="Confirmation" onClick="showPopup('Confirmation')"> </input>
<input type="button" value="Goods Receipt" onClick="showPopup('Goods Receipt)"> </input>
</td>
</tr>
</table>
</BODY>
</HTML>
-----------------------------------------
END OF CODE
-----------------------------------------
Test the IRPT. If XSL is correct, you will be able to see a table similar to below image depending on the data in query template.
STEP 2: Row Selection
When a row is selected, we would like to assign the value of the position to a global variable to make it easy for us to read values of selected row.
- Create a global variable: var selectedRowPosition="";
- Modify the XSL, append position attribute and onClick function to the radio button element.
-----------------------------------------
BEGIN OF CODE – Main IRPT
-----------------------------------------
<input type="radio" name="RadioButtonName" onClick="radioButtonSelected(this.id)">
<xsl:attribute name="id"><xsl:value-of select="concat('RadioButton_',position())"/></xsl:attribute>
</input>
-----------------------------------------
END OF CODE
-----------------------------------------
- Define radioButtonSelected() function in JavaScript Main.IRPT. We will set the focus to Input Box for entering quantity when radio button is selected.
-----------------------------------------
BEGIN OF CODE – Main IRPT
-----------------------------------------
function radioButtonSelected(buttonID){
var position = buttonID.substring(12);
selectedRowPosition=position;
document.getElementById('InpBox_'+position).focus();
}
-----------------------------------------
END OF CODE
-----------------------------------------
STEP 3: Define a generic showPopup() function, this function is called when Goods Issue, Goods Receipt and Confirmation buttons are clicked. We will open up a window pop-up to show the selected transaction name, order number, material, Qty to be posted and Description.
-----------------------------------------
BEGIN OF CODE – Main IRPT
-----------------------------------------
function showPopup(screenName){
var orderNum = document.getElementById('orderNum_'+selectedRowPosition).innerHTML;
var material = document.getElementById('orderNum_'+selectedRowPosition).innerHTML;
var Qty = document.getElementById('InpBox_'+selectedRowPosition).value;
var Description = document.getElementById('Desc_'+selectedRowPosition).value;
var myWindow=window.open('','','width=200,height=400');
myWindow.document.write("<p>"+screenName+"</p><br>"+"<p>Order Number : "+orderNum+"</p><br>"+"<p>Material : "+material+"</p><br>"+"<p>Qty : "+Qty+"</p><br>"+"<p>Description : "+Description+"</p>");
myWindow.focus();
}
-----------------------------------------
END OF CODE
-----------------------------------------
Select 3rd Row, provide quantity and description. Then mouse click on each of these buttons it should result in:
STEP 4: Now that we have an MII application which is working with mouse clicks. Let’s now Integrate Speech Recognition API to it to drive it with voice commands.
- While this API works for several languages and regional dialects, I am limiting this sample for just English and five widely used English Regional dialects. More information about languages in provided in the API documentation.
Add this code just below <body> tag
-----------------------------------------
BEGIN OF CODE – HTML
-----------------------------------------
<table border="0" width="100%" valign="top">
<tr>
<td> Select Language:
<select>
<option> English</option>
</select>
Select Regional Dialect:
<select id="selectCountry">
<option value="en-US"> United States</option>
<option value="en-GB"> United Kingdom</option>
<option value="en-CA"> Canada</option>
<option value="en-IN"> India</option>
<option value="en-ZA"> South Africa</option>
</select>
</tr>
<tr>
<td>
<button id="startID" onclick="startCapturingVoice(event)">
<img alt="Start" id="micImgID"src="micOFF.gif"></button>
<textarea rows="2" cols="75" id="convertedSpeechID" style="font-size:20" readonly></textarea>
<div id="messageID">Click on the microphone icon and provide voice command.</div>
</td>
</tr>
</table>
-----------------------------------------
END OF CODE
-----------------------------------------
- After adding the above code, the screen should look like:
STEP 5: Code to use Speech Recognition API.
- webkitSpeechRecognition() object does this magic. It provides the Speech Interface and it currently works with only Chrome browser v25 and above. “Webkit” is the vendor name as it is currently experimental.
- This is a standard code and there is no need to customize this.
Add this code in JavaScript section.
-----------------------------------------
BEGIN OF CODE – JavaScript
-----------------------------------------
var voice = new webkitSpeechRecognition();
var convertedSpeech = '';
var capturingVoice = false;
var ignoreEnding;
var timestamp;
voice.onstart = function() {
capturingVoice = true;
displayMessage('start_speaking');
document.getElementById("micImgID").src = 'micON.gif';
};
voice.onerror = function(event) {
if (event.error == 'no-speech') {
document.getElementById("micImgID").src = 'micOFF.gif';
displayMessage('speech_not_detected');
ignoreEnding = true;
}
if (event.error == 'audio-capture') {
document.getElementById("micImgID").src = 'micOFF.gif';
displayMessage('microphone_not_detected');
ignoreEnding = true;
}
if (event.error == 'not-allowed') {
if (event.timeStamp - timestamp < 100) {
displayMessage('microphone_blocked');
} else {
displayMessage('perm_denied');
}
ignoreEnding = true;
}
};
voice.onend = function() {
capturingVoice= false;
if (ignoreEnding) {
return;
}
document.getElementById("micImgID").src = 'micOFF.gif';
if (!convertedSpeech) {
displayMessage('initialize');
return;
}
displayMessage('');
};
voice.onresult = function(event) {
if (typeof(event.results) == 'undefined') {
voice.onend = null;
voice.stop();
upgradeBrowser();
return;
}
for (var i = event.resultIndex; i < event.results.length; ++i) {
convertedSpeech += event.results[i][0].transcript;
}
document.getElementById("convertedSpeechID").value = convertedSpeech;
callMIIFunctions(convertedSpeech);
};
function upgradeBrowser() {
startID.style.visibility = 'hidden';
displayMessage('not_supported');
}
function startCapturingVoice(event) {
if (capturingVoice) {
voice.stop();
return;
}
convertedSpeech = '';
voice.lang = selectCountry.value;
voice.start();
ignoreEnding = false;
document.getElementById("convertedSpeechID").value = '';
displayMessage('click_allow');
timestamp = event.timeStamp;
}
function displayMessage(messageKey) {
switch(messageKey)
{
case 'initialize':
document.getElementById("messageID").innerHTML="Click on the microphone icon and provide voice command.";
break;
case 'start_speaking':
document.getElementById("messageID").innerHTML="Start Speaking.";
break;
case 'speech_not_detected':
document.getElementById("messageID").innerHTML="Speech was not recognized.";
break;
case 'microphone_not_detected':
document.getElementById("messageID").innerHTML="Microphone was not found.";
break;
case 'click_allow':
document.getElementById("messageID").innerHTML="Click on Allow button above to enable microphone.";
break;
case 'perm_denied':
document.getElementById("messageID").innerHTML="Microphone cannot be used, permission is denied.";
break;
case 'microphone_blocked':
document.getElementById("messageID").innerHTML="Microphone is blocked, permission is denied.";
break;
case 'not_supported':
document.getElementById("messageID").innerHTML="This browser doesn't support Voice API, upgrade it to Chrome browser.";
break;
default:
document.getElementById("messageID").innerHTML="";
}
}
-----------------------------------------
END OF CODE
-----------------------------------------
STEP 6: JavaScript function to recognize customized voice commands.
- We are calling a function callMIIFunctions(convertedSpeech) in the above JavaScript code in voice.onResult and passing the text which is converted from speech (Voice command), we will use this converted Speech to recognize the command and perform required action is this function.
- In this function we are:
- First checking if “Confirmation” keyword exists in the voice command, if so, we call showPopup(‘confirmation’) function which we had defined earlier, similarly we call showPopup(‘Goods Issue’) and showPopup(‘Goods Receipt’) for “Issue” and “Receipt” voice commands respectively.
- Then we check for “Select Row number n” voice command, where n is position number between 1 and 9, if “Select Row” word exists, then we substring and get the last digit in the voice command and select the row specified based on the position.
- Then we check for only number in the voice command, if only number exists, number will be populated in Quantity input box.
- Then we check for “Description” keyword in the voice command, if this exists, we truncate description keyword and populate rest of the sentence in Description input box.
- You can customize this code to call any required JavaScript function based on any custom Voice command as per your requirement.
- NOTE: This is just a sample code mainly focusing on various ways to use voice commands in MII applications. Once you get the idea, you can optimize the code; there are better ways to implement and use it which is not in the scope of this documentation.
---------------------------------------------------
BEGIN OF CODE – callMIIFunctions function
---------------------------------------------------
function callMIIFunctions(convertedSpeech){
//alert(convertedSpeech);
if(convertedSpeech.indexOf("confirmation")!=-1){
showPopup('Confirmation');
}else if(convertedSpeech.indexOf("receipt")!=-1){
showPopup('Goods Receipt');
}else if(convertedSpeech.indexOf("issue")!=-1){
showPopup('Goods Issue');
} else if(convertedSpeech.indexOf("select row")!=-1){
selectRadioButton(convertedSpeech.substring(convertedSpeech.length-1,convertedSpeech.length));
} else if(!isNaN(convertedSpeech)){
document.getElementById('InpBox_'+selectedRowPosition).value=convertedSpeech;
} else if(convertedSpeech.indexOf("description")!=-1){ document.getElementById('Desc_'+selectedRowPosition).value=convertedSpeech.substring(11,convertedSpeech.length);
}
}
-----------------------------------------
END OF CODE
-----------------------------------------
STEP 7: JavaScript function to select Radio button:
- Since we already have showPopup() function defined, we just need to define a function to select radio button.
---------------------------------------------------
BEGIN OF CODE – selectRadioButton function
---------------------------------------------------
function selectRadioButton(position){
radioButtonSelected('RadioButton_'+position);
document.getElementById('RadioButton_'+position).checked=true;
}
-----------------------------------------
END OF CODE
-----------------------------------------
STEP 8: Voice Commands:
That is it. We are now done with coding, it’s time to have fun testing it and see the results.
- Make sure you are using Chrome Browser v25 and above.
- Select the language and Regional Dialect. Selecting correct regional Dialect will provide close to accurate results. If you are a native American English Speaker use United States, If you are a native Queen’s English Speaker then use United Kingdom, If you are an Indian like me use India.
- Click on microphone icon/button and start saying the voice commands.
- Voice Command: “Select Row Number 1” Result: Should select the first Row.
- Voice Command: “200” Result: Should populate Quantity 200 in the first Row’s quantity field
- Voice Command: “Description This is test description” Result: Should populate “This is test description” in first Row’s description field.
- Voice Command: “Confirmation” Result: Should show Confirmation pop-up with details mentioned above.
- Voice Command: “Issue” or “Goods Issue” Result: Should show Goods Issue pop-up with details mentioned above.
- Voice Command: “Receipt” or “Goods Receipt” Result: Should show Goods Receipt pop-up with details mentioned above.
- NOTE: You may have to “Allow” microphone every time you want to say a voice command. This can be avoided if sever is SSL enabled, this means that URLs begin with HTTPS.
- TIPS: Give a small pause between words while saying voice commands. Eg., Select <pause> Row <pause> Number <pause> 1. You may have to give a couple of tries if your voice is not being recognized properly.
THE END
References:
- SAP Managed Tags:
8 Comments
