It doesn't take a sledgehammer to crack a nut
A phrase that became popular in the 1950's in the US apparently - It refers to the idea, that you don't always need a huge solution/effort to solve a problem. Recently I joined up with a start-up consulting company and one of the things we needed was an SAP system for us to demo and practice our skills on. The company I had just left was (by the time I left) quite a big one, with its own servers, infrastucture and air conditioners that ran full tilt in the server room. As a small start up, we couldn't really afford that kind of infrastructure, so we had to think of other ways of doing it.
What we came up with was a solution where we ran our demo IDES system (including NW7.02 portal) on an Amazon Web Services EC2 instance. This meant we could get great performance (very powerful virtual hardware) and easy access. And what was more, when we weren't using it, we didn't have to pay for it! At less than $2 per hour to run our system, it's an incredibly cheap and well performing option.
But it's never as easy as it first looks
But we soon came across a bit of an issue. We wanted to be able to start and stop our instance at specific times. We wanted to start it in the morning, and shut it down at night. The problem was - how? My colleague Fendy Wongso, found an iPhone App (of course there was an apps for it!) that allowed you to start and stop an AWS EC2 instance.
We set the instance up to start the SAP instances when it was started. So far so good. Unfortunately stopping the EC2 instance whilst all our SAP systems were running (not to mention the underlying DB) was not a good idea! So another app was used to remote desktop onto the EC2 instance and trigger the SAP systems to shutdown, and then close down the instance. I've used an RDP app on my Xoom (Android ICS tablet) and it's not the easiest thing to do.
Fendy found an app that we could run on my home HTPC (which runs 24/7) to start the system every morning (well it was supposed to be every morning, but what with my son turning off the computer (that's what 2 year olds do!) and the utter dodgyness of my home internet connection (random outages that drive me crazy) it wasn't quite every morning!) We scheduled the system to shutdown at 7pm each night (we try to force people to get a work life balance - unfortunately this doesn't work with me because I know how to start it up again 🙂 But it was soon clear that either Fendy or I would be constantly logging on to either start or stop the instance outside of the standard hours as project needs dictated, or our team logged on to do some work (or in my case play).
I'd been looking at building a Google Apps Engine (GAE) application for a long time (I'd really wanted to use it to do some cool push notifications to Android phones actually - but that will have to wait a bit longer.) However, I'd never really found a need -as most of my development time was spent around SAP instances. But this sounded like something I could really use! Unlike AWS, the GAE doesn't give you your own server, but allows you to run some code on the Google App Engine infrastructure. There are lots of lovely explanations out there which go into more detail - but the most important detail for me was that if your app uses minimal resources, you don't have to pay a cent! Now that's something I like 🙂
So this was my design - in very rough...
Let me break it down a little.
The GAE is running some Java servlets. These can be "activated" by calling them. Some might be tempted to call them REST resources just because you can call them from a browser and get an response that you can understand, but then I might have to set Sasha on them to explain the difference. I have one Servlet called "StartInstance" and one called "StopInstance" one called "CheckInstance" and finally one called "CheckSystem".
These servlets are called by the HTML page that I have used as the default entry point of my GAE app. The HTML page just uses AJAX calls to the servlets to return data about the status of the SAP system and to stop/start it. As the page and the servlets are hosted on the same domain, there are no cross domain issues and it all works very nicely. I coded my servlets to respond in either XML or JSON - both of which are relatively easy to consume in ECMAScript (that's JavaScript for you old timers (oops I put Javascript in my diagram, guess I'm getting old too).
Here's a quick example of one of the functions that is getting some data and updating the screen.
var checkServerStatus = function() {
$
.ajax("checkinstance")
.done(
function(response) {
var status = $.parseJSON(response).status;
if (status == "Running") {
status = "SAP Instance Running"
$("#startButton").button().button("disable");
$("#stopButton").button().button("enable");
} else {
status = "SAP Instances not reachable - checking again in <span id=\"countdown\">30</span> secs"
setTimeout("checkServerStatus();", 30000);
doCountDown();
}
document.getElementById("status").innerHTML = "<b>"
+ status + "</b>";
})
}
The observant amongst you will have noticed the little $ signed splashed about - a clear indication that jQuery is being used to make my life as simple as possible.
The servlets would then talk to either AWS or the SAP ICF to do stuff.
The servlets can be triggered by anything that can talk to them - I've set up a cron job on the GAE (standard feature of GAE) to start the server every weekday at 7am. A very easy task, just include the cron.xml file below in the application deployed to GAE.
<?xml version="1.0" encoding="UTF-8"?>
<cronentries>
<cron>
<url>/startinstance</url>
<description>Start the demo system</description>
<schedule>every mon, tue, wed, thu, fri 07:00 </schedule>
<timezone>Australia/Melbourne</timezone>
</cron>
</cronentries>
Starting the server
So every weekday morning at 7, the cron job fires up the StartInstance servlet - what does it do? It calls AWS using the very useful HTTP based Query API. This API allows you to do pretty much anything to your AWS system, and returns the results as an XML message. The following is my code that puts together the "query" URL that starts my instance.
SignedRequestsHelper signer = SignedRequestsHelper.getInstance(
AccessDetails.getEndpoint(), AccessDetails.getAwsAccessKeyId(),
AccessDetails.getAwsSecretKey());
Map<String, String> params = new HashMap<String, String>();
params.put("Version", "2012-04-01");
params.put("Action", "StartInstances");
params.put("InstanceId.1", AccessDetails.getInstanceId());
String urlString = signer.sign(params);
URL url = new URL(urlString);
However, in order to get AWS to do anything for you, you have to authenticate yourself to it. AWS does this by making you sign all your requests to it with a secret key that only you and Amazon know.
Now unfortunately Amazon being as security conscious as they are, have recently updated all their security/signing algorithms - they have kindly made libraries available in all sorts of languages. Unfortunately the latest version uses HTTPClient (a very useful utility class for HTTP comms in Java). GAE doesn't support much of the code in HTTPClient and thus it doesn't work. On the other hand - they don't supply any versions of the old libs. So I went searching for some example implementations. And a found a few - although they were all so old that they implemented an authentication API that Amazon had switched off because of a security flaw...
A fair bit of bashing my head against the wall later (and finding a chunk of code that looked like I could base mine one) and I dug out Amazon's guide to the security API and built my own implementation. This was a very tricky task, as Amazon, unlike Google's OAuth Playground, doesn't really give you any tools to help you check if what you've built is working or not. You either get an error or you get in... For those that are interested here is my implementation. (It's under the Apache License as I based it on code that was licensed that way - so you'll not see this code creep into Code Exchange I fear!)
/**********************************************************************************************
*
* Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file
* except in compliance with the License. A copy of the License is located at
*
* http://aws.amazon.com/apache2.0/
*
* or in the "LICENSE.txt" file accompanying this file. This file is distributed on an "AS IS"
* BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations under the License.
*
* ********************************************************************************************
*
* Based on code found in the Sample Code area of
* Amazon Product Advertising API (which is almost the same as the EC2 version)
*
*
*/
package au.com.discoveryconsulting;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.net.URLEncoder;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
import java.util.SortedMap;
import java.util.TimeZone;
import java.util.TreeMap;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import org.apache.commons.codec.binary.Base64;
/**
* This class contains logic for signing requests to the Amazon EC2 Query API.
*/
public class SignedRequestsHelper {
/**
* All strings are handled as UTF-8
*/
private static final String UTF8_CHARSET = "UTF-8";
/**
* The HMAC algorithm required by Amazon
*/
private static final String HMAC_SHA256_ALGORITHM = "HmacSHA256";
/**
* This is the URI for the service, don't change unless you really know what
* you're doing.
*/
private static final String REQUEST_URI = "/";
/**
* The sample uses HTTP GET to fetch the response. If you changed the sample
* to use HTTP POST instead, change the value below to POST.
*/
private static final String REQUEST_METHOD = "GET";
private String endpoint = null;
private String awsAccessKeyId = null;
private String awsSecretKey = null;
private SecretKeySpec secretKeySpec = null;
private Mac mac = null;
/**
* You must provide the three values below to initialize the helper.
*
* @Param endpoint
* Destination for the requests.
* @Param awsAccessKeyId
* Your AWS Access Key ID
* @Param awsSecretKey
* Your AWS Secret Key
*/
public static SignedRequestsHelper getInstance(String endpoint,
String awsAccessKeyId, String awsSecretKey)
throws IllegalArgumentException, UnsupportedEncodingException,
NoSuchAlgorithmException, InvalidKeyException {
if (null == endpoint || endpoint.length() == 0) {
throw new IllegalArgumentException("endpoint is null or empty");
}
if (null == awsAccessKeyId || awsAccessKeyId.length() == 0) {
throw new IllegalArgumentException(
"awsAccessKeyId is null or empty");
}
if (null == awsSecretKey || awsSecretKey.length() == 0) {
throw new IllegalArgumentException("awsSecretKey is null or empty");
}
SignedRequestsHelper instance = new SignedRequestsHelper();
instance.endpoint = endpoint.toLowerCase();
instance.awsAccessKeyId = awsAccessKeyId;
instance.awsSecretKey = awsSecretKey;
byte[] secretyKeyBytes = instance.awsSecretKey.getBytes(UTF8_CHARSET);
instance.secretKeySpec = new SecretKeySpec(secretyKeyBytes,
HMAC_SHA256_ALGORITHM);
instance.mac = Mac.getInstance(HMAC_SHA256_ALGORITHM);
instance.mac.init(instance.secretKeySpec);
return instance;
}
/**
* The construct is private since we'd rather use getInstance()
*/
private SignedRequestsHelper() {
}
/**
* This method signs requests in hashmap form. It returns a URL that should
* be used to fetch the response. The URL returned should not be modified in
* any way, doing so will invalidate the signature and Amazon will reject
* the request.
*/
public String sign(Map<String, String> params) {
// Let's add the AWSAccessKeyId and Timestamp parameters to the request.
params.put("AWSAccessKeyId", this.awsAccessKeyId);
params.put("Timestamp", this.timestamp());
params.put("SignatureVersion", "2");
params.put("SignatureMethod", HMAC_SHA256_ALGORITHM);
// The parameters need to be processed in lexicographical order, so
// we'll
// use a TreeMap implementation for that.
SortedMap<String, String> sortedParamMap = new TreeMap<String, String>(
params);
// get the canonical form the query string
String canonicalQS = this.canonicalize(sortedParamMap);
String queryString = this.queryString(sortedParamMap);
// create the string upon which the signature is calculated
String toSign = REQUEST_METHOD + "\n" + this.endpoint + "\n"
+ REQUEST_URI + "\n" + canonicalQS;
// get the signature
String hmac = this.hmac(toSign);
// construct the URL
String url = "https://" + this.endpoint + REQUEST_URI + "?"
+ queryString + "&Signature=" + hmac;
return url;
}
/**
* This method signs requests in query-string form. It returns a URL that
* should be used to fetch the response. The URL returned should not be
* modified in any way, doing so will invalidate the signature and Amazon
* will reject the request.
*/
public String sign(String queryString) {
// let's break the query string into it's constituent name-value pairs
Map<String, String> params = this.createParameterMap(queryString);
// then we can sign the request as before
return this.sign(params);
}
/**
* Compute the HMAC.
*
* @Param stringToSign
* String to compute the HMAC over.
* @return base64-encoded hmac value.
*/
private String hmac(String stringToSign) {
String signature = null;
byte[] data;
byte[] rawHmac;
String base64Signature;
try {
data = stringToSign.getBytes(UTF8_CHARSET);
rawHmac = mac.doFinal(data);
base64Signature = new String(Base64.encodeBase64(rawHmac), UTF8_CHARSET);
signature = percentEncodeRfc3986(base64Signature);
} catch (UnsupportedEncodingException e) {
throw new RuntimeException(UTF8_CHARSET + " is unsupported!", e);
}
return signature;
}
/**
* Generate a ISO-8601 format timestamp as required by Amazon.
*
* @return ISO-8601 format timestamp.
*/
private String timestamp() {
String timestamp = null;
Calendar cal = Calendar.getInstance();
DateFormat dfm = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'");
dfm.setTimeZone(TimeZone.getTimeZone("GMT"));
timestamp = dfm.format(cal.getTime());
return timestamp;
}
/**
* Canonicalize the query string as required by Amazon.
*
* @Param sortedParamMap
* Parameter name-value pairs in lexicographical order.
* @return Canonical form of query string.
*/
private String canonicalize(SortedMap<String, String> sortedParamMap) {
if (sortedParamMap.isEmpty()) {
return "";
}
StringBuffer buffer = new StringBuffer();
Iterator<Map.Entry<String, String>> iter = sortedParamMap.entrySet()
.iterator();
while (iter.hasNext()) {
Map.Entry<String, String> kvpair = iter.next();
buffer.append(percentEncodeRfc3986(kvpair.getKey()));
buffer.append("=");
buffer.append(percentEncodeRfc3986(kvpair.getValue()));
if (iter.hasNext()) {
buffer.append("&");
}
}
String cannoical = buffer.toString();
return cannoical;
}
private String queryString(SortedMap<String, String> sortedParamMap) {
if (sortedParamMap.isEmpty()) {
return "";
}
StringBuffer buffer = new StringBuffer();
Iterator<Map.Entry<String, String>> iter = sortedParamMap.entrySet()
.iterator();
while (iter.hasNext()) {
Map.Entry<String, String> kvpair = iter.next();
buffer.append(kvpair.getKey());
buffer.append("=");
buffer.append(kvpair.getValue());
if (iter.hasNext()) {
buffer.append("&");
}
}
String queryString = buffer.toString();
return queryString;
}
/**
* Percent-encode values according the RFC 3986. The built-in Java
* URLEncoder does not encode according to the RFC, so we make the extra
* replacements.
*
* @Param s
* decoded string
* @return encoded string per RFC 3986
*/
private String percentEncodeRfc3986(String s) {
String out;
try {
out = URLEncoder.encode(s, UTF8_CHARSET).replace("+", "%20")
.replace("*", "%2A").replace("%7E", "~");
} catch (UnsupportedEncodingException e) {
out = s;
}
return out;
}
/**
* Takes a query string, separates the constituent name-value pairs and
* stores them in a hashmap.
*
* @Param queryString
* @return
*/
private Map<String, String> createParameterMap(String queryString) {
Map<String, String> map = new HashMap<String, String>();
String[] pairs = queryString.split("&");
for (String pair : pairs) {
if (pair.length() < 1) {
continue;
}
String[] tokens = pair.split("=", 2);
for (int j = 0; j < tokens.length; j++) {
try {
tokens[j] = URLDecoder.decode(tokens[j], UTF8_CHARSET);
} catch (UnsupportedEncodingException e) {
}
}
switch (tokens.length) {
case 1: {
if (pair.charAt(0) == '=') {
map.put("", tokens[0]);
} else {
map.put(tokens[0], "");
}
break;
}
case 2: {
map.put(tokens[0], tokens[1]);
break;
}
}
}
return map;
}
Once I could correctly sign my queries to AWS, I could call the interface to start the server. A task on the server was scheduled to run whenever the server started. That ran a batch file that called Zomoni.com (via an HTTPS wget call) to update the servers IP address against the DNS entry (dynamic DNS - a very cool solution). The batch then started the SAP instances. A couple of minutes later and it would be possible to connect to the SAP systems.
Stopping the server
So far, so cool - but the iPhone app (there was also an Android app, but I never bothered to buy it) could start the server just as well (OK it couldn't schedule that to happen... but!) The real problem was safely shutting down the server, whilst first stopping all the SAP instances. To do this, I went back to something that I know and love. A little ICF programming. I built an ICF service.
The service is simple enough - it just checks what you've called and whether you are the authorised user. (I restricted it to one communications user with an impossible to guess password.) From the GAE side it just uses basic authentication to get onto the SAP system - although only HTTPS access is open/allowed - so there should be minimal possibility of password interception. (I'm going to trust HTTPS for this - after all I do for my banking and pretty much everything else!). Within my GAE servlet the code to call the ECC system is below:
String urlString = "https://" + AccessDetails.getSapsystem()
+ "/check";
URL url = new URL(urlString);
URLConnection urlConnection = url.openConnection();
String plaintext = AccessDetails.getCheckuser() + ":"
+ AccessDetails.getUserpassword();
String authString = new String(Base64.encodeBase64((plaintext
.getBytes())));
urlConnection.setRequestProperty("Authorization", "Basic "
+ authString);
Once the connection is established - the handler takes over and deals with the query, I've included the whole handler here - it's not particularly pretty - but it gets the job done.
method if_http_extension~handle_request.
data: l_verb type string,
l_path_info type string,
begin of ls_response,
status type string,
end of ls_response,
lo_json type ref to zcl_json_document,
l_json type string,
l_command type text80.
l_verb = server->request->get_header_field( name = '~request_method' ).
l_path_info = server->request->get_header_field( name = '~path_info' ).
* Abort if not GET
if l_verb <> 'GET' .
server->response->set_header_field(
name = 'Allow'
value = 'GET' ).
server->response->set_status(
code = '405'
reason = 'Method not allowed' ).
return.
endif.
if l_path_info is not initial.
shift l_path_info left by 1 places.
endif.
* get details of request
translate l_path_info to upper case.
case l_path_info.
when 'CHECK'.
ls_response-status = 'Running'.
create object lo_json.
lo_json->set_data( ls_response ).
l_json = lo_json->get_json( ).
server->response->set_cdata(
exporting
data = l_json
).
server->response->set_header_field(
name = 'Content-Type'
value = 'application/json; charset=utf-8' ).
when 'SHUTDOWN'.
l_command = 'C:\RemoteShutdown\remote_shutdown.bat'.
if sy-uname <> 'SHUTDOWNUSER'. "#EC USER_OK
server->response->set_status(
code = '403'
reason = 'Only special user "SHUTDOWNUSER" is allowed to do this' ).
return.
endif.
call 'SYSTEM' id 'COMMAND' field l_command.
ls_response-status = 'Shutting down'.
create object lo_json.
lo_json->set_data( ls_response ).
l_json = lo_json->get_json( ).
server->response->set_cdata(
exporting
data = l_json
).
server->response->set_header_field(
name = 'Content-Type'
value = 'application/json; charset=utf-8' ).
when others.
server->response->set_status( code = '404'
reason = 'Cannot find that' ).
return.
endcase.
data: l_verb type string,
l_path_info type string,
begin of ls_response,
status type string,
end of ls_response,
lo_json type ref to zcl_json_document,
l_json type string,
l_command type text80.
l_verb = server->request->get_header_field( name = '~request_method' ).
l_path_info = server->request->get_header_field( name = '~path_info' ).
* Abort if not GET
if l_verb <> 'GET' .
server->response->set_header_field(
name = 'Allow'
value = 'GET' ).
server->response->set_status(
code = '405'
reason = 'Method not allowed' ).
return.
endif.
if l_path_info is not initial.
shift l_path_info left by 1 places.
endif.
* get details of request
translate l_path_info to upper case.
case l_path_info.
when 'CHECK'.
ls_response-status = 'Running'.
create object lo_json.
lo_json->set_data( ls_response ).
l_json = lo_json->get_json( ).
server->response->set_cdata(
exporting
data = l_json
).
server->response->set_header_field(
name = 'Content-Type'
value = 'application/json; charset=utf-8' ).
when 'SHUTDOWN'.
l_command = 'C:\RemoteShutdown\remote_shutdown.bat'.
if sy-uname <> 'SHUTDOWNUSER'. "#EC USER_OK
server->response->set_status(
code = '403'
reason = 'Only special user "SHUTDOWNUSER" is allowed to do this' ).
return.
endif.
call 'SYSTEM' id 'COMMAND' field l_command.
ls_response-status = 'Shutting down'.
create object lo_json.
lo_json->set_data( ls_response ).
l_json = lo_json->get_json( ).
server->response->set_cdata(
exporting
data = l_json
).
server->response->set_header_field(
name = 'Content-Type'
value = 'application/json; charset=utf-8' ).
when others.
server->response->set_status( code = '404'
reason = 'Cannot find that' ).
return.
endcase.
endmethod.
As you can see, I've got my handler class either returning a simple "I'm alive" if called with "check" or calling a batch script on the server if the "shutdown" method is called. The "I'm alive" call was made when the AWS query returned that the EC2 instance was running - and checked if the SAP ABAP server was running. Shutting things down was a tiny bit more complex as it involved a tiny bit of batch script - but it is really simple:
echo. |TIME |find "current" >> c:\RemoteShutdown\remote_shutdown.log
echo. |DATE |find "current" >> c:\RemoteShutdown\remote_shutdown.log
start c:\RemoteShutdown\shutdown_sap.bat
N.B. the "start" command in here - this allows it to run asynchronously - kinda important - as otherwise shutting down the ABAP stack, stops the shell command - which stops the shutdown working. The script the stops the servers is pretty basic too. (I'm no Basis person so I might be doing something wrong here - but the below does seem to work.)
D:\usr\sap\DAA\SYS\exe\uc\NTAMD64\stopsap.exe name=POR nr=40 SAPDIAHOST=localhost
D:\usr\sap\DAA\SYS\exe\uc\NTAMD64\stopsap.exe name=POR nr=41 SAPDIAHOST=localhost
D:\usr\sap\DAA\SYS\exe\uc\NTAMD64\stopsap.exe name=DEV nr=00 SAPDIAHOST=localhost
D:\usr\sap\DAA\SYS\exe\uc\NTAMD64\stopsap.exe name=DAA nr=97 SAPDIAHOST=localhost
shutdown -s -t 30 -c "System going down for daily shutdown!"
And that's it! It really is that simple.
Google Apps/GAE integration
Because as a company (like pretty much every small business I know) we are running Google Apps for our emails/collaboration/everything we need, I could very easily use Google Apps as my user authentication engine. After all, anyone in the company is allowed to start and stop the server, and only people in the company have access. In GAE it is a simple setting to allow a domain to be used for authentication.
and in the setup of the web.xml file of the application a couple of lines force authentication for every call to the app.
<security-constraint>
<web-resource-collection>
<web-resource-name>all</web-resource-name>
<url-pattern>/*</url-pattern>
</web-resource-collection>
<auth-constraint>
<role-name>*</role-name>
</auth-constraint>
</security-constraint>
And the future is bright, and cloudy.
The end result is a pretty secure mobile enabled way of controlling our company's demo system. My extremely basic html skills and use of jQuery means that the application runs on every browser we've bothered to test - which is basically every flavour of desktop browser, Android and iOS (strangely enough - no Blackberrys chez nous). I'm thinking of extending the solution so that users can schedule when they want the system to start-up and stop and running a simple cron job every 5 minutes on the GAE to check if a start-up/shutdown should be initiated. The potential to hook these kinds of cloud tech together to do simple little tasks that would otherwise mean having a server, space to put it, maintenance, etc. is so cool. Now I can run a HANA box in an AWS instance "next door", I wonder if should extend my solution to have a start/stop HANA buttons too ;-). But all of that will be on my to-do list. And with so much cool stuff to play with out there, not to mention my day job, it might be a while.
All of the code that is provided in the blog can be re-used AT YOUR OWN RISK. Where mentioned (the AWS authentication code in particular) some code is under Apache Licence, please check out what that actually means - it shouldn't stop you from using it. In all cases, please attribute back to me (Chris Paine) if you do decide to reuse any of the code. Other than that - I hope you get some use out of this blog, whether it just be an interesting (I may be being optimistic) read, or even implementing something like this yourself. If you've got any thoughts about how I could have done this better, or some ideas that might use a similar model, please do add a comment below. I always like hearing about what other people think. 🙂 And finally, all opinions, mistakes, grammatical and spelling errors, insulting and offensive turns of phrase are my own and shouldn't be used to judge the company I work for (Discovery Consulting) at all. However, if you like what you see, then great stuff!
- SAP Managed Tags:
6 Comments
