Hello again. Depending on whether joanna.chan has managed to get anything more out about all the fun that she's been having with the build of the actual mobile apps, this could be anything from the fourth to the tenth blog (although even if she does have enough material to write a truckload of blogs, time might be a blocker there) in our series about using SAP NetWeaver Cloud as the platform to build a lightweight yet secure mobile application.
Apologies - SERIOUS amounts of code ahead
Much as I love my silly videos and pretty pictures they can't really replace code snippets when it comes to reference and trying things out on your own. Although this isn't intended to be a simple to follow tutorial on how to develop a REST API using SAP NWCloud, it does go into quite some detail about what I have done, so others can hopefully learn from it. (And if there are any experts out there, perhaps suggest to me what I might have done better! :smile: )
But -Whiteboards RULE :wink:
Video is a bit of a code walk through of the various code snippets in this blog:
Again - SCN insist on making the size of the YouTube clips tiny by default and I haven't figured out how to change it - so you might want to open the video in a new window - http://www.youtube.com/watch?v=h_ELTs8_taU
First step security
As mentioned in my previous blog, getting the security of the communications between our mobile device and the cloud working was our first concern. We achieved this through the use of a very simple method - user name and password (albeit getting there was a bit complex - see my other blog for details.) By using HTTPS for all communications between the mobile device and the cloud even using basic authentication (unencrypted username and password) is quite secure.
Building a resource data model(or not)
One of the distinctions I like to make between RESTful APIs and SOA is that REST deals with resources and SOA services. A service might be "UpdateUsersSetting" whereas the resource is "UserSettings". This is a very basic distinction, there are more subtle but very important ones too. Whereas in SOA, I would publish a list the APIs that were available to me, this is the antithesis of REST. Roy Fielding (the "father" of REST) makes some pretty clear points about what is REST in this case. I'll quote a couple of points from one of his blogs - the highlighting is mine.
"I am getting frustrated by the number of people calling any HTTP-based interface a REST API. Today’s example is the SocialSite REST API. That is RPC. It screams RPC. There is so much coupling on display that it should be given an X rating.
...
A REST API must not define fixed resource names or hierarchies (an obvious coupling of client and server). Servers must have the freedom to control their own namespace. Instead, allow servers to instruct clients on how to construct appropriate URIs, such as is done in HTML forms and URI templates, by defining those instructions within media types and link relations. [Failure here implies that clients are assuming a resource structure due to out-of band information, such as a domain-specific standard, which is the data-oriented equivalent to RPC's functional coupling].
...
A REST API should be entered with no prior knowledge beyond the initial URI (bookmark) and set of standardized media types that are appropriate for the intended audience (i.e., expected to be understood by any client that might use the API). From that point on, all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations. The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly (e.g., code-on-demand). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]"
RESTful Documentation - or the lack of it
If you know me, you'll realise that the discovery that it isn't RESTful to document your APIs was a big thumbs up for me. However, life being as it is, it ain't that simple. What Roy is saying (it appears to me), is that your resources must be discoverable, and self documenting. One little used solution is the OPTIONS verb. This verb is used to ask your system what it does/doesn't support.
"The OPTIONS method represents a request for information about the communication options available on the request/response chain identified by the Request-URI. This method allows the client to determine the options and/or requirements associated with a resource, or the capabilities of a server, without implying a resource action or initiating a resource retrieval."
http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.2
Typically this is used for deciding if your resource supports CORS and what other verb (GET, PUT, POST, DELETE) are supported. I found a great post by Zac Stewart which describes using OPTIONS as a way of self documenting a REST API. I have to say, I like it! And if I were blessed with a little more time, I'd probably have gone down that path. As it was my documentation tended to be throwing the code at Joanna whenever she needed to figure out what I had done. (they say code should be self documenting - well this was just going to the next level 😉 )
But back to that non-existent data/resource/object model
Whilst I seemingly didn't need to document (on a piece of paper) my resource model, it was, however, required in order to build it!
Jo and I had some great fun with a whiteboard trying to figure out what resources we would need. What happened was that we ended up designing the Java object model and then seeing how it fitted with the resources we needed. In the end it was pretty simple:
The pairing "resource" is definitely NOT Idempotent so perhaps shouldn't even be part of the model... However, it does give the device the link to the user resource, from which all other resources are discoverable.
Colour coding here refers to how the resources are accessible, and how they can be retrieved.
Yellow = GET
Red = GET, PUT
Purple = GET, PUT, POST, DELETE
Brown = GET, PUT, POST, DELETE (but only from the Web interface not generally accessible)
Orange = special use case, as days, although we originally thought to have as a resource, ended up being brought into the top level entries resource, however, it was retained in my Java object model.
The data model, was a little simpler:
All the details that we needed in entries and days could be stored in the "Entry" detail.
Building Data Access Objects
Actually creating the database tables to store the attributes of all these object was a LOT of work. I was using JDBC, and one of the fun challenges when working with SAP NetWeaver Cloud is that each environment has different databases. So what might work on my local server, might not work when deployed to the cloud. The local server runs Apache Derby, whilst the cloud runs either HANA or MaxDB. To give you an idea of the hassle, here is my code for the storage of the individual timesheet line items.
package au.com.discoveryconsulting.timesheet.demo.persistance;
import java.sql.Connection;
import java.sql.DatabaseMetaData;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
import javax.sql.DataSource;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import au.com.discoveryconsulting.timesheet.demo.TimeSheetUser;
import au.com.discoveryconsulting.timesheet.demo.entries.TimeSheetDay;
import au.com.discoveryconsulting.timesheet.demo.entries.TimeSheetEntry;
public class TimeSheetEntriesDAO {
private static final String TABLE_NAME = "TIMESHEET_ENTRIES";
private static final String STMT_CREATE_TABLE_DERBY = "CREATE TABLE "
+ TABLE_NAME
+ " (ID VARCHAR (36) PRIMARY KEY, "
+ "USER_ID INTEGER, ENTRY_DATE VARCHAR (8), BILLABLE VARCHAR (1), "
+ "COMMENTS VARCHAR (200), DATE_MODIFIED VARCHAR (40), HOURS FLOAT, "
+ "START_TIME VARCHAR (10), END_TIME VARCHAR (10), "
+ "PROJECT_CODE VARCHAR (80), PROJECT_NAME VARCHAR (80), STATUS VARCHAR (20)"
+ ")";
private static final String STMT_CREATE_TABLE_MAXDB = "CREATE TABLE "
+ TABLE_NAME
+ " ( ID VARCHAR (36) PRIMARY KEY, "
+ "USER_ID INTEGER, ENTRY_DATE VARCHAR (8), BILLABLE VARCHAR (1), "
+ "COMMENTS VARCHAR (200), DATE_MODIFIED VARCHAR (40), HOURS FLOAT, "
+ "START_TIME VARCHAR (10), END_TIME VARCHAR (10), "
+ "PROJECT_CODE VARCHAR (80), PROJECT_NAME VARCHAR (80), STATUS VARCHAR (20)"
+ ")";
private static final String STMT_SELECT_ALL = "SELECT ID, USER_ID, ENTRY_DATE, BILLABLE, "
+ "COMMENTS, DATE_MODIFIED, HOURS, START_TIME, END_TIME, PROJECT_CODE, PROJECT_NAME, STATUS FROM "
+ TABLE_NAME;
private static final String STMT_SELECT_ALL_WHERE_USER_IS = STMT_SELECT_ALL
+ " WHERE USER_ID = ?";
private static final String STMT_SELECT_ALL_WHERE_ID_IS = STMT_SELECT_ALL
+ " WHERE ID = ?";
private static final String STMT_INSERT = "INSERT INTO "
+ TABLE_NAME
+ " ( ID, USER_ID, ENTRY_DATE, BILLABLE, "
+ "COMMENTS, DATE_MODIFIED, HOURS, START_TIME, END_TIME, PROJECT_CODE, PROJECT_NAME, STATUS ) "
+ "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)";
private static final String STMT_UPDATE = "UPDATE "
+ TABLE_NAME
+ " SET ENTRY_DATE = ?, BILLABLE = ?, "
+ "COMMENTS = ?, DATE_MODIFIED = ?, HOURS = ?, START_TIME = ?, END_TIME ?, PROJECT_CODE = ?, "
+ "PROJECT_NAME = ?, STATUS = ? " + "WHERE ID = ?";
private static final String STMT_SELECT_SETTINGS = "SELECT ID FROM "
+ TABLE_NAME + " WHERE ID = ?";
private static final String STMT_DELETE_ID = "DELETE FROM " + TABLE_NAME
+ " WHERE ID = ?";
private DataSource dataSource;
Logger logger = LoggerFactory.getLogger(TimeSheetEntriesDAO.class);
/**
* Create new data access object with data source.
*/
public TimeSheetEntriesDAO(DataSource newDataSource) throws SQLException {
setDataSource(newDataSource);
}
/**
* Get data source which is used for the database operations.
*/
public DataSource getDataSource() {
return dataSource;
}
/**
* Set data source to be used for the database operations.
*/
public void setDataSource(DataSource newDataSource) throws SQLException {
this.dataSource = newDataSource;
checkTable();
}
/**
* drop the table.
*/
public void drop() throws SQLException {
Connection conn = dataSource.getConnection();
try {
PreparedStatement pstmt = conn.prepareStatement("DROP TABLE "
+ TABLE_NAME);
int dropResults = pstmt.executeUpdate();
logger.debug("Executed drop results " + dropResults);
return;
} finally {
if (conn != null) {
conn.close();
}
}
}
/**
* Add/Update user to the table.
*/
public void updateOrAddEntry(TimeSheetEntry entry) throws SQLException {
Connection conn = dataSource.getConnection();
// only add if entry does not already exist
try {
PreparedStatement pstmt1 = conn
.prepareStatement(STMT_SELECT_SETTINGS);
pstmt1.setString(1, entry.getId());
ResultSet rs = pstmt1.executeQuery();
logger.debug("Checking for existing entry with id " + entry.getId()
+ rs.getStatement().toString());
if (!rs.next()) {
if (entry.getId() == null || entry.getId().isEmpty()) {
String id = UUID.randomUUID().toString();
entry.setId(id);
}
logger.debug("Adding entry "
+ entry.getJsonObject("/").toString());
PreparedStatement pstmt = conn.prepareStatement(STMT_INSERT);
pstmt.setString(1, entry.getId());
pstmt.setInt(2, entry.getAssociatedDay().getAssociatedUser()
.getId());
pstmt.setString(3, entry.getAssociatedDay().getDay());
if (entry.isBillable()) {
pstmt.setString(4, "Y");
} else {
pstmt.setString(4, "N");
}
pstmt.setString(5, entry.getComments());
pstmt.setString(6, entry.getModified());
pstmt.setDouble(7, entry.getHours());
pstmt.setString(8, entry.getStartTime());
pstmt.setString(9, entry.getEndTime());
pstmt.setString(10, entry.getProjectCode());
pstmt.setString(11, entry.getProjectName());
pstmt.setString(12, entry.getStatus());
pstmt.executeUpdate();
logger.debug("Added entry " + entry.toString());
} else {
// entry already exists - update entry instead
logger.debug("Updating entry "
+ entry.getJsonObject("/").toString());
PreparedStatement pstmt = conn.prepareStatement(STMT_UPDATE);
pstmt.setString(1, entry.getAssociatedDay().getDay());
if (entry.isBillable()) {
pstmt.setString(2, "Y");
} else {
pstmt.setString(2, "N");
}
pstmt.setString(3, entry.getComments());
pstmt.setString(4, entry.getModified());
pstmt.setDouble(5, entry.getHours());
pstmt.setString(6, entry.getStartTime());
pstmt.setString(7, entry.getEndTime());
pstmt.setString(8, entry.getProjectCode());
pstmt.setString(9, entry.getProjectName());
pstmt.setString(10, entry.getStatus());
pstmt.setString(11, entry.getId());
pstmt.executeUpdate();
logger.debug("Updated entry " + entry.toString());
}
} finally {
if (conn != null) {
conn.close();
}
}
}
/**
* Get user's entries from the table using user id.
*/
public List<TimeSheetEntry> selectUsersEntries(int userId)
throws SQLException {
Connection conn = dataSource.getConnection();
try {
PreparedStatement pstmt = conn
.prepareStatement(STMT_SELECT_ALL_WHERE_USER_IS);
pstmt.setInt(1, userId);
ResultSet rs = pstmt.executeQuery();
logger.debug("Getting users timesheet entries " + rs.toString());
List<TimeSheetEntry> entries = new ArrayList<TimeSheetEntry>();
while (rs.next()) {
entries.add(getEntryFromResults(rs));
}
return entries;
} finally {
if (conn != null) {
conn.close();
}
}
}
public TimeSheetEntry selectEntry(String id) throws SQLException {
Connection conn = dataSource.getConnection();
TimeSheetEntry entry = null;
try {
PreparedStatement pstmt = conn
.prepareStatement(STMT_SELECT_ALL_WHERE_ID_IS);
pstmt.setString(1, id);
ResultSet rs = pstmt.executeQuery();
logger.debug("Getting users entry " + id + rs.toString());
if (rs.next()) {
entry = getEntryFromResults(rs);
}
return entry;
} finally {
if (conn != null) {
conn.close();
}
}
}
public int deleteEntry(String id) throws SQLException {
Connection conn = dataSource.getConnection();
try {
PreparedStatement pstmt = conn.prepareStatement(STMT_DELETE_ID);
pstmt.setString(1, id);
int deleteResults = pstmt.executeUpdate();
logger.debug("Executed delete results " + deleteResults);
return deleteResults;
} finally {
if (conn != null) {
conn.close();
}
}
}
private TimeSheetEntry getEntryFromResults(ResultSet rs)
throws SQLException {
TimeSheetEntry t = new TimeSheetEntry();
t.setId(rs.getString(1));
// create "dummy" timesheet day - actual will be assigned in model
// layer
TimeSheetDay day = new TimeSheetDay();
TimeSheetUser user = new TimeSheetUser();
user.setId(rs.getInt(2));
day.setAssociatedUser(user);
day.setDay(rs.getString(3));
t.setAssociatedDay(day);
t.setBillable(rs.getString(4).equals("Y"));
t.setComments(rs.getString(5));
t.setModified(rs.getString(6));
t.setHours(rs.getDouble(7));
t.setStartTime(rs.getString(8));
t.setEndTime(rs.getString(9));
t.setProjectCode(rs.getString(10));
t.setProjectName(rs.getString(11));
t.setStatus(rs.getString(12));
return t;
}
/**
* Check if the settings table already exists and create it if not.
*/
private void checkTable() throws SQLException {
Connection conn = null;
logger.debug("checking table");
try {
conn = dataSource.getConnection();
if (!existsTable(conn)) {
logger.debug("table does not exist");
createTable(conn);
}
} finally {
if (conn != null) {
conn.close();
}
}
}
/**
* Check if the settings table already exists.
*/
private boolean existsTable(Connection conn) throws SQLException {
DatabaseMetaData meta = conn.getMetaData();
ResultSet rs = meta.getTables(null, null, TABLE_NAME, null);
while (rs.next()) {
String name = rs.getString("TABLE_NAME");
if (name.equals(TABLE_NAME)) {
return true;
}
}
return false;
}
/**
* Create the settings table.
*/
private void createTable(Connection conn) throws SQLException {
String query = getDbSpecificQueryForTableCreation(conn);
logger.debug("creating table with sql" + query);
PreparedStatement pstmt = conn.prepareStatement(query);
pstmt.executeUpdate();
}
/**
* The statement for creating the table with the ID column is DB platform
* dependent. You can modify the sample code to run against other DB
* platforms.
*/
private String getDbSpecificQueryForTableCreation(Connection conn)
throws SQLException {
String databaseProductName = conn.getMetaData()
.getDatabaseProductName();
if (databaseProductName.equals("Apache Derby")) {
return STMT_CREATE_TABLE_DERBY;
} else if (databaseProductName.equals("SAP DB")) {
return STMT_CREATE_TABLE_MAXDB;
}
throw new SQLException("The JDBC sample is not yet implemented for \""
+ databaseProductName + "\".");
}
}Note the fun I had storing/retrieving "billable" which is a Boolean in my data object, but has to be a character in my database.
Some learnings on DAO
I used the examples from the SAP NetWeaver Cloud "Consuming SAP NetWeaver Cloud Persistence Service" documentation as the basic for my DAOs. Perhaps I should have thought about using JPA, but at some point in the future I want to be able to code with HANA, and as I currently understand that means using JDBC as the means of accessing the DB, so I'd better get used to it! :smile:
The JDBC DAO examples are quite good, but I replicated a bunch of code in each one. What I should have done was to define an abstract super-class that implemented the reused methods, like " getDbSpecificQueryForTableCreation(Connection conn)". That way my code would have been a bit more readable.
Coding a "drop" into each DAO.
When I was building my application I often found that I had forgotten to add a field to the database, or it was the wrong length, etc. This wasn't a problem, I'd just adjust the DAO and restart my local server. Restarting the local server had the effect that the database was completely refreshed. Once I had deployed my application to the cloud however, this was a different matter. Stopping and starting the cloud server/application does NOT refresh the database. Even removing( un-deploying) the application from the SAP NWCloud and then redeploying it does NOT refresh the database. (This is a particularly interesting observation as it has implications if you ever want to remove your application from the cloud, removing the application does not remove the data!).
My solution was to code a servlet "BuggerTheDb" which called a drop method on all the DAOs. This way I could refresh the database and have the code recreate the tables with the correct parameters.
I tweeted about this and Harald Mueller responded with an interesting observation about Liquibase:
@wombling nice! You would think jpa would have provisions for such basic things.;-)
— Jason Scott (@js1972) January 17, 2013
@wombling @js1972 check this blog on liquibase scn.sap.com/community/deve…
— Harald Mueller (@har_mueller) January 17, 2013
@wombling @js1972 agree, i personally would use liquibase for all DDLs and changes in DB tables, creation and evolution of all versions.
— Harald Mueller (@har_mueller) January 18, 2013
https://twitter.com/har_mueller/status/292178654077788160
I'm not sure I agree with Harald about using Liquibase for all my table definition, certainly it would mean that all support users would need training/understanding that additional metadata was creating and maintaining the tables. But it would be quite useful if I ever did need to make a change to a productive system. Certainly worth bearing in mind.
Using a generated key is simple, but can be a PITA
In the JDBC tutorial the table is created with a generated key. This is great to ensure that you don't have to worry about doing it yourself, however quickly becomes very difficult if you need to refer to that particular DB entry again. Code needed to be written to select the entry based on other fields which should be unique, but obviously weren't guaranteed by the database to be so. In particular in a REST based API where I want to have a unique resource identifier for each entry and want to return this on the creation of the entry, it is quite tricky if you allow the database to do that identifier generation for you. By the time I created the timesheet line entry DAO that I listed above, I'd got wise to this problem so I was creating the key identifier value myself (which made it much easier to return on creation of the entry.
Naming standards for your DAO methods make sense (and make life easier)
By adopting a similar naming standard for similar methods across the different DAOs it was easier to reuse and adapt code. At the data model level (one up from the DAOs and the class I used to access all the DAO methods) it is worth remembering that Java supports method overloading which means that store(objectTypeA) can run completely different code from store(objectTypeB).
Here is a code snippit from my UserDataStore model class that shows this:
public class UserDataStore {
Logger logger = LoggerFactory.getLogger(UserDataStore.class);
private TimeSheetUsersDAO userDAO;
private TimeSheetUserMobileAppsDAO deviceDAO;
private UserSettingsDAO settingsDAO;
private UserSettingsDaysDAO settingsDaysDAO;
private UserSettingsProjectsDAO projectsDAO;
private TimeSheetEntriesDAO entriesDAO;
private void init() {
try {
InitialContext ctx = new InitialContext();
DataSource ds = (DataSource) ctx
.lookup("java:comp/env/jdbc/DefaultDB");
if (ds == null) {
logger.error("datasource not found");
}
userDAO = new TimeSheetUsersDAO(ds);
deviceDAO = new TimeSheetUserMobileAppsDAO(ds);
settingsDAO = new UserSettingsDAO(ds);
settingsDaysDAO = new UserSettingsDaysDAO(ds);
projectsDAO = new UserSettingsProjectsDAO(ds);
entriesDAO = new TimeSheetEntriesDAO(ds);
} catch (NamingException e) {
logger.error("Naming Exception" + e.getMessage());
} catch (SQLException e) {
logger.error(e.getMessage());
}
}
public void store(TimeSheetUser user) {
if (userDAO == null) {
init();
if (userDAO == null) {
logger.error("database access not created!");
return;
}
}
try {
userDAO.updateOrAddUser(user);
if (user.getId() == 0) {
user.setId(userDAO.getUserIdForUser(user.getUserId(), user
.getUserType().toString()));
}
if (!user.getMobileDevices().isEmpty()) {
// need to also store any changes to the devices
Iterator<TimeSheetMobileApp> deviceIter = user
.getMobileDevices().iterator();
while (deviceIter.hasNext()) {
TimeSheetMobileApp device = deviceIter.next();
deviceDAO.updateOrAddDevice(device);
}
}
if (user.getSettings() != null) {
logger.debug("creating settings in DB");
settingsDAO.updateOrAddSettings(user.getSettings());
if (user.getSettings().getId() == 0) {
user.getSettings().setId(
settingsDAO.getSettingsIdForUser(user.getId()));
}
settingsDaysDAO.updateOrAddSettings(user.getSettings()
.getFriDetails());
settingsDaysDAO.updateOrAddSettings(user.getSettings()
.getSatDetails());
settingsDaysDAO.updateOrAddSettings(user.getSettings()
.getSunDetails());
settingsDaysDAO.updateOrAddSettings(user.getSettings()
.getMonDetails());
settingsDaysDAO.updateOrAddSettings(user.getSettings()
.getTueDetails());
settingsDaysDAO.updateOrAddSettings(user.getSettings()
.getWedDetails());
settingsDaysDAO.updateOrAddSettings(user.getSettings()
.getThuDetails());
}
} catch (SQLException e) {
logger.error(e.getMessage());
}
}
public int store(TimeSheetMobileApp device) {
if (userDAO == null) {
init();
if (userDAO == null) {
logger.error("database access not created!");
return 0;
}
}
try {
deviceDAO.updateOrAddDevice(device);
if (device.getId() == 0) {
device.setId(getDeviceFromPairCode(device.getPairingKey())
.getId());
}
return device.getId();
} catch (SQLException e) {
logger.error(e.getMessage());
}
return 0;
}
public void store(DeviceSettings settings) {
if (settingsDAO == null) {
init();
if (settingsDAO == null) {
logger.error("database access not created!");
return;
}
}
try {
settingsDAO.updateOrAddSettings(settings);
if (settings.getId() == 0) {
settings.setId(settingsDAO.selectUsersSettings(
settings.getAssociatedUser().getId()).getId());
}
settingsDaysDAO.updateOrAddSettings(settings.getFriDetails());
settingsDaysDAO.updateOrAddSettings(settings.getSatDetails());
settingsDaysDAO.updateOrAddSettings(settings.getSunDetails());
settingsDaysDAO.updateOrAddSettings(settings.getMonDetails());
settingsDaysDAO.updateOrAddSettings(settings.getTueDetails());
settingsDaysDAO.updateOrAddSettings(settings.getWedDetails());
settingsDaysDAO.updateOrAddSettings(settings.getThuDetails());
return;
} catch (SQLException e) {
logger.error(e.getMessage());
}
return;
}
public void store(DeviceSettingsProjects project) {
if (projectsDAO == null) {
init();
if (projectsDAO == null) {
logger.error("database access not created!");
return;
}
}
try {
projectsDAO.updateOrAddProject(project);
return;
} catch (SQLException e) {
logger.error(e.getMessage());
}
return;
}
public void store(TimeSheetEntry entry) {
if (entriesDAO == null) {
init();
if (entriesDAO == null) {
logger.error("database access not created!");
return;
}
}
try {
entriesDAO.updateOrAddEntry(entry);
return;
} catch (SQLException e) {
logger.error(e.getMessage());
}
return;
}Dealing with complex objects makes some things more difficult, but helps too
In the example code above, one of the methods stores a TimeSheetUser entry in the database. Now part of the TimeSheetUser object is the user settings. There is a 1:1 relationship between these two objects. I could have just stored all of them in the same database table. As it was, however, I used 3 tables - this was because this better reflected my object model of the data where I had:
As settings themselves could be updated separately from the user record, it seemed to make sense to store them separately. This said - there certainly does not need to be a one to one mapping between resources and database tables! Part of RESTful design is ensuring that the model exposed makes sense and isn't just a representation of the underlying database (which is why I get so annoyed when people talk about exposing HANA through OData/Gateway and then mention REST in the same breath...) My other purpose in having multiple tables (especially the settings per day) was to avoid having to type in huge numbers of fields making my code more maintainable and easier to read! Imagine I could have had a table with:
MondayHours, TuesdayHours, ...
MondayDefaultStart, TuesdayDefaultStart ...
etc.
Or just one table that has the user's settings id as a key field that I can use to retrieve all 7 days' worth of defaults. Add to this that I for ease of manipulation/representation I had the days as a separate data object, it just made sense!
Anyway - back to my main point here (I do like to get distracted :smile: ) updating the user object also updates the settings for the user (including any day defaults). In code that I haven't listed above (because there is such a thing as code overload) I retrieve the associated user when settings are read, and the associated settings when user is read. This makes it very easy for me in my code to reference the data that my application needs. I just have a reference to the object, all the data model is completely abstracted away so I don't need to worry about it.
I built my object model to give me maximum reuse and ease of dealing with the data, my data model was built around that, and the resources were then built leveraging the power of very easy to use object model - but represented the resource based view that made sense for my application.
I'm not sure if others would describe this as best practice or not! But to me, building my object model such that I could code easily, then the data model, then the resource model made for the easiest build of the solution.
Building a servlet
The way that data gets into and out of a Java SAP NWCloud app is through servlets. Servlets have a bit of a dirty name due to their association with JSP - indeed, if you go to the Wikipedia page on Java Servlet, the first thing you see is a great big diagram on how JSPs work. That isn't what you want for a RESTful API!
When considering using this framework for a RESTful API we step back a bit and remember that we aren't only serving HTML as responses from the servlets (or not at all in my case) but whatever media type makes sense.
POST a new entry, GET the data, PUT it back
The first thing I needed to implement in my servlets was a means of authenticating the user making the request and checking if they had the authorisation to the data/action requested. I've described this in my blog Linking to and managing a mobile app using a simple solution in SAP NetWeaver Cloud.
There are 4 main verbs that my servlets deal with - GET (retrieve a resource), PUT (update a resource), POST (create a resource) and DELETE (remove a resource). You might think it make sense to start with GET as this would seem to be the simplest. (Actually in most cases DELETE is the easiest), but unfortuately you need to have a resource before you can read it - so we start with POST.
Here is an example from my time sheet line entries servlet, which I'll work through.
protected void doPost(HttpServletRequest request,
HttpServletResponse response) throws ServletException, IOException {
AllowCORS.addCORSHeaders(request, response, "GET, PUT, POST, DELETE");
try {
authoriseAndValidate(request);
logger.debug("entry found");
// entry already exist - cannot be created.
response.setStatus(303);
response.setHeader("Location", request.getRequestURL().toString());
} catch (EntryNotFoundException e) {
// update the response - in this case we want that entry is not
// found
logger.debug("entry not found - creating");
try {
TimeSheetEntry entry = getEntryFromRequest(request);
// store in db
userData.store(entry);
// send email
UserMailSender.addEntryForMail(entry);
outputResponse(entry, request, response);
} catch (JsonParsingException jsonExp) {
logger.debug("problems parsing JSON - POST");
response.sendError(400, "Error parsing JSON");
} catch (UnauthorisedAccessException e1) {
logger.info("No Authorisation for associated user - POST");
response.sendError(403, "No Authorisation for associated user");
}
} catch (UnauthorisedAccessException e) {
logger.info("Unauthorised Access attempt - POST");
response.setStatus(403);
}
}
Firstly I set the headers of the response to allow CORS (but that's another blog).
Then I try to authenticate the current user and retrieve the entry that is referenced by the resource URL.
private TimeSheetEntry authoriseAndValidate(HttpServletRequest request)
throws UnauthorisedAccessException, EntryNotFoundException {
TimeSheetUser myUser = AuthenticateUser.getUserIfValid(request,
userData);
if (myUser != null) {
TimeSheetEntry entry = null;
// are we requesting a particular entry? (hope so!)
String myURI = request.getRequestURI();
String myPath = myURI.substring(myURI.indexOf("entry") + 5);
if (!myPath.isEmpty()) {
// attempt to parse entry id out of this path
logger.debug("path value: " + myPath);
try {
String parsedEntryId = myPath.substring(1);
if (!parsedEntryId.isEmpty()) {
logger.debug("parsed value: " + parsedEntryId);
TimeSheetEntry requestedEntry = userData
.getEntryFromId(parsedEntryId);
if (requestedEntry == null) {
throw new EntryNotFoundException();
}
if (requestedEntry.getAssociatedDay()
.getAssociatedUser().getId() == myUser.getId()
|| myUser.isAdmin()) {
entry = requestedEntry;
} else {
throw new UnauthorisedAccessException();
}
} else {
throw new EntryNotFoundException();
}
} catch (NumberFormatException e) {
logger.error(e.getLocalizedMessage());
throw new EntryNotFoundException();
}
} else {
throw new EntryNotFoundException();
}
return entry;
} else {
throw new UnauthorisedAccessException();
}
}In my case if I'm trying to create the entry, finding an existing entry would be bad - which is why I then redirect the user to GET the existing resource using a 303 return code. The convention with HTTP 303 is to include the header "Location" with the value of the resource.
If the entry does not exist we read the body of the POST to get the data out. I use GSON, the Google JSON library, for doing this. Hopefully the following code is reasonably easy to read:
private TimeSheetEntry getEntryFromRequest(HttpServletRequest request)
throws JsonParsingException, IOException,
UnauthorisedAccessException {
TimeSheetEntry entry = new TimeSheetEntry();
BufferedReader reader = new BufferedReader(new InputStreamReader(
request.getInputStream()));
StringBuilder sb = new StringBuilder();
for (String line; (line = reader.readLine()) != null {
sb.append(line);
}
String bodyText = sb.toString();
logger.debug(bodyText);
// lots to read here
try {
JsonObject entryJSON;
entryJSON = (new JsonParser()).parse(bodyText).getAsJsonObject();
// check that id is same as resource location (or that resource
// location is blank)
String myURI = request.getRequestURI();
String myPath = myURI.substring(myURI.indexOf("entry") + 5);
String parsedEntryId = "";
if (!myPath.isEmpty()) {
// attempt to parse entry id out of this path
logger.debug("path value: " + myPath);
try {
parsedEntryId = myPath.substring(1);
} catch (Exception e) {
// no resource URL used - just a blank - that's OK
}
}
String idJson = entryJSON.get("id").getAsString();
if (!idJson.equals(parsedEntryId) && !parsedEntryId.equals("")) {
logger.error("Resource id used " + parsedEntryId
+ " and JSON do not agree");
throw new JsonParsingException();
}
entry.setId(idJson);
entry.setBillable(entryJSON.get("billable").getAsBoolean());
entry.setComments(entryJSON.get("comments").getAsString());
entry.setModified(entryJSON.get("modified").getAsString());
String hoursText = entryJSON.get("hours").getAsString();
if (hoursText.equals("")) {
entry.setHours(0);
} else {
entry.setHours(entryJSON.get("hours").getAsDouble());
}
entry.setProjectCode(entryJSON.get("projectCode").getAsString());
entry.setProjectName(entryJSON.get("projectName").getAsString());
entry.setStartTime(entryJSON.get("startTime").getAsString());
entry.setEndTime(entryJSON.get("endTime").getAsString());
entry.setStatus(entryJSON.get("status").getAsString());
TimeSheetDay day = new TimeSheetDay();
day.setDay(entryJSON.get("date").getAsString());
entry.setAssociatedDay(day);
TimeSheetUser user = AuthenticateUser.getUserIfValid(request,
userData);
int userId = entryJSON.get("associatedEntriesId").getAsInt();
if (user.getId() == userId || user.isAdmin()) {
TimeSheetUser entryUser = new TimeSheetUser();
entryUser.setId(userId);
entryUser.addTimeSheetDay(day);
} else {
throw new UnauthorisedAccessException();
}
return entry;
} catch (NullPointerException jsonParseExcp) {
logger.debug("Error Parsing JSON \n" + bodyText);
throw new JsonParsingException();
}
}It does seem like a lot of effort to go to get the body of the request as a String - and then parse out the values from the JSON representation. But it works. One thing I did pickup from building these servlets was that catching/throwing lots of different exceptions was a nice way to handle all the problems that might be caused by the inbound JSON being malformed.
Since I wanted my solution to be as flexible as possible, all URLs returned are relative to the URL that has been requested (so if you request from chris.wombling.com:8080/TimeSheet/entry then the response will also be to that server). This is very useful when doing things like OAuth, where an address like http://localhost:8080 is not acceptable to many, but adding a line to my hosts file:
127.0.0.1 chris.wombling.comfixes that problem :smile:
Once the entry has been successfully read, it's a simple matter to actually create it in the database (as our data access class manages the call to the relevant DAO).
In this particular example I also send an email to the user of the application confirming that the entry was successfully received.
Finally I output a JSON representation of the object that has just been created. I use the same routine to return the data for the results of the GET and PUT calls too.
private void outputResponse(TimeSheetEntry entry,
HttpServletRequest request, HttpServletResponse response)
throws IOException {
String myURL = request.getRequestURL().toString();
String rootURL = myURL.substring(0, myURL.lastIndexOf("/entry") + 1);
JsonObject entryJson = entry.getJsonObject(rootURL);
// convert for output
String jsonOutput = entryJson.toString();
response.getWriter().println(jsonOutput);
response.setContentType("application/json");
}I have hidden the logic for the creation of the JSON within my class representing the time sheet line entry, and return a JsonObject rather than a String, as this allows me to use it to do things like create arrays of entries.
The logic for the GET and PUT methods is very similar, just that they actually act on the referenced resource rather than just throwing it away :).
DELETE and what to send back
When I first started writing my DELETE implementations they were by far the easiest of the four verb to implement. I just checked if the resource was there, then if it was, I deleted it. Done! No need for any response body to be returned as the object no longer existed. Here I made two basic errors!
- If you don't actually return a body on an HTTP request and it is successful, you should return status 204 not a status 200 (oops).
- Given that cross domain AJAX calls in jQuery must be asynchronous, it is quite difficult to tell what item has been successfully deleted unless you respond with some sort of detail about what has just been deleted.
So now all my DELETEs send back a very simple bit of JSON e.g.:
{
"id": "86325706-8750-4abd-bb5a-6340c0509e4a",
"deleted": true
}protected void doDelete(HttpServletRequest request,
HttpServletResponse response) throws ServletException, IOException {
AllowCORS.addCORSHeaders(request, response, "GET, PUT, POST, DELETE");
try {
TimeSheetEntry entry = authoriseAndValidate(request);
// delete project
userData.delete(entry);
JsonObject deleteResponse = new JsonObject();
deleteResponse.addProperty("id", entry.getId());
deleteResponse.addProperty("deleted", true);
String jsonOutput = deleteResponse.toString();
try {
response.getWriter().println(jsonOutput);
response.setContentType("application/json");
response.setStatus(200);
} catch (IOException e) {
logger.error(e.getLocalizedMessage());
}
} catch (EntryNotFoundException e) {
logger.info("Project not found - DELETE");
response.sendError(404, "Project not found");
} catch (UnauthorisedAccessException e) {
logger.info("Unauthorised Access attempt - DELETE");
response.setStatus(403);
}
}Testing
If I was good (which I rarely am), I would have looked at creating JUnit tests for all my methods. Although I admire the idea of test driven development (TDD) I think I probably need to be better at building NWCloud apps and all the intricacies involved before I give it a go. (I'm sure Alisdair Templeton will disagree with me here...) Instead I took the standard SAP approach and ran a few macro-level tests :wink: I used a Chrome application called Postman - which really made life doing the testing so much easier.
I could store a library of tests in an easy to sort/find collection (and I could even export them to install them on another machine - there is even a cloud sharing option, but that's for public sharing only). It formats the result of the calls so that the condensed JSON that is output by the servlets becomes human readable. One of the other things I like about it is the ability to set environment variables like the address of the server - so I can just select which environment (my local machine, or the NW trial cloud account) I want to use and I don't have to cut and past URLs all over the place. Being able to view and set the header (and an easy way to set basic authentication into the header) was a big plus too. :smile:
Looking through my code you'll see extensive use of "logger.debug("blah");" throughout. This was how I figured out what was, and wasn't working in my code. I'd use Postman to call the servlet and then check the logs to see what had happened.
Of course, this isn't a replacement for some decent automated testing ala JUnit style. But for the simple API that I've built it has worked well so far.
Making it work from a mobile device - CORS
Once my wonderful (my opinion) servlets were working and happily GETing, PUTting, POSTing and DELETEing we tried to integrate to the mobile app that Jo was writing in PhoneGap - see her blog about that. However we came across some issues, and they certainly weren't obvious. Rather than try to cover them here, Jo and I will be collaborating on a blog which covers just this topic. I'll update once the blog is posted - in the meantime search CORS!
Sharing the love - or at least the code
Recently Joanna and I (along with quite a few others for the SAP space in Melbourne) attended a conference where they almost didn't mention SAP at all (someone had to spoil it by mentioning how much they liked HANA in one of the database sessions - it was the speaker not us!). The conference was YOW2012 and its tag line is "For Developers By Developers". It's a great way to find out what's happening outside of the SAP world and perhaps bring a bit of it home. One session that I really enjoyed was Zach Holman's session on how GitHub manages to manage its staff without managers. We had some interesting discussions afterwards on how we could replace org charts with something better - but that's another blog! Anyway, coming back to the office, I decided that I'd try to use GitHub to help me work on different machines and share my code with different users. I even managed to get it working with my MacBook Pro (although I still prefer using Eclipse on my PC (might be the big 27inch monitor, with two 19 inch monitors on either side actually :smile:
However, being able to "check in" my code and have others working on the same code at the same time was really cool.
I would strongly recommend to any developer setting out to build code using Java on the SAP NetWeaver Cloud that they take the time and effort to learn how to use Github. If you don't mind your code being public it is free - there's a price point you can't argue with! It really helped me by being able to with a few clicks copy down and deploy the latest version of my code to Joanna's machine for example. Also when I went into the office and wanted to use my laptop, I could - and even update files, safe in the knowledge that I could merge these onto my home PC later.
In summary - main learnings
Building a reasonably complex application that needed authentication and authorisation checking was quite a challenging task. I certainly would not have wanted to start out with this being my first NWCloud application! Defining your Java object model so that it is easy for you to use it is very important. Database storage of those objects can come later.
Using the JDBC data storage of NWCloud is painful. Remember that your application isn't a copy of the tutorial and think about how you are going to manage your data entries. Remember to code into your solution some way of dropping or adjusting the tables once they have been created on the cloud server.
Reuse as much as possible. This is simple, but by reusing the same method names and concepts and using inheritance and abstract classes, generating that "one more" servlet become much simpler.
GitHub is a great tool for sharing code, it's definitely worth looking at for Java NWCloud development.
Stuck? It is likely that someone else has already solved the problem that you are looking at. There are an awful lot more Java developers out there than ABAP ones. What you're attempting here isn't locked into that Web Dynpro Java wrapper that no-one else understands. Likely case is that there is an open source solution for your problem e.g. GSON, BCrypt, Scribe etc. Go and have a look. Get friendly with StackOverflow - it is your new best friend, and much more likely to have an answer than any SCN forum (unless it is 100% NWCloud based in which case I've had some fantastic response from the forum :smile: )
Disclaimers, apologies, and more ramblings
Once again, I only aim to please, but in case I don't, all the errors, omissions, and flaws are mine, my dear employer is welcome to take credit for the rest. The code samples provided in the blog are in no way guaranteed to be working or fit for purpose. You learn from them and use them entirely at your own risk. I'm sorry I didn't get around to doing more whiteboard magic for the video for this blog, but I could really figure out what to put in it. Next time you meet me, let's have a beverage (depending on time of day will decide what type) and we can discuss what I should have done.
Coming up next in this series of blogs (I'm not sure in what order).
- Sending emails from the cloud (but not millions of them)
- CORS - what that means and how Chrome, iOS and Android interpret that
- Synchronising data between multiple devices and using offline storage
- Building a web administrator front end for handling timesheets (including an extraction to MS Excel)
- Making a simple web QR code redirector
- Interfacing timesheet entries with SAP CATS via Gateway and OData
- Joanna finally admits that Android is a superior OS compared to iOS (I live in hope).
If there is anything you have any questions on about this particular blog, please feel free to leave a comment. I'd also like to know if people appreciate having this much code in a blog, or should I cut it down and go for more discussion type stuff. Please let me know!
- SAP Managed Tags:
8 Comments
