Hi everyone!
For those who have ever read any of my blogs, hello again..
Those previous blogs are for iOS native apps, but this one is for all the SMP3 OData SDK platforms. If you have gone through one of those platforms SDKs, you should have been on a good track of happy app development.
The common discussion while gaining development experience is about the offline store performance with large OData collections. Even though it is originally designed for fairly large volume for mobile devices, there are some tips you should be aware of - hopefully this will help your development.
<updated>Apr 29th, 2015</updated>
Problem: Your app with offline store has some performance issue with large volume of data. The initial creation of the store and the refresh time takes longer than you had expected.
Solution: Try out the several practical tuning tips below.
Discussion:
When you encounter the situation, here's the checklist of tips & remarks:
1. Measure the OData size and time
2. The log trace of offline component with DEBUG level to figure out the bottleneck
3. How OData is implemented – Server Side Paging
4. How OData is implemented – Delta Tracking/Query
5. Using Server Side Paging and Delta Query
6. Possible performance improvement with Caching
Let's take a look at one by one.
1. Measure the OData size and time
Current web browser provides very useful tools built-in. By using developer tools, we can measure the precise data size easily, such as content size, network time..etc to gain solid idea what’s going on. Here's my example (by Chrome) capture fetching 10000 entities. The content size is 20.3MB and it gets compressed in 618KB size during the transfer, and it took 18.20 seconds during HTTP GET. (Latency is like a ping call)
If you compare the payload sizes which has the same amount of the entity numbers but double numbers of parameters in each entity, you'll see the total size of data transfer gets nearly double (of course). So you can examine how the properties of each entity can affect the data transfer size and time.
And - Payload in JSON format is lot smaller than XML.
2. The log trace of offline component with DEBUG level to figure out the bottleneck
Bump up the trace level for offline component. Go to "LOGS" > "SETTINGS" in admin console.
Change the log level of the Offline component to "debug" and the trace "Enabled".
Now the offline component is ready to show the useful info.
The info provides very fine details of the offline store, you'll get clear idea what's happening behind the scenes. The timestamp data also tells you where's the bottleneck is.
3. How OData is implemented – Server Side Paging
As already discussed here, your offline store would fail to initialize if the OData collection in the defining request is too large. In that case you need to implement $skiptoken (aka Server Side Paging). Most of OData producer frameworks should support $skiptoken.
Once you implement it, in general, you should experiment a bit with the page sizes until you find a size that balances the backend performance and the response size. (For example, one of my personal experiment was 1000 entities in one page was slower than 5000.)
How to Implement $skiptoken with TravelAgency table in SAP Gateway
4. How OData is implemented – Delta Tracking/Query
Delta Query implementation in the OData producer is effective way to boost up the overall end-to-end performance. A quick explanation from the offline store perspective is written here. Just like $skiptoken, most of the OData producers should support delta query, so please consult respective documentation on how to implement.
SAP Gateway could have two major options to implement Delta Query:
- The approach based on Syclo xChange framework. Calculating deltas at modification time - The ABAP system tracks relevant changes when they occur. At request time, the deltas are already prepared and thus available. On the one hand this approach requires small coding and configuration steps. On the other hand it’s more scalable and has an optimized overall performance.
How To Implement Lightening Fast OData Services with Exchange Table
- The approach based on delta determination at request time - where the system compares old and new state to find out which records have been changed/deleted. The implementation effort is small but it does not optimize the performance of the backend - this needs to “compare” all the entity to calculate the delta - that means, the more records you have in the full collection, the longer the response time of the request.
How to Implement Basic Delta Query Support in SAP Gateway
The H2G on Integration Gateway: How To Use the Custom Coding for JDBC Connections in Integration Gateway
5. Using Server Side Paging and Delta Query
You want to use both? Good idea.
How To Use Delta Query and Server Side Paging with Soft State
6. Possible performance improvement with Caching
Together with Delta Tracking, Caching is also referred here:
"To minimize the load on the OData Producer and possibly improve performance, cache data that is shared by numerous users on SAP Mobile Platform Server. "
Essentially what you need to do is to tweak the parameters for the defining request by uploading .ini file to the SMP server. You'll find the params for cache is pretty optimized by default, so you might not see a real difference even if you use this feature - hence I described it as "possible" performance improvement.
One remark about the parameter is about "indexed_type". It is not for caching but it is used primarily for $orderby and for $filter clauses when executing requests from the client. So when your offline store deals with a high volume of data, your local $orderby or $filter query execution (remember - offline store CRUD operations get queued locally until the next flush or refresh, it acts like an OData producer) might get faster.
Tip: (Once you configured the Offline component Trace as written in 2) Log lines containing “MODATA_EP” describe the operations of the delta calculations in MobiLink. In particular lines with MODATA_EP_REQ_OPT show the options that are used.
Okie that's all for now. I'll update this content whenever new & important info comes in!
See you in the next blog,
Ken
H2Gs - "How do we build the best OData services for offline scenario?" Series
- SAP Managed Tags:
2 Comments
