Mobile app developers do not have to buy a mobile device to actually run their app on it and test it. You get simulators for free. You can test your development on iOS, Android, WM or Blackberry without spending money on a physical device. As soon as your project gets more concrete and you identify your target group, you also start testing your development with real devices. Why on a real device? After all, you don’t want stumble over a bug that only manifests in a real device. There still may be a hardware / software / mobile network combination that break the code. That’s not different when you develop with SUP.
Here is the good news: SUP supports the main mobile operating systems: iOS, Android, BlackBerry, Windows Mobile. Here is the bad news: Sybase did not test SUP on physical mobile devices. For the Object API and iOS, the documentation details what was tested:
- iPhone 4 (iOS 5.1), iPad and iPad 2 (iOS 4.3)
This means that the iPhone 4S and new iPad are not tested. If you now suspect that Sybase has no money to buy new devices from Apple, take a look at the page for the hybrid web container. The list of tested devices lists more combinations:
- iPad (iOS 4.3.x), iPad 2 (iOS 4.3, iOS 5.1), iPad 3 (iOS 5.1), iPhone 4S (iOS 5.1), iPhone 4 (iOS 4.3.5), iPod (iOS 5.0).
Pad 3 is now tested, as is the iPod and iPhone 4S.
What does that mean?
If you decide to develop your SUP app for iOS 5.1, using the Object API and an iPhone 4S, you will run your app on a device that wasn’t tested. It’s supported, but not tested. You are transforming yourself into a beta tester.
Other mobile operating systems
The next problem you run into is how mobile devices are sold: they are customized. The operator puts some custom apps and other stuff on it, and for non-iOS devices you get a different UI/UX from venders like Samsung or HTC. As Sybase supports for instance Android, does it support the customized Android from Samsung? It is too much to ask a company to test against every OS modification, but the best sellers should be tested.
And what about the simulators mentioned in the beginning? What simulators Sybase used to test SUP? On the BlackBerry page you can download almost every device / OS RIM released. Now: what did Sybase tested? Yes, Object API for BB OS 5.x, 6.x 7.x is supported, but did Sybase test the device using the simulator? That leads to another question:
How to interpret the phrase tested in the online documentation? A physical device was tested or just the simulator? Or both?
What about Windows Mobile devices? A use case for them is field services. And that means that you won’t use a WM device you can buy in a shopping mall store. It’s these heavy, brick-like mobile devices like the one here made for outdoor usage, with a keyboard, scanner and RFID reader. Use case: offline scenario. For SUP this means: MBOs.
Sybase does not list one of these in the tested device section. As these devices are slightly more expensive than consumer WM phone, it would be nice to have a test suite before buying them in large quantities.
Test suite
It’s impossible for Sybase to test all devices as it is for others: here is an example for phonegap and jquery mobile. There is a test guide available on SCN that lists some test tools (most only available to Sybase and consulting) for you app, not for SUP. How to know that the device chosen for the solution works with SUP, before start writing code? What if only after a few months into the project an error gets identified and traced to the actual device / OS / SUP combination? How to get a test suite to test the correct execution of SUP on the mobile device?
What can currently be done is to use the sample applications and run them manually to find out if they work on the device. But that’s far from having SUP passing (automatic) tests to ensure a functional SUP on the device.
Here is an idea SAP: release a test suite for SUP and together with a developer license people can start testing SUP with their simulator and physical device. They will test their device using a real internet connection, different OS, configurations, locations, providers, and so on. This way SAP can use the crowd to test SUP in an unbeatable way: real life.
Update
Idea at IdeaPlace created: https://cw.sdn.sap.com/cw/ideas/11555
Comments