Friday, January 10, 2014

New Site, New Blog

I've decided that I want to try something else for blogging, so I've gotten my own domain name and switched to Ghost.  My new blog will be found at http://blog.danlew.net/.

The posts here will remain up until the end of time, but all new content going forwards will be found over there.

Monday, January 6, 2014

Testing on Android (Part 2): Functional Tests

Here's part two of my series on Android testing.  This time, I'm going to wade into the huge number of functional testing frameworks that exist.  The differentiation I made between unit testing and functional testing is that the functional tests operate the actual device (as in, you could watch it running through a test).
So Many Frameworks

As you can see in my spreadsheet, there are a ton of functional testing frameworks out there.  I don't have any inclination to test all of them.  Here's what I haven't tried myself, for various reasons:
  • Non-Java - if my code and unit tests are going to be in Java, I might as well make my functional tests in Java as well.
  • Recorder-based testing - While this might be good for a less technical tester, I'd prefer exact control over my tests.
  • Superseded or discontinued - Some frameworks are old and have been surpassed by more modern frameworks.  Others have been discontinued.
This leaves just a few frameworks to investigate.

Robotium

I was already somewhat familiar with Robotium.  It runs off of the official Android testing framework, but adds the capabilities needed to actually run through an entire app.

It's been around for years, and so it's fairly stable.  It's also straightforward; you use the Solo class to run through your app, one step at a time.  It requires compiling along with the app's source, which could be a negative for some.  Overall, it's a solid functional testing solution.

uiautomator

UiAutomator is a more recent offering from Google.  I found it to be lacking and difficult to use.

First, it depends on more recent APIs, so it can only be used on Jelly Bean and above - you can't use it to test backwards compatibility.  Second, the deployment is more complex than other frameworks.  There's no simple script to put the code you wrote onto a device - you have to write a script yourself because it involves a few dynamic commands (depending on how you named your classes).

Third, it just feels like it's not quite there.  For example, simply launching my app from the test was a struggle.  The suggested start is to begin on your home page then page towards your app to launch it, except the code for doing so is broken.  Instead I had to use some hacky code which used "am start" to launch my Activity.

It does have a few unique advantages.  First, it can be written entirely black-box; you don't need access to the source code.  You use uiautomatorviewer to examine how an app is written and write tests based on that.  Second, it runs on the entire OS, so you can really test cross-app interaction.  If you need either of these, then I might check it out; but for my purposes this is a non-starter.

Espresso

Espresso is the latest toolkit from Google, and it's so awesome I have to wonder if the reason uiautomator is having issues is because everyone was actually working on Espresso.

It's got a different paradigm from other testing frameworks.  It starts with onView() or onData() with Hamcrest matchers to find your View/AdapterView.  Once found, you can then either perform an action or check some assertions.

The reason for the onView()/onData() setup is that Espresso examines the UI thread to know when processing is done.  A common problem with functional testing on Android is not knowing when a View will show up, so you end up with a lot of sleeps in your tests.  With Espresso you skip all of that, so it's blazing fast!

It's not all sunshine and roses.  It doesn't handle animations well (in fact, the expected setup is to turn all your animations off with dev opts).  I also found initial understanding harder, having to learn how to use Hamcrest to match my Views.  But once I got past those, the testing was so fast and stable that it blew me away.

Appium

I had started to try out Appium, as it seemed to present another way of functional testing, but ultimately gave up.

Maybe it was because I tried uiautomator, robotium, and Espresso before getting to Appium, but the setup was just overwhelming.  There was no "hello, world" sample I found that could take me from zero to testing.  There was a lot of documentation I found that listed out about five things I had to install on my machine (and have play together nicely) before I could start.

If my entire job was automation testing, or I knew selenium very well already, then I might consider Appium because then the time investment might pay off.  But as an Android developer who wants to dabble with testing on the side, I don't want to have to deal with getting this environment setup (not to mention the pain I'd be putting others through to replicate my work).

Conclusion

I would take my thoughts with a grain of salt; I haven't used any of these long enough to really get into the nitty gritty.  That said, I think my future functional tests will be written in Espresso; its design paradigm makes for focused, solid testing.  Plus, its basis on the Looper to determine when to continue makes it lightyears faster than the other frameworks could hope to be.  I hope it continues to be supported by Google because I like it.

Wednesday, December 18, 2013

Who Goes First App

Continuing with my series of boardgame-related side projects, I've written a small holo-themed app for determining who goes first called (unsurprisingly) Who Goes First.  As with most side projects it's also open source.  I released this app a while ago but thought I'd mention it here.

There wasn't much complex about it; mostly I was pleased with how easy it is to make a good-looking app with holo theming (even if you're not very talented in the design department).

The only interesting thing I learned relate to the arrow pointing towards who goes first.  If you need more than six arrows, then it'll just show "X#" inside of the arrow instead.  This required scaling the text size depending on how long the text was.  I ended up using a Region for the arrow (defined via the Path used to construct the arrow) and another Region for the text and test if they overlapped; if they did, shrink text size until they don't.  You can check out what I'm talking about in the actual ArrowView.

The one problem I've run into is when there are multiple people at the table with this app.  The app does not use networking to sync their random seeds, so you can end up with multiple people going first.  I guess my next project will have to be a "Who Runs Who Goes First First" app.  :)

Monday, December 16, 2013

Testing on Android (Part 1): Unit Tests

I've recently been doing research on Android testing.  I'm quite new to this business; automated testing has (for the longest time) felt unnecessary.  But I've been slowly convinced that the benefits outweigh the time costs - as long as you do it in an efficient manner.  To that end, I set out to evaluate the many, many Android testing frameworks to see which ones seemed to save the most time.

I began by summarizing all the tools/services I could find in a spreadsheet (let me know if I'm missing anything).  Beyond that, I am going to do a series of posts going into more detail about different parts of Android testing:
Let's start with unit testing.

Unit Tests

There's no reason you can't use normal JUnit 4 testing for Android applications... as long as you stay away from anything Android.

Normally you compile against the SDK's android.jar, which contains nothing but stubbed methods that throw exceptions when run.  When you actually upload your APK to a device, it uses the device's implementations of all those stubs.  As a result, when running normal unit tests in your IDE, you get no access to those framework implementations (instead receiving mountains of exceptions).  This is not a big deal if you're testing some simple functionality that doesn't touch Android itself.

Pros:
  • Fast and easy
Cons:
  • Cannot use any Android framework classes
The Android Testing Framework

The Android testing framework is the official method of unit testing on Android.  It loads your application onto a device, then runs JUnit-based test suites.  Since it runs on the actual OS you can use the Android framework as you normally would in your application and can conduct a series of realistic tests that way.

Ostensibly the testing framework is unit testing, but the slowness of having to fully compile and upload your app onto a device before executing any tests makes testing slow.  Plus, you have to make sure you've got a device attached or an emulator running.  As a result, I might consider the testing framework for semi-regular tests (e.g., whenever you push a new commit, or nightly tests) but I would have trouble using them while actively developing.

Pros:
  • Access to the Android framework
Cons:
  • Slow
  • Requires attached device or running emulator
  • Uses JUnit 3 (instead of the newer JUnit 4)
Robolectric

Robolectric is a project that unifies the speed of unit testing with the ability to access the Android framework.  It does this by implementing all those stubs with mocked classes.

Having tried it out, it is lightning fast and works as expected.  I'm also impressed with the amount of active development on it - this is a rapidly improving framework.  However, the active development does take a toll; documentation is a bit lacking, plus some new versions of Robolectric break things in previous versions.  Plus, it can't mock everything - for example, inter-app communication - since it's not on an actual Android OS.  That said, the benefits here far outweigh the negatives when it comes to unit testing Android.

Pros:
  • Fast
  • Can access mocked Android framework
  • Actively developed
Cons:
  • Not the true Android framework
  • Not everything is mocked
  • Lacking documentation
Conclusion

I'm a fan of what Jason Sankey said on StackOverflow about tiered unit testing: Prefer pure unit tests, then Robolectric, then the Android testing framework.  The tests get harder/slower the higher the tier, but sometimes you need it.

Thursday, December 5, 2013

Thoughts on the Current State of Android IDEs

I've repeatedly been getting the question "which IDE should I use for Android?" recently so I'm writing up a brief summary.  Here's my current thoughts on the state of Android IDEs (circa December 2013):

Eclipse + ADT

The original Android IDE, officially supported by Google.

  • Supports the most features for creating/building Android apps.
  • Abundant documentation, both from Google and from years of people asking questions about it on StackOverflow.
  • Many 3rd party tools, since it's been the main player for years.
  • Relatively stable; your project won't suddenly stop building when you update your sources.

IntelliJ IDEA + Android Plugin

A Java IDE that has an Android plugin built for it by IntelliJ.  This is the one IDE I haven't used extensively, so take my comments with a grain of salt.

  • A fan favorite; people rave about IntelliJ (especially those who were burned in one way or another by Eclipse).
  • As stable (if not more so) than Eclipse, because it lacks some of the built-in instabilities in Eclipse (aka, randomly crashing every once in a while).
  • Does not support NDK.

Android Studio + Gradle

Android Studio is a fork of IntelliJ IDEA with gradle-based build support.

  • Gradle is the future of Android build processes, so using it you'll be one step ahead of the curve.
  • Since it's still heavily in development Android Studio is quite unstable.  It's gone from rarely working to only stabbing you in the back every once in a while, but use of it requires extra maintenance work.
  • Does not support all features, like NDK or lint.  (There are workarounds, though.)
  • Improves weekly because that's about how often they push out updates.  If you love cutting edge technology that's a pro, if you like stability that's a con.  (You can switch off of the "canary" update channel if you're more conservative.)
Conclusion

  • If you need all Android features (like NDK support), use Eclipse.
  • If you hate Eclipse, use IntelliJ IDEA.
  • If you love gradle builds or want to plan for the future and are willing to put in some extra effort, use Android Studio.
Also, if you've got anything to add let me know - I'd be curious if there's some important pieces I'm leaving out, as I am far from an IDE expert.

Tuesday, December 3, 2013

Streaming JSON Parsing Performance Test: Jackson vs. GSON

A few years ago, when speed and memory became a concern in one of my Android apps, I implemented some streaming JSON parsing using Jackson.  At the time I chose Jackson over Gson because the benchmarks seemed to show it was significantly faster*; however, I've heard that recent versions of Gson rival Jackson for speed.  I want to re-examine the benchmarks again to make sure I'm choosing the right library.

The best benchmark I found is at json-benchmark.  There were a number of other benchmarks out there but they didn't appeal to me for various reasons, either because they didn't provide source, the benchmark didn't seem very fair, or it didn't test on the Dalvik VM.

Unfortunately, json-benchmark's last documented test is from two and a half years ago.  I decided to try to reproduce these tests against the latest libraries, Jackson 2.3 and Gson 2.2.4.  There were a few trials getting the test to run again:
  • The JSON benchmarking code has moved to libcore, so I checked out and referenced the latest benchmarks from there.
  • The ParseBenchmarks code to test Jackson and GSON directly was removed, so I had to re-add it (using the same old code, but with some package names updated).
  • vogar uses old assumptions about where "dx" (and related tools) are located; it still thinks they are in $ANDROID_HOME/platform-tools.  Rather than fix and recompile I just made my files match its assumptions for a bit.
Once I got it running, I ran the tests on a Nexus 7 (2nd generation), running Android 4.4.  Here's the results:

        document            api     ms linear runtime
          TWEETS ANDROID_STREAM  19.82 ===
          TWEETS JACKSON_STREAM  11.47 ==
          TWEETS    GSON_STREAM  17.98 ===
          TWEETS       GSON_DOM  39.00 =======
          TWEETS       ORG_JSON  30.86 =====
          TWEETS       XML_PULL  26.08 ====
          TWEETS        XML_DOM  85.32 ===============
          TWEETS        XML_SAX  33.83 ======
    READER_SHORT ANDROID_STREAM   4.60 =
    READER_SHORT JACKSON_STREAM   2.99 =
    READER_SHORT    GSON_STREAM   4.36 =
    READER_SHORT       GSON_DOM   8.07 =
    READER_SHORT       ORG_JSON   6.42 =
    READER_SHORT       XML_PULL   6.75 =
    READER_SHORT        XML_DOM  14.84 ==
    READER_SHORT        XML_SAX   5.54 =
     READER_LONG ANDROID_STREAM  38.24 =======
     READER_LONG JACKSON_STREAM  18.12 ===
     READER_LONG    GSON_STREAM  43.81 ========
     READER_LONG       GSON_DOM  72.34 =============
     READER_LONG       ORG_JSON  52.47 =========
     READER_LONG       XML_PULL  65.53 ============
     READER_LONG        XML_DOM 160.02 ==============================
     READER_LONG        XML_SAX  47.37 ========

Of course your mileage may vary depending on what exactly you're parsing, but with somewhere between 1.5-2x the speed I think I'll stick Jackson when I need my JSON parsing to be fast.

* Speed isn't everything; if Gson was only marginally slower than Jackson I still would have preferred it; I find their streaming parsing paradigm easier to use.

Monday, November 25, 2013

Sample Android Project: Movies

There's a sample application that I worked on recently with talented designer Chris Arvin for a talk we gave at DroidCon UK 2013.  I also repurposed part of it for the talk about animations I gave at AnDevCon a few weeks later.

The app is open source and you can check it out here: https://github.com/dlew/android-movies-demo

You can get the APK here: https://github.com/dlew/android-movies-demo/releases/



The interactions required a bit of interesting engineering so I wanted to discuss some of it here.

ViewPager and Decor

One of the key interactions we wanted was the app to feel like a ViewPager so that it was an interaction users were familiar with except that the result isn't quite the same - instead of paging, content would slide out from underneath other content.  This is not something ViewPager is inherently designed to do.

There were two options open to me.  One was to rip out all of ViewPager's event handling code and make my own custom View, which was not an appealing prospect.  The other was to find some way to manipulate ViewPager itself.

It turns out there is an interface ViewPager finds special called Decor.  If a View implements Decor, then ViewPager treats it as a special View that can remain on top of it the ViewPager.  This is normally for implementing your own tabs, but in this case I imported my own copy of ViewPager (since Decor is hidden normally) and made my entire UI a Decor View.

It's a neat trick that would work for any app that wants ViewPager event handling without paging Views, though I think if you wanted a less hacky solution you'd write your own event code.

Custom Views Everywhere

I've gone from avoiding custom Views like the plague to fully embracing them.

The key realization is that custom Views do not need to handle every situation.  If you're looking at framework custom Views as reference you will be overwhelmed quickly.  For example, during measurement framework Views have to handle WRAP_CONTENT, MATCH_PARENT, and everything in between; but if you know your View is always going to be MATCH_PARENT you can greatly simplify all your code.

The movies sample app is a pile of custom Views.  First there's SlidingRevealViewGroup, which generically shows one View slide out from another.  On top of that is built MovieRowView, which has the specific Views that we want to display.  Then there's SlidingPairView, which is a set of two SlidingRevealViewGroups that creates the side-by-side effect seen in the app.

I also needed a few other custom Views to shore up some other issues.  CenteringRelativeLayout just adjusts the film cover so that, as it shrinks in size, it still looks centered.  SlidingListView was required for performance; we needed to manipulate the rows directly when sliding, instead of constantly notifying of data changed.

Performance Tricks

The coolest part of the whole app is the slide: how the cover becomes smaller and the content slides out from underneath.  All of this was achieved through translation of Views.  I took advantage of the fact that ViewGroups don't render their children outside of their own bounds.  By translating content I could hide them outside the clip bounds.

When a slide starts, it throws practically everything into hardware layers then just slides Views left/right.  Moving around pre-rendered content is fast and as a result the paging is silky smooth.

Nowadays I'm a huge fan of those basic properties of Views (translation, scale, rotation and alpha).  They're the core reason to support ICS+ only; with these properties you can take your interactions to the next level.

Drawbacks

The drawback of the solution I came up with is that it creates a ton of overdraw.  This hurts performance, especially (as far as I can tell) when rendering a new row while scrolling up/down.  I sacrificed scrolling performance for paging performance.  Perhaps there is a way to achieve both, but I'm pretty much done with this sample for now.

The rounded corners could've been implemented in a much better fashion than an overlaid Drawable, but with the limited time before the presentation I had, I could not come up with a better solution.