State Your Destination Posts

This is part 2 in a series of posts about creating a mobile web app for browsing music databases. Part 1 can be found here.

The first task in building the front end was testing my API to make sure I knew what was being returned by Rovi and that it had everything I wanted. I added some test JavaScript to the default MVC view that would call my API. It was a bit of trial and error going through the data and seeing where I needed to adjust my requests on the back end. My plan was to simply copy the test code into the official script files. The Rovi service itself is easy to use and well documented.

Next was setting up the base AngularJS implementation. I fired up Google to try and find a good online example of how to structure the app. The web site has a tutorial so I started stepping through it. But as I began to have questions on how to do certain things, and what the best practices are, I noticed the code I found online differed from what the tutorial was doing. More searching uncovered tools like angular-seed and angular-enterprise-seed. They were comprehensive but included way too much stuff to absorb for someone just learning the framework. They seem to be more for large scale web applications. I eventually came up with what looked like a good way to set up my module, controller, and service declarations, along with the source file structure to use. I followed suggestions from places like yeoman.io and various others found online. I don’t know if it’s exactly what is considered good by the Angular community but it’s close. My main app script ended up looking like this:

'use strict';

// The name of the module for the main app must match the ng-app attribute of the  tag in the
// startup page. The contents of the array are dependencies for the app:
//   ngRoute:                  Provides routing support
//   ngAnimate:                Provides animation support
//   ngSanitize:               Sanitizes text that contains markup before binding it to a view, needed
//                             for artist bios and album reviews which need to have line breaks and
//                             possibly hyperlinks to other views
//   jmdobry.angular-cache:    Custom caching implementation for use with the $http service
//   musicBrowserControllers:  Module which will hold all controllers for the app
var musicBrowserApp = angular.module('MusicBrowserApp', [
    'ngRoute',
    'ngAnimate',
    'ngSanitize',
    'ui.bootstrap',
    'jmdobry.angular-cache',
    'musicBrowserControllers'
]);

var musicBrowserControllers = angular.module('musicBrowserControllers', []);

// /                        Home page, currently redirected to the search page
// /search                  Page that shows the search UI
// /search/artist/          Shows the results of an artist search for
// /search/album/           Shows the results of an album search for
// /search/song/            Shows the results of a song search for
// /artist/                 Shows the details for the artist represented by
// /artist//full-bio        Shows only the complete bio for the artist represented by
// /album/                  Shows the details for the album represented by
// /album//full-review      Shows only the complete review for the album represented by
// /genre/                  Shows the details for the genre represented by
// /style/                  Shows the details for the style represented by
// /options                 Page for changing app options
musicBrowserApp.config(['$routeProvider', '$provide', function ($routeProvider, $provide) {
    $routeProvider.when('/', { templateUrl: 'views/search.html', controller: 'SearchCtrl', title: "Search" });
    $routeProvider.when('/search', { templateUrl: 'views/search.html', controller: 'SearchCtrl', title: "Search" });
    $routeProvider.when('/search/artist/:searchTerm', { templateUrl: 'views/artistSearch.html', controller: 'ArtistSearchCtrl', title: "Artist Search" });
    $routeProvider.when('/search/album/:searchTerm', { templateUrl: 'views/albumSearch.html', controller: 'AlbumSearchCtrl', title: "Album Search" });
    $routeProvider.when('/search/song/:searchTerm', { templateUrl: 'views/songSearch.html', controller: 'SongSearchCtrl', title: "Song Search" });
    $routeProvider.when('/artist/:id', { templateUrl: 'views/artist.html', controller: 'ArtistLookupCtrl', title: "Artist" });
    $routeProvider.when('/artist/:id/full-bio', { templateUrl: 'views/artistBio.html', controller: 'ArtistLookupCtrl', title: "Artist Bio" });
    $routeProvider.when('/album/:id', { templateUrl: 'views/album.html', controller: 'AlbumLookupCtrl', title: "Album" });
    $routeProvider.when('/album/:id/full-review', { templateUrl: 'views/albumReview.html', controller: 'AlbumLookupCtrl', title: "Album Review" });
    $routeProvider.when('/style/:id', { templateUrl: 'views/style.html', controller: 'StyleLookupCtrl', title: "Style" });
    $routeProvider.when('/genre/:id', { templateUrl: 'views/genre.html', controller: 'GenreLookupCtrl', title: "Genre" });
    $routeProvider.when('/options', { templateUrl: 'views/options.html', controller: 'OptionsCtrl', title: "Options" });
    $routeProvider.otherwise({ redirectTo: '/' });
}]);

// Any startup code needed by the app should go here
musicBrowserApp.run(['$rootScope', '$http', '$angularCacheFactory', function ($rootScope, $http, $angularCacheFactory) {
    $http.defaults.headers.common["Accept-Encoding"] = "gzip,deflate";

    $rootScope.$on("$routeChangeSuccess", function (event, currentRoute, previousRoute) {
        // Change page title based on the current route
        $rootScope.title = currentRoute.title;
    });

    // Create a custom cache for our data, and set the $http service to use it for its caching
    $angularCacheFactory('dataCache', {
        // Items added to this cache expire after 60 minutes
        maxAge: 3600000,
        // This cache will clear itself every two hours
        cacheFlushInterval: 12000000,
        // Items will be deleted from this cache right when they expire
        deleteOnExpire: 'aggressive'
    });

    $http.defaults.cache = $angularCacheFactory.get('dataCache');
}]);

What to say about AngularJS? I like it. The framework strives to make it easy to separate your app logic from your markup from your data access, and largely succeeds. It includes tons of built-in stuff to further that goal. I ended up creating two custom services, one for common code and one for data access. The latter looked like this:

musicBrowserApp.factory('mbData', ['mbCommon', '$http', function (mbCommon, $http) {
    var curInstance = this;

    Object.defineProperty(curInstance, "maxShortDescriptionLength", {
        value: 150,
        writable: false,
        enumerable: true,
        configurable: true
    });

    curInstance.searchForArtist = function (query, size, offset) {
        var url = "api/search/artist/" + encodeURIComponent(query) + "?size=" + size + "&offset=" + offset;

        return $http.get(url, { cache: true }).
            success(function (data, status, headers, config) {
                var result = JSON.parse(data.Content);

                if (result.searchResponse) {
                    data.searchResult = result.searchResponse.results.map(function (element) {
                        setPrimaryImage(element, mbCommon.placeholderImageMedium);
                        return element;
                    });
                }
            });
    };
    
    curInstance.lookupArtist = function(id) {
        var url = "api/artist/" + encodeURIComponent(id);

        return $http.get(url, { cache: true }).
            success(function (data, status, headers, config) {
                var result = JSON.parse(data.Content);
                var primaryImage;
                var formattedBioText;
                var mbConfig = mbCommon.getConfiguration();
                var styleIndex;

                for (var i = 0; i < result.name.musicGenres.length; i++) {
                    styleIndex = getIndexOfId(result.name.musicStyles, result.name.musicGenres[i].id);

                    if (styleIndex > -1) {
                        result.name.musicStyles.splice(styleIndex, 1);
                    }
                }

                if (result.name.discography) {
                    if (mbConfig && mbConfig.albumChrono) {
                        result.name.discography.reverse();
                    }

                    for (var i = 0; i < result.name.discography.length; i++) {
                        if (result.name.discography[i].year) {
                            result.name.discography[i].year = mbCommon.formatDate(result.name.discography[i].year, true);
                        }

                        if (result.name.discography[i].images && result.name.discography[i].images.length > 0) {
                            primaryImage = result.name.discography[i].images[0].url;
                        }
                        else {
                            primaryImage = mbCommon.placeholderImageSmall;
                        }

                        result.name.discography[i].primaryImage = primaryImage;
                        result.name.discography[i].formattedType = "Album"

                        if (result.name.discography[i].flags && result.name.discography[i].flags.indexOf("Compilation") > -1) {
                            result.name.discography[i].formattedType = "Compilation";
                        }

                        if (result.name.discography[i].type === "Single" || result.name.discography[i].type === "EP") {
                            result.name.discography[i].formattedType = "SingleOrEP";
                        }
                    }
                }

                if (result.name.isGroup) {
                    result.name.originLabel = "Formed:";
                    result.name.endLabel = "Disbanded:"
                }
                else {
                    result.name.originLabel = "Born:";
                    result.name.endLabel = "Died:"
                }

                if (result.name.active) {
                    for (var i = 0; i < result.name.active.length; i++) {
                        result.name.active[i] = mbCommon.formatDate(result.name.active[i], true);
                    }
                }

                if (result.name.birth) {
                    var newDate = mbCommon.formatDate(result.name.birth.date, false, true);

                    if (newDate === "") {
                        result.name.birth.date = "N/A";
                    }
                    else {
                        result.name.birth.date = newDate;
                    }
                }

                if (result.name.death) {
                    var newDate = mbCommon.formatDate(result.name.death.date, false, true);

                    if (newDate != "") {
                        result.name.death.date = newDate;
                    }
                }

                setPrimaryImage(result, mbCommon.placeholderImageLarge);

                if (result.name.musicBio) {
                    formattedBioText = replaceRoviLinks(result.name.musicBio.text);
                    formattedBioText = formattedBioText.split("\r\n").join("");
                    result.name.musicBioFormatted = formattedBioText;
                }

                if (result.name.headlineBio) {
                    result.name.headlineBioFormatted = replaceRoviLinks(result.name.headlineBio);
                }
                else {
                    if (result.name.musicBio) {
                        result.name.headlineBioFormatted = getShortDescription(result.name.musicBioFormatted);
                    }
                }

                data.lookupResult = result.name;
            })
    };

    curInstance.searchForAlbum = function (query, size, offset) {
        var url = "api/search/album/" + encodeURIComponent(query) + "?size=" + size + "&offset=" + offset;

        return $http.get(url, { cache: true }).
            success(function (data, status, headers, config) {
                var result = JSON.parse(data.Content);

                if (result.searchResponse) {
                    data.searchResult = result.searchResponse.results.map(function (element) {
                        if (element.album.images && element.album.images.length > 0) {
                            element.album.primaryImage = element.album.images[0].url;
                        }
                        else {
                            element.album.primaryImage = mbCommon.placeholderImageMedium;
                        }

                        return element;
                    });
                }
            })
    };

    curInstance.lookupAlbum = function (id) {
        var url = "api/album/" + encodeURIComponent(id);

        return $http.get(url, { cache: true }).
            success(function (data, status, headers, config) {
                var result = JSON.parse(data.Content);
                var primaryImage;
                var formattedReviewText;

                if (result.album.originalReleaseDate) {
                    result.album.originalReleaseDate = mbCommon.formatDate(result.album.originalReleaseDate, false, true);
                }

                if (result.album.duration) {
                    result.album.durationFormatted = mbCommon.formatDuration(result.album.duration);
                }

                if (result.album.images && result.album.images.length > 0) {
                    primaryImage = result.album.images[0].url;
                }
                else {
                    primaryImage = mbCommon.placeholderImageLarge;
                }

                result.album.primaryImage = primaryImage;

                if (result.album.primaryReview) {
                    formattedReviewText = replaceRoviLinks(result.album.primaryReview.text);
                    formattedBioText = formattedReviewText.split("\r\n").join("");
                    result.album.primaryReviewFormatted = formattedReviewText;
                }

                if (result.album.headlineReview) {
                    result.album.headlineReviewFormatted = replaceRoviLinks(result.album.headlineReview.text);
                }
                else {
                    if (result.album.primaryReview) {
                        result.album.headlineReviewFormatted = getShortDescription(result.album.primaryReviewFormatted);
                    }
                }

                if (result.album.tracks && result.album.tracks.length > 0) {
                    for (var i = 0; i < result.album.tracks.length; i++) {
                        result.album.tracks[i].durationFormatted = mbCommon.formatDuration(result.album.tracks[i].duration);
                    }
                }

                data.lookupResult = result.album;
            })
    };

    curInstance.searchForSong = function (query, size, offset) {
        var url = "api/search/song/" + encodeURIComponent(query) + "?size=" + size + "&offset=" + offset;

        return $http.get(url, { cache: true }).
            success(function (data, status, headers, config) {
                var result = JSON.parse(data.Content);

                if (result.searchResponse) {
                    data.searchResult = result.searchResponse.results.map(function (element) {
                        if (element.song.images && element.song.images.length > 0) {
                            element.song.primaryImage = element.song.images[0].url;
                        }
                        else {
                            element.song.primaryImage = mbCommon.placeholderImageMedium;
                        }

                        return element;
                    });
                }
            })
    };

    curInstance.lookupStyle = function (id) {
        var url = "api/style/" + encodeURIComponent(id);

        return $http.get(url, { cache: true }).
            success(function (data, status, headers, config) {
                var result = JSON.parse(data.Content);
                var items;

                if (result.styles) {
                    items = result.styles;
                }
                else {
                    items = result.subgenres;
                }

                data.lookupResult = items.map(function (element) {
                    element.formattedDescription = replaceRoviLinks(element.description);
                    return element;
                });
            })
    }

    curInstance.lookupGenre = function (id) {
        var url = "api/genre/" + encodeURIComponent(id);

        return $http.get(url, { cache: true }).
            success(function (data, status, headers, config) {
                var result = JSON.parse(data.Content);

                data.lookupResult = result.genres.map(function (element) {
                    element.formattedDescription = replaceRoviLinks(element.description);
                    return element;
                });
            })
    }

    return curInstance;
}]);

The data service uses the built-in Angular $http service and does any required massaging of the data before handing it off to the controller that called it. The controllers then set various properties of the current scope as needed.

I created several different views based on what needed to be shown; one for artist search results, one for data on a specific artist, etc. Whenever I ran into a case where I needed the markup to be different based on the data, I was pleasantly surprised to find an Angular directive that would allow it to be driven by the model. Things like ng-show and ng-href were invaluable. The general rule in the Angular world is that you shouldn’t make any changes to the DOM in your controllers, and if you find yourself reaching for jQuery you might be doing something wrong. I’m happy to say I didn’t have any need to use jQuery to manipulate the DOM.

Next is incorporating animations for view transitions, adding something to the options page, and filling out some missing features.

Update:
As of Dec 31, 2021, long-term support for AngularJS has ended. And so the code above isn’t useful anymore, and the Music Browser is currently offline while I give it a complete overhaul using a different front end framework (maybe Angular?) and connect it to a different source of music metadata using a new back end built on Python. I greatly enjoyed using the framework for both personal projects and in my job for several years. May it rest in peace.

It’s been over a year since I last touched my bookmark browser, and it needed some attention. There were a couple of outstanding bugs to deal with, plus it seemed like a good time to update the various libraries it uses.

One bug that occasionally cropped up is logging in for the first time and changing to the bookmark page, but having nothing show up. If I did a full refresh of the page then everything would be fine. I never quite figured out what was going on, until now. I assumed there was something I wasn’t doing right with Knockout, so I tried moving the code that does the bindings from the PageBeforeShow event for the bookmark page to right after the AJAX call that gets the data. But that didn’t help. Turns out it was something simple: the bookmark container is initially hidden, and I was never showing it after the first successful login, assuming the app was started in a logged-out state. The solution was showing the container right after the bindings were applied.

And speaking of content not showing right, it suddenly became necessary to add an explicit jQuery Mobile refresh call on the list that holds the bookmark data, like this:

$("#bmMain").listview("refresh");

Without such a call none of the jQuery Mobile styles would be applied unless I did a full page refresh. Not sure why, but whatever. The latest version did resolve a weird style issue with the bookmark menu list showing a double border around each list item, so that’s good.

This is part 1 in a series of posts about creating a mobile web app for browsing music databases.

In my continuing quest to up my web/mobile game, I decided to build a web app for searching the All Music Guide database. There are a number of music metadata repositories on the web, some robust and some paltry. The one I like the best is run by Rovi (now Tivo) and powers All Music. They had an iOS app that allows you to access all parts of the data store. It was great, but the search portion is not very user friendly and doesn’t always work the way you expect.

My goal was to create a basic search form that allows you to look up an artist, album, or song title. The native app had an all-in-one search feature where it tried to dynamically show you results for what it thought you were looking for, but it often failed to return what I wanted. I understand the ease and utility of having a single field for different types of information, but I wanted to run specific searches.

The first step was creating an API that would essentially wrap the calls to Rovi’s RESTful API. I didn’t want to interface directly with Rovi for several reasons: to allow the results to be formatted differently if I wanted, to make it easier to switch to a different data store in the future, and so I wouldn’t have to allow cross-origin requests. I went with the standard Web API project in Visual Studio 2013.

Getting the routing to work properly was the only real hurdle with the API. The default Visual Studio template sets up routes that include /api/<controller>. But I just wanted /api and not the <controller> part. It’s tricky because the base controller class is ApiController. The MVC convention is controller class names are of the form <my_ctl_name>Controller and then my_ctl_name becomes part of your route. Having my own class called ApiController wasn’t possible, so I called it SiteApiController. But how to tell all requests to use that controller? Enter WebApiConfig.

I removed all the default routes and added two new ones: one for searching and one for lookups. I was able to specify the exact controller class and a route template that only included /api and not a controller designation. Bonus points to Microsoft for allowing lots of route configuration options.

Even though I only needed to make single, synchronous requests to Rovi, I used HttpClient to do it. There might be a need in the future to make multiple simultaneous requests to build a query result, if so it will be easy to make them asynchronous. The next step was the front end.

I’ve been waiting to upgrade my iPhone 4S to iOS 6 until there was a maps app available other than the new native one that Apple built. And by ‘other’ I really mean one from Google. I only use maps for directions about half the time; the other half is spent searching, which is precisely where the Apple app falls down hard. So it was with great pleasure that I installed the official Google native maps app shortly after it was released. It’s beautiful, and definitely works as well as the pre-iOS 6 one. A couple days after loading it I pulled the iOS 6 trigger. Everything I used on a regular basis was working great…or so I thought.

I use my bookmark browser every day. It has an option to refresh bookmark data each time you start a new session, but I’m not using it at the moment. That means I must manually refresh the data, which I do every week or so. But after the OS upgrade the refresh process started acting weird. Normally it takes a few seconds to get the data from Mozilla Sync. Once the data is downloaded the last refresh timestamp is updated along with the total bookmark count. What was happening is the refresh seemed to happen instantly. It would update the timestamp but the count never changed. I knew this was not right because I happened to do a bunch of recent bookmark organizing and the number should have been very different.

I found that clearing all website data for Safari would allow the refresh to work properly. But that blows away any data from other web sites, which is bad. I finally got annoyed enough that I opened up the code to see if maybe I was doing something wrong. My investigation lead to the cause of the problem: Apple had messed up again.

Here is what seems to be happening, based on this really popular question at Stack Overflow: Apple is servicing POST requests from its cache rather than sending them to the server if no cache-control header is included and the request data is unchanged. It appears to be a bug. The AJAX call the bookmark browser makes to the server uses the same parameters each time: username, password and key. So the browser was basically not making the call to the server, and my post-request code was executing as if it had.

The solution was to set the ‘cache-control’ header in the POST to ‘no-cache’. It seems to have solved the problem. The good news is an official fix is supposed to be in the up-and-coming iOS 6.1. It seems like version 6 is on track to be the most problematic iOS release yet.

As I was putting the final touches on my Bookmark browser project, I came across this:

blog.mozilla.org/services/2012/08/31/retiring-firefox-home

That’s delicious, because I strongly suspected Mozilla wasn’t interested in improving Home or fixing any issues with it. Yet another reason I wanted to build a replacement. They have published the iOS Sync client source code that Home was built upon at GitHub. It would be fun to use that code and learn Objective-C while building a new app, but first I need to get a Mac. Maybe next year…

Update:
It seems a developer has taken that ball and run with it. He posted an app in the App Store that is essentially the same as Home. I’ll be giving it a try.

Later update:
OK, this is lame. He basically took the Firefox Home source code, removed all references to Mozilla, slapped on a new name, and submitted it to the App Store. All the same flaws are there, including the re-arrangement bug that I hate.

This is part 4 in a series of posts about creating a web-based replacement for the Firefox Home iOS app. Part 3 can be found here.

Now that I had an ironclad way of getting bookmark data, I needed to display it and provide a clean method of navigating through it. I looked around the web for a suitable client-side data binding solution, and KnockoutJS seemed like a good way to go. I wasn’t familiar with the MVVM pattern but thought it would be a good learning experience. In a nutshell, the way I understand MVVM is you have a model of your data that is kept strictly separate from the client-side view of it. A view model lives on the client that takes in the data itself and knows where to put it in the UI (admittedly, I may be simplifying or leaving something out).

A quick note about how Firefox organizes bookmarks: it stores them in two high-level containers, the bookmarks toolbar and the bookmarks menu. No big mystery where these are: it’s the built-in toolbar for single-click access to bookmarks, and the Bookmarks menu in the main menu bar at the top of the window.

I set up an unordered list for the bookmarks that looked like this:

<div id="bookmarkContainer">
    <ul id="bmMain" class="bookmarkList" data-role="listview" data-divider-theme="e">
        <li id="toolbarDivider" data-role="list-divider">Bookmarks Toolbar</li>

        <!-- ko foreach: BookmarksToolbar -->
        <li data-bind="bookmarkItemType: ItemType" data-icon="false">
            <div class="imageBlock"><img class="ui-li-icon" alt="Item icon" width="16px" height="16px" /></div>
            <div class="nameAndLocationBlock">
                <div data-bind="text: Name"></div>
                <div class="locationBlock" data-bind="text: Location"></div>
            </div>
        </li>
        <!-- /ko -->

        <li id="menuDivider" data-role="list-divider">Bookmarks Menu</li>

        <!-- ko foreach: BookmarksMenu -->
        <li data-bind="bookmarkItemType: ItemType" data-icon="false">
            <div class="imageBlock"><img class="ui-li-icon" alt="Item icon" width="16px" height="16px" /></div>
            <div class="nameAndLocationBlock">
                <div data-bind="text: Name"></div>
                <div class="locationBlock" data-bind="text: Location"></div>
            </div>
        </li>
        <!-- /ko -->
    </ul>
</div>

I went with the foreach binding in Knockout, since it made the most sense for what I was doing. I briefly tried the template binding but it turned out to be more involved than necessary. I initially tried to have a single unordered list that held everything, including the dividers for toolbar bookmarks and menu bookmarks. But I had a hard time changing the theme for the divider items on the fly, which I wanted to be a different color. Knockout lets you define binding event handlers, like this one:

ko.bindingHandlers.bookmarkItemType = {
    init: function (element, valueAccessor, allBindingsAccessor, viewModel, bindingContext) {
        if (viewModel.ItemType === 0) {
            $(element).jqmData('icon', 'arrow-r');
            $(element).find("a").attr("href", "#");
            $(element).find("a").attr("onclick", "doNavigation(this);");
            $(element).find("img").attr("src", "Images/folder.png");
            $(element).find(".locationBlock").css("display", "none");
        }
        else if (viewModel.ItemType === 1) {
            $(element).find("a").attr("href", viewModel.Location);
            $(element).find("a").attr("target", "_blank");
            $(element).find("img").attr("src", "Images/bookmark.png");
            $(element).find(".locationBlock").css("display", "block");
        }
    }
};

Here I’m basically setting attributes on the elements in each list item based on whether it’s a directory or an actual bookmark. I tried modifying the data-theme attribute used by JQM to indicate a list divider, but the change never seemed to get applied. Fortunately Knockout lets you apply the binding logic to a subset of items in a ul via containerless control flow syntax. There was some duplication of markup, but not enough to worry about.

The various samples at knockoutjs.com and the web at large had two different ways of setting up the view model: as a JavaScript variable or a function. I went the function route since I needed to do a bit of data manipulation before assigning the bookmark data. Knockout has the capability of updating the UI automatically when the bound data changes, such as in response to a user clicking on something. That sort of fit my usage pattern, though I would be updating the data via code. The mechanism Knockout uses is observables, and you simply declare them in your view model. Mine looked like this:

var bookmarksViewModel = function () {
    var self = this;
    var currentBookmarks = JSON.parse(localStorage.getItem("CurrentBookmarks"));
    self.BookmarksToolbar = ko.observableArray(currentBookmarks.BookmarkItems[0].BookmarkItems);
    self.BookmarksMenu = ko.observableArray(currentBookmarks.BookmarkItems[1].BookmarkItems);

    self.setBookmarks = function (node) {
        if (node) {
            self.BookmarksToolbar(node.BookmarkItems);
            self.BookmarksMenu([]);

            $("#toolbarDivider").hide();
            $("#menuDivider").hide();
        }
        else {
            // We are back at the topmost level, so show the toolbar and menu bookmarks
            self.BookmarksToolbar(currentBookmarks.BookmarkItems[0].BookmarkItems);
            self.BookmarksMenu(currentBookmarks.BookmarkItems[1].BookmarkItems);

            $("#toolbarDivider").show();
            $("#menuDivider").show();
        }
    }
};

The initial state of the app would always be to show everything in the bookmarks toolbar followed by everything in the bookmarks menu, which matches how Firefox Home shows things. I set up two observable arrays holding each of those sets. The trickiest part was how to go about updating that data when the user wanted to navigate into a directory. The rule in Knockout is you only apply the bindings once, then let the framework update the UI for you when things change. And since references to the two observable arrays will be maintained by the framework, I would need to replace their contents rather than assign completely new arrays.

Several failed attempts ensued. I tried using the removeAll() method that is available on observable arrays, then the push() method which is also available on observable arrays but is slightly different than the native JavaScript one (it turns out to only take single elements rather than an array of elements). I finally found the best way to do it via a blog post from Ryan Niemeyer, who totally has the title of Knockout expert locked up.

So the setBookmarks() function in my view model allows me to update the bound data at any time. I wrote the following function to handle the navigation when the user selected a directory rather than an actual bookmark:

function doNavigation(sender) {
    var nodePath = $(sender).attr("nodePath").split("\\");
    var newHeader;

    if (sender.id === "backButton") {
        var newPath;
        nodePath.pop();

        if (nodePath.length === 0) {
            return;
        }

        if (nodePath.length === 2) {
            newPath = "Root";
            newHeader = "Bookmarks";
        }
        else {
            newPath = nodePath.join("\\");
            nodePath.shift();
            newHeader = nodePath[nodePath.length - 1];
        }

        $("#backButton").attr("nodePath", newPath);
    }
    else {
        nodePath.shift();
        newHeader = nodePath[nodePath.length - 1];
        $("#backButton").attr("nodePath", $(sender).attr("nodePath"));
    }

    var currentBookmarks = JSON.parse(localStorage.getItem("CurrentBookmarks"));
    var curNode = getNode(currentBookmarks.BookmarkItems, nodePath);
    ko.dataFor($("#bookmarkContainer")[0]).setBookmarks(curNode);
    $("#bmHeader").html(newHeader);

    // For some reason the 'slide' transition doesn't work right when using it in a call
    // to changePage. It does the animation, but then doesn't show the page content
    $.mobile.changePage($("#Bookmarks"), { allowSamePageTransition: true, transition: "slidefade" });
}

The bookmark data itself was an object that contained a list of directories and bookmarks. Each directory had a list of subdirectories and bookmarks, while bookmarks had just a name and URL. The list was stored as an array on the client, so it was just a matter of finding the right one to use in my Knockout view model. The idea in doNavigation() was to take the path to the node the user just pressed and get the bookmark items assigned to it, or if Back was pressed get the bookmark items for its parent.

Each time the user selected a bookmark a new page would be opened, and if they selected a directory I needed to bind to the proper array in my main data object. I wrote the following function to search that object to find the exact items to display:

function getNode(items, nodePath) {
    var curDir = nodePath.shift();
    var node = null;

    for (var i = 0; i < items.length; i++) {
        if (items[i].Name === curDir && items[i].ItemType === 0) {
            // We know to stop when we've found the final directory in the node's path
            if (nodePath.length === 0) {
                node = items[i];
                break;
            }
            else {
                node = getNode(items[i].BookmarkItems, nodePath);
                break;
            }
        }
    }

    return node;
}

Putting it all together gave me a reasonable imitation of the navigation in Firefox Home. The speed of the navigation wasn’t too bad, though I have an idea of how I might improve it (Update: the ‘idea’ was a CSS rule to reduce the animation duration. It didn’t work, but I suspect it might be because my rule is incomplete). I’d like to get as close to native app response time as possible. The full source is posted at GitHub.

I feel I have a fairly good grasp on jQuery Mobile now. It would be interesting to try to incorporate PhoneGap into the project and see how it works.

This is part 3 in a series of posts about creating a web-based replacement for the Firefox Home iOS app. Part 2 can be found here.

What I wanted to do next was write enough code to save Sync settings and load bookmarks, log out as the current Sync user and wipe out the locally stored bookmarks and credentials in the process, and refresh bookmark data, all without any problems of any kind. Basically everything the user could do on the settings page. I planned to do as much work on the client as possible, since that is where the bookmark data would be stored. When the user saved new credentials I wanted them to be directed to the bookmark page, and if they were simply refreshing they would stay on the settings page but would see an updated count of their bookmarks.

The biggest issue was doing the form submissions and getting the UI to behave accordingly, including when an error occurred on the server. What was tricky was the fact that to get bookmark data from the Sync servers I made an Ajax call to a server-side function, and JQuery Mobile relies mainly on Ajax calls to do its thing. That coupled with the fact that my Ajax call was non-blocking made the UI flow have a bunch of niggling problems. Mostly it wouldn’t show the right things at the right time.

The best way I found to make it all work correctly was to disable the JQuery Ajax handling of the button clicks. I added data-ajax=”false” to the form element, then canceled the posting to the server and did everything manually. Essentially I took JQuery out of the equation. In keeping with the credo of the framework that you work with pure HTML elements and not server-based ones, I replaced the asp:Button controls with input elements and made the form element a regular client element, like so:

<div id="Settings" data-role="page">
    <div data-role="header" data-theme="b">
        <h5>Settings</h5>
    </div>

    <div class="smallText" data-role="content" data-theme="b">
        <div class="msgPanel">&nbsp;</div>
        <form id="settingsForm" action="Default.aspx" method="post" data-ajax="false">
            <div id="LoggedOut">
                <div class="bottomPadding20"><input id="txtUserName" type="text" data-mini="true"></div>
                <div class="bottomPadding20"><input id="txtPassword" type="password" data-mini="true"></div>
                <div class="bottomPadding20"><input id="txtSyncKey" type="password" data-mini="true"></div>
                <div class="alignCenter"><input id="Save" type="button" value="Save" data-icon="check" data-mini="true"></div>
            </div>

            <div id="LoggedIn" class="alignCenter hideMe">
                <table class="statsTable">
                    <tbody>
                        <tr>
                            <td class="labelColumn">&nbsp;</td>
                            <td class="fieldColumn">&nbsp;</td>
                        </tr>
                        <tr>
                            <td class="labelColumn">&nbsp;</td>
                            <td class="fieldColumn">&nbsp;</td>
                        </tr>
                        <tr>
                            <td class="labelColumn">&nbsp;</td>
                            <td class="fieldColumn">&nbsp;</td>
                        </tr>
                    </tbody>
                </table>

                <input id="Logout" type="button" value="Logout" data-icon="delete" data-mini="true"> <br><input id="Refresh" type="button" value="Refresh" data-icon="refresh" data-mini="true">
            </div>
        </form>
    </div>

    <div data-role="footer" data-theme="b">
        <div data-role="navbar">
            <ul>
                <li><a href="#Home" data-role="button" data-iconpos="bottom" data-icon="home">Home</a></li>
                <li><a href="#Bookmarks" data-role="button" data-iconpos="bottom" data-icon="star">Bookmarks</a></li>
                <li><a href="#Settings" data-role="button" data-iconpos="bottom" data-icon="gear">Settings</a></li>
            </ul>
        </div>
    </div>
</div>

The Save onclick handler looked like this:

function Save_OnClick() {
    var userName = $("#txtUserName").val();
    var password = $("#txtPassword").val();
    var syncKey = $("#txtSyncKey").val();

    if (!userName) {
        displayMessage("User name cannot be blank", "Settings");
        return false;
    }
    if (!password) {
        displayMessage("Password cannot be blank", "Settings");
        return false;
    }
    if (!syncKey) {
        displayMessage("Sync key cannot be blank", "Settings");
        return false;
    }

    clearMessagePanel("Settings");
    $.mobile.showPageLoadingMsg();
    loadBookmarks(userName, password, syncKey, "Save");

    return false;
}

It called this common function for getting bookmark data:

function loadBookmarks(userName, password, syncKey, action) {
    $.ajax({
        type: "POST",
        url: "Default.aspx/LoadBookmarks",
        contentType: "application/json; charset=utf-8",
        data: "{'userName':'" + userName + "', 'password':'" + password + "', 'syncKey':'" + syncKey + "'}",
        dataType: "json",
        headers: {"LoadAction":action},
        error: function (error) {
            var resp = JSON.parse(error.responseText);
            displayMessage(resp.Message, "Settings");
        },
        success: function (data) {
            localStorage.setItem("CurrentBookmarks", JSON.stringify(data.d));

            if (action === "Refresh") {
                localStorage.setItem("BookmarkCount", data.d.Tag);
            }
            else if (action === "Save") {
                localStorage.setItem("UserName", userName);
                localStorage.setItem("Password", password);
                localStorage.setItem("SyncKey", syncKey);
            }
        }
    });
}

I added a handler for the jQuery ajaxCompleted event and used that to handle the post-retrieval stuff, like hiding the input controls or updating the bookmark count.

function ajaxCompleted(e, xhr, settings) {
    if (settings.url === "Default.aspx/LoadBookmarks") {
        if (xhr.status < 400) {
            switch (settings.headers["LoadAction"]) {
                case "Save":
                    $.mobile.changePage("#Bookmarks");
                    break;
                case "Refresh":
                    loadSettingsPage();
                    break;
            }
        }

        $.mobile.hidePageLoadingMsg();
    }
}

At this point I tried everything out in Electric Plum to get an idea of how it might look on an actual iPhone. It was fairly good, only a couple of cosmetic issue came up that might not even be issues on a real phone.

Next, listviews and data binding.

This is part 2 in a series of posts about creating a web-based replacement for the Firefox Home iOS app. Part 1 can be found here.

I set up my bookmark browser web app project and started reading through the excellent jQuery Mobile site. I planned to have a home page, a page for settings, an About page, and one to show the actual bookmarks. So I created separate .aspx pages, each set to use a single master page, and added a page template to each one. The master page had a link to a CSS file I created and links to all the jQuery scripts hosted at code.jquery.com. It also had a single JQuery Mobile header and footer since I didn’t want to have to duplicate markup over several pages. I put in a few text boxes on the settings page and added a button for saving Sync credentials to local storage. I made it so when your credentials were saved, it would hide the input controls via JavaScript and show a button for refreshing the data.

When I ran it and tried out the navigation, things were not working as expected. The page state wasn’t being saved when I would browse from the settings page to the home page and back, meaning my hiding and showing of elements was failing. I had put links to each page in the main footer, and they would take you to the respective page, but old content was being shown. I was adjusting styles between each build, and those changes weren’t being reflected properly. In short, it was a mess.

After more reading through the JQuery Mobile site, I realized I was going to have to change my approach. Probably the most unique thing about JQuery Mobile is the navigation paradigm. It’s very different from what I was used to. Each time it needs to access content, it will make an AJAX call behind the scenes and insert that content into the DOM. There are ‘transitions’ between pages, but they are not pages in the sense of separate files in your project. They can be, but everything ends up in one big container anyway.

I ended up using multi-page templates, all housed in default.aspx. I ditched my master page and set up separate div elements for each page. I had to duplicate the header and footer markup, which I’m not thrilled about, but it isn’t much and I might find a better solution in the future. I experimented with using true single-page templates, but they just didn’t work how I wanted, whereas in the multi-page scenario everything flowed as I expected. DOM size wasn’t too much of a concern because apart from the page for bookmarks, the other pages have little markup in them. It may be possible to incorporate a master page into the navigation model, but I’d rather press on with the actual functionality.

Next, how to handle transitions to the current page, and others in the DOM.

This is part 1 in a series of posts about creating a web-based replacement for the Firefox Home iOS app.

I installed the Firefox Home iOS app shortly after I got an iPhone 4S in late 2011. It provided access to my 1,500+ bookmarks from my phone, and it has a nice feature where it can be set to use Safari as the browser rather than a UIWebView. However, that version, and still the current one as of this writing, has some sort of bug which is horribly annoying.

Occasionally I would add a new bookmark from my desktop, Firefox Home would refresh its data, and then its ordering of bookmarks would get messed up. It was most prevalent in the Bookmark Toolbar directory, in which I have subdirectories and individual bookmarks. I have all of the subdirectories positioned at the far left end of the toolbar in Firefox itself, and they are shown at the top of the main UI in Firefox Home. But when this problem occurred, they would get shuffled around in the UI with no apparent rhyme or reason. It would even happen if I didn’t explicitly add something to the toolbar directory. I haven’t been able to figure out the exact circumstances that cause it, but it has happened enough that I decided to create a replacement, as a personal project.

Unfortunately I don’t own a Mac or know Objective-C, so I decided to create a mobile web app using ASP.NET, C# and jQuery Mobile, and to document the process. My first goal was to successfully get data out of my own Sync account. Sadly, the Sync API is not as well documented as it could be. What documentation does exists is fairly spread out, and there aren’t any canonical examples of how to write clients for it. Plus it has a complex security design; a specific series of decryption steps are needed to read data from a Sync account.

Through some Google searching I stumbled upon a sample program a guy had written in Python that did all the proper encryption stuff and was able to retrieve bookmarks. I managed to duplicate the process in C# after some trial and error, to the point where I could see decrypted data from my own account. It was then that I decided, on a lark really, to search the web to see if anybody had written any .NET wrappers for the Sync API. And wouldn’t you know, Pieter De Rycke had done just that. In fact, he had written a complete Windows Phone version of Firefox Home. So I decided that reinventing the wheel wasn’t necessary, and I used the bits of his project that wrapped the essential calls to the Sync API. Thanks Pieter. (Update: the original pre-Sync 1.5 code is no longer around, and he eventually updated the wrapper to support the authentication changes in version 1.5 of the Sync service, which corresponded to the release for Firefox 30).

My goal for the front end of my app was to mimic the Firefox Home UI as much as possible. Initially I figured I would do most of the processing on the server then simply use JQuery Mobile to make the UI mobile-friendly. But there was more to it than that.

In 2004 I was working at a job which involved projects that made heavy use of VBA; from an Access application that would literally build an entire PowerPoint presentation from scratch, to Excel workbooks that would do all sorts of things that Excel was probably never designed to do. None of them would have been possible without the very thorough and extensive COM-based API that Microsoft makes available for all its Office applications.

The one place where the Office API falls down though is in web applications. Being single threaded, the code does not play nice in that kind of environment, and it is not officially supported by Microsoft (though it is not strictly impossible to use it that way). Recently I needed to have an ASP.NET web application of mine open an Excel template and add some data. I figured I would need to do something like build a separate web service to handle the Excel processing, using the same API I know and sort of love. Then I discovered Microsoft had published an SDK specifically for interacting with the XML-based default file formats introduced in Office 2007. And to my delight, they provided a managed DLL that would integrate nicely with ASP.NET. Life was good.

Or so I thought. Turns out the new API is not as nice as the COM-based original. The main issue I had is it takes a very XML-centric approach to doing things, which made it hard to implement the specific Excel things I wanted, like cell formatting, copying rows, etc. The old API was much more closely aligned with the individual Office apps, making it fairly straightforward to utilize whatever document features you wanted via code. Fortunately I wasn’t the first to feel like the new way was harder than it had to be. I recently came across ClosedXML, which is a .NET wrapper around the new API that makes doing Excel-specific things so much easier. Many thanks to MDeLeon for his excellent creation.