Chapter 6: Client Data Management and Caching

Article
08/28/2012

Introduction | Client-Side Data Design | Data Manager - Benefits of the Data Manager Abstraction, Data Request, Data Request Options, Performing a Data Request, Performing Ajax Requests | Data Cache - Adding and Retrieving Cached Data, Removing a Data Cache Item | Summary | Further Reading

Introduction

Web applications are intended to present data to the user. In rich, interactive, client-centric applications like Mileage Stats, users expect the application to respond quickly to mouse gestures, page transitions, and the saving of form data. Delays caused by data retrieval or saving data can negatively impact the user's experience and enjoyment of the site. Therefore, carefully managed data retrieval and caching is critical to today's web applications.

Note

The term data caching used in this documentation refers to the caching of client-side JavaScript objects within the DOM (document object model) rather than using the browser's built-in page caching mechanism.

A sound client-side data management strategy is critical to the success of a web application and will address fundamental concerns such as:

Maintainability. Writing clean maintainable JavaScript code requires skill, discipline, and planning. The Mileage Stats data implementation addresses maintainability by providing a simple object for centralized data access that other application objects use to execute data requests and to cache the results.
Performance. Client-side caching and prefetching of data plays a key role in achieving application responsiveness from the user's perspective. Eliminating unnecessary data calls to the server enables the browser to process other tasks, such as animations or transitions, more quickly.. Mileage Stats addresses these performance concerns by caching data returned from the server, using prefetching to acquire data that a user is likely to view, and using Ajax to perform asynchronous data calls to the server.
Scalability. Client-side objects should avoid making repeated requests to the server for the same data. Unnecessary calls to the server require additional server-side resources, which can negatively impact the scalability of your application. Mileage Stats uses client-side data caching to increase scalability.
Browser support. Your choice of a data caching mechanism will influence which browsers your application can support. Rather than employing a browser-specific feature such as local storage, Mileage Stats caches data using a generic JavaScript object caching mechanism so that older browsers such as Windows® Internet Explorer® 7 are supported.

In this chapter you will learn:

How Mileage Stats manages and centralizes application data requests.
The benefits of a client-side data manager and how data requests are abstracted.
How to improve application performance by caching and prefetching data.

The technologies and libraries discussed in this chapter are Ajax, JavaScript, jQuery, and jQuery UI Widgets.

Note

Data validation on the client and server is covered in Chapter 11, "Server-Side Implementation."

Client-Side Data Design

The Mileage Stats data solution centers on the data manager, which handles client-side data requests and manages the data cache. The diagram below shows the relationship of the client-side JavaScript objects to one another and to the server JSON (JavaScript Object Notation) endpoints.

Mileage Stats client-side data architecture

Hh404101.a35421be-c17f-493c-af62-dc78031c857f(en-us,PandP.10).png

Mileage Stats objects use URLs when requesting data from the server. URLs were chosen because their use simplifies the data manager's design by providing a mechanism to decouple the server JSON endpoints from the data manager's implementation.

The URL contains the JSON endpoint and, optionally, a data record key value corresponding to the requested object. The URL typically aligns with the UI elements the object is responsible for. For example, the reminders widget uses "/Reminder/JsonList/1" to retrieve the reminders for the vehicle with the ID of 1.

When data is requested from the data manager, it returns the data to the caller and optionally caches it. The caching of data improves the performance of the application because repeated requests to the server for the same data are no longer necessary.

In addition to data caching, Mileage Stats also prefetches the chart data. The chart data is prefetched on the initial page load because there is a reasonable expectation that users will use the Charts page to compare their vehicles. The prefetching of this data enables an instant response when the user navigates to the Charts page.

In your applications, the amount of data you elect to prefetch should be based on the volatility of the data, the likelihood of the user accessing that data, and the relative cost to get that data when the user requests it. Of course, the number of concurrent users and the capabilities of your web and database servers also play a role in this decision.

Data Manager

All Ajax data requests are routed through the dataManager JavaScript object contained in the mstats-data.js file. The data manager is responsible for performing data requests and managing interactions with the data cache. The data manager has a simple public interface that exposes two methods: sendRequest, which processes Ajax calls to the server, and resetData, which removes a requested item from the data cache.

The next three sections examine the benefits of the data manager abstraction, look at how data requests are initiated by jQuery UI widgets, and show how those requests are executed by the data manager.

Benefits of the Data Manager Abstraction

Abstracting the data request implementation as a data manager object provides an injection point for data caching, which is a cross-cutting concern. Data requestors get the full benefit of data caching without taking on another dependency or implementing additional code. Isolating the data cache also makes it much easier to change the data cache implementation because only the data manager has a direct dependency on it.

The data manager improves application testability by allowing developers to unit test data requests and data caching in isolation. The data manager also facilitates changing the application over time. Evolution of an application is required not only after its release but during development as well. For example, the team added a feature to the data manager that would have required modifying all the Ajax request code. Had the data manager not been implemented, the change would have been riskier and potentially costlier.

The team discovered that when the website was deployed to a virtual directory as opposed to the root directory of the web server, the URLs in the JavaScript code had not taken the virtual directory into account. The fix for this problem only had to be applied to the data manager, which saved development and testing resources. This feature is discussed in the "Performing Ajax Requests" section below.

Data Request

Client-side data requests in Mileage Stats are initiated by jQuery UI widgets and JavaScript objects, and performed by the data manger. The data manager sendRequest method has a signature similar to the signature of the jQuery ajax method. Widgets making requests set their calls up as if they are calling jQuery ajax, passing an options object that encapsulates the URL, a success callback, and optionally an error callback or other callbacks such as beforeSend or complete.

Data Request Options

When a widget is constructed, the options provided supply the methods needed to execute a data request or to remove an item from the data cache. Externally configuring widgets removes tight coupling and hard-coded values from the widget. Widgets can also pass their options, like sendRequest, to other widgets they create.

This technique of external configuration enables Mileage Stats widgets to have a common data request method injected during widget construction. In addition to run-time configuration, this technique also enables the ability to use a mock implementation for the data request methods at test time.

In the following code, the summaryPane widget is constructed, and its sendRequest and invalidateData options are set to corresponding data manager methods. The summary widget does not make any data requests; instead, these two methods will be passed into child widgets created by the summary widget.

// Contained in mileagestats.js
summaryPane = $('#summary').summaryPane({
  sendRequest: mstats.dataManager.sendRequest,
  invalidateData: mstats.dataManager.resetData,
  publish: mstats.pubsub.publish,
  header: header
});

In the code snippet below, the summary widget constructs the child widget, statisticsPane, and passes the above sendRequest and invalidateData data manager methods as options. Setting these options replaces the default implementation defined in the statistics widget for making data requests. Now, when the statistics widget performs a data request, the method defined in the data manager will be executed.

// Contained in mstats.summary.js
_setupStatisticsWidget: function () {
    var elem = this.element.find('#statistics');
            
    this.statisticsPane = elem.statisticsPane({
        sendRequest: this.options.sendRequest,
        dataUrl: elem.data('url'),
        invalidateData: this.options.invalidateData,
        templateId: '#fleet-statistics-template'
    });
},

The dataUrl option is the URL or endpoint for the data request. The url value is stored below in the data- attribute in the HTML. The statisticsPane widget is attached to and is queried by the elem.data method call seen above. Externally configuring data endpoints prevents you from having to hard-code knowledge about the server URL structure within the widget.

// Contained in \Views\Vehicle List.cshtml
<div id="statistics" class="statistics section" 
     data-url="@URL.Action("JsonFleetStatistics","Vehicle")">
...
</div>

Performing a Data Request

The sendRequest method has the same method signature as the jQuery ajax method, which takes a settings object as the only argument. The _getStatisticsData method, below, passes the sendRequest method an object literal that encapsulates the url, success, and error callbacks. When the Ajax call completes, the appropriate callback will be invoked and its code will execute.

// Contained in mstats.statistics.js
_getStatisticsData: function () {
    var that = this;
    that.options.sendRequest({
        url: this.options.dataUrl,
        success: function (data) {
            that._applyTemplate(data);
            that._showStatistics();
        },

        error: function () {
            that._hideStatistics();
            that._showErrorMessage();
        }
    });
},

The above pattern simplified the Mileage Stats data request code because it does not need to know about the data caching implementation or any other functionality that the data manager handles.

Now that you understand how widgets and JavaScript objects initiate a data request, let's examine how the data manager makes the Ajax request to the server and see how data caching is implemented.

Performing Ajax Requests

The data manager sendRequest method is used to request data from the server and in some places it is used to execute a command on the server, such as fulfilling a reminder. Because the jQuery ajax method signature is the same for requesting as well as posting data, the team chose to implement a single method for Ajax calls to the server. In addition to success and error callbacks, the sendRequest method has an option to cache the request or not. By default, requests are cached.

Mileage Stats has two use cases where data is not cached: data requests that only post data to the server, and the Pinned Sites requests. Results of the Pinned Sites requests are not cached because these requests are only initiated by events after data has changed. Because Pinned Sites only refreshes its data after a change, the data request needs to get fresh data from the server.

The diagram below illustrates the logical flow of a data request. The data manager services the request by first checking if the request should be cached and, if so, checks the cache before making a call to the server. Upon successful completion of the request, the resulting data will be returned to the user and added to the cache according to the option.

Data request

Hh404101.aed9afe1-8508-4c7a-b0ac-564eb8092e32(en-us,PandP.10).png

Now let's look at the code that implements the functionality of the above diagram. The sendRequest method, shown below, first modifies the URL to account for the virtual directory the website is deployed to by calling the getRelativeEndpointUrl function. Using the modified URL, it attempts to retrieve the requested data from the data cache. The options are then merged with the data manager's default options. If the caller wants the data cached, and data was found in the cache, it's immediately returned to the caller. If the data is not found, the jQuery ajax call is made. If successful, and the caller requested the data to be cached, it is added to the cache and the caller's success callback is invoked. If an error occurs and the caller implemented an error callback, it will be invoked. If a global Ajax error handler has been defined, it will be invoked after the error callback.

Note

The jQuery ajax method can be configured at the global level to define default options as well as default event handlers. Mileage Stats defines the global Ajax error handler shown above.
For more information about how Mileage Stats implements the global Ajax error handler, see the "User Session Timeout Notification" section in Chapter 10, "Application Notifications."

// Contained in mstats.data.js
sendRequest: function (options) {
  // getRelativeEndpointUrl ensures the URL is relative to the website root.
  var that = mstats.dataManager,
            normalizedUrl = mstats.getRelativeEndpointUrl(options.url),
            cachedData = mstats.dataStore.get(normalizedUrl),
            callerOptions = $.extend({ cache: true },
                                       that.dataDefaults,
                                       options, 
                                       { url: normalizedUrl });

  if (callerOptions.cache && cachedData) {
    options.success(cachedData);
    return;
  }

  callerOptions.success = function (data) {
    if (callerOptions.cache) {
      mstats.dataStore.set(normalizedUrl, data);
    }
    options.success(data);
  };

  $.ajax(callerOptions);
},

Note

getRelativeEndpointUrl is a utility method in the mstats.utils.js file that is used to modify the URL passed in the argument, inserting the virtual directory the website is installed under. This is necessary because the virtual directory is not known until run time.

Data Cache

The Mileage Stats data manager uses an internal data cache for storing request results; the data cache is only accessed by the data manager. Making the data caching internal to the data manager allows the caching strategy to evolve independently without affecting other JavaScript objects that call the data manager.

The data cache is implemented using a JavaScript object named dataStore that is contained in the mstats-data.js file. Other data cache storage locations could include the DOM, a browser data storage API or a third-party library. The dataStore JavaScript object was implemented because Mileage Stats supports Internet Explorer 7, which does not support the HTML5 web storage specification.

Adding and Retrieving Cached Data

Note

The Mileage Stats dataStore object is scoped to the page, not the browser window. Consequently, when the page is fully reloaded from the server, the cached data will no longer be cached since the data cache object will be recreated when the page loads.

Mileage Stats integrates client-side data caching into the data manager's sendRequest method implementation that was described in the previous section.

Internally, the dataStore is implemented using name/value pairs. It exposes three methods: get, to retrieve data by a name; set, to cache data by a name; and clear, to remove data corresponding to a name.

// Contained in mstats.data.js
mstats.dataStore = {

  _data: {},

  get: function (token) {
    return this._data[token];
  },

  set: function (token, payload) {
    this._data[token] = payload;
  },

  clear: function (token) {
    this._data[token] = undefined;
  },
};

Removing a Data Cache Item

In addition to the data manager retrieving and adding data to the cache, it also provides the resetData method for removing cached data by URL.

// Contained in mstats.data.js
resetData: function (endpoint) {
  mstats.dataStore.clear(mstats.getRelativeEndpointUrl(endpoint));
}

Mileage Stats objects call the resetData method when client-side user actions make the cached data invalid. For example, when a maintenance reminder is fulfilled, the requeryData method shown below will be called by the layout manager widget. When designing your data architecture, it is important to consider which client-side actions should invalidate the cache data.

// Contained in mstats.statistics.js
refreshData: function () {
  this._getStatisticsData();
},

requeryData: function () {
  this.options.invalidateData(this.options.dataUrl);
  this.refreshData();
},

The requeryData method first invokes the invalidateData method, passing the URL of the cache item to remove. The invalidateData method is an option on the statistics widget, which was passed the data manager's resetData method when the widget was created. Now that the data cache item has been removed, the next call to refreshData will result in the data manager not locating the cached data keyed by the URL, and then executing a request to the server for the data.

Summary

In this chapter, you have learned about the design, benefits, and implementation of a centralized client-side data manager that executes all Ajax requests and manages the caching of data. This approach simplifies testing, facilitates application or external library changes over time, and provides a consistent pattern for objects to follow to execute data requests.

You have also learned how Mileage Stats keeps its widgets free from external dependencies and the hard-coding of server URLs by constructing and configuring them externally. This approach of injecting external dependencies increases the flexibility and maintainability of the JavaScript code, and the absence of hard-coded server URLs prevents the creation of brittle JavaScript code.

Chapter 6: Client Data Management and Caching

Introduction

Client-Side Data Design

Data Manager

Benefits of the Data Manager Abstraction

Data Request

Data Request Options

Performing a Data Request

Performing Ajax Requests

Data Cache

Adding and Retrieving Cached Data

Removing a Data Cache Item

Summary

Further Reading

Additional resources