Archive

Archive for March, 2012

Care and feeding of large JavaScript projects

March 16, 2012 Leave a comment

Tetsuo reminds us that projects allowed to grow out of control often become difficult to manage.

When building a webapp in JavaScript, you are using JavaScript in a very different way than it was originally intended when the language was designed.  Case in point for this discussion: most languages designed for big projects provide some sort of mechanism for every source file to describe any other source files it depends upon.  C/C++ has the “#include” directive, Java and C# have a strict class design that allows the compiler to determine where to find every source file, and Ruby has the “require” statement.

The JavaScript language has none of these techniques inherent in the language.  Once you are writing code in a JavaScript file, there is no easy way to pull in other JavaScript files that your current file depends on.  Fortunately, JavaScript developers have come up with several mechanisms to solve this problem.

In the old days before Visual Studio, C/C++ programmers used a tool called a ‘Makefile’ to tell the compiler which source files were needed to build a program (kids, ask your parents).  The Makefile was, as its name implies, a separate file the programmer would need to open and add a new source file to every time a new file was written to add to the project.  It was a cumbersome process and prone to error, but it’s also the easiest pattern for a JavaScript programmer to follow.  If you’re building a web-based application, you can use the index.html page as your “Makefile” and add all JS files you need to the <head> element in your index page to have the browser load all needed files, like so:

<!DOCTYPE html>
<html lang="en">
<head>
	<meta charset="UTF-8">
	<title>One webapp to rule them all</title>
	<script src="thirdParty/theirResourceA.js"></script>
	<script src="thirdParty/theirResourceB.js"></script>
	<script src="lib/myResourceA.js"></script>
	<script src="lib/myResourceB.js"></script>
	<script src="lib/myResourceC.js"></script>
	<script src="webApp.js"></script>
	<script type="text/javascript">
		var app = new WebApp();
		window.addEventListener('load', function(){app.run();}, false);
	</script>
</head>
<body>
	<div>One does not simply hack into Mordor.</div>
</body>
</html>

There are a few problems with this approach. As your project grows, this list of files will become cumbersome and unwieldy. It will always be a pain for your developers to open the index.html file to add any new files to the architecture, and there’s no easy way to tell which files depend on which other files, which becomes especially onerous if you need to load the files in a specific order to make sure that the files that depend on certain other files are loaded after those files they are dependent upon. Finally, once your webapp is ready for production, you are probably going to want to run all these files through a minifier anyway, at which point you’ll need to edit the index.html file to load only the minified version of the app rather than all the resource files separately.

Don't let this happen to your project.

Fortunately, there are tools like RequireJS which allow us with a minimal amount of restructuring to declare our JavaScript dependencies within each JavaScript file. This makes the dependency tree much clearer which in turn makes it much easier to move a subset of code over to new projects. Developers no longer need to manage a resource list in an external file like the index.html file. And finally, when the project is complete, RequireJS has tools that assist in minifying the entire project for production.

Check out more on RequireJS below the cut

Read more…

WebSocket Latency Unveiled, News at 11

March 9, 2012 3 comments

As promised in the previous post discussing WebSockets, gentle readers, I have recorded a video showing just how darn fast WebSockets are, even when going through a proxy system like Pusher.

Behold!


Now on YouTube at: http://www.youtube.com/user/JustinSHouk

Building multiplayer HTML5 games – Part 2: WebSockets

March 7, 2012 1 comment

Flynn also failed to write good network support in his architecture.

In the last post we discussed how digital multiplayer games have evolved, various strategies for getting different game clients to keep their games in sync, and why the core HTTP request/response architecture of the web was fundamentally flawed when it came to multiplayer games.  In this post, we’ll dig into the HTML5 solution to this problem and how to use it in desktop or mobile web applications.

WebSockets are a new technology that was written in response to JavaScript developers using AJAX calls to keep checking back with the server to see if anything new had come in.  It was clear that the current design was not meeting the needs of modern Web 2.0 applications, which wanted to leave a connection with the server open for as long as the web page was up so that any new information from the server could instantly be read by the JavaScript running on the client and displayed.

At the lowest levels, WebSockets do piggyback on top of the World Wide Web infrastructure (hence the name). They follow a URL scheme, similar to the ubiquitous HTTP definition.  Since the underlying protocol is different from HTTP, however, the URL scheme begins instead with a new label: ‘ws‘ (for unencrypted streams) and ‘wss‘ (for encrypted streams). For example:

ws://yourserver.com/yourwebsocketresource

If you’re running your own web servers, you’ll need to have a program running on the backend to handle WebSocket requests such as these.  Apache’s ActiveMQ project is one example of WebSocket-aware server software. A full discussion of the pros and cons of various server configurations and software packages to handle WebSockets is beyond the scope of this post.

WebSocket technology is new enough that browser support when running your client HTML5 application should not be assumed.  To assist in this, there are several JavaScript libraries that wrap the WebSocket calls in their own API and thus can downgrade the connection to an HTTP polling connection if the client application is running on a browser that does not support WebSockets, and we’ll dig into one of those wrappers momentarily.  In the HTML5 mobile space, WebSockets were added to Mobile Safari in iOS 4.2 (though the WebSocket spec supported is an out-of-date specification, so make sure your servers are compatible with the old spec if you want to support iOS devices), but as of this writing are not supported on the default Android browser. WebSockets are supported in the newly-announced Chrome for Android beta, which only runs on Android 4.0.

If you are running in an environment that supports WebSockets, interacting with one on the client side is delightfully simple. Read more…

Building multiplayer HTML5 games – Part 1: A History

March 2, 2012 4 comments

The Matrix is a system, Neo

Along with the recent performance upgrades in the major browsers’ implementation of the HTML5 canvas, there is one other big piece of the puzzle falling into place that will provide HTML5 games with an environment that allows them to look and feel just as much like a “real game” as a game developed for Xbox Live Arcade or Steam, particularly where multiplayer is concerned. That piece is the WebSocket, which finally provides web applications with genuine realtime 2-way communication with game servers and/or other devices. But before we get to a detailed discussion of WebSockets, let us take a look at the history of networking as it relates to games and see how we got to where we are today.

A talks to B, B talks to A

In the beginning, there were two game clients who wanted to talk to each other. My personal first experience with this was a little first person shooter called Doom. The clients each ran their own copy of the game and in multiplayer mode, each copy of the game sent messages back and forth over the network so you could see where the other players’ little guys were running around and could try to blow them up before you got blown up. You know, the usual.

When Blizzard Entertainment first set up Battle.net to allow Diablo players to find each other over the Internet and play together, they followed a similar simple networking model. Each client ran a copy of the whole game and sent sync up messages to the other players so you could keep track of what your friends were doing. As long as everyone followed the rules, this was fine. However, it quickly became clear that since each client was the final authority of the game running on that client, it was very hard to catch cheaters who hacked their local copy of the game to give their character an unfair advantage.

Clients talk to the server, the server talks to the clients

Enter the client/server model. In this system, the game is designed from the ground up not to trust the client. Since clients can be hacked, servers are now used to run the central game processing, with clients only having very limited control over their environment. The messages a client sends to the game world become severely restricted in this environment as well. Take, for example, a game where a player is supposed to only move up to three squares a turn. Where before, in an environment where a client might broadcast to the other players in the game, “My player is now in location 200, 200!” and the rest of the clients simply accepted this message, a hacker could easily bypass the three-square-max rule on a hacked client where the rule was not enforced. In a client/server environment, the server verifies all messages that are sent in by the client. In this way, a hacked client that tries to bypass the three-square-max restriction would get caught by the server when the client attempts to claim their player is in a location that should be impossible for the player to reach in one turn. At the very least, the illegal message can be rejected by the server and never passed on to the other clients playing the game.

Battle.net took on just such an enforcer role for Diablo 2, which is why players were given the choice to play Battle.net-only characters, who could be considered trusted by other players to have never been hacked, or they could play “local” characters with whom Battle.net did not play a watchdog enforcer role, and thus the characters may or may not have been given unfair advantage through hacking the game. Even the Blizzard behemoth World of Warcraft, at its core, uses a similar model, though hundreds of clients are allowed to connect to a World of Warcraft server at once (actually several different servers make up a World of Warcraft game world, which is why players see a loading screen when they move from continent to continent in the game–the loading screen masks the hand-off of the character across different game servers).

Client talks to server. Gets one response. Hangs up. Client talks to server again. Gets one more response. Hangs up.

On the HTML side of the world, client/server communication was around long before it became clear that client/server networking was required for trustworthy multiplayer gaming. The basic form of communication between a web page and its web server has always been something called “HTTP”, which is designed around the core idea that the web browser will ask for a single resource–a web page, an image file, a video stream, etc–then hang up the connection after the resource is delivered. In the course of loading this blog post, your web browser issued several HTTP requests: one for the post itself, and one for each image embedded in the post, as well as some extra requests invisible to the reader.

As it turns out? For games, although you have the same client/server architecture the multiplayer game world migrated to, this is a terrible system. Although it already has a server in place ready to be used as the game enforcer to protect players from cheaters using hacked clients, there is a fundamental problem in the “hang up after every request” part of the HTTP design. For multiplayer games, you don’t want to hang up. Ever. You want to keep that line open so that you can be in a constant state of communication with the server: keeping it up to date with what your client is doing and getting back constant updates on what the other clients are doing. In HTTP-land, that means a whole lot of wasted bandwidth as clients constantly set up new connections to provide new updates to the server and check for updates from other players. It also means web servers have to set up and tear down a lot of connections that may not actually contain any data if nothing has happened in the game since the last time the client checked.

TL;DR: HTTP was never intended for this kind of communication.

Which leads us, at last, to WebSockets: a new protocol that allows web pages to open a connection to a server and keep that connection open, both for sending and receiving data, for long periods of time. In my next post, we will explore WebSockets in detail and see what options HTML5 developers have today to leverage this new technology.

Follow

Get every new post delivered to your Inbox.