(CSS is for programming, not for pasting)

When the Closure Tools were first released a little over two years ago, they gave web developers the ability to organize and optimize their JavaScript and HTML in a new way. But there was something missing, namely, a tool to help manage CSS.

You see, the nature of CSS runs contrary to the DRY principle that is exhibited in good software engineering. For example, if there is a color that should be used for multiple classes in a stylesheet, a developer has no choice but to copy-and-paste it everywhere because CSS has no concept of variables. Similarly, if there is a value in a stylesheet that is derived from other values, there is no way to express that because CSS also lacks functions. Common patterns of style blocks are duplicated over and over because CSS has no macros. All of these properties of CSS conspire to make stylesheets extremely difficult to maintain.

To this end, we are excited to introduce the missing piece in the Closure Tools suite: Closure Stylesheets. Closure Stylesheets is an an extension to CSS that adds variables, functions, conditionals, and mixins to standard CSS. The tool also supports minification, linting, RTL flipping, and CSS class renaming. As the existing Closure Tools have done for JavaScript and HTML, Closure Stylesheets will help you write CSS in a maintainable way, while also empowering you to deliver optimized code to your users. We hope you enjoy it! Please let us know what you think in the discussion forum.

By Michael Bolin, Open Source Engineer

Cross posted from the Google Open Source Blog.

This is the last in a series of blog posts on how Closure Compiler decides what properties. Part 1 was about the --compilation_level=ADVANCED_OPTIMIZATIONS flag and part 2 was about renaming policies that didn't work. This blog post will be a bit of a grab bag of newer tools for better property renaming.

Using Type Checking to Fill in Missing Externs

Your externs file should declare all the properties that are defined outside of your program. If you haven't done this before, then finding all those properties can be a pain. Fortunately, Closure Compiler can point us in the right direction by checking for missing properties.

There are a few ways to turn on the missing properties check. For example, you can specify the flags:
--warning_level=VERBOSE --jscomp_warning=missingProperties
or
--jscomp_warning=checkTypes
The compiler will try to find places where you've used dot access (foo.bar) to read a property that can't possibly be defined on that object, perhaps because you've forgotten to declare it in your externs.

Notice that this check is subtly different than compiler checks in more static languages, like Java. The Java compiler uses a "must-define" approach: the property bar must be declared on all possible values of foo (except null), or it will be a compiler error. Closure Compiler uses a "may-define" approach. It only requires that some possible value of foo has a property bar. For example, if you have:

function f(x) {
  return x.apartment;
}

Closure Compiler will emit a warning if the apartment property is not assigned anywhere in the program. Similarly, if you have:

/** @param {Element} x */
function f(x) {
  return x.apartment;
}

the compiler will emit a warning if the apartment property is not assigned on any object that could possibly be an Element. It will not emit a warning if apartment is assigned on some specific subtype of Element, like HTMLTextAreaElement.

As you can see, you don't need complete type annotations to use this check, but more type annotations will get better results.

Using Type Analysis for Better Renaming

If a property is listed in the externs file under the "All Unquoted" naming policy, that property will not be renamed anywhere in the program. This is a bummer. We went through a lot of trouble to make the Closure Library events API consistent with the native browser events API. But because we gave the methods the same name as extern methods, those methods can't be renamed. Could we use type information to differentiate between method calls on user-defined objects from method calls on browser-defined objects?

We can. Closure Compiler's Java API has two options: disambiguateProperties and ambiguateProperties.

Disambiguation means that Closure Compiler will look at all accesses to a property
x in the program. If two types have a property xx// externs file
/** @constructor */ function Element() {}
Element.prototype.id;

// user code
/** @constructor */
function MyWidget() {
this.id = 3;
}

Disambiguate properties will rename this to something like:

/** @constructor */
function MyWidget() {
  this.MyWidget$id = 3;
}

By design, disambiguate properties gives things very verbose names, on the assumption that they will be given shorter names by the "All Unquoted" naming policy. This makes it easier to debug, because you can turn off this optimization independently of "all unquoted" property renaming.

Disambiguate properties allows us to rename some properties that are in the externs file, but it creates a new problem: there are more unique properties, which makes the gzipped code bigger. To solve this problem, we use ambiguateProperties to minify the number of unique properties. Ambiguate properties will look at two property names on different objects such that there's no chance those objects will appear in the same variable. Then it will give those properties the same name.

Disambiguate and ambiguate properties are very conservative. They will only rename things if they are reasonably sure that it's safe to do so. (They can never be absolutely sure, because you could always pass in external objects that violate the declared types.) These optimizations only make sense when used in conjunction with "All Unquoted" renaming.

Short Names Aren't Necessarily Better

So far in this series, we've been assuming that short names make your binary smaller, and will always be better than long names.

That makes sense if you're sending your entire JS file across the network. But what if your visitors had old versions of your JS in their cache? You might want to figure out what version they have, and send them only the parts that had changed.

This is called delta encoding. If the compiler is choosing the shortest possible names for your properties, then small changes to your JS may change every compiled property in the output. The delta between two versions may be as large as the original files.

What we really want is a way to tell the compiler, "give these properties the shortest possible names, unless you've seen them before, and in that case give them the same name you gave them last time." The compiler has 4 flags for this:

--variable_map_input_file
--variable_map_output_file
--property_map_input_file
--property_map_output_file

The output maps from one compilation can be used as the input map for the next compilation. Although the compiler cannot guarantee that a property will be renamed the same way in both binaries, it will make a best-effort attempt to do so.

That covers most of our major property renaming policies. If you have ideas for more, let us know at the Closure Compiler discussion group.

Posted by Nick Santos, Software Engineer

This is the second in a series of blog posts on how Closure Compiler decides what properties to rename with the --compilation_level=ADVANCED_OPTIMIZATIONS flag. Part 1 talks about the current algorithm. This blog post will focus on property renaming policies that we tried that didn't work so well.

In the beginning, we tried to use coding conventions to decide when to rename things. Uppercase property names (foo.MyMethod) were renamed, and lowercase property names were not renamed. This didn't work well. One man's internal code is another man's external library. Sometimes you really didn't want uppercase property names to be renamed. Changing your code to use this convention meant breaking your API.

Later, we tried to move towards "smarter" algorithms, ones that did not require the programmer to be aware of renaming policies. These were called the "heuristic" property renaming policies. These algorithms looked at the entire input to the compiler, and tried to find all writes and reads to a specific property. If it saw at least one write to the property, and was reasonably sure that all the reads of that property came from those writes, then it renamed them all.

In small apps, heuristic renaming policies worked well. They were not very powerful, but they were easy to migrate to. Even when you didn't declare all the properties on external objects in the externs file, you'd usually still be ok. There would be no property writes to that property name, so the compiler wouldn't try to rename it.

But for medium to large apps, these advantages were a curse. Consider the following code:

/** @param {Object} json Some external JSON.
function f(json) {
return json.estate;
}
window['__receive_json'] = f;

// ...

// in some other code base
Foo.prototype.estate = 3;
f(new Foo());

If these were the only two appearances of the property estate in your binary, the compiler would rename it. The compiler can't tell that you're calling f from external code, and that you expect estate to be preserved.

You could have this piece of code that worked for years and years. Then, somebody who you never met could add Foo.prototype.estate in a different part of the codebase. It would break your code for no obvious reason, and the breaking change would be difficult to track down. When we have common JavaScript libraries, this becomes orders of magnitude more problematic. Adding Foo.prototype.estate could break any of the 25 products that depend on your library in subtle and difficult-to-debug ways.

Even if you did find the problem, how would you work around it? If this is shared code, then changing json.estate to something like json['estate'] might break other projects that depend on it, because their binaries do expect estate to get renamed.

Because of these problems, most projects that use Closure Compiler do not use heuristic renaming algorithms. But heuristic renaming wasn't a total failure. We learned some useful lessons:

• If the compiler looks at your whole program to determine whether a property should be renamed, then that means a change in one part of the program can change property renaming in an unrelated part of the program.
  
• If your code is shared across projects, then you probably want the property to be renamed in all projects or none of them.
  
• When renaming properties, it's better to be transparent and 90% accurate than to be cryptic and 99% accurate.
  

Could we use these lessons to develop a better renaming algorithm? We'll talk about this more in Part 3.

Posted by Nick Santos, Software Engineer

When you use Closure Compiler's --compilation_level=ADVANCED_OPTIMIZATIONS flag, the compiler will try to rename properties on your objects. For example, it may rename x.longPropertyName to x.a.

Because property renaming is a complex topic, we're going to split this discussion up into three blog posts. Part 1 is about the property renaming that you get with ADVANCED_OPTIMIZATIONS. Part 2 will be about other property renaming algorithms we've tried that didn't work so well. Part 3 will be about property renaming algorithms that we're currently experimenting on and are available from the Java API.

If you're using Closure Compiler's Java API, you have more fine-grained control over what renaming the compiler does. The API treats variable renaming (foo.bar -> a.bar) and property renaming (foo.bar -> foo.a) as completely independent optimizations. You can choose a variable renaming policy and a property renaming policy. The best property renaming policy, "All Unquoted," is what you get when you use ADVANCED_OPTIMIZATIONS. Most large Google projects use it. It significantly changes how we write JavaScript.

In the general case, a compiler can't rename properties at compile-time. You simply don't have enough information to try. There will always be objects that come from external sources that the compiler can't see (like JSON responses from the server), and property names that are undecidable. (Consider the expression foo[undecidableFunction()] = function(){};.) So property renaming can never be perfect. There will always be rules and gotchas.

Before we talk about the best property renamer, "All Unquoted," we have to define what we mean by "best." Usually, we use three criteria.

1. Power: How much smaller does it make your code?
   
   
2. Failure Cases: If it renames properties incorrectly, how easy is it to figure out what went wrong? How easy is it to fix the issue? If you make a change, how confident can you be that it won't break renaming?
   
   
3. Migration: How easy is it to update a legacy codebase so that it can take advantage of property renaming?
   

"All Unquoted" renaming was designed to optimize for #2: making the failure cases easy to debug and correct. By design, the algorithm is transparent and simple.

1. If a property is in the externs file, don't rename it.
   
   
2. If the property appears in quotes, don't rename it.
   
   
3. Otherwise, rename it.
   

For example, if you write the code:

var obj = {
alice: true,
'bob': true
};
obj.claire = true;
obj.document = true;
window['obj'] = obj;

then you will get something that looks like this:

var a = {
a: true, // alice was not in quotes or in the externs file
bob: true // bob was in quotes
};
a.b = true; // claire was not in quotes or in the externs file
a.document = true; // document was in the externs file
window.obj = a; // obj was in quotes

If you use the --debug flag, the same properties still get renamed, but now it will be much easier to see what the original names were:

var $obj$$ = {
$alice$: true,
'bob': true
};
$obj$$.$claire$ = true;
$obj$$.document = true;
window['obj'] = $obj$$;

This has some nice features that help debugging and development. It's straightforward for the average programmer to look at the compiled output and figure out why the compiler is renaming something, and what the name should be. If we decide that we don't want the property "claire" to be renamed, then we can change obj.claire to obj['claire'].

Furthermore, this convention makes it easier to read and refactor code written by a large team of JavaScript developers. If Bob is using "All Unquoted," and he has a method defined as Foo.prototype.methodA = fn, then Alice can easily search for all calls to methodA in the codebase. She doesn't have to worry about opaque accesses to the method, like obj['method' + 'A'], because she knows that the compiler will break those accesses anyway.

Another way to look at it is that we use the dotted access (obj.alice) for compile-time property lookups, and array access (obj['alice']) for run-time property lookups.

But there are some big downsides to "All Unquoted." As you saw above, any property in the externs file cannot be renamed anywhere in the program. When your methods have the same names as native browser methods, they won't be renamed. It's not as powerful as it could be.

Even worse, it is difficult to safely convert a legacy codebase to use this renaming policy. Because you never had to declare your "external" properties, you probably didn't, and it's a bear to find them all.

Could we do better? Maybe there's additional information we could leverage to get fewer false positives and false negatives? Perhaps we could look at the type annotations, or at other property assignments in the program? We'll get to those questions in the next two posts.

Posted by Nick Santos, Software Engineer

In a past blog post, we talked about how Closure Compiler makes it easier to share common JavaScript by ensuring that everybody declares what variables they own.

Global variable uniqueness is something that we added checks for fairly early on. Some restrictions are less obvious. In this blog post we’ll talk about the this keyword.

Suppose Nathan wants to outsource hot dog eating contests, so that we don’t need an instance of Nathan to run one. To do so, he takes runHotDogContest and unpackHotDogs off the prototype, and makes them into static methods. The code now looks like this:

Nathan.runHotDogContest = function(player, boxes) {
 for (var i = 0; i < boxes.length; i++) {
   player.eatHotDogs(this.unpackHotDogs(boxes[i]));
 }
};

I want to run 2 hot dog contests, so naturally, I write:

var run = Nathan.runHotDogContest;
run(joey, crateA);
run(kobayashi, crateB);

The code fails with an error that unpackHotDogs is not defined on this. The reason is subtle. To the person writing Nathan.runHotDogContest, this and Nathan seem like interchangeable variables. And as long as you’re testing them by invoking them as:

Nathan.runHotDogContest(joey, crateA);
Nathan.runHotDogContest(kobayashi, crateB);

then they will be interchangeable.

In fact, this is a secret argument to the function. It’s much like any other function argument, except that it appears on the left side of the function call instead of on the right side. So Nathan.runHotDogContest(...) passes Nathan as the this argument, and run(...) passes null as the this argument (which JavaScript helpfully coerces to window).

To make Closure Library usable, the library authors and the library users need to agree on what the arguments to a function mean. This includes secret arguments like this. So we’ve agreed that only constructors and prototype methods may use this, and the this argument must be an instanceof the relevant constructor. Closure Compiler enforces this in --warning_level VERBOSE, and will often emit a warning if it sees this used in a dangerous place.

If you’re writing a function that expects an unusual this context, you can explicitly document it with the @this annotation, and the warning will go away.

Good luck with all of this!

Posted by Nick Santos, Software Engineer

When collaborating with lots of engineers as we do here at Google, it’s important to keep a consistent coding style. To this end, we recently open sourced the Google JavaScript Style Guide.

Today we’re happy to announce we’re open sourcing a tool that will help you follow that style guide with minimal manual effort - the Closure Linter.

Take for example, this code:

var x = 10
var y=20;

for(var i = 0;i < 10; i++ ) {
  x += i;
   y -= i;
}

var z = [10, 20,];

x = y + z[0]
    + 10;

When we run gjslint --strict fixme.js we get

Line 1, E:0010: (New error) Missing semicolon at end of line
Line 2, E:0002: Missing space before "="
Line 2, E:0002: Missing space after "="
Line 4, E:0002: Missing space before "("
Line 4, E:0002: Missing space after ";" in for statement
Line 4, E:0001: Extra space before ")"
Line 6, E:0006: (New error) Wrong indentation: expected any of {2} but got 3
Line 9, E:0121: Illegal comma at end of array literal
Line 12, E:0120: Binary operator should go on previous line "+"
Found 9 errors, including 2 new errors, in 1 files (0 files OK).

Even better, if we run fixjsstyle --strict fixme.js, 7 of the 9 errors are automatically fixed for us!

We hope you can use the Closure Linter to improve your style consistency while simultaneously saving time. More information on how to get started can be found on the How to Use Closure Linter page.

We welcome comments and questions in the discussion forum.

Posted by Robby Walker and Andy Perelson, Software Engineers

The Closure Compiler you know is very different than the Closure Compiler that’s familiar to Google engineers. The defaults at Google are much stricter. It forbids duplicate global variable declarations, performs type-checking, and restricts a number of other patterns that are common in non-Closure-style JavaScript. By default, the “Google build” of Closure Compiler is like running the "open source build" of Closure Compiler with the --warning_level VERBOSE flag.

Why? Almost every contributor to Closure Tools works on web applications. We want to be able to share code. Part of that means we should be able to fix the shared code, and integrate it into all our apps quickly so that, for example, fixes for IE9 will make it out to production within a reasonable time. To integrate those changes safely, we need to be confident that they won’t break anything. Compiler restrictions help us do that. Sometimes those restrictions are obvious...sometimes less so.

Suppose Joey writes some library code:

Joey.prototype.eatHotDogs = function(hotDogs) {
  for (i = 0; i < hotDogs.length; i++) {
    this.eat(hotDogs[i]);
  }
};

Next summer, Nathan writes an application to feed Joey HotDogs by the boxful:

Nathan.prototype.runHotDogContest = function(player, boxes) {
  for (i = 0; i < boxes.length; i++) {
    player.eatHotDogs(this.unpackHotDogs(boxes[i]));
  }
};

Surprisingly, when Nathan runs his code in a browser, it tries to feed Joey an infinite number of hot dogs and crashes the browser.

The problem, of course, is that JavaScript "helpfully" declares variables for you if you forgot. No one declared i, so a variable i got magicked into the global scope. When Joey originally wrote his code, he tested it and it worked fine. But when Nathan tried to use it, he found that eatHotDogs() was "resetting" the i in his loop every time he called it.

In this example, Nathan was directly calling the problematic method, and the bug was relatively easy to spot. In the general case, these bugs are "non-localized" —missing var in one file may trigger a bug in an unrelated file that hasn’t been touched in years. Or it may cause a bug years later in unrelated new code. For a library shared across many codebases, non-localized bugs can stop progress quickly because it’s so hard to know where to start looking.

In VERBOSE mode, Closure Compiler enforces that all variables must be declared with the var or function or catch keywords. Global variables may only be declared once. The Google JS style guide has naming conventions to make these collisions even less likely.

This rule has some corollaries that have a big impact on how we write JavaScript. For one, there must be a way to declare variables that are "owned" by the surrounding page, and not by the code under compilation. So Closure Compiler has externs files, which allow you to declare external variables. Conversely, external variables may not conflict with your code. If you use the default externs, you will not be able to name a variable Node because it will conflict with DOM Nodes.

Try running Closure Compiler with --warning_level=VERBOSE mode and see what you find in your code!

Posted by Nick Santos, Software Engineer