Jetpack, HTML5 pushState and unsafeWindow

I wrote a cross-browser extension to replace the ads in the sidebar of Facebook with content of my own. Facebook uses entirely client-side navigation: new data for the next view is loaded via Ajax, pushState is fired, and the next view is rendered.

Unfortunately, Firefox Jetpack is still lagging behind the support that Chrome extensions have for navigation events. In Chrome, it was simple to set up a listener for URL changes:

var match = ""; // the url to match
var callback = function(tabId, changeInfo, tab) {
    if (changeInfo.url.indexOf(match) != -1 || changeInfo.url == match) {
        chrome.tabs.executeScript(null, {file:"lib/jquery-1.8.3.min.js"}, function() {
            chrome.tabs.executeScript(null, {file:"scripts/bgs.js"}); // the script to execute
        });
    }
};

chrome.tabs.onUpdated.addListener(callback);

Jetpack was a little more involved. The closest listener available is called pageshow, but it’s a far cry from Chrome’s onUpdate. Ultimately, the best choice ended up being to extend the window’s history object in JavaScript to add a listener for pushState events.

var match = ""; // the url to match
var pushState = history.pushState;
unsafeWindow.history.pushState = function(state) {
    if (typeof history.onpushstate == "function") {
        if (state.indexOf(match) != -1 || state == match) {
            history.onpushstate({state: state});
        }
    }
    return pushState.apply(history, arguments);
}

unsafeWindow.history.onpushstate = function(state) {callback();};

Similar to the onpopstate event, this method implements onpushstate by monkey patching the native history object. While using the unsafeWindow object is frowned upon (and rightly so), I’m limiting my exposure by restricting the plugin to facebook.com, and chances are, they won’t change their codebase to exploit my extension.

I did find that I needed to add a small delay with setTimeout before I loaded my content, after pushState event was handled. This allowed the DOM time to process the changes for the new view.

Installing Homebrew on Mac OSX 10.9 Mavericks

Note: This is a hack. Read the comments, specifically Daniel Dunbar’s, for a better way to do this. Do not use this method anymore.

After installing the developer preview of Mac OSX Mavericks, I started installing my usual list of apps, tools and tweaks. First on the list is always Homebrew, my favorite command line tool. It took a little finessing to get it running, but it works fine, despite the warnings about 10.9 being unsupported.

Before trying to install any packages, I installed the developer preview of Xcode 5 to get the command line tools for compiling. Without the command line tools, you get errors like this trying to install git: git-credential-osxkeychain.c:131: error: ‘protocol’ undeclared (first use in this function)

Unfortunately, it seems that Xcode 5 is doing something different with the command line tools. They are not available for download as they were in Xcode 4.

I also tried using Kenneth Reitz’s wonderful osx-gcc-installer, but Homebrew complained about mismatched compiler versions when using the Mountain Lion installer.

The trick is to download the command line tools from developer.apple.com and spoof the OS version as 10.8 so that it can be installed. Simply change two lines in /System/Library/CoreServices/SystemVersion.plist:

1
2
3
4
<key>ProductUserVisibleVersion</key>
<string>10.8</string>
<key>ProductVersion</key>
<string>10.8</string>

You may have to reboot after changing the version. Be sure to change it back to 10.9 after installing the command line tools, otherwise things get quirky.

Kaleidoscope: The Most Powerful Diff Tool for Mac OS X

Since diff was first published in 1974, file comparison tools have been a staple of any developers toolbelt. Especially when using git, I run diffs all the time. My default GUI was Apple’s FileMerge app (bundled with Apple Developer Tools). I say was because it’s been completely blown out of the water by Kaleidoscope 2.

Kaleidoscope by Black Pixel surpasses any other diff tool I’ve ever used. It’s stunningly beautiful and intuitive. Powerful too. It can compare text, folders and even images. It nails color highlighting, and gives you three different views for comparing files (Block, Fluid and it’s new “Unified” view).

The best part? It’s fully integrated with git (and SVN and Mercurial). That means that commands like git difftool or git mergetool can use Kaleidoscope to compare and merge files. After installing Kaleidoscope and it’s command line tool, ksdiff, you can run git difftool -t Kaleidoscope to compare files in the app. Alternatively, set it as the default difftool and mergetool by adding the following to your git config:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[difftool "Kaleidoscope"]
  cmd = ksdiff --partial-changeset --relative-path \"$MERGED\" -- \"$LOCAL\" \"$REMOTE\"
[diff]
  tool = Kaleidoscope
[difftool]
  prompt = false

[mergetool "Kaleidoscope"]
  cmd = ksdiff --merge --output \"$MERGED\" --base \"$BASE\" -- \"$LOCAL\" --snapshot \"$REMOTE\" --snapshot
  trustExitCode = true
[mergetool]
  prompt = false
[merge]
  tool = Kaleidoscope

It’s a little pricey at $70, but it’s worth every penny. It’s even on sale 50% off until Jan 30!

Next time you see Automatic merge failed; fix conflicts and then commit the result you’ll be grateful to have Kaleidoscope in your toolbelt.

Autocompletion in Git

Do you like hitting tab to autocomplete long file name and commands? Do you use git? Then you need to install this right now! Git doesn’t autocomplete commands or branch names, but you can enable it with this script.

Grab it from here: https://raw.github.com/git/git/master/contrib/completion/git-completion.bash

Save it in ~/.git-completion.bash and add this line to your ~/.bash_profile: source ~/.git-completion.bash

You are now a git speed ninja. You’ll wonder what to do with all your extra time (and why git doesn’t include this be default).

Thanks to Mohammed Buttu and simont on StackOverflow.

Automatically Activate Virtualenv

I find myself typing source venv/bin/activate a lot when switching between Python projects. I added this function to my .bash_profile. It checks if there’s a directory named venv (or virtualenv or whatever you use). If there is, it activates the virtual environment automagically.

1
2
3
4
5
6
function cd {
    builtin cd "$@"
    if [ -d "venv" ] ; then
        source venv/bin/activate
    fi
}

Connecting Things That Are Inherently Disconnected

As much as we live in an overly connected world, our data online lives in seperate silos. Everything is locked down behind logins and API-less websites. That’s why startups and web apps that connect these disconnected silos are so exciting to me. I’m a huge fan of IFTTT. Olark. Thread.is. Zapier. Rapportive. Filepicker.io. HelloFax. Boxcar. Card.io. EasyPost. Less Neglect. Instapaper. Timbre.

The common theme here is that these apps accomplish things that we feel we should be able to do. After all, our services and data are in the cloud and should be able to tap in to each other. Sometimes it reminds me of a misconfigured router in my networking class years ago: no matter how hard I try, I can’t ping the IP of the computer next to me.

This part of the internet is broken. It’s about time we start to fix it.

Just-In-Time Architecture Scaling

Any discussion about the architecture of a software project leads to arguments over the best way to achieve scalability. Essentially, it comes down to finding a happy medium between the time required to develop a scalable architecture and how scalable that architecture will be.

On one theoretical extreme, we could build a simple prototype of our software over the weekend, capable of supporting 100 users. Once we pass 100 users, we would have to restructure the architecture. On the other side, we could spend a year building rock solid software, capable of supporting 100 million users.

This is where the idea of Just-In-Time (JIT) scaling comes in. Once you’ve reached a certain amount of users on the first iteration, you can afford to spend more time developing the next iteration. Scaling can be done through iterations. Avoid premature optimization.

Facebook first ran on PHP and Twitter ran on Rails. Stackoverflow used C#. Although the choices of language or database are the most often questioned at the initial stage, these are not very important questions. Build with the stack that you know. Reaching the limits of the stack used in the first iteration is A Good Problem to have.

Even before you reach the limits of iteration one, you can start development on a more robust, scalable architecture. By always building ”just enough”, you can minimize development time while staying ahead of the scalability curve. Keep in mind that, no matter what scripting language the prototype was built in, once your software scales to hundreds of thousands of users, you will likely be running on heavily optimized low-level code.

Of course, there’s no need to completely rewrite your entire codebase every time your growth multiplies. By modularizing development of different components, and by using internal APIs and queues, you can scale distinct parts of the software independently. These design questions are the questions that are important to ask.

Pro Tip: Manage Cron Like a Boss

Managing cron can be a little intimidating to the uninitiated. How do you load jobs in to cron? How do you know what’s running? How can you check on each job’s status? It’s a pity that the first tutorial result on Google for cron was written in 1999. A lot of what it recommends is outdated. For the most part, you should never have to mess with /etc/crontab. This is the system crontab, don’t touch it unless you know what you’re doing.

Easiest way to load new jobs

Create a file called .cron in your home folder. Edit the list of cron jobs in this file, then load it in to cron with the crontab command:

1
$ crontab ~/.cron

Note: This clears all existing jobs. Back them up first with the command below.

Easiest way to see what jobs are loaded

If you want to check what jobs are already loaded in cron, or generate a .cron file for the first time, run:

1
$ crontab -l

Optionally output this to the .cron file:

1
$ crontab -l > ~/.cron

Easiest way to check a job’s status

With cron’s default settings on most machines, this isn’t an easy one. On Ubuntu, cron messages are logged to the /var/log/syslog file. There’s ways to set up logs for each cron job, but if you just want a simple way to make sure your nightly backups are running, check out Dead Man’s Snitch.

DMS is an awesome web app that shows the status of each of your cron jobs, and how long ago it last ran. It does cost $19/month for more than one job, but if you aren’t handy with the command line, it’s worth it to keep an eye on your cron jobs.

All you have to do is add a curl request to a unique URL after each cron job runs. Dead simple:

1
$ run_backup && curl https://nosnch.in/34ljkh3d

If you have any questions about Dead Man’s Snitch, send a tweet to @DeadMansSnitch. They are awesome. If you have questions about cron, cron jobs, or anything else, feel free to tweet me @nathancahill.

Cron is a powerful tool, and once you understand how it works, it’s easy to use. Now, go ahead and cron ALL OF THE THINGS!

Using RAM Drives

After getting a new Macbook Pro with 16GB of RAM (G.Skill, not from Apple, highly recommended), I starting looking for ways to put it to use. One of my favorite hacks is mounting a virtual disk with a portion of the RAM. Once you create a RAM drive, you can take advantage of the theoretical 10GB/s read/write speed to RAM. Everything stored in the RAM is temporary and will disappear on unmount or reboot.

It’s pretty simple to do, just run this command in the Terminal:

1
$ diskutil erasevolume HFS+ "ramdisk" `hdiutil attach -nomount ram://4194304`

The number at the end of the command is the number of 512 byte sectors for the disk. For a 2GB disk: 2GB = 2147483648 bytes. 2147483648 / 512 = 4194304 disk sectors.

You’ll need to run the command every time you restart the computer. So drop it in an Automator app and add it to your Startup Items.

One great use for a RAM drive is to use it as Photoshop’s scratch disk. I played with the size a little and found the sweet spot to be around 4GB, which might be a little big if you have 8GB of RAM instead of 16GB.

Other cool things you can do with a very fast, but temporary disk:

  • Store a working copy of large files for instant access
  • Run portable apps entirely in the memory
  • Get a small linux distro like Puppy Linux and running it from a VM. Entire operating system in memory.

If you’re running an SSD, you can get even more benefits out of storing stuff in memory. Set your log files to write to the SSD, along with anything else that writes to the drive frequently. This reduces the number of writes to the SSD and boosts the longevity of your drive.

Have another cool use for a RAM drive? Comment below or tweet me @nathancahill.

Hackers and Hustlers: Where Next?

As we’re getting close to 1000 users on Hackers and Hustlers, I’d like to start a discussion about the future of H&H.

We used to have excellent discussions on here. A year ago and 600 people less, it was not unusual to see 40+ comment discussions, there was even talk of Facebook needing threading in comments. We were opinionated and expressive. Links posted tended to be relevant to the group’s goal: creating a startup culture in Michigan.

Lately, H&H has been swamped by all kinds of link posts and very little discussion or response. The volume of posts has more than tripled, but discussion and participation has decreased inversely. And the Start Garden endorsement requests are starting to feel like Farmville spam.

I know that this is a well documented phenomena in online communities. Digg, Youtube, Slashdot, Reddit and Hacker News have all followed the same pattern. What starts as a curated or exclusive ecosystem (and quickly gains popularity because of that), is inevitably degrade to chatter and spam.

Nathan Bashaw and I talked several times at the beginning of this year about the direction H&H was heading. We had just hit 600 members and it was the only reason that I ever logged in to Facebook. We talked about the possibility of H&H growing too big for itself, but we hoped that quality discussions and links would float to the top. Instead, they seems to get buried too quickly.

What’s next? Where do we go from here? We are still a solid community of michiganders, hacking and hustling our way through the startup life. Props to Bashaw for adding Kyle Mulka, Ryan Goins and Nate West as admins. You’ve all been doing a great job keeping everything running smoothly. And I’m excited for the H&H stickers too, they are awesome. If you haven’t yet, go check them out here.

So here’s what I suggest: Let’s participate more in discussions and let’s post stuff that’s more relevant to the group. Let’s see the projects you guys are working on, instead of the latest trend on HN! Don’t just ask for an endorsement on Start Garden, post a Show H&H of your prototype or even just your landing page. Give feedback, mentor, and encourage others. And for gods sake, post jobs to hackersandhustlers.org. Thoughtful discussion online (gasp) is not dead!