Since diff was first published in 1974, file comparison tools have been a staple of any developers toolbelt. Especially when using git, I run diffs all the time. My default GUI was Apple’s FileMerge app (bundled with Apple Developer Tools). I say was because it’s been completely blown out of the water by Kaleidoscope 2.

Kaleidoscope by Black Pixel surpasses any other diff tool I’ve ever used. It’s stunningly beautiful and intuitive. Powerful too. It can compare text, folders and even images. It nails color highlighting, and gives you three different views for comparing files (Block, Fluid and it’s new “Unified” view).

The best part? It’s fully integrated with git (and SVN and Mercurial). That means that commands like git difftool or git mergetool can use Kaleidoscope to compare and merge files. After installing Kaleidoscope and it’s command line tool, ksdiff, you can run git difftool -t Kaleidoscope to compare files in the app. Alternatively, set it as the default difftool and mergetool by adding the following to your git config:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

[difftool "Kaleidoscope"]
  cmd = ksdiff --partial-changeset --relative-path \"$MERGED\" -- \"$LOCAL\" \"$REMOTE\"
[diff]
  tool = Kaleidoscope
[difftool]
  prompt = false

[mergetool "Kaleidoscope"]
  cmd = ksdiff --merge --output \"$MERGED\" --base \"$BASE\" -- \"$LOCAL\" --snapshot \"$REMOTE\" --snapshot
  trustExitCode = true
[mergetool]
  prompt = false
[merge]
  tool = Kaleidoscope

It’s a little pricey at $70, but it’s worth every penny. It’s even on sale 50% off until Jan 30!

Next time you see Automatic merge failed; fix conflicts and then commit the result you’ll be grateful to have Kaleidoscope in your toolbelt.

Do you like hitting tab to autocomplete long file name and commands? Do you use git? Then you need to install this right now! Git doesn’t autocomplete commands or branch names, but you can enable it with this script.

Grab it from here: https://raw.github.com/git/git/master/contrib/completion/git-completion.bash

Save it in ~/.git-completion.bash and add this line to your ~/.bash_profile: source ~/.git-completion.bash

You are now a git speed ninja. You’ll wonder what to do with all your extra time (and why git doesn’t include this be default).

Thanks to Mohammed Buttu and simont on StackOverflow.

I find myself typing source venv/bin/activate a lot when switching between Python projects. I added this function to my .bash_profile. It checks if there’s a directory named venv (or virtualenv or whatever you use). If there is, it activates the virtual environment automagically.

1
2
3
4
5
6

function cd {
    builtin cd "$@"
    if [ -d "venv" ] ; then
        source venv/bin/activate
    fi
}

As much as we live in an overly connected world, our data online lives in seperate silos. Everything is locked down behind logins and API-less websites. That’s why startups and web apps that connect these disconnected silos are so exciting to me. I’m a huge fan of IFTTT. Olark. Thread.is. Zapier. Rapportive. Filepicker.io. HelloFax. Boxcar. Card.io. EasyPost. Less Neglect. Instapaper. Timbre.

The common theme here is that these apps accomplish things that we feel we should be able to do. After all, our services and data are in the cloud and should be able to tap in to each other. Sometimes it reminds me of a misconfigured router in my networking class years ago: no matter how hard I try, I can’t ping the IP of the computer next to me.

This part of the internet is broken. It’s about time we start to fix it.

Any discussion about the architecture of a software project leads to arguments over the best way to achieve scalability. Essentially, it comes down to finding a happy medium between the time required to develop a scalable architecture and how scalable that architecture will be.

On one theoretical extreme, we could build a simple prototype of our software over the weekend, capable of supporting 100 users. Once we pass 100 users, we would have to restructure the architecture. On the other side, we could spend a year building rock solid software, capable of supporting 100 million users.

This is where the idea of Just-In-Time (JIT) scaling comes in. Once you’ve reached a certain amount of users on the first iteration, you can afford to spend more time developing the next iteration. Scaling can be done through iterations. Avoid premature optimization.

Facebook first ran on PHP and Twitter ran on Rails. Stackoverflow used C#. Although the choices of language or database are the most often questioned at the initial stage, these are not very important questions. Build with the stack that you know. Reaching the limits of the stack used in the first iteration is A Good Problem to have.

Even before you reach the limits of iteration one, you can start development on a more robust, scalable architecture. By always building ”just enough”, you can minimize development time while staying ahead of the scalability curve. Keep in mind that, no matter what scripting language the prototype was built in, once your software scales to hundreds of thousands of users, you will likely be running on heavily optimized low-level code.

Of course, there’s no need to completely rewrite your entire codebase every time your growth multiplies. By modularizing development of different components, and by using internal APIs and queues, you can scale distinct parts of the software independently. These design questions are the questions that are important to ask.

Managing cron can be a little intimidating to the uninitiated. How do you load jobs in to cron? How do you know what’s running? How can you check on each job’s status? It’s a pity that the first tutorial result on Google for cron was written in 1999. A lot of what it recommends is outdated. For the most part, you should never have to mess with /etc/crontab. This is the system crontab, don’t touch it unless you know what you’re doing.

Easiest way to load new jobs

Create a file called .cron in your home folder. Edit the list of cron jobs in this file, then load it in to cron with the crontab command:

1

$ crontab ~/.cron

Note: This clears all existing jobs. Back them up first with the command below.

Easiest way to see what jobs are loaded

If you want to check what jobs are already loaded in cron, or generate a .cron file for the first time, run:

1

$ crontab -l

Optionally output this to the .cron file:

1

$ crontab -l > ~/.cron

Easiest way to check a job’s status

With cron’s default settings on most machines, this isn’t an easy one. On Ubuntu, cron messages are logged to the /var/log/syslog file. There’s ways to set up logs for each cron job, but if you just want a simple way to make sure your nightly backups are running, check out Dead Man’s Snitch.

DMS is an awesome web app that shows the status of each of your cron jobs, and how long ago it last ran. It does cost $19/month for more than one job, but if you aren’t handy with the command line, it’s worth it to keep an eye on your cron jobs.

All you have to do is add a curl request to a unique URL after each cron job runs. Dead simple:

1

$ run_backup && curl https://nosnch.in/34ljkh3d

If you have any questions about Dead Man’s Snitch, send a tweet to @DeadMansSnitch. They are awesome. If you have questions about cron, cron jobs, or anything else, feel free to tweet me @nathancahill.

Cron is a powerful tool, and once you understand how it works, it’s easy to use. Now, go ahead and cron ALL OF THE THINGS!

After getting a new Macbook Pro with 16GB of RAM (G.Skill, not from Apple, highly recommended), I starting looking for ways to put it to use. One of my favorite hacks is mounting a virtual disk with a portion of the RAM. Once you create a RAM drive, you can take advantage of the theoretical 10GB/s read/write speed to RAM. Everything stored in the RAM is temporary and will disappear on unmount or reboot.

It’s pretty simple to do, just run this command in the Terminal:

1

$ diskutil erasevolume HFS+ "ramdisk" `hdiutil attach -nomount ram://4194304`

The number at the end of the command is the number of 512 byte sectors for the disk. For a 2GB disk: 2GB = 2147483648 bytes. 2147483648 / 512 = 4194304 disk sectors.

You’ll need to run the command every time you restart the computer. So drop it in an Automator app and add it to your Startup Items.

One great use for a RAM drive is to use it as Photoshop’s scratch disk. I played with the size a little and found the sweet spot to be around 4GB, which might be a little big if you have 8GB of RAM instead of 16GB.

Other cool things you can do with a very fast, but temporary disk:

• Store a working copy of large files for instant access
• Run portable apps entirely in the memory
• Get a small linux distro like Puppy Linux and running it from a VM. Entire operating system in memory.

If you’re running an SSD, you can get even more benefits out of storing stuff in memory. Set your log files to write to the SSD, along with anything else that writes to the drive frequently. This reduces the number of writes to the SSD and boosts the longevity of your drive.

Have another cool use for a RAM drive? Comment below or tweet me @nathancahill.

As we’re getting close to 1000 users on Hackers and Hustlers, I’d like to start a discussion about the future of H&H.

We used to have excellent discussions on here. A year ago and 600 people less, it was not unusual to see 40+ comment discussions, there was even talk of Facebook needing threading in comments. We were opinionated and expressive. Links posted tended to be relevant to the group’s goal: creating a startup culture in Michigan.

Lately, H&H has been swamped by all kinds of link posts and very little discussion or response. The volume of posts has more than tripled, but discussion and participation has decreased inversely. And the Start Garden endorsement requests are starting to feel like Farmville spam.

I know that this is a well documented phenomena in online communities. Digg, Youtube, Slashdot, Reddit and Hacker News have all followed the same pattern. What starts as a curated or exclusive ecosystem (and quickly gains popularity because of that), is inevitably degrade to chatter and spam.

Nathan Bashaw and I talked several times at the beginning of this year about the direction H&H was heading. We had just hit 600 members and it was the only reason that I ever logged in to Facebook. We talked about the possibility of H&H growing too big for itself, but we hoped that quality discussions and links would float to the top. Instead, they seems to get buried too quickly.

What’s next? Where do we go from here? We are still a solid community of michiganders, hacking and hustling our way through the startup life. Props to Bashaw for adding Kyle Mulka, Ryan Goins and Nate West as admins. You’ve all been doing a great job keeping everything running smoothly. And I’m excited for the H&H stickers too, they are awesome. If you haven’t yet, go check them out here.

So here’s what I suggest: Let’s participate more in discussions and let’s post stuff that’s more relevant to the group. Let’s see the projects you guys are working on, instead of the latest trend on HN! Don’t just ask for an endorsement on Start Garden, post a Show H&H of your prototype or even just your landing page. Give feedback, mentor, and encourage others. And for gods sake, post jobs to hackersandhustlers.org. Thoughtful discussion online (gasp) is not dead!

Looking for a high performance, powerfull process manager for a Python project I’m working on, I stumbled on Circus on this excellent benchmark blog post. After running my own benchmark tests, I agree that the Circus + Chaussette + Meinheld stack is the way to go. High concurrency, fast response time, and socket support are the features that pulled me in. I’m switching off of Supervisord, because: a) Circus integrates directly with ZeroMQ, and b) Gunicorn + Supervisord requires two levels of process management: Supervisor controls gunicorn, and gunicorn in turn watching it’s own worker processes. Circus keeps everything on the same level.

circus.readthedocs.org has excellent documentation for getting started with Circus. I did run in to a couple caveats though:

Circus doesn’t start on boot

I started testing Circus in the command line by running circusd circus.ini. I quickly switched to running it as a service in Upstart, using this etc/init/circus.conf file:

1
2

start on filesystem and net-device-up IFACE=lo
exec /usr/local/bin/circusd /etc/circus.ini

This script just waits for the file system and networking to become available, then it runs circusd with my config file in /etc/circus.ini. Easy enough.

Getting Circus workers to work with virtual environments

Although Circus workers have awesome properties like env, copy_env and copy_path (which all work great when running from a local folder), this falls apart when starting the daemon from Upstart. I looked at my $PATH variable in an active environment, and copied it into the worker config:

1

env = PATH=/path/to/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,VIRTUAL_ENV=/path/to/venv

Circus Web Console behind Nginx with Sockets

Circus has a sweet web console to manage processes and workers. By default, it runs on port 8080, and uses websockets to push stats on CPU, memory and socket reads for each running process. The web console should never be publicly available, it allows arbitrary commands to be executed on the server. The preferred way to password protect the console is to use Nginx like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

server {
    listen  8001;
  location ~/media/*(.jpg|.css|.js)$ {
      alias /usr/local/lib/python2.7/dist-packages/circus/web/;
  }

  location / {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_redirect off;
      proxy_pass http://127.0.0.1:8080;
      auth_basic            "Restricted";
      auth_basic_user_file  /etc/nginx/htpasswd;
  }
}

I have the web console process listening on 8080: it’s serving both the website and the socket connection. Notice the issue? Nginx doesn’t support websockets! So I’m running Nginx on port 8001, and the web console processs on 8080. And this is where Varnish comes in: Varnish is a caching proxy, but I’ll just use it to multiplex port 8002 to two seperate backends. If the connection is a websocket, route it directly to 8080. If it’s the website, switch the backend to Nginx on port 8001:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

backend default {
    .host = "127.0.0.1";
    .port = "8001";
}

backend socket {
    .host = "127.0.0.1";
    .port = "8080";
    .connect_timeout = 1s;
    .first_byte_timeout = 2s;
    .between_bytes_timeout = 60s;
}

sub vcl_pipe {
     if (req.http.upgrade) {
         set bereq.http.upgrade = req.http.upgrade;
     }
}

sub vcl_recv {
    if (req.http.Upgrade ~ "(?i)websocket") {
        set req.backend = socket;
      return (pipe);
    }
}

A couple days ago, if you asked me how to add webfonts to my website, I would have immediately said Font Squirrel. One of the reasons that Font Squirrel is awesome is their @font-face generator. It is the most popular solution for generating webfonts and font CSS online.

That’s all about to change. See, there are several limitations to using an online generator for fonts. After uploading two or three fonts, it’s obvious that Font Squirrel would be not be able to handle hundreds of font files. A web browser simply isn’t built to support that. Previewing generated fonts in the browser on the fly is also impossible. There’s also the licensing to worry about: Font Squirrel only accepts fonts that are “legally eligible for web embedding.”

FontPrep solves all of this. FontPrep is a native Mac OS X app that prepares your fonts for the web. At its core, it generates webfont bundles with full browser compatibility. After dragging and dropping a TTF font on to the app, the whole bundle can be downloaded, including WOFF, SVG and EOT formats, and the snippets of CSS needed to add these fonts to a website. And that’s just scraping the surface of what the app can do:

FontPrep can spin up a local server running a font testbed in the browser. After adding fonts to FontPrep, the entire webfont can be previewed in the browser. Every glyph in the typeface is rendered, as well as a waterfall of font sizes and a text area for testing the font with custom text. Font size is easily changed as well.

According to the creators, Brian and Matthew Gonzalez, FontPrep was created with the goal of making an “easy to use, drag & drop OSX app which makes our lives as developers much easier when it comes to working with fonts.” They succeeded: not only is FontPrep highly functional, it is also beautiful. Admittedly, their app have less options than Expert mode on Font Squirrel, but they are working on adding more support. Meanwhile, head over to FontPrep.com and pick up the app. It’s free for the next 6 hours, after that it costs $5.