The emperor’s new clothes were built with Node.js

There are plenty of people lambasting Node.js (see the infamous “Node.js is cancer”) but proponents tend to misunderstand the message and come up with irrelevant counterpoints. It’s made worse because there are two very different classes of people that use Node.js. The first kind of people are those who need highly concurrent servers that can handle many connections at once: HTTP proxies, Websocket chat servers, etc. The second are those who are so dependent on JavaScript that they need to use JS across their browser, server, database, and laundry machine.

I want to address one-by-one all of the strange and misguided arguments for Node.js in one place.

Update: Please keep sending in anything I missed! Moderation of comments will probably continue to lag behind, but I do read them and will continue fixing and tightening up this article as best as I can.

TL;DR: what’s clothing that doesn’t use threads and doesn’t block (anything)?

Node.js is fast!

This is actually too imprecise. Let’s break it down into two separate claims:

Continue reading The emperor’s new clothes were built with Node.js

OpenStreetMap provider CloudMade shuts its doors on small users

(Original email at bottom.)

CloudMade, a company selling mapping services (many based on OpenStreetMap data) that competed head-to-head with Google, let its users know that as of May 1st, they’ll stop serving anyone who’s not on an enterprise plan. This is rather sad, because they were one of the main alternatives for custom OpenStreetMap tiles.

Their map tiles definitely left something to be desired. The OSM data that they were using seems to have been last refreshed around the time Steve Coast left (maybe that’s a wee bit of an exaggeration) and the rendering was never very polished—ugly icons and labels getting cut off on tile boundaries. But for $25/1M tiles (with the first 500k free), could you really complain?

CloudMade even listed Steve Coast, founder of OpenStreetMap, as a co-founder. Steve Coast left in 2010, and it was hard to tell what the company was trying to become. Now, we see that they’re gunning for enterprise services, along the lines of Navteq and TomTom. Instead of dealing with small fries like us, they’re apparently focusing on bigger deals like providing data for hardware and consumer electronics.

Maybe they just got tired of my emails to support asking why this or that was broken or when they’d update their data. Now, we’re left with almost no options for custom hosted OSM tiles. MapBox is one popular choice, but their online map customizer is elementary compared to CloudMade’s (and CloudMade’s was not super advanced). MapBox also have stricter terms of how their map tiles can be used. No proxying/caching of MapBox tiles is allowed, for example, especially since they charge based on usage.

CloudMade helpfully gave some alternative providers for us small fries to switch to. Still, one less provider means more risk when using a hosted provider. For example, who are we going to turn to when MapQuest decides to shut off its routing services?

Here’s to hoping people will step up and fill the gap that CloudMade is leaving. Us little users who will only pay a couple hundred dollars per month will then have somewhere else to go.

This is what came through today:

Hi [username],

We want to let you know about some changes we’re making to the CloudMade APIs. As of May 1st we’re switching to an enterprise model that supports the medium to large sized users of the CloudMade APIs. As part of this transition we’ll stop serving Map Tile, Geocoding, Routing and Vector Stream Server requests coming from your API keys below as of May 1st, unless you take action.

Your active CloudMade API keys are: W,X,Y,Z

If you wish to continue using the CloudMade services after April 30th you’ll need to upgrade to an enterprise plan. Enterprise plans are available for customers with 10,000,000 or more transactions per month. The plans include dedicated hosting, custom SLAs, 24×7 support from a named customer support representative and custom data loading. You can find out more about upgrading and request more information on the Web Portals page.

If your monthly usage is less than 10,000,000 transactions, or you don’t wish to upgrade to an enterprise plan, you should take action to update the app or website that’s using the CloudMade API keys shown above to use an alternative provider. There are a number of alternative providers of Map Tiles, Geocoding and Routing services based on OpenStreetMap data, for example:

– Mapquest (Map Tiles, Routing, Geocoding)

– MapBox (Styled Map Tiles)

Thanks for using CloudMade’s APIs over the past months and years. If you don’t switch to an enterprise plan, we wish you a smooth transition to the new service provider you choose.

[…]

Disclaimer: Nothing written here represents my employer in any way. I am/was a mostly satisfied user of many OSM-based services out there, including MapBox, MapQuest, and CloudMade.

Good things happen when you subtract datetimes in MySQL

Of course, you know that “good things” and “MySQL” don’t go together. File this one under the category of “small ways in which MySQL is broken”.

Let’s fire up MySQL 5.1.72-2-log or 5.5.34-log.

mysql> create temporary table blah
    -> (alpha datetime, beta datetime);
Query OK, 0 rows affected (0.01 sec)

mysql> describe blah;
+-------+----------+------+-----+---------+-------+
| Field | Type     | Null | Key | Default | Extra |
+-------+----------+------+-----+---------+-------+
| alpha | datetime | YES  |     | NULL    |       |
| beta  | datetime | YES  |     | NULL    |       |
+-------+----------+------+-----+---------+-------+
2 rows in set (0.00 sec)

OK, so we have two datetimes in a table. Let’s try adding a row:

mysql> insert into blah (alpha, beta)
    -> VALUES ('2014-01-01 03:00:00', '2014-01-01 03:00:37'); 
Query OK, 1 row affected (0.00 sec)

What happens if we try subtracting two datetimes?

mysql> select alpha, beta, beta - alpha from blah;
+---------------------+---------------------+--------------+
| alpha               | beta                | beta - alpha |
+---------------------+---------------------+--------------+
| 2014-01-01 03:00:00 | 2014-01-01 03:00:37 |    37.000000 |
+---------------------+---------------------+--------------+
1 row in set (0.00 sec)

So we got the number of seconds between the two datetimes. Let’s try that again with two datetimes a minute apart.

mysql> insert into blah (alpha, beta)
    -> VALUES ('2014-01-01 03:00:00', '2014-01-01 03:01:00');
Query OK, 1 row affected (0.00 sec)

mysql> select alpha, beta, beta - alpha from blah;
+---------------------+---------------------+--------------+
| alpha               | beta                | beta - alpha |
+---------------------+---------------------+--------------+
| 2014-01-01 03:00:00 | 2014-01-01 03:00:37 |    37.000000 |
| 2014-01-01 03:00:00 | 2014-01-01 03:01:00 |   100.000000 |
+---------------------+---------------------+--------------+
2 rows in set (0.00 sec)

So, 100 seconds in a minute? Yikes. Obviously, this isn’t how you’re supposed to subtract datetimes in MySQL. But the great part is that it kind of works! You get a number back that correlates to the actual interval of time between the two, and if you’re measuring lots of small intervals, you might not notice that your data is between 100% and 167% of what it should be. Excellent puzzle to drive a junior dev crazy!

Wait, any reasonable database would have known that we were making a mistake, right?

mysql> show warnings;
Empty set (0.00 sec)

tuntuntun – Combine Multiple Internet Connections Into One

GitHub repo: https://github.com/erjiang/tuntuntun (proof of concept status)

I was trying to play Minecraft by tethering over a Sprint data connection but was having awful random latency and dropped packets. The Sprint hotspot seems to only allow a limited number of connections to utilize the bandwidth at a time – a download in Chrome would sometimes stall all other connections. This was a huge problem in Minecraft, as loading the world chunks would stall my movements, meaning that I could teleport somewhere and die to an enemy by the time the map finished loading.

I’ve been seeing the idea of channel bonding here and there for a while, and it always seems like a cool idea without any popular and free implementations. Most of the approaches, though, were restricted to assigning different connections to different network interfaces. Essentially, a connection to YouTube might go over one link, while a software download might go out another. This works OK if you’re trying to watch YouTube while downloading updates, and it works great for many-connection uses like BitTorrent but in this case, I wanted to create a single load-balanced connection. So, I created tuntuntun.

Somewhat like a VPN

tuntuntun diagram

This requires the help of an outside server to act as the proxy for all of the connections, because most current Internet protocols require connections to originate from one address. The idea is to establish a connection to the proxy using a custom protocol that allows data to be split between two links. Tuntuntun works by encapsulating IP traffic in UDP packets, and does this in userspace using tun interfaces. A tun interface is a virtual network interface that has a userspace program as the other end. This means that any packets sent to the tun are read by the userspace program, and anything that the userspace program writes to it becomes “real” packets in the kernel.

Continue reading tuntuntun – Combine Multiple Internet Connections Into One

Googling for bugs: hundreds of date-formatting mistakes

Here’s something really easy to screw up:

<?php
echo date("Y-m-d H:m:s");
?>

Spot the mistake? The “m” format code can only mean one thing, which is “months”, not “minutes”. Yet, when you’re writing code in a hurry, it’s so easy to quickly write this code and assume it works. After all, it raises no errors or warnings, and always generates valid datetime strings. It’s just wrong.

Googling for ‘site:github.com “Y-m-d H:m:s” php’ returns hundreds of examples of mistakes. I tried to fix a couple instances but realized that there’s so much abandoned code on Github that it would’ve been useless.

This is a prime example of something that static analysis could warn about: just look for “H:m:s” or “h:m:s” in the first argument to date. This mistake isn’t limited to PHP, of course, as this Java example shows.

PHP’s wonderful pseudo-functions

There is a pseudo-function in PHP called empty, that tells you whether a variable is “empty” for someone’s definition of empty. The empty string, zero, null, and undefined variables are all considered to be empty.

<?php
$x = null;
echo empty($x); // prints "1"
echo empty($nothere); // prints "1"

Well, how did they make it work for undefined variables? They made empty a language construct, not a function, so that its argument isn’t evaluated before the empty check happens. This leads to some great errors:

$x = 0;
echo empty($x); // prints "1"
echo empty(0); // PHP Parse error:  syntax error, unexpected T_LNUMBER in php shell code on line 1

Or better yet,

function y() { return 0; }
echo y(); // prints "0"
echo empty(y()); // PHP Fatal error:  Can't use function return value in write context in php shell code on line 1

PLT is basically magick.

Getting the Sparkfun EL Escudo to work

I’m really disappointed in Sparkfun. I recently purchased the EL Escudo shield for driving EL wire with a microcontroller, and woe be upon anybody who doesn’t do hours and hours of Internet research before plugging the thing in. There were 3 years of questions from poor customers with only a few answers, and the few answers that were there were often contradictory. Here are my conclusions from my Internet research, and the amazing Will Byrd helped me finally get it to work. Long story short, Sparkfun sets you up to fail and hasn’t fixed things in 3 years.


Continue reading Getting the Sparkfun EL Escudo to work

Ubuntu works great with the MSP430 Launchpad

I got my MSP430 TI Launchpad more than a year ago simply because of the price. It was (and still is) $4.30 (with free shipping!) for a development board, two MSP430 microcontrollers, external crystal, and USB cable. Unfortunately, Windows was the only supported OS at launch time with a couple proprietary bundled IDEs. It still is the only officially supported OS, but setting up Ubuntu 11.10 and newer for the msp430 has become trivial:

sudo apt-get install gcc-msp430 gdb-msp430 mspdebug

To test your Launchpad, try compiling “blink” from https://github.com/mrothe/ti-launchpad.

The build process goes something like:

msp430-gcc -Os -mmcu=msp430x2012 -o main.elf main.c
msp430-objcopy -O ihex main.elf main.hex # generate hex file
mspdebug rf2500 "prog main.hex" # download to the launchpad

Do mind your O, o, and 0.

Using the Chez Scheme debugger

Achtung! Chez Scheme and Petite Chez Scheme do not give the same debugging information. Petite Chez Scheme is often much less helpful than Chez Scheme, and is probably half the reason why nobody actually tries debugging programs in Chez Scheme.

Here’s a sample, buggy program:

(define myreverse
  (lambda (ls)
    (if (null? ls)
      '()
      (append (myreverse (cdr ls)) (cons (car ls) '())))))

(define mylist (cons 1 (cons 2  (cons 3 (cons 4 5)))))

(display (myreverse mylist))

Let’s try loading it in Chez Scheme.

$ scheme
Chez Scheme Version 8.0
Copyright (c) 1985-2010 Cadence Research Systems

> (load "example.ss")
Exception in car: 5 is not a pair
Type (debug) to enter the debugger.
> 

Obviously, (debug) invokes the debugger. Once we get in there, however, it’s fairly cryptic.

debug> ?
Type i  to inspect the raise continuation (if available)
     s  to display the condition
     c  to inspect the condition
     e  or eof to exit the debugger, retaining error continuation
     q  to exit the debugger, discarding error continuation
debug>

Usually, inspecting the continuation is the most helpful thing to do, because once we’re in there, we can get a stack trace to find out where the error occurred. The command to print out the stack frames is sf.

debug> i
#<continuation in myreverse>                                      : sf
  0: #<continuation in myreverse>
  1: #<continuation in myreverse>
  2: #<continuation in myreverse>
  3: #<continuation in myreverse>
  4: #<continuation in myreverse>
  5: #<system continuation in map>
  6: #<system continuation in compile>
  7: #<system continuation>
  8: #<system continuation>
  9: #<system continuation in dynamic-wind>
  10: #<system continuation in dynamic-wind>
  11: #<system continuation in $reset-protect>
  12: #<system continuation in new-cafe>
#<continuation in myreverse>                                      : 

Knowing the stack trace, we can move down the stack frames using down, which is important if the debugger drops you in an error handler under the actual source of the problem. This information is often enough to debug your program. From here, we know that the error occurred when we were five instances deep in myreverse, so the problem was with the end of the list. But if you wanted more specifics, you can type s for show:

#<continuation in myreverse>                                      : s
  continuation:          #<continuation in myreverse>
  procedure code:        (lambda (ls) (if (null? ls) (quote ()) ...))
  call code:             (cdr ls)
  free variables:
  0. ls:                 5

Now we know for certain that we attempted (cdr ls) but ls was actually 5, and (cdr 5) is wrong.

Let’s try example2.ss:

(define pass-me-a-closure
  (lambda (thunk)
    (break)
    (thunk)))

(pass-me-a-closure
  (lambda ()
    (let loop ((x 5)
               (a 1))
      (if (zero? x)
        a
        (loop (sub1 x) (* a x))))))

Notice that we’ve set a breakpoint in the code using (break). This triggers the debugger, but only if we’re running Scheme interactively. If you try running scheme example2.ss directly, it will exit when it hits the breakpoint.

At the break point, we do the usual inspect and sfow frames.

[erjiang@silo ~]$ scheme
Chez Scheme Version 8.4
Copyright (c) 1985-2011 Cadence Research Systems

> (load "example2.ss")
break> i
#<continuation in pass-me-a-closure>                              : sf
  0: #<continuation in pass-me-a-closure>
  1: #<system continuation>
  2: #<system continuation>
  3: #<system continuation in dynamic-wind>
  4: #<system continuation in dynamic-wind>
  5: #<system continuation in $reset-protect>
  6: #<system continuation in new-cafe>
#<continuation in pass-me-a-closure>                              : s
  continuation:          #<system continuation>
  procedure code:        (lambda (thunk) (break) (display (thunk)))
  call code:             (break)
  free variables:
  0. thunk:              #<procedure>
#<continuation in pass-me-a-closure>                              : 

Right now, it doesn’t actually tell us what thunk is besides just #<procedure>. We need to rnspect that particular free variable by its number or name by issuing r 0 or r thunk and then printing the code.

#<continuation in pass-me-a-closure>                              : r 0
#<procedure>                                                      : c
(lambda () ((letrec ((loop (lambda (...) (...)))) loop) 5 1))     :

Notice that the let-loop form has been expanded (remember, the macro expander has already run) and on top of that, the part we actually care about has been elided. Also, the current object is now the code, instead of #<procedure>. Because of that, we can pretty-print the current object, and it will pretty-print the entire object.

(lambda () ((letrec ((loop (lambda (...) (...)))) loop) 5 1))     : p

(lambda ()
  ((letrec ([loop (lambda (x a)
                    (if (zero? x) a (loop (sub1 x) (* a x))))])
     loop)
    5
    1))

You can always go back up to the parent object from here.

In Chez Scheme, inspect is also available as a procedure that will launch the debugger and inspect whatever is passed to it. For example, writing (inspect thunk) into the above code would immediately launch the debugger on thunk. Likewise, doing (call/cc inspect) will launch the debugger and inspect the current continuation (which you can then trace).

Now you should know how to enter the debugger, inspect the stack frames, and inspect variables inside the stack frames, including the procedure code and call code. When in doubt, randomly hit keys or ask for help? Expect a pop quiz over this.

Arrow Endianness: How GNOME got sorting backwards

Conventions are important, even when it seems like it doesn’t matter one way or the other. Just look at all the people who complain about the nerve of Steve Jobs to place the close-window button on the left.

There’s no reason why the left side is better than the right side. Convention, however, says that you must choose one way and live with it, or else you’ll be forever forced to constantly convert between left-handed and right-handed traditions. Just ask anyone who’s ever had to write low-level networking code. When sending an integer, it doesn’t matter whether the big end or little end goes first, but if you don’t agree with the other party, your message will come out garbled.

So imagine my discomfort when I have to sort files in Nautilus or sort processes in System Monitor.

Continue reading Arrow Endianness: How GNOME got sorting backwards