Thursday, June 23, 2016

Use the freakin' debugger

This post is by a guy who does "not use a debugger". That's stupid. Using a friendly source-level debugger (Visual Studio, XCode, Eclipse) to step line-by-line through working code is what separates the 10x programmers from the wannabes. Yes, it's a bit of a learning hurdle, and creating "project" files for small projects is a bit of a burden, but do it. It'll vastly improve your coding skill.

That post quotes people like Rob Pike saying that stepping line-by-line is a crutch, that instead you should be able to reason about code. And that's true, if you understand what you are doing completely.

But in the real world, you never do. Programmers are constantly forced to stretch and use unfamiliar languages. Worse yet, they are forced to use unfamiliar libraries. Documentation sucks, there's no possible way to understand APIs than to step through code -- either watching the returned values, or compiling their source and stepping into it.

As an experienced programmer, it's true I often don't step through every line. The lines I understand completely, the ones I can fully reason about, I don't bother. But the programmer spends only a small percentage of their time on things they understand -- most of the time spent coding is noodling on the things they don't understand, and that's where the debugger comes in.

And this doesn't even take into account that in the real world, where programmers spend a lot of time working on other people's code. Sometimes the only way to figure it out is to set a breakpoint and run the unit test until it reaches that point.

Programmers fetishize editors. Real programmers, those who produce a lot of code, fetishize debuggers, both the one built into the IDE for debugging working code, and also the specialized tools for diagnosing problems in buggy code.

Seriously, if you are learning to program, learn to use the debugger in the integrated IDE. Step line-by-line through every line of code, until you grok it.. Microsoft's Visual Code is a good system for debugging JavaScript (which is a good starting language to learn). You'll thank me later when you are pulling down seven figures as a 10x programmer.

Wednesday, June 22, 2016

Reverse Turing testing tech support

So I have to get a new Windows license for a new PC. Should I get Windows 10 Home or Windows 10 Professional? What's the difference?

So I google the question, which gives me this website:

Ooh, a button that says "Download Table". That's exactly what I want -- a technical list without all the fluff. I scroll down to the parts that concern me, like encryption.

Wait, what? What's the difference between "Device Encryption" and "BitLocker"? I though BitLocker was Device Encryption?? Well, the purchase screen for Windows 10 has this friendly little pop out offering to help. Of course, as a techy, I know that such things are worse than useless, but I haven't tried one in a while, so I thought if I'd see if anything changed.

So up pops a chat window and we start chatting:

So at first he says they are the same. When I press him on the difference, he then admits they are different. He can't read the document I'm reading, because it's on a non-Microsoft "third party" site. While it's true it's on "", that's still a Microsoft site, but apparently he's not allowed to access it. I appears Microsoft firewalls their access to the Internet so jerks like me can't social engineer them.

So he goes on to find other differences:

At this point, he's acting as a Markov bot, searching Microsoft's internal site with the terms I give him, then selecting random phrases to spew back at me, with no understanding. Support for TPM has nothing to do with the difference.

Finally, he admits he can't answer the question, and offers to send me to more technical people:

I know this isn't going to work, but I'm in this far, and already planning to write a blog post, I continue the game.

At this point, I've learned to be more precise in my questioning:

It takes him awhile to research the answer. During this time, with more careful sleuthing with Google, I find the real answer (below). But eventually he comes back with this:

Like the previous person, it's all gibberish. He looked up my phrases, then spewed back random sampling of tech terms.

So I did figure eventually what "Device Encryption" was. It's described in this Ars Technica post. It's designed for when Windows is installed on tablet-style devices -- things that look more like an iPad and less like a notebook computer. It has strict hardware requirements, so it's probably not going to work on a desktop or even laptop computer. It requires a TPM, SSD, no fans (at least while sleeping), and so on.

The tl;dr is for people in my situation with desktop computer, Win10-Home's "Device Encryption" won't work -- it only works for tablets. If I want full disk encryption on my desktop, I'll need "Win10-Pro".

The idea is that in the future, tech support will be replaced AI bots that use natural language processing to answer questions like. But that's what we already have: tech support search text, finds plausible answers they don't understand, and regurgitates them back at us.

In other words, when the Turing Test is finally won, it's going to be in tech support, where a well-designed bot will outperform humans on answering such questions.

Saturday, June 18, 2016

Tesla review: What you need to know about charging

Before you buy an electric car, you need to understand charging. It’s a huge deal. You think it works almost like filling the gas tank. It doesn’t. Before going on long trips, you first need to do math and a bit of planning.

The Math

Like BMW model numbers indicate engine size, Tesla model numbers indicate the size of the battery, so my "Tesla S P90D" has a 90kwh (killowatt-hour) battery, with a 286mile range. Their lowest end model is the “Tesla S 60”, which has a 60kwh hour battery, or a 208mile advertised range.

In the United States, a typical plug is a 120volt circuit with a maximum of 15amps. Doing the math, this is how long it’ll take for me to recharge the battery:

That’s right, 1.4 days (or 2.1 days for a 90kwh car). This is the absolute worse case scenario, mind you, but it demonstrates that you have to pay attention to charging. You can't simply drive up to a station, fill up the tank in a couple minutes, and drive away.

Let’s say you live in Austin, Texas, and you have a meeting in Dallas. You think that you can drive up to Dallas in your new Tesla S 60, let the car charge while you are in the meeting, and then drive home. Or, maybe you have dinner there, letting the car charge longer. Or maybe you even stay overnight.

Nope, even 24 hours later, you still might not have enough charge left to get home. At 195 miles, it's at the range of the 60kwh battery, which would take more than a day to recharge using a normal electric circuit.

Faster Charging

That was a worst case scenario. Luckily, you probably won’t be charging using a normal 120volt/15amp circuit. That’s just the emergency backup if all else fails.

In your home, for high-watt devices like ovens, air conditioners, and clothes dryers, you have higher wattage circuits. The typical max in your home will be a 240volt/50amp circuit. It has a different power connector than a normal circuit, thicker wires, and so forth. Doing the math on this sucker, you get:

For our 190 mile drive, then, you can except to drive to Dallas, charge during the meeting and dinner for 5 hours, then you’ll have enough juice to get back home.

When you buy a Tesla, the first thing you’ll do is hire and electrician, and for $1000 to $5000, pay them to install this high-end circuit in your garage or car port. Since you garage is usually where the circuit breaker is located anyway, it’s usually the low-end of this range. You have to choose either the NEMA 14-50 plug, which can be used to power any electric car, or the Tesla HPWC (“High Power Wall Charger”) that just bundles the cord and everything together, making it easier to charge. Just back into your garage, get out of the car, pull off the cord, and plug it in. Real simple.

Standard NEMA 14-50 plug.
Different layout so you don't accidentally plug the wrong thing into it and blow a circuit.
Tesla proprietary wall charger.
Now for our trip to Dallas, though, we have a problem. While we can get the right charging circuit at home, we might not be able to find one on the road. How common can they possibly be? They sound like they'll be hard to find.

Well, no. Electric cars have become a thing. People are into them, unnaturally so. Even if you haven't noticed they EV (electric vehicle) plugs around you, they are everywhere. People are way ahead of you one this.

In our story of driving to Dallas, the first thing you'll do is visit, and lookup where to find charging stations. You'll find a ton of them, even in oil-rich cities like Dallas:

On the left coast, like California, it's insane. Chances are if you go to a business meeting, you'll find one in the parking lot, if not that building, then one next door. Drive in, go to the meeting, have some drinks or dinner afterwards, and you'll be able to drive home on a full charge.

Note that these charging stations primarily use the J1772 plug, a standard that all electric cars support. Your car comes with the standard electrical plug, the NEMA 14-050, and a J1772, so you can use any.

Also note that these charging stations are either run for profit, or part of a network. Even if the charging station is free, you still have to become a member. The most popular network nationwide is ChargePoint, but which one is most popular in a city varies. You may have to join a couple networks (I've just joined ChargePoint -- they have a free one down the street in a park, so I drive there and go for a walk and suck free juice).

These are sort of a franchise network. Somebody owns the parking space. They sign up with a network like ChargePoint, buy their unit and pay for installation, then get payments back from ChargePoint when they use the parking space. Since some businesses want to encourage you to visit, they don't charge you.

Ideally, all these charging stations should deliver max power. In practice, they are usually a bit weaker. Luckily, you can read people's reviews online and figure that out before you go.

One thing I want to stress is there is that charging is parking. The cost of electricity is negligible, that's not what they are charging your for. Instead, they charge your for time. Almost always, you'll be charge for how much time you leave your car parked there, not how much power you use.

As a Tesla owner, you can use these plugs, but also special Tesla plugs. Tesla makes up to two HPWC chargers available for free to business owners, especially hotels. They call this "destination charging", because you charge once you reach your destination. This is rather awesome as a Tesla owner, that you get vastly more options to charge than normal electric cars.

Level 1 and Level 2 charging

When you look at a charging map, like this one from ChargePoint, you'll see it mentions different "levels" of charging. What does this mean?

Level 1 means the standard 120volt/15amp, 1.8kw plug that you have for all electrical devices. Business (or even homes) that have an external plug will often put themselves on the map, but you don't care, since the charging at this level is so slow.

Level 2 means anything faster than Level 1, using the J1772 connector usually. There are a wide range of charging speeds. Original Nissan Leaf could only charge at 3.3kw, so a lot are in that range. More cars can deal with 6.6kw, so some are in that range. Only the Tesla and a Mercedes model go to the full 10kw, so many chargers don't support that much juice.

"Level 2 Tesla", in the map above means the HPWC mentioned above. They have appeared in the last 6 months as Tesla has aggressively pushed them to businesses, though it's usually just hotels. These may be 10kw (40amp), but may go up to 20kw (80amp). Note your car can only handle the 10kw/40amp speeds unless you upgrade to 20kw/80amp dual-charger.

"Level 2 NEMA" didn't use to be on the charging maps 6 months ago, but have appeared now. From what I can tell, a big reason is that when businesses put in a Tesla HPWC, they also put in the NEMA plug because it's really cheap, and allows them to attract more cars than just Teslas (though many don't come standard with that plug). Another reason this exists is because camping parks usually have the plug. You drive in with your campter/trailer, then hook up to the local electricity with this plug. You can also drive in with your Tesla. Indeed, the back of your car is big enough to sleep in.

The next options I'm going to describe below.

DC Fast Charging

Electricity from our walls is AC, or alternating current. Batteries, however, deal only with DC, or direct current. When you plug in your car, a device inside called the charger must convert the current. That's why cars have limitations on how much juice they consume, they are limited by the charger's capacity.

An alternative is to place the charger outside the car, where it can be much bigger, and feed DC direct current to the car. There is a lot of complexity here, because the car's computers need to talk to the external charger many times a second in order to adjust the flow of current. It's as much as a computer networking protocol as it is a copper connection for power.

If you look at your car, you'll see that the regenerative braking often charges the battery at 50kw, which is already many times faster than the AC Level 2 chargers mentioned above. We know the battery pack can handle it. Indeed, according to Tesla, SuperChargers can charge at a rate of 120kw, or 170 miles driving range in 30 minutes of charging.

Tesla has a network of SuperChargers placed along freeways in the United States so that you can drive across country. The idea is that every couple hours, you stop for 30 minutes, relax, drink some coffee, take a pee, and charge your car for the next stage.

In our driving scenario above, there's a SuperCharger in Waco, halfway between Austin and Dallas with 8 stalls available:

From Waco, it’s 76 miles to Dallas, meaning if you fully charge there, you can make it to Dallas and back to Waco without recharging – though it’s cutting it a bit close with the 60kwh model. Though, if your destination is the east side of Dallas, then maybe going through Corsicana and using their SuperCharger would be easier.

The SuperCharger is Tesla's solution to the problem but there are two other standards. There is the Asian standard known as CHAdeMO, and the European standard often called the COMBO adapter, as the DC component is combined with the J1772 AC standard. The CHAdeMO was the early standard coming with Japanese cars, but the COMBO adapter appears to be the winning standard in the long run. Tesla doesn't support either of these standards. These other standards are also far behind the Tesla in charging speed, maxing out around 60kw (and usually less), whereas the Tesla does 120kw.

As I mentioned, the direct DC charging dumps power into the battery as fast as the battery can take it -- which means it's the battery that now becomes the bottleneck. As you know from experience charging phones and laptops, charging to 50% full, but it seems to take forever to get from 90% to 100%. The same is true of your Tesla.

So here's a trick. The 60kwh car actually ships with the 75kwh battery. Tesla is allows you to "upgrade" your car later to the higher range for $9k, which will consist of just Tesla turning on a switch enabling the entire battery. But, for SuperCharging, it's still a 75kwh battery. 60 is 80% of 75. That means, you can charge to 100% full in 40minutes rather than 75minutes.

Charging to 100% lowers the lifetime of the battery, to Tesla recommends you only charge to 80% anyway. Again, with the 60kwh battery, charging to 100% means only 80% anyway. Thus, you get most of the benefits of a larger battery without paying for it. If you truly want extra range, you might consider the 90kwh upgrade instead of going from 60kwh to 75kwh.

The new Model 3s that come out in a few years won't have free SuperCharger access. That may also be true of the cheapest Model S (they keep changing this). You should check on this.

Off-peak Charging

Among the many revolutions recently has been smart electrical meters. They can now bill you by time-of-day when you consume power, charging you more during peak hours, and less during off-peak. If you haven't been paying attention to this, you are still probably on the old billing plan. You might want to look into changing.

Where I live, they charge $0.01 (one cent) per kilowatt-hour during off-off-peak, between 11pm and 7am. That's insane, compared to $0.20 elsewhere in the country.

Your Tesla can be configured to when it charges, so you get home, plug it in, and it won't start charging right away, but wait until off-peak or off-off-peak to start charging.


Electric charging math has given me a new appreciation for the power of gasoline. Filling up your gas tank is the equivalent of charging at multiple megawatt speeds – something electric cars will never get close to. Driving at the range limits of the car requires planning – you just can’t do it without doing some math.

Ethereum/TheDAO hack simplified

The news in the Bitcoin world is the Ethereum/DAO hack. I thought I'd write up a simplified explanation.

What is Bitcoin?

I'm sure you know, or have an idea what Bitcoin is, but I'll explain it again in terms that are important here.

Bitcoin is just a public ledger (the "blockchain"), of all transaction there ever was. This ledger is huge (80-gigabytes) and growing, but Moore's Law says computers grow even faster, so that shouldn't be a problem.

Each entry in the ledger says to move the coins received in these previous entries, and give them to this recipient. In other words:
move these coins I received there, to this guy here
In the future when that guy spends the coin in a new transaction, he'll refer back to this transaction here. That's why it's called a "chain" -- every transaction refers to a previous one, back to the original creation of the coins.

Actually, these transactions aren't fixed as simply moving money around. Instead, a script is used, written in  a bitcoin-specific programming language that can do things like add, multiply, and compare numbers. However, it's not a full programming language. It can't call functions or execute loops. It's complex enough to do allow some creativity, such as allowing transactions to be escrowed by a third party, but it can't do much more than that.

What are alt-coins?

After Bitcoin took off, other people started to create their own variations.

Most of these variations have offered no meaningful improvement over Bitcoin. They instead are just popular among speculators who pump and dump, hoping to buy in at a low price and sell to some fool at a higher price.

There are only two alt-coins that have actually made an improvement: ZeroCoin and Ethereum. ZeroCoin works by making transactions anonymous. Ethereum is the alt-coin that is being discussed here.

What is Ethereum?

Ethereum is a variation of Bitcoin that uses a full programming language instead of a simple scripting language. The reason for this is that Ethereum supports smart contracts.

Let's say that we want to organize a small conference. We need 100 people to sign up and pay/deposit money, so we can rent a hotel and such. But if not enough people sign up by a certain date, then the deposits need to be refunded. With Ethereum, we can write in a JavaScript-like language to code up this contract. It'll guarantee that everyone will get a ticket to the conference, or everyone will get their money refunded, depending on how many sign up.

That's a simple example. The possibilities are endless, which has got a lot of people really excited. Which, of course, makes the believers insufferable among non-believers in the system.

What is The DAO?

DAO stands for decentralized autonomous organization. It's notionally like the example above for getting funds for a small conference, except that it includes much more. Members buy shares in the DAO, and according to the number of shares they have, can vote on things. The dreamers have the idea they'll replace Democracy and run entire countries this way.

There are many types of DAOs on the Ethereum blockchain, but one in particular is TheDAO, which is some sort of DAO-based mutual-fund/investment-fund. You buy in, vote on which investments the fund should make, then reap the returns. It looks like a big organization, but it actually just runs as bits of code and data within the Etherium blockchain.

This investment scheme has gobbled up 10% of Ethereum coins, or $100 million worth out of Ethereum's $1 billion ecosystem.

When you want to leave the fund, you split from it. This includes your share of unspent Ethereum coins, but also tokens representing returns on investments you were part of, but which haven't yet paid out. That's why you "split" rather than "cash out", you need your own DAO shard to track those investments until they pay out.

How did The DAO get hacked?

When a member exits the investment scheme, they call the function name splitDAO(). There are two issues.

The first is that the member will supply some of their own code with the transaction. Among the things that code will do is tell the DAO code how to transfer Ethereum coin. It's a necessary feature, part of Bitcoin as well.

The second issue is that Ethereum code is recursive. That means when a function is running, it may call itself a second time.

The bug is that when splitDAO() is called, it will then call the recipients code to transfer Ethereum coin, after which the recipients code will call splitDAO() again before finishing. This causes the process to repeat itself, transferring more Ethereum coin, then calling splitDAO() again, which calls the hacker's code, which calls splitDAO(), which calls the hacker's code, and so on. The process will continue endlessly, until it drains all of TheDAO's coin.

When you split like this, they still force you to wait 27 days before you get your Etherium coin. Thus, the hacker doesn't actually have use of it for another month.

How can they recover the stolen money?

They can't -- at least not without destroying the entire principle of cryptocurrencies. It's like trying to cure cancer with a Howitzer.

One solution is to roll-back the blockchain before the theft. Of course, that means screwing over everybody who made a transaction since then. You'd be screwing people out of $1 million in order to compensate the theft of $100 million. This is, of course, the type of corrupt thinking that gets us into banking failures in the real world, as we screw over everyone else in order to protect those banks who are too big to fail.

Another solution is to update the Ethereum code to blacklist this address, or better yet, insert a magic key that will give control over those funds back to TheDAO.

The problem with changing the code is that it forks the blockchain. Transactions are added to the chain by miners -- a decentralized group of people. It can include you, if you want to run mining software on your computer. Those miners who update their code to the change will be working off a different blockchain than those who don't. The blockchain will repair itself if 51% of the miners update to the latest code (a soft-fork), but it'll be hopeless broken if they don't (a hard-fork of two irreconcilable chains).

What does this all mean?

I'm a crypto-anarchist. The entire point of cryptocurrencies to get around corrupt humans. And that's what trying to repair this problem is -- corruption. It's a violation of TheDAO's own contract, which says the code is the contract, not to be superseded by human re-interpretation. It's tacit acknowledgement that TheDAO deserves special treatment that would not be given to smaller makers of contracts that have similar recursion bugs. That the miners vote on it (by choosing which software to run) is besides the point, nobody else with Ethereum's favoritism would be able to get miner consensus. It's a profound betrayal of trust that those maintaining the code would even consider helping TheDAO.

Obviously, the complexity of Ethereum is a huge security issue that will likely be repeatedly exploited in the future. This "recursion" issue demands that everybody writing contracts needs to write code extraordinarily carefully. In hindsight, it's obvious that recursion shouldn't have been an allowed feature, but it likely can't be fixed now without a hard-fork. Also, in the future, we'll likely see additional problems as dangerous as the recursion issue.

The fascinating thing now is that in the past, people hired lawyers to review complicated contracts. In the future, they'll need to hire hackers. After a contract is signed, I'm now motivated to hire a very good hacker that will keep reading the code until they can find some hack to my advantage.

In any case, the original concept of TheDAO is useless utopian nonsense. The original Bitcoin was created by people who actually understood a lot about currency. TheDAO was created by people who are hopelessly naive about investing, who then put the system in the hands of trained monkeys. This isn't "wisdom of the crowds", as they proposed, but "ignorance of the mob". If the hacker hadn't put a stop to this nonsense, it would have slid into some sort of Ponzi scheme, getting its creators arrested a couple years from now.

Update: The hacker is bribing the miners not to fork [*], by promising to give the miners a third of Etherium. This is possible with the protocol, to create transactions that takes money from the blacklisted address and donate it directly to whoever mines a block, as a sky-high transaction fee. Choosing to fork would mean forgoing these payments. Thus, the hacker has a power to corruptly subvert the system to counterbalance Etherium's power.


Official statement by Ethereum code maintainers declaring their intention to blacklist/soft-fork

More detailed walk through of splitDAO()

Bloomberg "Smart contracts were dumb", a cool look at the issue

Monday, June 13, 2016

Scanning for ClamAV 0day

Last week an 0day was released for ClamAV. Well, not really an 0day so much as somebody noticed idiotic features in ClamAV. So I scanned the Internet for the problem.

The feature is that the daemon listens for commands that tell it to do things like scan files. Normally, it listens only locally for such commands, but can be reconfigured to listen remotely on TCP port 3310. Some packages that include ClamAV sometimes default to this.

It's a simple protocol that consists of sending a command in clear text, like "PING", "VERSION", "SHUTDOWN", or "SCAN
So I ran masscan with the following command:

masscan -p3310 --banners --hello-string[3310] VkVSU0lPTg==
Normally when you scan an address range (/0) and port (3310), you'd just see which ports are open/closed. That's not useful in this case, because it finds 2.7 million machines. Instead, you want to establish a full TCP connection. That's what the --banners option does, giving us only 38 thousand machines that successfully establish a connection. The remaining machines are large ranges on the Internet where firewalls are configured to respond with SYN-ACK, with the express purpose of frustrating port scanners.

But of those 38k machines, most are actually things like web servers running on odd ports. 51 machines running VNC, 641 machines running SSH, and so on.

To find specifically ClamAV, I send a command using the --hello-string feature. I send the text "VERSION", which must be encoded with base64 on the command-line for masscan (in case you need to also send binary).

This finds 5950 machines (i.e. 6k) that respond back with a ClamAV signature. typical examples of this response are:

At first I thought the date was when they last updated the software, maybe as a page. Roughly half had dates of either this morning or the day before. But no, it's actually the dates when they last updated their signatures.

From this we can conclude that roughly half of ClamAV installations are configured to auto-update their signatures.

Roughly 2400 machines (nearly half) had the version 0.97.5. This was released in June 2012 (four years old). I'm thinking some appliance maker like Barracuda bundled the software -- appliances are notorious for not getting updated software. That hints at why this non-default configuration is so common -- it's not users who made this decision, but the software that bundles ClamAV with other things. Scanning other ports gives me no clues -- they appear all over the map, with different versions of SSH, different services running, different SSL versions, and so on. I thought maybe "mail server" (since that'd be a common task for ClamAV), but there were only a few servers, and they ran different mail server software. So it's a mystery why this specific version is so popular.

I manually tested various machines with "SCAN foo". They all replied "file not found", which hints that all the units I found are vulnerable to this 0day.

As for other things, I came across a bunch of systems claiming to be ChinaDDoS systems:


This sort of stuff shouldn't exist. The number of ClamAV systems available on the public Internet should be zero.

Even inside a corporate network, the number should be 0. If that stuff is turned on, then it should be firewalled (such as with iptables) so that only specific machines can access it.

Two important results are that half the systems are really old (EOLed, no longer supported), and only half the systems have the latest updates. There's some overlap -- systems with latest signature but out-of-date software.

Tuesday, June 07, 2016

No, Musky, Feudalism is best for Mars

Recently, the press fawned all over Elon Musk's comments at a conference. Among them was Musk's claim that "direct democracy" would be the best system, where citizen's vote directly for laws, rather than voting for (corrupt) representatives/congressmen. This is nonsense. The best political system would be feudalism.

There is no such thing as "direct democracy". Our representatives in congress are only the first layer on top of a bureaucracy. Most rules that restrict us are not "laws" voted by congress but "regulations" decided by some bureaucrat.

Consider the BP Gulf Oil spill, as an example. It happened because oil companies got cozy with their regulators, the minerals Management Service (MMS), part of the Department of the Interior. The bureaucrats had a dual mandate: to protect the environment, and to promote economic activity. Oil companies lobbied them to risk the environment in favor of profits.

Consider  Obamcare's controversial mandate that health insurers must pay for abortions. This was not part of the law pass by congress, but a decision by the bureaucrats in charge of all the little details in carrying out the law.

Consider the Federal Communication Commission (FCC) regulation of the Internet. It bases its power to regulate the Internet on laws that essentially predate the Internet as we know it.

No matter how ideal this "direct Democracy" of Musk's, you are still going to leave most decision making in the hands of a bureaucracy. This is especially true on space flight to Mars. If something's wrong with the air system, you want a technician making quick decisions to fix it. Otherwise, people would suffocate long before they had a chance to vote on the issue. Technicians must be trusted with important decisions, like jettisoning that one pod killing 10 people in order to save the remaining 100.

No matter the political system, you are going to have the bureaucracy making tactical, day-to-day decisions. You are also going to have an upper tier, making long term strategic decisions. It's how all political systems work, from monarchies to "direct democracy". They largely just change the names of the bureaucrats, rather than being substantively different.

The corruption in Democracies doesn't necessarily come from those in power, but from the voters themselves. Voters are idiots and vote like idiots. That's why you have candidates like those of the U.S's current election season -- populist demagogues preying on people's ignorance proposing solutions that educated people believe to be unworkable. The majority of voters have never taken an economics class, do not understand foreign policy, or have any other qualification to make the decisions they make.

Instead of education, voters overwhelming decide what's best for themselves, not dispassionately what's best for society as a whole. College students vote for free college. Old people vote for social security and health care. Mothers vote for child leave and child care. Racists vote to keep unwanted types out of their community. And so on. That's corruption at it's core.

As de Tocqueville is famous for noting, democracy only lasts up to the point that 51% of the population realizes they can vote to just take everything away from the other 49%. You call it corruption, but our current system allows a member of the 49% to lobby congress so that they don't get screwed by the 51%.  Indeed, that's what most lobbyists do -- they aren't asking for special favors from the government so much as trying to alleviate special punishments. It's a sort of corruption defending themselves from the voter's corruption.

As the famous quote goes, "Democracy is the worst form of government -- except for all the others". It's a horrible system, it's just we haven't found any better.

But in space exploration, the old rules no longer apply. We can imagine better political systems.

Overwhelmingly, the best system is "vote with your feet". In the future, billionaires will creation space stations around Earth and out in the asteroid belt. Yes, they will be absolutely dictators in their own artificial worlds, but they can't be too evil. They'll be competing for people to come work and live on their space stations.

Such a system doesn't work well on Earth because the barrier in changing countries is just too high. Consider the European Union as an example, where citizens of one country can move to any country they wish. They don't, because they are tied to the language and culture of their own community. In space exploration, such barriers to movement don't exist. Space will look more like the United States, where people do move around a lot, and who do move to the state that they like best. Sure, the culture of the South is different from New York, which is different from the Midwest, which is different from Texas, which is different from the West Coast, but these are tiny cultural/language barriers compared to those that have stopped movement in the past.

Thus, people will vote -- vote by deciding which space station is best. Those who want free health care will go to those space stations. Those who want more money in their pockets now will go to those stations without free health care.

The benefit of the "vote with your feet" system is that there's no coercion. Democracies are always backed up with a police state that coerces you, at the point of a gun. to conform to what the majority has decided. On Earth, you have to submit, because in most countries leaving just isn't a viable option.

But imagine the petit dictators like Musky in his space station. If he tries to coerce people, they'll simply leave. Following laws will therefore always be voluntary -- take it or leave it.

And you will be able to leave. You might imagine that Musky might just surprise the inhabitants and seal all the airlocks, enslaving them all the sudden. That might happen, but only once. Then all the other space station dictators will get together, agree on some sort of "big charter" guaranteeing people rights, such as to leave any space station, and agree that if any member violates the charter, the rest of will just break in from the outside, freeing the people.

Another plausible scenario is that billionaires try to trick people into slavery. A good example is Uber, which provides new workers with cheap loans for new cars. The worry among activists is that it then "enslaves" the worker, because they have to keep working for long hours at low pay in order to satisfy the agreement with Uber -- and agreement they signed without realizing the consequences. That can happen in the future where workers can never leave the space station until they've paid off their debts -- which they can never do in a system rigged against them.

This might happen, but as you'll note above, it can also happen under the current system. Word gets around. Uber has to deal more fairly with its workers who hear such stories, and billionaire dictators of space stations will likewise have to deal a bit more fairly.

Such a system won't be just a billionaire (or corporation) as the dictator with everyone serfs below them. Monolithic corporations are a disaster. Instead, space stations will outsource. They'll have a life support company managing life support. They'll have a propulsion company managing the rockets. They'll contract with a food service company. They me dictators of their own little worlds, but they'll still have to deal with banking corporations outside their worlds -- just like how monarchies in midevil times had to borrow money from banks to conduct their little wars.

What I'm getting at here is that the best political system for space exploration looks a lot like feudalism, though one full of yeomen (who were free to move about) rather than serfs (tied to the land). It's the only morally defensible system of government -- nobody is coerced to follow laws they dislike, but is able to vote with their feet, and choose the laws that best suite them. Those with bad rules will suffer, those who make good rules will prosper.

Monday, June 06, 2016

Instrumenting masscan for AFL network fuzzing

This blog post is about work in progress. You probably don't want to read it.

So I saw this tweet today:

As it turns it, he's just fuzzing input files. This is good, he's apparently already found some bugs, but it's not a huge threat.

Instead, what really needs to be fuzzed is network input. This is chronic problem with AFL, which is designed for inserting files, not network traffic, into programs.

But making this work is actually pretty trivial. I just need to make a tiny change to masscan so that instead of opening a libpcap adapter, it instead opens a libpcap formatted file.

This change was trivial, successfully running it is tough. You have to configure the command-line so all IP addresses match up with the libpcap file content, which is a pain. I created a sample lipcap file and checked it into the project, along with a help document explaining it. Just git clone the project, run make, then run this command line to see it run for yourself:

bin/masscan --nobacktrace --adapter file:data/afl-http.pcap --source-ip --source-port 6000 --source-mac 00-11-22-33-44-55 --router-mac c0-c1-c0-a0-9b-9d --seed 0 --banners -p80 --nostatus
If you run on the command-line, it appears to return immediately. I say "appears" because there's actually a 10 millisecond wait. That limits fuzzing speed to a 100 attempts per second, rather than thousands per second. That's a tougher change, so I'll have to get around to fixing that, but in the meanwhile, you can just run a bunch of AFLs in parallel to get around this.

But when I try to run AFL, it's not working at the moment. In instead get this error:

As you can see, the command that returns in 10ms is now hanging when run under AFL, which says that it doesn't return in 1000ms. Using the '-t' option to increase the timeout doesn't help. Running masscan in some other way, such as parsing configuration files, works just fine.


So I changed to where I "join" threads cleanly, so that the entire thing can run cleanly without every having to stop and wait. However, this creates a second problem not AFL refused to run because it's crashing instead of hanging. AFL suggests that it might be an out-of-memory issue, and that I should increase memory. So I bumped up memory and now it's running.

This memory issue might be what the problem was all along. Masscan assumes big scanning and sets up some large data structures at the start, so it may exceed the 50-megabyte assumed by AFL.

So now I have it running, fuzzing HTTP server response input:

But this isn't really success. The pcap file is 1986 bytes long. However, AFL has "trimmed" the input file down to 0.20%, which is the first 4 bytes of the file. This is just testing the libpcap library at this moment, and the fact that it supports multiple file types determined by the "magic" string in that first 4 bytes. I need to figure out how to make it fuzz starting deeper in the file, not at byte 0.


I got the LLVM version working on Raspberry Pi 3 Odroid C2 ARM-64, so naturally I need to spread the work across 4 cores.

The Odroid has slightly faster CPUs than the Raspberry Pi 3, but mostly importantly, it's got 64-bit Linux available to it, which the Raspberry Pi 3 apparently still doesn't.

Thursday, June 02, 2016

My fellow Republicans: don't support Trump

Scott Adams, the creator of the Dilbert comic strip, has a post claiming a Trump presidency wouldn't be as bad as people fear. It's a good post. But it's wrong.

Trump is certainly not as bad as his haters claim. Trump not only disables the critical-thinking ability of his supporters, but also of his enemies. In most conversations, I end up defending Trump -- not because I support him as a candidate, but because I support critical-thinking. He's only racist sometimes, most of the time I love his political incorrectness.

But with all that said, he would indeed be a horrible president. As a long-term Republican, I'd prefer a Hillary Clinton presidency, and I hate Hillary to the depths of my soul. She's corrupt, and worst of all, she's a leftist.

But there's a thing worse than being a leftist (or right-winger) and that's being a "populist demagogue". Populist demagogues tell you that all your problems are caused by them (you know, those people), and present unrealistic solutions to problems. They appeal to base emotion and ignorance.

When nations fail because of politics, it's almost always due to populist demagogues. Virtually all dictators are a "man of the people", protecting the people's interests against the powerful (somehow, the dictators themselves are never part of the "powerful", since by definition, they are "of the people"). We see that in Venezuela right now, whose economy has crashed with oil prices (50% of their GDP was oil exports). The leader is making everything worse by running the playbook of bad populist policies. For example he's printing money, which first year economics textbooks tell you causes inflation, then blaming the resulting inflation on the United States and the CIA manipulating prices. That's the essence of populism: they pursue horrible policies, but blame the consequences on them.

In a Trump presidency, bad results that educated people know is caused by the government policy will instead be blamed on Mexico, China, and so on. The worst things get, the more crowd will cheer on Trump's and congress's bad policies, the more they punish Mexico and China, and the more they make bad policies worse.

Consider the $15 minimum wage promoted by Bernie Sanders, a hateful populist demagogue who is, if anything, worse than Trump. Hillary wanted $12.

Why not $18? Why not $25? Why not $100/hour minimum wage? Presumably, there are some negative thingies that happen the more you hike minimum wage. Presumably, there are some educated people out there who have studied this problem and can measure these things.  And there are. An example is this non-partisan, Congressional Office of Management and Budget (OMB) analysis of raising minimum wage to $10.10. It describes numerous positive and negative effects, none of which fits in a demagogic sound bite.

Raising the minimum wage has broad popular support, even among Republicans, because few are educated enough to appreciate the downsides. But yet, it doesn't get raised. The only explanation by populists like Bernie, or Trump, is that there must be some conspiracy (such as by Wall Street billionaires) that prevents the minimum wage from being raised. The truth is that our political leaders are basing their decision on things like the OMB report. They are basing their votes on an educated analysis of the policy, not on corruption and bribes from Wall Street. Note that there is no right or wrong answer to raising the minimum wage. There are reasonable people on both sides. It's just that this true debate based on education is far different than the public debate, which is based on emotion and ignorance.

Trade, which both Bernie and Trump oppose, is the same way. Educated people are for it, because it's such an obvious benefit to the population as a whole. Yet, special interests exploit the ignorance of the populace, which is why most people oppose trade.

Again, since anti-trade policies are so obviously a crowd pleaser, populace demagogues explain why such policies aren't adopted by blaming the vast conspiracy of the powerful, like Chinese lobbyists and Wall Street executives who want to move factories to Mexico.

Again, there's really no right and wrong answer. I oppose the latest "trade" deals like TPP and TTIP because they expand regulation rather than reduce tarifs, for example. I also appreciate that while benefits of trade exceed the costs, the costs of the change are often born unfairly by some groups.

The point isn't that you should support trade and oppose raising the minimum wage. Instead, the point is that populists present things as moral issues that transcended educated thought, and that when these policies are opposed by reasonable, educated people, the populist creates conspiracy theories explaining their opposition. Their power rests on the quality of their conspiracy theories.

All politicians are a little populist in this regard. The current one is President Obama. Yet when Obama has failed at his populist policies, like closing Gitmo, he blames the Republicans only a little bit. He hasn't gone scorched-earth populist-demagogue on them.

The only danger to a Democracy is such populist demagoguery. We see how Alexis Tsipras was elected on a wave of populism, and proceded to make the Greek debt crisis much worse. We see how the populist leader of Venezuela is making his oil crisis much worse. When the educated opposed policies for smart reasons, ignorant crowds overran them. The educated soon learned to keep quiet.

The same will happen with a Trump presidency. When a crisis happens, and a crisis will always happen, his will revert to populist demagoguery. He'll sweep aside any informed, rational debate on the issue. And as we've seen with the Republican politicians who have meekly agreed to Trump's candidacy, very few politicians will have the backbone to stand up to him. Republicans are already mute on criticism of Trump, and Democrats so frothing at the mouth in hatred Trump that nobody listens to them, either.

Trump is unforgivably racist (though barely so, not the white supremacist his enemies claim). Trump is a crappy businessman, not nearly successful as he claims. The few successes he's had are based on flim-flam, the faulty belief in his success. He's a con man, not a good manager. He's not the negotiator he claims, international politics works much different than negotiating price for building materials. On the world stage, everyone will laugh at him.

But all of these things can be forgiven, because most candidates suck just as much. Instead, the thing that makes Trump dangerous is his populist demagoguery. Historically, it this more than anything else that destroys democracies and make people's lives worse off.

Drumpf: this is not how German works

In our willingness to believe any evil of Trump, some have claimed his original name was "Drumpf". This isn't true, this isn't how the German language works. Trump has the power to short-circuit critical thinking in both his supporters and his enemies. The "Drumpf" meme is just one example.

There was no official pronunciation or spelling of German words/names until after Trump's grandfather was born. As this The Guardian article describes, in the city ("Kallstadt") where Trump's grandfather was born, you'll see many different spellings of the family name in the church's records. like "Drumb, Tromb, Tromp, Trum, Trumpff, Dromb" and Trump. A person might spell their name different ways on different documents, and the names of children might be spelled different than their parent's. It makes German genealogy tough sometimes.

During that time, different areas of German had different dialects that were as far apart as Dutch and German are today. Indeed, these dialects persist. Germans who grow up outside of cities often learn their own local dialect and standard German as two different languages. Everyone understands standard German, but many villagers cannot speak it. They often live their entire lives within a hundred kilometers of where they grew up because if they go too far away, people can no longer understand them.

The various German dialects, sub-dialects, and accents often had consistent language shifts, where the same sound is pronounced differently across many words. For example, words that in English have a 'p' will in German have 'pf" instead, like the word penny becoming Pfennig, or pepper becoming Pfeffer.

Kallstadt is located in the Pfalz region of Germany, or as they pronounce it in the local dialect, Palz. You see what I'm getting at, what is 'pf' in German is 'p' (like English) in the local dialect. Thus, you'd say "Trump" if you were speak Pfalz dialect, or "Trumpf" if you were speaking standard German.

It's like the word for stocking, which in standard German is Strumpf. In documents written around that time in the Pfalz region, you'd find spellings like StrumpStrumpf, StrumpffStrimp, and Stromp. Both the vowels and the last consonant would change (according to a Pfalz dictionary I found online).

Friederich Trump was born in 1869, in a time when Germany was split into numerous smaller countries. The German Empire that unified Germany was created in 1871. The counsel to standardize the language and spellings was 1876. Friederich emigrated to America in 1885. In other words, his birth predates the era in which they would've standardized the spelling of names.

From the records we have, "Trump" was on his baptism record, and "Trump" is how he spelled his name in America, but "Trumpf", with an 'f' was on his immigration form. That's perfectly reasonable. The immigration officer was probably a German speaker, who asked his name, and spelled it according to his version of German, with an 'f'.

This idea of an official spelling/pronunciation of a name is a modern invention, with the invention of the modern "state" and "government officials". It didn't exist back when Friederich was born. His only birth record is actually his baptismal record at the local church.

Thus, Trump's name is spelled "Trump". It was never officially spelled any other way in the past. It was never "changed". Sure, you'll see church documents and stuff with different spellings, but just how all words and names were handled back then. Insisting that he's "Drumpf" is ignorant -- it's not now the German language works.

Update: Somebody named Gwenda Blair wrote book on Trump's family, which claims the name comes from Hanns Drumpf, who settled in Kallstadt in 1608. But they can't connect the dots. That's because right after, the 30 years war happened. It's a famous event in Germany because it burnt most of the church records. Most all German family trees can be traced back to the 30 years war -- but no further.

It's probable they were related. It's possible that Hanns was even an ancestor of Trump who at one time spelled his name "Drumpf". But that's still not the official spelling, because that's not how German worked at that time.

Update: According to Snopes, it's true that "Donald Trump's ancestors changed their surname from Drumpf to Trump". Snopes is wrong, because they are morons. The correct answer is "there's no record of a name change". The fact that difference sources make conflicting claims should've been proof enough for Snopes that there's no evidence to support the claim.

BTW, my great-grandmother is "Pennsylvania Dutch", most of who came from that same region. I may be distantly related to Trump.

Also BTW, isn't weird that we are talking about his grandfather born 150 years ago? His grandfather was 36 when his son was born, and his father was 41 when Trump was born. Three generations back, and we are already in a pre-historical era -- that is to say, the era where we had writing, but not standardized spelling.