Google ISS Prank

As April fool’s day aproaches, I noticed an odd item in my Google Analytics page. After visiting their realtime stats, it appears I have 41 active visitors to my website from the International Space Station.

ISS on Google Analytics

Hive CSV SerDe

In the past I’ve used a hive CSV SerDe originally from bizo but one complaint that often arose is some operations failed due to type errors.  The original SerDe produces strings objects for operations. This leads to odd issues with various hive operations. Due to this, I updated the hive SerDe to support integer types as well. The fork is on github at dmaust/csv-serde.

Tip: EC2 DNS Records

It is common after launching a new instance in EC2 to assign a domain name to it.  A useful practice I have found is to create a CNAME to the amazon provided public hostname.  This address will resolve to the internal address when inside EC2, and to the public address outside EC2.  By using this hostname, you can ensure the best performance and lowest cost by always hitting the server with the optimal address.

EC2 Pricing

This Y-Combinator article <> says it best.  I have used EC2 since 2009 and the on-demand price of an m1.small has started at $0.10/hour.  Since the performance of the instance has seen little change, you would expect after one and a half years have elapsed the price to have fallen to 0.05/hour, and after 3 years $0.025/hour.  Instead the current price is $0.065 per hour.

When I initially subscribed to EC2 it was a great deal.  Now the pricing is becoming excessive.  With the advent of OpenStack, I see Amazon being forced to lower their prices.

IPv6 enable a website

Recently it has become more pressing to migrate to IPv6 due the diminishing IPv4 address space. Because of this, services like Hurricane Electric are growing in popularity to allow IPv6 tunnels to be created for various applications.

Typically a website owner who wishes to provide IPv6 connectivity has had two choices:

  • Use a web host that has native IPv6 connectivity.
  • Use a tunnel brokering service to bring in IPv6 connectivity to their server over IPv4.

While these are great approaches for many users in an dedicated hosting environment with flexible firewalls, there are many situations where a tunnel is not permitted. Many smaller sites are on shared servers in which custom network configurations are not permitted.  Also some hosts utilize firewalls too restrictive for IPv6 tunnels.  One example of this is Amazon Web Service’s EC2, in which only TCP, UDP, and ICMP can be opened.  In such an environment, most tunnel broker’s solutions fail.

For these cases I’ve introduced a new solution. will allow IPv6 connectivity by forwarding HTTP requests to the existing server.  It supports X-Forwarded-For and X-Real-IP headers which allow the backend web server to identify the real IP of the users.  Also it supports a dynamic HTTP Host field for your site, so the solution will work in virtual hosting systems in which only the hostname is unique.

The service is still in beta and is providing service free during the beta.  The only requirement is the ability to modify DNS records so that an AAAA record can be added.


Why we encourage everyone to make highly leveraged investments to be made?

The Investment Opportunity

If you asked many people if they would buy a stock on margin paying only one-tenth the price, have the potential to receive its gains, but be responsible for paying for its losses, they would be hesitant to say the least.  Add the fact that there is a minimum investment of $80,000 being leveraged to purchase a $400,000 asset, and they would think you’re crazy.  Luckily due to regulations passed after the Great Depression this is not legal.  Currently a 50% initial margin is required for individuals to purchase stocks.

However, there is a market where it is not only legal to do this but it is commonplace.  There is a large group of people who routinely take on such a risk; they have the hope of becoming wealthy, but if their investment moves against them, many will throw in the towel and refuse to cover their losses.  We call them homeowners.

Buying a house with a mortgage places the buyer at tremendous financial risk.  With a 10% drop in price, the buyer has lost all equity provided by the 10% down payment.  With a 20% drop in price the buyer owes the bank the amount of their original down payment.  With a 50% drop, many consider a “strategic mortgage default,” by walking away from their mortgage since they are now $160,000 underwater on their $400,000 home.

Strategic Mortgage Default

Since the collapse of the housing market there have been numerous individuals who have opted for strategic mortgage default.  Many states have non-recourse mortgages. Within these states, borrowers are freed from their losses (debt) when they choose a strategic mortgage default while the bank is left to take the loss.  Because of this situation, many Americans feel the potential upside for a house to appreciate in value outweighs the potential loss from a down housing market.  If individuals lose, they can walk away with a bruised ego and a poor credit history, but without a burden of debt to repay.  Because of the way many view mortgages, there is a tremendous upside, with little downside.  It’s clear why this is a great investment for individuals, and why banks are making it harder and harder to take out a mortgage.  The benefit to the individual is tremendous, while the risk is low.  The benefit to the bank is low, but the risk is tremendous.

A House of Cards

With mortgages in default, how can banks remain solvent?  In the Great Depression, banks became insolvent as losses stacked up and margin calls become ineffective.  This led to a collapse in banking which required Government programs initiate a recovery.  What solutions exist?

A Disturbing Solution

One possibility would be to raise the initial margin requirement (down payment) on homes, and allow banks to make margin calls on loans.  This would very easily allow banks to ensure a borrower will not default.  If the price of a house falls, the borrower will be required to provide additional margin.  If they fail, the bank will be authorized to sell the house immediately, and the former homeowner will have lost everything.

This will protect the bank in the event of a drop in price: the homeowner may be broke, but he will not owe the bank anything.  This will ensure the banking system does not risk a substantial loss on defaults… or does it?

In a falling housing market, bank sales will increase the supply of houses for sale, in a market where demand is likely to be weak already.  This will create a long squeeze driving down house prices, which will drive more and more homeowners to ruin.  This will continue until every mortgage has been called by banks until the market finally hits bottom.  At this point the damage to the economy will be tremendous, and possibly be worse than the collapse of the banking industry that this solution seeks to prevent.

A Better Solution

Instead of forcing homeowners to accept the possibility of a margin call, why would we allow them to make such an investment in the first place with a directional risk they should not be willing to accept.  Instead we should limit the risk by allowing a homeowner to reduce their margin requirement by taking on a hedge to their position of being long on a house.

In other industries futures are often used to effectively reduce risk of unexpected changes in price.  Home buyers should not only be offered protection from a potential loss; it should be required.  While stock traders are not permitted to go long on a stock without a substantial margin requirement, homeowner should not be permitted to be long on his house without the same margin requirement.  If someone purchases a house, their mortgage contract should include the requirement to provide a hedge against a falling housing market.

Home buyers should be required to do one of the following things:

  • short housing index futures to reduce their overall position
  • buy options providing downside protection.
  • deposit a reasonable margin requirement

Of course, options would carry a premium, but the strategy would allow the buyer to continue to benefit from a rise in the value of their house.  Futures on the other hand would absorb both losses and gains making the homeowners equity rise only as they pay off their mortgage, but it would eliminate the have the possibility of rising or declining equity due to a change in the market.

If the home buyer decides to deposit a margin requirement, a maintenance margin would still be required to protect the lender.  If the equity in the home falls under the maintenance margin, the borrower would be required to deposit additional money to act as margin, or use a hedge to reduce their position.

By creating a lending system that enables this by choice, laws that require this system, or a combination of both, we can separate homeownership from the speculation and risk required to purchase a home for living one’s life and raising a family.

Homeowners will continue to gain equity as they repay their mortgage, and are removed from the tribulations of the housing market.  Homeowners will still have the choice to speculate on the price of their house, but only as long as they can afford to take the loss, and not place the lender in jeopardy.


For this system to be effective, it is necessary that futures markets exist which accurately follow the price of a buyer’s house. Since prices vary by region, multiple index futures would need to exist. A new type of home depreciation insurance could be established to provide protection for an individual house. This insurance would need to cover any potential difference between the broader market that the hedge is based on and the value of an individual house.

This insurance must be designed to protect against only a loss of value due to market decline, but should not provide protection against loss of value due to poor maintenance. Maintenance should continue to be the homeowner’s responsibility, and this may present complications in implementation. A risk would still exist that lenders could lose money due to poor maintenance, but it is not the same level of risk assumed by a widespread decline in the broad housing market that causes huge losses in many homes.  Also, if the loss of value is due to poor maintenance, lenders should be able to pursue repayment from the former homeowner.