While taking the Coursera “Developing Data Products” course, I developed a tool for comparing performance on standardized tests between California Schools. Today I ported tool to python and published to http://www.dmaust.net/schools/.
As April fool’s day aproaches, I noticed an odd item in my Google Analytics page. After visiting their realtime stats, it appears I have 41 active visitors to my website from the International Space Station.
While reviewing bitly’s API, I cam across a call for “high value” links. http://dev.bitly.com/data_apis.html#v3_highvalue.
After wondering what this this call produces, I made a test script to pull the current top thirty high value links and display them in a human readable format. The results are available at http://dmaust.net/cgi-bin/hot_bitly.py.
In the past I’ve used a hive CSV SerDe originally from bizo but one complaint that often arose is some operations failed due to type errors. The original SerDe produces strings objects for operations. This leads to odd issues with various hive operations. Due to this, I updated the hive SerDe to support integer types as well. The fork is on github at dmaust/csv-serde.
While comparing hosting options I decided to record unixbench benchmarks on a few of the servers and other machines of which I have access. My results are available on my github repo as well as tests from others. https://github.com/dmaust/unixbench-results
It is common after launching a new instance in EC2 to assign a domain name to it. A useful practice I have found is to create a CNAME to the amazon provided public hostname. This address will resolve to the internal address when inside EC2, and to the public address outside EC2. By using this hostname, you can ensure the best performance and lowest cost by always hitting the server with the optimal address.
This Y-Combinator article <http://news.ycombinator.com/item?id=1460725> says it best. I have used EC2 since 2009 and the on-demand price of an m1.small has started at $0.10/hour. Since the performance of the instance has seen little change, you would expect after one and a half years have elapsed the price to have fallen to 0.05/hour, and after 3 years $0.025/hour. Instead the current price is $0.065 per hour.
When I initially subscribed to EC2 it was a great deal. Now the pricing is becoming excessive. With the advent of OpenStack, I see Amazon being forced to lower their prices.
I have just posted my Mining calculator. Just put in your hash rate (in Mhps), and it will calculate your expected revenue.
This Thursday Marc Bevand (Bitcoin miner and GPGPU expert), Stephen Gornick (author of bitcoinminer.com), and myself be speaking on Bitcoins as part of a panel at the UNIX Users Association of Southern California. It should be interesting, especially after the recent events in Bitcoin security due to the events at MtGox.
For more information visit: http://bolthole.com/uuala/
Recently it has become more pressing to migrate to IPv6 due the diminishing IPv4 address space. Because of this, services like Hurricane Electric are growing in popularity to allow IPv6 tunnels to be created for various applications.
Typically a website owner who wishes to provide IPv6 connectivity has had two choices:
- Use a web host that has native IPv6 connectivity.
- Use a tunnel brokering service to bring in IPv6 connectivity to their server over IPv4.
While these are great approaches for many users in an dedicated hosting environment with flexible firewalls, there are many situations where a tunnel is not permitted. Many smaller sites are on shared servers in which custom network configurations are not permitted. Also some hosts utilize firewalls too restrictive for IPv6 tunnels. One example of this is Amazon Web Service’s EC2, in which only TCP, UDP, and ICMP can be opened. In such an environment, most tunnel broker’s solutions fail.
For these cases I’ve introduced a new solution. Ipv6MadeSimple.com will allow IPv6 connectivity by forwarding HTTP requests to the existing server. It supports X-Forwarded-For and X-Real-IP headers which allow the backend web server to identify the real IP of the users. Also it supports a dynamic HTTP Host field for your site, so the solution will work in virtual hosting systems in which only the hostname is unique.
The service is still in beta and is providing service free during the beta. The only requirement is the ability to modify DNS records so that an AAAA record can be added.