I recently bought a Roku 3 media player from Amazon, taking advantage of the latter's free international shipping offer.
This new Roku, the latest in a line of low-priced devices from the company, is essentially a streaming media device that connects to your TV and allow you to watch content streamed over the network. Featuring a dead-simple interface - be it on-screen or in your hands - the Roku makes browsing for and consuming content in the living room virtually effortless. It's the promise of 'Smart TVs' - actually delivered.
Unfortunately, if you are outside the USA, much of the content that the Roku promises to stream is unavailable -- locked away behind geographic restrictions. You can use something like the Plex Media Server to stream your local media to it - and it does an excellent job of that. In fact, you can even treat this tiny device as just the best way to get your Plex content on your living room television.
Aside: My earlier effort at using the Raspberry Pi as a living room media center didn't quite work out due its inability to output surround sound to my home theater.
There's a veritable cottage industry of services that have sprung up around mechanisms that let users bypass these content geo-restrictions. One such mechanism utilizes the DNS system to enable access. In this category are services like Unblock-Us and UnoDNS. I'll just point you to one of these services for an explanation of how this works.
If you read that explanation, you know that it boils down to using these services are your DNS provider. This does come with certain downsides. Specifically:
- You must trust these services not to indulge in malicious DNS spoofing.
- You must be okay with the results returned by these services for the rest of the Internet's domain names.
While #1 in the above list may be dismissed as a paranoid concern, the second is a legitimate, everyday concern. As an example, some of these services delegate to Google's DNS for resolving names that aren't core to their service (i.e. not netflix.com, hulu.com, etc.). I personally don't use Google DNS because I've had issues with them not resolving my work place's remote-access domain to the correct load-balanced server.
Simply put, I'd be loath to use one of these services as my primary DNS.
One nice middle ground would be to configure the Roku to use, say, UnoDNS as the DNS provider and continue using my preferred DNS provider for all the other the devices in my network.
Unfortunately, the Roku provides no way to specify the DNS server to use!
If I had a hackable router, there's some iptables trickery that could be put to use:
iptables -t nat -I PREROUTING -i br0 -s <roku.ip.addr> -p udp --dport 53 -j DNAT --to <dns.ip.addr>
iptables -t nat -I PREROUTING -i br0 -s <roku.ip.addr> -p tcp --dport 53 -j DNAT --to <dns.ip.addr>
The above (disclaimer: untested!) sets up a firewall rule that redirects all requests originating from the Roku device and bound for port 53 (the DNS service port) to a designated DNS server. All other DNS traffic is unaffected.
Alas, I don't have a router that lets me set such custom firewall rules.
Which brings me to Dnsmasq, the solution I settled on. Dnsmasq is a simple DNS server that's suitable for small intranets. In essence, you can configure two kinds of mapping rules in Dnsmasq:
- hostname -> IP address
- hostname -> DNS server
The first kind, obviously, sets a static mapping from a hostname to an IP address. This is the basic DNS functionality and you can imagine have ten to fifteen such rules being sufficient for a small intranet with a dozen computers.
The second kind of rule sets a mapping between a hostname and the DNS server to query to get the IP for that hostname. So if I had a rule like so:
server=/netflix.com/69.197.169.9
It means that when there's a request to fetch the IP address of the domain name netflix.com, forward that request to the DNS server at 69.197.169.9.
That's all I need! I can configure one of the geo-block-bypassing services as the upstream DNS for a list of domains that host the content I am interested in and then use my regular DNS server for all other domains!
Now comes the question of where to run this dnsmasq server. As I mentioned in the aside above, my Raspberry Pi is no longer serving as a media center so I re-purposed it as a super-silent, power-sipping energy efficient Linux server running just dnsmasq!
Here's a quick howto:
- Install Raspbian on the RPi.
- Set it up to use a static IP, say
192.168.0.2
- Install dnsmasq:
sudo apt-get install dnsmasq
- Create a custom mapping rules config file for dnsmasq
- Copy the new rules file to
/etc/dnsmasq.d/
- Setup your main router's WAN settings to use
192.168.0.2 as the DNS server.
That's pretty much it. Restart all devices and check if the DNS changes have taken effect. All devices in your intranet should now be using the RPi as a DNS server. To check if dnsmasq is working correctly, browse to a configured domain (say hulu.com) and check whether you get blocked or not. If you aren't, congratulations!
Because I am still experimenting with the DNS bypass service providers as well as the list of domains that require special handling, I wrote a quick Python script that generates the requisite dnsmasq configuration file. I've shared it in my Bitbucket repository in case anyone finds it useful.
As of this writing, this is the dnsmasq config file that I am using.
Now to figure out what's the most optimum way to populate my Netflix queue!
I had to spend way too much time last night recovering from a stupid mistake made nearly three years ago. A quick flashback: three years ago, my home computer died (mass suicide by the capacitors on the motherboard) and I bought a Dell Inspiron to replace it. As I noted in that post, I popped in an HDD taken from the dead PC into the new Dell and installed Ubuntu on it.
I made one critical error at that point: I installed Grub, the boot loader, on the Win 7 disk instead of the Linux disk. Looking back, I can't even recall if the Ubuntu installer gave me a choice in that regard, but I should've been more diligent.
Fast forward to this week: the Ubuntu running disk started giving SMART errors (not the other SMART) warning me of impending death. Luckily, the disk is still under warranty even under the hard disk cartel's industry-wide reduced warranty period of three years instead of five years.
So in preparation of returning the disk for replacement, I spent some time configuring & preparing for use the long-neglected Windows 7 installation. Windows 7, BTW, is much nicer than I expected; but let's leave that train of thought for another day. The final step in the process was to pull out the Linux disk and restore the Dell to its pure Windows existence.
That's when the three year old mistake came back to bite me:
error: no such partition
grub rescue>
Essentially, the Windows 7 boot loader on the Win 7 disk was gone and replaced by Grub which now couldn't find its second stage loader which was installed on the failing hard disk. In simpler terms, I needed both disks in the PC to boot Windows.
Restoring the Win 7 boot loader ranges from trivial to tricky depending on which side of the retail Windows license holder or OEM Windows license holder line you fall. It is trivial because all you need is a Win 7 CD with which you boot the computer, go into the rescue console and type out a sum total of two commands. It is tricky because Dell, like most other PC manufacturers, subscribes to the logic that it is too expensive to ship a 5$ Windows 7 installation CD with a 1000$ computer.
Microsoft thankfully seems to care about their end users more than Dell and as a work-around to the cheapo behaviour of their OEMs, have added a feature to Windows 7 that allows you to create a rescue or recovery CD from any Win 7 installation. As luck would have it, I still have a few dozen shiny coasters left over from the pre-flashdrive era. I fetched one from storage, popped it into the CD/DVD burner and had a rescue CD ready in a couple of minutes.
But Dell had another card up its sleeves. While the computer seemed to boot from the CD, it eventually errored out throwing the number 0X4001100200001012 in my face. A bit of Googling revealed that this error code seems to appear only on rescue CDs created from Win 7 computers sold by Dell.
Thankfully, some kind souls on the Internet have provided instructions on how to create a USB boot disk using the Win 7 rescue CD. Following those instructions, I was finally able to boot into the recovery environment using a flashdrive and restore the boot loader.
Once Windows was booting properly, I kicked off a full disk erase process using the zeroing feature in Seagate's Seatools utility and went to bed. This morning, I pulled out the dying disk and went down to the Seagate distributor's office to return the disk and initiate a warranty replacement.
TL;DR: be careful where you install Grub!
Software is the only business in which adding extra lanes to the Golden Gate bridge would be called maintenance. -- David Tilbrook
I had to figure this out today to automate some log cleanup:
- find all files in the current directory
- that are older than 30 days
- but not in directories
DIR1, DIR2 or DIR3
- print the matching file names
- and delete them
This is what I came up with:
find . -regextype posix-extended -type f -a -not -regex "^\.\/(DIR1\/|DIR2\/|DIR3\/).*" -a -mtime +30 -exec rm -vf {} \;
I tried doing combinations of -wholepath but it just didn't work. Further, I didn't want to use grep to filter out the directories because there's no corresponding flag in grep to handle the -print0 output from find.
I am currently working on migrating a bunch of shell/perl reporting scripts from Solaris to Linux. Clearly, a trivial task since it's all POSIX ... until it's not!
Some of the reports do things which involve dumping data from one database, importing it into a temp table in another database and then doing further processing of the data there. Normally, this would be done using Sybase's bcp utility; you would bcp out from the source database & then bcp in at the target database. However, bcp out works only on full tables or views. You can't provide a select query and get the output of that query in a bcp in compatible format.
The scripts I was working on were relying on some Solaris executable to dump data from a select query in a format that was compatible for import using a bcp in command. Since I didn't have the source code of this binary (typical!), here's how I reproduced its functionality.
First, consider the isql query:
$ isql -U$USER -P$PASSWORD -S$SERVER -I$INTERFACES -o$OUT_FILE << EOF
select * from SUPERHEROES where WEAKNESS like '%Kryptonite%'
EOF
Which results in:
$ cat $OUT_FILE
ID NAME ALIAS POWER WEAKNESS
--- ----------- ------------- -------------------- --------------
12 Superman Kal-El Flight,X-Ray Vision Kryptonite
13 Supergirl Kara Zor-El Flight,X-Ray Vision Kryptonite
(2 records selected)
But in order for the output to be importable by bcp in, it has to look like this:
12|Superman|Kal-El|Flight,X-Ray Vision|Kryptonite
13|Supergirl|Kara Zor-El|Flight,X-Ray Vision|Kryptonite
In effect, we need to:
- Lose the header rows
- Use a better delimiter instead of whitespace
- Trim the field values of extra whitespace
- Lose the footer rows
For requirements 1 & 2, we can modify the isql statement as follows:
$ isql -b -s"|" -w9999 -U$USER -P$PASSWORD -S$SERVER -I$INTERFACES -o$OUT_FILE << EOF
select * from SUPERHEROES where WEAKNESS like '%Kryptonite%'
EOF
The -b flag removes the header rows and the -s flag asks isql to use the pipe character as the field delimiter in the output. The -w flag is added for good measure: it specifies the width of each output line. The default value of 80 characters is likely to cause each output row to be split over multiple output lines.
The output now looks something like this:
|12 |Superman |Kal-El |Flight,X-Ray Vision |Kryptonite |
|13 |Supergirl |Kara Zor-El |Flight,X-Ray Vision |Kryptonite |
(2 records selected)
The footer can be removed by:
$ head -n -2 $OUT_FILE > ${OUT_FILE}.new
$ mv ${OUT_FILE}.new $OUT_FILE
Update (29/Nov/2012) : Just use the set nocount on option in your SQL query which will skip printing of the footer.
And finally, we are left with the task of removing all the extra whitespace around field values. Here's a quick & dirty Python script that does the job:
#!/usr/bin/env python
"""
A simple utility that takes isql generated output
and converts it into a format suitable for import
via 'bcp in'.
The isql output should ideally be generated using
the following flags:
isql -b -w9999 -s"|"
The delimiter is especially important since this
script assumes pipe ("|") as the field delimiter.
__WARN: The input file will be overwritten!__
Aside: This whole file could probably be replaced
by a one line sed/awk incantation, if I knew how.
"""
import shutil
import sys
DELIM = "|"
NL = "\n"
if len(sys.argv) != 2:
print >> sys.stderr, """Usage:
isql2bcp.py <input_file>"""
sys.exit(1)
infile = sys.argv[1]
outfile = infile + ".new"
in_fp = open(infile,'r')
out_fp = open(outfile,'w')
for line in in_fp:
# Strip each line of leading & trailing DELIM
# as well as trailing newline
line = line.strip(DELIM+NL)
tokens = line.split(DELIM)
# Remove the extra whitespace surrounding each token
tokens = [t.strip() for t in tokens]
line = DELIM.join(tokens)
out_fp.write(line + NL)
in_fp.close()
out_fp.close()
shutil.move(outfile, infile)
The script can then be invoked as:
Resulting in output that can be imported using bcp as:
$ bcp $TEMPDB..$TABLE in $OUT_FILE -c -t"|" -I$INTERFACES -S$SERVER -U$USER -P$PASSWORD
That's it. I hope this will be useful to someone!
