Local File Repository with .htaccess

So, I run a web server.  I do not pay for hosting.  Maybe one day when I’m rich and famous I’ll have the need to offload my computing and security needs to a third party, but for now it’s the joy of having full control over the hardware that both feeds and permits my curiosity of technology.  Hey, the title of this post was not misleading.  If you’re actually reading this, then you must share some of these interests.

With all web-accessible content comes the need for access control.  Normally I handle this through the operating system’s administration panel, but a need arose in which this wasn’t as practical as I had hoped.  Here’s why:

  • No one can ever remember their login credentials
  • The web GUI is processor-intensive, and therefore slow (especially on mobile devices), leading to user impatience
  • the web GUI doesn’t play nice with mobile OSes in general
  • Mounting network shares is a lot of trouble for a single file (also: see the first bullet)
  • Access control management is a pain, especially when it’s a new user who doesn’t necessarily need access to the server for anything else
  • The files were meant to  be shared with anyone on the LAN, who presumably would already have been authenticated by me or else they wouldn’t be on my LAN in the first place
  • When a file needs to be downloaded, and the client doesn’t need to upload anything, few methods are easier and more universal than good ol’ HTTP

Based on these limitations and my needs, I determined the best solution was to create a file repository that was devoid of separate access control, restricted to the local LAN.  Only people on my LAN could access the files, and any people on my LAN by default would have de facto permission to access them (and not those on the guest subnet).

Fortunately from experience, once I identified these needs, I knew of the solution, though it did take a little research.  Any web server has individual configuration files which can be applied at the directory level:

  • IIS has “web.config”
  • NGINX has “nginx.conf”
  • Apache has “.htaccess”

apacheNatrually I would never be caught dead using IIS, although I was forced to use it for a prior job.  But my server, Linux-based of course, leverages NGINX with an Apache backend.  I had a working familiarity of Apache, and I had already dabbled with .htaccess and .htpasswd files before, as well as modifying the Apache config files to allow their overrides, so this seemed like the best option.

Still with me?  Okay good.  I created a new directory “/public_LAN/” and with the server’s own text editor, created the directory’s own .htaccess file.  And my god why do OSes have to be so difficult with non-standard file extensions?  I know why: some idiot will mess with a critical config file or open malware, but why can’t I turn it off?  I used to be able to edit any file type I wanted with older Apple OSes, but it seems that now it’s forbidden completely.  So no, I couldn’t just open my HTML editor and save a text file as .htaccess because that’s an usupported extension.  Whatever.

Into this file I placed:

Options +Indexes
ErrorDocument 403 “<h1>403 Forbidden</h1><p>This page is restricted to internal LAN access only:<br><a href=’http://192.168.0.106/public_LAN/’>192.168.0.106/public_LAN/</a></p>”
order deny,allow
allow from 192.168.0.
allow from 10.8.0.
deny from all

BAM!  Okay, triumphant interjection aside, what does this mean?  I will explain, else I risk bastardizing the value of this post:

Options+Indexes : This command enables directory browsing.  Web servers always have this off by default for security reasons, but since I was going to use the directory for the very purpose of browsing files within, I needed to turn it on.  This is how you do it.

external
Attempted external access

ErrorDocument 403 “<h1>403 Forbidden</h1><p>This page is restricted to internal LAN access only:<br><a href=’http://192.168.0.106/public_LAN/’>192.168.0.106/public_LAN/</a></p>” : This is optional, but it adds a custom 403 error page (for this directory only).  In short, mine says that if it’s triggered, the user isn’t inside the LAN and therefore can’t go there.  Attentive readers will notice that I neither link to a TLS connection, nor use the domain name.  More on this later.

order deny,allow : This sets the precedent that all access will be denied by default first, then checked for conditions under which access is allowed.

allow from 192.168.0. : This line is the condition under which access will be granted.  It is the first 3 blocks of the main LAN IP address.  This includes any client IP address that begins with these 3 blocks–which will be anything on my LAN (excluding the guest network).

allow from 10.8.0. : This is the second set of LAN IP addresses to allow.  In my case, this is the subnet for anything connected to the LAN via VPN.  I wanted this available to VPN clients too as the VPN is handling the authentication and encryption parts already for any remote access.

deny from all : Finally, any client that doesn’t meet the above conditions will be denied access.

public lan
Old-school HTTP directory listing

Okay, now the two elephants in the room, and all the technical babble.  First is the lack of encryption.  Ultimately I determined that this wasn’t necessary, as any file access would be strictly over the LAN.  If there are untrusted devices on my primary network, then I have bigger problems to deal with.  Also, I can’t service HTTPS without a domain name being used for the connection, since no Certificate Authority will issue a valid TLS certificate to a private IP address, so I’d have to use a certificate that won’t pass a browser’s domain name validation–in itself not a problem, but then it warns the client of a potential security risk, which the client may not understand, thus inciting panic and undermining the entire point of this project–seamless ease of use.  Also, as mentioned before, any remote access will be tunneled through a VPN, so any data that makes it to the outside web will be encrypted anyway.  Second, domain name validation isn’t possible for the above reason, but also because I can’t access this directory via the domain name anyway (okay, I can, but only by local IP–VPN clients still perform a DNS lookup for the host IP, making the client appear to the server that it’s outside the LAN), or the server will see the request coming from the WAN IP, and not the LAN IP.  Therefore, it will block the request.  I could add the WAN IP to the whitelist, but it’s not static and if it ever changes I’ll have to update the .htaccess file again.  Also the authentication side of domain name validation is moot while accessing via LAN IP, as it won’t be feasibly faked unless some rogue device is attempting ARP spoofing–again something which, if happening, means I have bigger problems to deal with.  Whew, done.

Obviously this isn’t high-end security, but it’s reasonably effective.  I wouldn’t use this method to conduct crime, but if I need to say, give a somewhat sensitive file to a guest and it’s too big to email, and I don’t want that file publicly accessible, then this is a pretty good solution.  Keep your data safe!

–Simon

Spam and Botnets

Remember those days of Nigerian princes and overseas lotteries?  The ones who just needed a little bit of financial assistance, who would reward you in turn for your efforts with profit a hundred fold?  Or the cheap Viagra?  Or the young Asian girls who want to meet just you?

Nigerial Lottery
This image isn’t racist, is it?

I’d like to sigh nostalgically and say “Those were the days” except, apparently these are still those days.  Something on the Internet has survived multiple decades.  Go figure.

I run my own email server, and in so doing, need to open certain ports in order to receive email.  One of these ports is port 25–the Simple Mail Transfer Protocol port.  In other words, it’s the default port upon which email moves.  Now, in order to receive most email, I have to open this port, even though I don’t generally use it for my own purposes, preferring newer TLS-by-default port 465, among others.  Technological details aside, I only have port 25 open by necessity, and I don’t use it myself.

But, because it’s universal, botnets continually scan the Internet for servers with this port open.  With modern computational power, it takes a surprisingly short amount of time to scan all the available IPv4 address space.  Consequently, I’m regularly identified as a host with open port 25.

What does this mean?  Generally nothing, except these automated botnets hope that I haven’t bothered to take basic precautions.  Upon seeing the open port, the botnet then attempts to log in, using various default credentials (e.g. Admin, User, root).  Very quickly they move on, but still, I find this irritating.

Unfortunately there isn’t much I can do about it, other than blacklisting by default all non-US IP addresses (and any countries to which I’m aware family is currently traveling), and any IP address which previously failed to log in.  But, there are still a lot of IP addresses.  And with no recourse, I decided to vent my frustrations by posting a list of offenders.  It is worth a moment to do a Whois and find their geographical regions, if nothing else.  And if one of these is you, it’s time for a malware scan:

198.12.93.218
198.23.132.250
205.234.153.210
5.39.219.214
46.166.160.153
193.189.117.88
155.133.18.178
23.95.24.162
46.105.120.50
151.80.147.113
212.129.4.178
151.80.147.144
38.87.45.116
52.22.59.41
209.95.52.130
80.11.96.236
166.176.251.239
195.154.116.169
96.43.128.14
195.154.119.141
195.154.105.115
50.116.123.186
104.238.129.26
118.193.179.177
195.154.110.230
122.224.248.250
203.171.31.60
31.170.104.245
220.244.5.154
111.204.219.197
175.100.189.174
111.68.98.136
180.250.9.52
177.39.152.250
59.127.51.128
184.74.44.51
173.189.252.21
50.252.84.9
70.15.249.139
173.164.154.100
69.199.239.200
63.223.116.37
173.13.117.142
71.10.87.50
23.246.213.202
104.238.141.153
104.168.145.83
51.255.235.154
104.168.141.86
107.179.40.46
45.76.81.226
23.254.215.249
46.218.164.132
96.255.34.171
138.197.1.145
195.154.103.205
195.154.77.202
62.210.25.5
74.113.139.17
23.254.211.205
176.183.204.200
65.245.57.3
192.86.34.108
45.32.203.111
144.217.213.132
66.194.234.110
207.118.200.111
185.81.158.149

–Simon

WiFi Woes

WiFi sucks.  I mean, it’s awesome, in theory, but it kinda sucks, though I can’t think of an appropriate metaphor to explain why.  Let’s say, it’s trying to take a drink from a sprinkler when a crowd of people are allWiFi gathered around it and fighting.  Now let’s compare that to, say, a water tower, where everyone in that same crowd gets their own spigot.  Everyone attaches a hose, and everyone’s happy.  This latter explanation is wired Ethernet.

Okay, that was a horrible explanation, and I hate analogies.  Analogies are a means to add context to an explanation when parties involved don’t have the prerequisite knowledge, but they always end up sounding like a twisted politician’s manipulative words.  Anyway, as with all technical explanations, there is no shortage of people on the Internet who can explain this better, so go Google the background information if you need.  As usual, my post is anecdotal, because, overly-researching a project adds to the complexity, which tends to dissuade me from even attempting it.  So instead, I will simply tell you that the following project is reasonably straightforward and even an amateurish attempt will reap huge benefits.

With WiFi, you have a much more restrictive set amount of bandwidth, and that bandwidth is shared by all devices on that particular hotspot.  But wait–it gets better.  That hotspot is also sharing that particular range of radio spectrum with every other hotspot in range, and any other device licensed to operate within that spectrum, which is just about every wireless consumer appliance.  So in practice, the general throughput you could ever hope to achieve with a hotspot is roughly a third of the theoretical maximum.  1300 Mbps?  More like 450.

Ubiquiti
A pretty commercial hotspot

Also, ironically, the main point of using WiFi is for mobility, and a major problem with WiFi is in its mobility.  When the connected device moves, it experiences latency.  When the device switches between hotspots on the same network, there’s a delay while it renegotiates a connection.  In my house, in order to cover the property in its entirety, I have 3 hotspots.  Adding to the complexity is the changing technology, while maintaining legacy support, so I have 802.11b-ac, on both 2.4GHz and 5GHz bands.  And, while this is all done transparently, it still experiences delays.  There are commercial products that claim to handle the transition better, as do consumer “mesh” WiFi products, and I’m sure they do, but you pay for it, and for consumer use I just don’t yet find the price point cost-effective.

Additionally, mobile devices also connect to cellular data service.  That means, when roaming between hotspots, the device also has to to consider whether it’s going to connect to a new hotspot, or the cellular service.  Quite often I walk from my living room to the garage, go to load a podcast, and the phone says something like this:

‘Oh, we’re walking out of range of that hotspot now, guess I’ll switch to cellular, hang on a sec.  Oh wait!  There’s another hotspot here, but I already started negotiating an IP from the cellular network.  Okay, hang on a sec, I’ll connect to this other hotspot.’

Of course, this takes only seconds, but that’s enough of an irritation when in the middle of loading something to rouse ire.  Plus, in my house, whenever the Internet isn’t immediately accessible for whatever reason, it’s my fault.  Worse is when the connection drops while already involved in something.  I pray every time my wife loads up a Destiny raid that the connection remains stable.

And sometimes, when a device boots or wakes from sleep, it can stall while deciding on where to connect.  Such was the case with the Apple TV in our bedroom.  So after months of dropped connections, despite having installed a new hotspot 10 feet away, I did what I had been gradually doing throughout the house:  I installed an Ethernet jack.

And this is where one would ask: “Simon, aren’t there hundreds of online guides from people with far more experience than you on how to do this?”  And I would answer yes, there most certainly are.  But unlike those people, this type of work is not relevant to my career, so the observations I offer are free of prejudice, as they represent no meaning to my ego.  So, no lengthy arguments here about industry standards.  If that’s what you’re after, shoo.

Let us begin.  I will explain what I did and why.  First, I acquired a 1000 ft box of CAT6, UTP, CMR, solid core Ethernet cable.  Here is why I chose this:

  • 1000 feet is a standard unit to purchase, easy to find, and economical in price.  It seemed like a good place to start, based on my estimates.
  • CAT6 is the most recent official rating for Ethernet cable.  The price difference for the quantity I was purchasing was negligible, so logic dictated that I buy the more modern cable.
  • UTP.  This means it’s unshielded twisted pair.  Technically, all Ethernet cable is twisted pair, so whatever (phone lines are twisted pair, so the tech is old, although whoever installed the land lines in my house just draped individual pairs of wire willy-nilly through the basement–that couldn’t have been very good audio fidelity).  As for the shielding, well, that’s usually reserved for niche applications.  I didn’t even see shielded wire available during my search.
  • CAT6
    This cable has no idea how much data it’s going to carry

    CMR.  This is the rating on the insulation.  It means it’s rated for riser applications, so safe to run up through dead space into walls.  The rating is for fire-retardant purposes.  Plenum wire is designed for air ducts.  For residential applications, I wasn’t concerned.  If the basement is on fire to the point that the wires are burning, I’ll have bigger problems than how quickly the fire will eat through the wire.  Google is your friend if you want the details.

  • Solid core wire is the standard for Ethernet runs.  Banded wire is used for patch cables.  Basically, wire that doesn’t move should be solid, and wire that connects devices to jacks should be banded.  Solid can break from repeated bending, apparently.  I didn’t discover this until I had already crimped patch wires for every wired device.  I suppose time will tell if this was a bad idea or not.  But if wires break, I can always buy new.

There, that’s settled.  There is no reason to further discuss wire types.

Running the wire was straightforward, and the boxes are designed for easy spooling.  Simply place the box at one end of the run and pull it as needed.  This comes up a lot so I’ll mention it: use no more than 25 pounds of pulling force.  I suppose this means that everyone has a very accurate sense of quantifying force?  I guess just avoid using it to swing like a monkey from the rafters and you should be good.

up
Stupid support beam

I found CAT6 passes easily through a 3/8 inch opening, so drilling is pretty simple.  I also live in a ranch with a full basement, so all I had to do was measure carefully, then drill up into the wall-space from below.  I can’t imagine how difficult it must be to fish this stuff through multiple stories of drywall.  You’ll see in the photo where I hit the damn support beam and had to re-drill.  Also the drill battery died and I had to borrow the neighbor’s.  Why is a project never simple?

Once through, I drilled a rectangular hole for the box with a Dremel.  CAT6 is also stiff enough that I was able to simply reach into the wall space with a hook and pull the wire out.

punch
Don’t do it this way

The next step was to punch down the wires into the jack.  This was wonderfully simple–just follow the supplied instructions and use the included tool.  Choose either the A or B configuration.  Apparently B is the US standard, but A has minor advantages over B.  Since I was wiring everything myself, I figured I’d go with A then.  The important thing is to stick with either A or B once you start.  Also, I discovered later that you’ll want to leave the wires twisted as much as possible, and that the wires are supposed to feed from the inside of the jack punches to the outside, not the other way around as I did in the photo.  Maybe one day I’ll re-punch properly, but the line tested at 780+ Mbs, and seeing as the Apple TV only has a fast Ethernet NIC (100 Mbs maximum), there seemed little point to fuss over technique.

The other end was then terminated at the router.  For simplicity, I just crimped it down into an RJ45 plug, though maybe one day I’ll get a proper patch panel installed.

Final notes:

If possible, use a dedicated Ethernet switch, rather than the router’s internal switch.  Chances are the router’s switch doesn’t have as much resources dedicated to the switch part.  Also, I was going to run out of ports anyway.

Every time the connection hops between devices, you lose throughput.  I opted to only run a single wire to each drop, and then connect another switch to give me the ports I needed.  Professional installations use one main switch and run however many wires are needed to each drop.  I didn’t want to drill that many holes, or deal with that much wire.  But the consequence was that my 760 Mbs connection dropped to a little more than 450 when I tested a file copy to the NAS, which sent the signal through 2 switches.  Offhand it looks like you lose half of the throughput for each jump, but that’s only based off of two hops so I don’t know how that actually scales.  Still, each device is guaranteed that much static bandwidth, and half a gig isn’t too shabby.

data center
My aspiring data center

Despite the drawbacks, the connections are solid.  I’ve even taking to plugging my work laptop into the Ethernet to speed up the VPN, with much success.  And besides, all this tech does look damn cool.

–Simon

Password Entropy

Passwords, ugh.  The very word causes pain.  It invokes feelings of aggravation and despair, memories of fighting computers and IT admins.  And still, despite their flaws, we have yet to universally assign any other means of simple authentication, so we’re stuck with them.

And, we constantly argue over what makes a good password.  In the midst of this debate, one man and his famous comic surged through the internet:

https://xkcd.com/936/

XKCDIf you don’t know of XKCD, shame on you.  Go there now and revel in its wonderfully sophisticated humor.

In summary, the argument’s premise is that words, being easier to remember, are better suited for passwords as their method of authentication relies upon human memory.  And indeed they would be, but it would take significantly more words than could feasibly fit into a password field.  Why?  Because of dictionary attacks.

A dictionary attack works by guessing known words.  Even if the words are obscure, they are known.  I will elaborate:

For this example, I will use the word hello.  Hello is 5 letters.  The logic behind using words for passwords is their per character entropy.  5 letters, all lowercase, represents 26^5 possible combinations, or 11,881,376.  With dictionary attacks, however, the word in itself becomes a single “character”.  According to the Oxford English Dictionary, there are 171,476 in-use words in the English language.  This translates a password, consisting of one word, to having one of 171,476 possibilities–significantly less than the 11,881,376 previously mentioned.

So why not stack random words?  Well, in the comic’s given example of correcthorsebatterystaple, there are 4 words.  171476^4=864,596,308,417,753,000,000 (approximately, since Excel is truncating numbers to 15 significant figures).  So we’ll say 8.65E+20 (using Excel notation).  How secure is this?  I honestly have no idea.  This is where the argument turns ugly.  So I will pass on forcing an opinion upon you and instead stick to providing information.

Looking at the ASCII chart, there are 95 usable characters.  Taking 95^X then, the tipping point is 11 characters.  This, compared to the above example, has 5.69E+21 possible combinations, significantly more entropy, yet significantly fewer characters (11 vs. 25).

The argument then would be to add more words to the password.  And I would agree, except all too often we encounter password field limits.  And besides, how many random words for how many websites could you remember anyway?  Once you fail to remember one, you completely lose the benefit of the word method, in which case why not make a higher-entropy password instead?

Searching the Internet for consensus on password size, I find the general rule is to use 12 characters.  This equates to 5.40E+23 combinations for ASCII, which means we need five words to achieve at least that number of combinations with the word method.  That’s a lot of words to remember for every website.

Another point that bears mentioning is that we need to consider the lowest possible entropy denomination, so word length does come into play, although not significantly.  Specifically, a word has to be at least 4 letters long, otherwise its number of combinations falls below 171,476 (26^3=17,576).  Therefore, if you think you can get away with stringing together 5 short words, you’re only getting the combined strength of the letters themselves, meaning you’d need 17 letters to at least meet the entropy of a 12-character ASCII password.  And remember, you don’t get more entropy by using longer words, so correcthorsebatterystaple is 8.65E+20, not 2.37E+35.

Entropy
We’re gonna need a bigger column

And no, mixing lowercase and capitalized letters, or even number substitutions, does not impact a word’s entropy in a meaningful way, as dictionary attacks are aware of this trick.

Finally, the word method is assuming any word in the OED might be used, when in practice there are estimated to be only 100,000 common English words.  I was erring in favor of the word method, but in practice it’s much weaker than this math suggests.

Conclusion: in order to supply enough random words to a password chain to achieve the minimum industry-recommended level of entropy, you would need to supply 5 uncommon words, which will likely defeat its own purpose of being memorable, not to mention it will likely exceed the password length limit of many servers.

Whatever method you choose to use, I think it’s safe to say that we can all agree that passwords just plain suck, and with the exponentially increasing computational power of Moore’s Law, it’s only going to get worse.

–Simon

Doppler Radar Website

If you’re like most people, when you decide to check the weather, you look at a forecast, likely supplied by your mobile device.  If you’re more inquisitive and want a visual aid, you visit a website and check the radar.  That probably satisfies your curiosity and you quickly move on to significantly more important aspects of your life.

Doppler
This–this sucks

But if you’re like me, you become irrationally irritated because, well, there seems to be no good way to view the radar.  Chances are, if you go to your local news’ website, they have a poor quality, buggy, slow to load page with the current national or regional radar.  Why this is so difficult to pull off baffles me.  It’s probably due to the extraneous features that no one will ever use, like the option to view the last 24 hours.  Who the hell cares what the weather was like 24 hours ago?  Also there’s probably a plethora of scripts that have to load on the page, adding to the already script-laden overhead from 3rd party served advertisements.  Often do I miss the days when websites contained text, images, hyperlinks, and nothing else.

But nostalgia aside, I considered what I wanted such a radar page to do.  My list was short: I wanted it to show the current radar as of the last update and not the interactive history, not the last 30 minutes that always cycles through on infinite loop with no way to disable.  I wanted nothing else on the page, and I wanted it to take up the full page.  Finally, I wanted it to update as the new radar become available.

NOAABut first things first–and a question that always bothered me: where do news stations get their national radar, because surely that has to be a shared network?  Fortunately, it didn’t take long to figure out.  If it was a nationally-shared network of data, then it had to be government-owned.  It had been around too long and was too big for it not to be government-controlled (look at where national internet access is headed).  I didn’t even have to Google this, for I already knew of this governmental entity: the National Oceanic and Atmospheric Administration:

http://www.noaa.gov/

Digging through this site, I discovered that the national Doppler radar was indeed freely available for public access, and updated automatically:

https://radar.weather.gov/

As I write this post I notice that it’s even https now, go figure.

I found the image I was after, the national real-time Doppler image:

https://radar.weather.gov/ridge/Conus/full.php

But, problems still remained.  For one, the image is too large for a normal browser window.  Also it contains the NOAA logo that uses up space.  And, the image is static and doesn’t update unless the page is refreshed.

Fortunately, with a little bit of web-development know-how, it is quite possible to link to the image externally, as the gracious NOAA has not disabled image-linking.  Looking at the page’s source code, I saw that the radar image was delivered from this URL:

https://radar.weather.gov/Conus/RadarImg/latest.gif

Easy-peasy.  I simply just inserted that image with one line of code into my own page’s body:

<p align=”center”><img src=”https://radar.weather.gov/Conus/RadarImg/latest.gif” height=”650″ width=”1157″ align=”center”></p>

I also centered and resized it to my liking.  Then it occurred to me that I really don’t need to see the national radar.  I was more interested in the regional radar.  This became slightly more involved, but still doable with a little CSS, so I inserted this little bit of styling information into my page’s head, forgoing adding the image to the body:

<style type=”text/css”>
body
{
background: url(“https://radar.weather.gov/Conus/RadarImg/latest.gif”) no-repeat left center white;
background-size: 4500px 2400px;
background-position: -2500px -550px;
}
</style>

I set the image to be the page’s background, then through trial and error resized it to zoom, then offset it so as to center approximately upon the Midwest.

One last problem remained, and that was the automatic updates.  This was solved with a simple meta refresh:

<meta http-equiv=”refresh” content=”60″>

Now the page refreshes every minute, which automatically pulls the latest radar image at that time.

Now, at long last, I can have real-time Doppler radar imagery delivered without fluff, and automatically updated.  Plus, if I put the window in kiosk mode, it covers the entire monitor so I can feel like a meteorologist!  Or…old man, depending on who sees me doing this.

Here are the sites if you wish to use mine and forego any coding:

https://moorheadfamily.net/weather.html

https://moorheadfamily.net/weatherohio_kiosk.html

Now place a wireless thermometer next to your monitor and you won’t even have to open the curtain to determine the weather is not to your liking!

–Simon