Installing Macports and md5deep on OSX

When it comes to installing packages and you have to make a decision between MacPorts or Homebrew I first went and visited their home pages (linked previously) and couldn’t make a decision based on what I saw to go either way. I didn’t want to install both, so I googled a bit more. I suggest you do the same since I don’t want people doing something JUST because I said so, go make your own decision :P

Having said that, I based my decision primarily on these points:

  1. the number of packages available in MacPorts (almost 25K) vs Homebrew’s (3K)
  2. the ease of Apache/PHP updates
  3. I’m comfortable with the command line and file/system permissions

Why do I want to install macports?

Because I’m a dunce at compiling, configuring, making stuff myself. And I’ve wanted md5deep for too long.

Now to install all I typed was:

sudo port install md5deep

and it was done.

It really was that simple.

ZFS on Ubuntu 14.04

I’ve completed a fresh install of Ubuntu on an old box that use to contain solaris. The system disk died and upon installing a new (old) drive and then installing ubuntu, I found that two of the disks still in the computer were part of a zfs raid. I’m not sure how many disks were in the raid, but I’m curious as to what was on this file system that has been shutdown in inaccessible for more than 7-8 years. (I later confirmed that most files on the system are from 2008 and earlier).

There were only 4 available sata connectors to the board. Two were used for the drives in there, it didn’t take long to find the matching drives to plug in.

Installing ZFS

I began with installing zfs by following the instructions here:
Install ZFS on Ubuntu—Server as Code

Installing SSH Server

I also had to install an SSH Server because this is on a box located remotely. Follow any generic install of Open SSH Server. The one I used was SSH/OpenSSH/Configuring.

Because my server is behind a firewall and not publicly accessible, I haven’t worried too much about logon via SSH Keys. I have done it before, but this box is only going to be a temporary install, but I do recommend you do that. A couple of other related tutorials to passwordless ssh key access to servers:
https://help.ubuntu.com/community/SSH/OpenSSH/Keys << a good resource, includes troubleshooting
http://www.mccarroll.net/blog/rpi_cluster2/index.html
https://www.howtoforge.com/tutorial/ssh-and-scp-with-public-key-authentication/
https://www.raspberrypi.org/documentation/remote-access/ssh/passwordless.md

Installing Samba

I also installed samba, following these instructions How to Create a Network Share Via Samba Via CLI (Command-line interface/Linux Terminal) – Uncomplicated, Simple and Brief Way!

With SSH, Samba and the ZFS modules installed, configured and running… let’s try and rebuild this raid :)

Let’s have a look!

disks:

lsblk
NAME                         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                            8:0    0 111.8G  0 disk 
├─sda1                         8:1    0   243M  0 part /boot
├─sda2                         8:2    0     1K  0 part 
└─sda5                         8:5    0 111.6G  0 part 
  ├─ubuntu--vg-root (dm-0)   252:0    0 110.6G  0 lvm  /
  └─ubuntu--vg-swap_1 (dm-1) 252:1    0  1016M  0 lvm  [SWAP]
sdb                            8:16   0 698.7G  0 disk 
├─sdb1                         8:17   0 698.6G  0 part 
└─sdb9                         8:25   0     8M  0 part 
sdc                            8:32   0 698.7G  0 disk 
├─sdc1                         8:33   0 698.6G  0 part 
└─sdc9                         8:41   0     8M  0 part 
sdd                            8:48   0 698.7G  0 disk 
├─sdd1                         8:49   0 698.6G  0 part 
└─sdd9                         8:57   0     8M  0 part 
sde                            8:64   0 698.7G  0 disk 
├─sde1                         8:65   0 698.6G  0 part 
└─sde9                         8:73   0     8M  0 part 
sr0                           11:0    1  1024M  0 rom

and for pools specifically:

$ sudo zpool import
   pool: solaraid
     id: 10786192747791980338
  state: ONLINE
 status: The pool is formatted using a legacy on-disk version.
 action: The pool can be imported using its name or numeric identifier, though
	some features will not be available without an explicit 'zpool upgrade'.
 config:

	solaraid                                       ONLINE
	  raidz1-0                                     ONLINE
	    ata-WDC_WD7500AACS-00C7B0_WD-WCASN         ONLINE
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT         ONLINE
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT         ONLINE
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT         ONLINE

Ok, this PC only had 4x SATA drives and it appears I’ve found the correct drives. Things are looking good from the start.

Let’s do it!

:~$ sudo zpool import solaraid
:~$ sudo zpool status
  pool: solaraid
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Wed Nov 11 21:37:59 2015
    11.5M scanned out of 1.63T at 1.15M/s, 412h58m to go
    2.60M resilvered, 0.00% done
config:

	NAME                                           STATE     READ WRITE CKSUM
	solaraid                                ONLINE       0     0     0
	  raidz1-0                              ONLINE       0     0     0
	    ata-WDC_WD7500AACS-00C7B0_WD-WCASN  ONLINE       0     0     0
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0     0
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0     0
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0     2  (resilvering)

errors: No known data errors

YOUCH! 412HOURS… That’s 17 days! I gave it a couple of seconds to stabilise and ran it again, and came up with an error:

:~$ sudo zpool status
  pool: solaraid
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-9P
  scan: resilvered 2.60M in 0h0m with 0 errors on Wed Nov 11 21:38:24 2015
config:

	NAME                                           STATE     READ WRITE CKSUM
	solaraid                                ONLINE       0     0     0
	  raidz1-0                              ONLINE       0     0     0
	    ata-WDC_WD7500AACS-00C7B0_WD-WCASN  ONLINE       0     0     0
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0     0
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0     0
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0     2

errors: No known data errors

Reboot and Ubuntu bootup fail

(long winded fluff about nothing, you can scroll down to “Resilvering” if you’d like to skip this)

I do recall I had issues with this raid, I could certainly go the route of upgrading it first and trying it again, but once I did a

tree -L 2 /solaraid/

and seeing things I had long thought were gone, I’m going to back this up first :)

The only problem is, the raid is installed in a box with only a 10/100 network card :(

I’ll let it run overnight taking off only what I need, and see how we go. This has been a good find

At this point I was operating in the house and the server is located off-site. I had several ssh/terminal windows open to the box and as I was working away I kept getting the message to reboot the system. I issued the relevant reboot command and set off a ping to tell me when it came back online… It didn’t come back online.

I went to the server and found it was still booting. This was after more than half an hour and eventually it gave up and crashed and restarted again.

For the next couple of hours I could not get the drive to boot and I was blaming the old boot drive, but after eventually getting into the “Try Ubuntu” mode from the DVD I found that one of the drives were not being reported in the system. Another was coming up as totally unknown and two were seen as part of a set. It took several hours to get to the bottom of it. Eventually thru the BIOS I could see one of the drives weren’t being detected.

A couple of sata cable changes and swapping power cables around and I was back in business.

Resilvering

:~$ sudo zpool status
  pool: solaraid
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-9P
  scan: scrub in progress since Thu Nov 12 03:12:54 2015
    1.73G scanned out of 1.63T at 32.3M/s, 14h40m to go
    12.5K repaired, 0.10% done
config:

	NAME                                           STATE     READ WRITE CKSUM
	solaraid                                ONLINE       0     0     0
	  raidz1-0                              ONLINE       0     0     0
	    ata-WDC_WD7500AACS-00C7B0_WD-WCASN  ONLINE       0     0     0
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0     0
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0    13  (repairing)
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0     0

errors: No known data errors

Over the next few minutes I kept polling the status and it was picking up speed.

  scan: scrub in progress since Thu Nov 12 03:12:54 2015
    24.3G scanned out of 1.63T at 84.5M/s, 5h31m to go
    12.5K repaired, 1.46% done
  scan: scrub in progress since Thu Nov 12 03:12:54 2015
    32.1G scanned out of 1.63T at 97.0M/s, 4h47m to go
    12.5K repaired, 1.93% done
  scan: scrub in progress since Thu Nov 12 03:12:54 2015
    56.9G scanned out of 1.63T at 102M/s, 4h29m to go
    12.5K repaired, 3.42% done
  scan: scrub in progress since Thu Nov 12 03:12:54 2015
    184G scanned out of 1.63T at 137M/s, 3h4m to go
    184K repaired, 11.01% done

It’s starting to slow down again, and we’re seeing more errors!

  scan: scrub in progress since Thu Nov 12 03:12:54 2015
    195G scanned out of 1.63T at 93.6M/s, 4h28m to go
    354K repaired, 11.68% done

The next day and something had gone wrong. I’m still unsure what happened, but the whole `solaraid` drive became unresponsive.. Where it had got to the evening(/morning before) at 195GB in the resilvering is where it was when I checked it later today. And the drive was otherwise not responding. I remotely tried to reboot and again it hanged.

At this present time, I’m still putting it down to hardware, but it’s really unknown what’s at the root of the problem.

It’s been back up and running for a while now and it’s present status is:

  scan: scrub in progress since Thu Nov 12 03:12:54 2015
    1.41T scanned out of 1.63T at 160M/s, 0h23m to go
    1.23M repaired, 86.89% done
config:

	NAME                                    STATE     READ WRITE CKSUM
	solaraid                                ONLINE       0     0     0
	  raidz1-0                              ONLINE       0     0     0
	    ata-WDC_WD7500AACS-00C7B0_WD-WCASN  ONLINE       0     0     0
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0     0
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0   251  (repairing)
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0     6  (repairing)

errors: No known data errors

When I captured the above, I hadn’t realised the process was almost finished until I pasted and read over it.

  scan: scrub in progress since Thu Nov 12 03:12:54 2015
    1.62T scanned out of 1.63T at 158M/s, 0h0m to go
    1.29M repaired, 99.46% done

As I write this the process has been running for exactly 24 hours. We have 0.5% left.

The final capture:

:/solaraid$ sudo zpool status 
  pool: solaraid
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-9P
  scan: scrub repaired 1.29M in 12h0m with 0 errors on Thu Nov 12 15:13:05 2015
config:

	NAME                                    STATE     READ WRITE CKSUM
	solaraid                                ONLINE       0     0     0
	  raidz1-0                              ONLINE       0     0     0
	    ata-WDC_WD7500AACS-00C7B0_WD-WCASN  ONLINE       0     0     0
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0     0
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0   291
	    ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT  ONLINE       0     0    12

errors: No known data errors

The resilvering has checked every checksum in 1.6TB of data and repaired 1.29MB. The process took exactly 12 hours (with a reboot thrown in there JUST to push the boundaries a little bit).

Next we’re to get any data off that we want… Let’s grab some directory information
To be continued…

XKCD, orbital speed and travelling to Mars

I was scanning stack exchange and came to a VERY interesting question about travelling to Mars (as a kid I was HUGELY interested in space—and even at one time dreamed of being an astronaut :/).

The question seemed to have merit and upon reading it even I said, “Yeah, why don’t they do THAT?”

Even after reading the answers I didn’t really get it (actually, I only partially skimmed some of the comments), but somewhere in there was an XKCD link.

I thought ANY XKCD on this subject has to be cool and so I clicked on it and was given my first lesson in Orbital Speed

But before clicking on it, I had read one comment from Beska that suggested to read such-and-such about it and it would go “from ‘somewhat mysterious’ to ‘obvious'”. I didn’t read such-and-such, but I thought, in xkcd typical humour, it might actually shed some light, and it did!

Inspiration_Mars_trajectory.svg_
source: http://www.extremetech.com/wp-content/uploads/2013/03/Inspiration_Mars_trajectory.svg_.png

First, the guys question:

Inspiration2
it sounds like a reasonable question

He asked, why take such a long route to Mars, we are virtually flying from one extreme of the inner solar system, to the other extreme and back again, plus some. Why don’t we just fly from Earth’s orbit to Mars’ orbit and wait for it to come?
Sound reasonable?

I thought so. I did realise he was looking at it too simplistically, but I had no idea why. I made an assumption that it was going to be a “spacial awareness” thing and that his question was missing THAT point.

By spacial awareness I mean factors outside of a two dimensional representation of a three dimensional scenario within its own three dimensional scenario.

Ok, I didn’t say it was a SOUND argument! :P

I’ve heard the term “micro-gravity” with reference to the International Space Station and how the astronauts aren’t really weightless and I understand that, but what I didn’t understand was the true nature of orbital speed and the impact it has on the effects of how we feels the Earth’s gravity, nor COULD I get my head around that this “micro-gravity” is about 90% (according to xkcd) of the Earth’s actual pull, which always led me to ask, “then how can they be floating at all?!”

Until now.

Even when I finished reading the article, I got a better idea of orbital speed, and understood it as it related to the Earth, but not to the scenario posed by the original poster. We were FREE of the Earth’s orbit, so why not wait in the depths of Space and wait for Mars (there’s a HUGE hint—MASSIVE hint—in both that statement AND this hint-of-a-hint! :D).

So after reading it, I had a little understanding, and maybe for you, it’s much easier for you to grasp it. But it was while I was typing this post and going to and fro the image above that I had the “OH, YEAH!” moment. Then I started thinking about the micro-gravity equaling 90% of the Earth’s full gravitation effect and started relating the motion of the ISS to that of an aircraft. Why is there no orbital speed effect and micro-gravity/apparent weightlessness there? (because it’s aerodynamics and nothing to do with maintaining an orbit, well, it kinda is an orbit, but aided by the volume of air passing around the wing—aerodynamics).

And somewhere in this post I did have an epiphany and it all DID make sense. And Beska was right.

I started this post as an email to a friend, and the reason I started it was to point out the song fact right at the end of the XKCD article, and to spread an interesting topic. But now, it’s more about the fact that I learned something very interesting tonight, something I didn’t know, something I will never use, but something that is still interesting just the same.

Oh, and just in case you want to read the XKCD article (you may as well, it is very enlightening), feel free to come back and checkout the “Trip to Mars” route map and look at that BIG yellow dot in the middle :D

XKCD article on Orbital Speed

Stack Exchange question: Why don’t we take the shortest route to Mars

sudo please

I stumbled across this alias quite a few months ago, but it was under a different name then.

I wanted to find it today and Google searches brought up a whole heap of malicious finds on it, aimed at targeting the unsuspecting newbie to linux. Obviously there are some very dangerous linux commands and so I’m not going to repost them here. Or maybe.

In linux, when you need special permissions to run a command, you’ll get a “permission denied” error. The standard response to that is to “sudo !!” (pron: sudo bang bang) which runs the last command again. Everyone who’s ever used linux will know this can get frustrating.

The I found “fuckit” (or it could have been just “fuck”, I can’t remember which).

When you forget to type sudo, and you’re prompted your permission has been denied, you can issue the expletive whereby the command runs as expected. The funny thing about this, is that when this does happen, it is the sort of thing you are probably thinking, and the expletive does it’s job.

In looking for it today, I found that someone else uses the “please” alias instead. There’s no difference, they both do the same job. It’s just how you feel you want to talk (type) to your computer at the time.

To achieve this, you need to create an alias:

alias please='sudo $(fc -ln -1)'
alias fuck=please

Only include the second line if you feel so inclined. I did because initially you can be all like “please?? Oh thank you understanding computer.” but after a day of permission denied errors, you might be more inclined to just yell at the screen, in which case, the second line is more appropriate.

Creating the alias alone won’t make it persistent. To do that, add the alias to the end of your ~/.bashrc file, or in my case, I’m running ubuntu and you can add it to ~/.bash_aliases instead.

FBI posts PSA on IoT

FBI-PSA Banner

INTERNET OF THINGS POSES OPPORTUNITIES FOR CYBER CRIME


says the FBI.

That’s a fair call. It does.

But it also puts the onus of security around these IoT devices on the user, not the manufacturer or developer.

I’m not sure where I land on this, because on the one hand I’d like to see that the manufacturer/developer (“them”) made sure the product was in fact safe, and if it’s not safe, then to make it so.

On the other hand, it’s good for the end user (“us”) to be proactively involved in their own security.

I’m not saying “we” (as a collective user base) need to fully understand it, but we should understand its implications, and what we could do to prevent bad things happening.

The problem is hardly anybody really understands security and even the need to change default passwords, or keep software up to date, let alone router settings and firewalls. And I don’t really like the thought of saying, “it’s ‘your’ fault,” even though clearly, it is, when someone has installed the latest so-called “virus scanning software” which itself is malware.

If “we” weren’t so stupid, you wouldn’t need warnings like this. But people every day are still falling prey to Nigerian scams and the like, or losing their identity or all their finances. It is very sad. But it’s more sad when it’s your own fault.

Security starts at home.

I don’t know who said that, but it’s true. Security starts with you. Be wise to the goings on of your devices, or really, don’t use them. If you don’t know why something “needs” the internet, turn it off until you do. A bit of education here goes a long way.

Maybe we need a “help out those who can’t help themselves, on the internet” day, where we all lend a hand to those less tech-savvy. Unfortunately, that’s almost everyone I meet.

Read the full PSA on the FBI’s Internet Crime Compliant Center (IC3) website (which incidentally will inform you that October is “National Cyber Security Awareness Month“).

The Casio F-91W

UPDATE: I had to add the comparison image between the iPhone 3G (2008) and the watch (1991).

F91W watch from CasioI had one of these watches!

It was my first ever digital watch. Oh how I loved thee.

There was a time I would not walk outside the house without wearing a watch, but sadly that changed in 2009 when I moved to Thailand and found that everything ran on Thai Time, and the need for a watch just didn’t seem to be so important.

Even upon moving back to Australia, I have worn a watch intermittently, and although I did give some smart watches the time of day, I have gone back to my non-watch wearing ways and have to say, I don’t really miss them. I almost ALWAYS have a phone on me and the time features prominently somewhere on the home screen.

In a search for this article I found a comparison between this watch (released in 1991) and the original iPhone 3G (2008).

comparison between Casio F-91W and the iPhone 3G
tongue in cheek humour, we all know the watch is better :)

Over the years I have owned both digital and analog watches (predominantly digital, of course!), but what I never knew was the dangerous connection this watch has with terrorism!

A watch, ahead of it’s time, both in features and nerdiness, it was adopted by certain groups for its ease of integration as an IED and notably a large number of terrorists with explosives training have been apprehended whilst wearing this watch.

The Casio F-91W is still being made today, but given the evil connection, I don’t know that I’d be flaunting it…

Or maybe I would.

That is, if I still wore watches.