journalctl -xef -u some.service

systemctl status … will display an excerpt.

If the error is displayed when starting Caddyserver from a console, but is not shown in the logs if an initdaemon such as systemd is used (which logs stdout and stderr), then Caddyserver’s authors have forgotten to Flush stdout and/or stderr before exiting.

found: https://caddy.community/t/cant-start-caddy-with-systemd-service-file/3447

I wish I’d found this little gem a long time ago!

Psychology in pricing

When you see $9.95, do you think:

  • WOW, $9
  • That’s almost $10
  • That’s $10

When I see fuel is 174.9c although I may say one seventy four in my head (simply because that’s what I’m reading), I recognise the .9 to mean “close enough to one”, and so I think in my head $1.75.

Maybe I think differently to most (I don’t think that’s really the case), but when I see this on facebook, I can’t help but think the writer is an idiot. 

That caption alone: “Try paying $2”. I’m already stuck. I don’t see anything that’s $2, I’m already seeing $3. I’m not even seeing $2.99, that THREE WHOLE DOLLARS. (you AREN’T going to get any change!)

My second thought is, “YES! TRY paying $2 a litre and you’ll be met with a stare followed by a call to the local cops because you’re trying to steal a third of the fuel you just put in your tank!”

And when I read, “which is nearly $3,” my brain is screaming “it IS three dollars!”

Somebody needs a lesson in maths.

That’s not my job, someone needs to go back to school. 

They could also just watch some YouTube videos!

If that’s all too dificult for you, check out these 9.99999̅ reasons. 

Cheers!

Flush DNS cache in Ubuntu

According to quite a few users on Stack Exchange, Ubuntu doesn’t cache DNS queries, but I got into an awkward situation where I couldn’t hit an internal server because my installation was convinced my internal server was being hosted on an external IP. 

It wasn’t entirely incorrect, because the server is (sometimes) available externally, but it’s only as needed. I have a few web serviced that need to be reached from outside and although I could use different ports, for ease, I just change the rules according to what service I want to use that day. I know, it’s very awkward and all I really need to do is set up a reverse proxy, but I am yet to get that done. It’s on “the list”! lol

I am using pfsense and the way I have it set up, the services I develop on need to resolve internally and externally. That means when I type https://subdomain.example.com/ and I am inside my network, I need it to resolve to 10.0.0.10 (for example), but when I am not on the network I need it to resolve to my public IP address.

pfsense handles this perfectly.

What I didn’t realise, is that because I mis-configured something, enquiries locally were hitting my public IP address and being hosted that way. Meaning, my DNS request was being directed to my external IP address instead of my internal one. When I changed the order of the rules to allow a different service access to the HTTPS port, I could no longer access my original service. 

Long story, short: my internal server was being accessed by my computer via the public IP address instead of the internal one. 

I fixed the entries in the firewall and now if I did nslookups and digs on the URL, the correct internal IP was being returned. 

Cool!

But it didn’t work. I still couldn’t access the service I needed, because Ubuntu was still accessing it from the external IP address.

Not Cool!

I restarted the resolver in pfsense in case for some reason it didn’t stick. But that wasn’t it. As I said, diging the firewall was returning the correct IP address, but pinging it was returning the external one. This took some time for me to work out that it was Ubuntu that had cached the URL, and it wasn’t the firewall that was now stopping me from accessing that page. 

Let me show you by example:

subdomain2.example.url (all URLs and IP addresses have been changed to protect the innocent) is responding with an internal IP address.

dave@home:~$ dig subdomain2.example.url

; <<>> DiG 9.11.3-1ubuntu1.1-Ubuntu <<>> subdomain2.example.url
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63440
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;subdomain2.example.url.			IN	A

;; ANSWER SECTION:
subdomain2.example.url.		1655	IN	A	192.168.1.78

;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Sun Aug 26 23:58:37 AEST 2018
;; MSG SIZE  rcvd: 57

subdomain1.example.url is returning a public IP address (that’s not desired).

dave@home:~$ dig subdomain1.example.url

; <<>> DiG 9.11.3-1ubuntu1.1-Ubuntu <<>> subdomain1.example.url
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2906
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;subdomain1.example.url.		IN	A

;; ANSWER SECTION:
subdomain1.example.url.	7112	IN	A	69.68.67.66

;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Sun Aug 26 23:58:48 AEST 2018
;; MSG SIZE  rcvd: 61

I knew this wasn’t quite right, but I couldn’t work out exactly why. So I asked my router where it thought the domain belongs:

dave@home:~$ dig @192.168.1.1 subdomain1.example.url

; <<>> DiG 9.11.3-1ubuntu1.1-Ubuntu <<>> @192.168.1.1 subdomain1.example.url
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1846
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;subdomain1.example.url.		IN	A

;; ANSWER SECTION:
subdomain1.example.url.	3600	IN	A	192.168.2.111

;; Query time: 0 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Mon Aug 27 00:08:02 AEST 2018
;; MSG SIZE  rcvd: 61

As you can see, the router knows the correct internal IP address, but Ubuntu is getting it from somewhere else. 

dave@home:~$ dig subdomain1.example.url

; <<>> DiG 9.11.3-1ubuntu1.1-Ubuntu <<>> subdomain1.example.url
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 33482
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;subdomain1.example.url.		IN	A

;; ANSWER SECTION:
subdomain1.example.url.	6551	IN	A	69.68.67.66

;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Mon Aug 27 00:08:09 AEST 2018
;; MSG SIZE  rcvd: 61

Then when I googled and found the above referenced Stack Exchange post and most answers were saying Ubuntu does not cache the DNS request, I just got confused. Not being confident of what I was looking at, I didn’t understand that the server IP address (127.0.0.53#53) was a loop back address for the DNS. I knew the port meant that it was a DNS, but I didn’t understand where it was being served from. Yes, I should have quickly realised it was only a loop back address, but I thought it was coming from the router (because of the DNS port entry).

hence my confusion. When I asked the router directly, it was reporting correct. But the computer was getting it from somewhere else.

And that TTL was killing me. I knew if I rebooted the computer it would more than likely fix the issue (it would have), but I had too many things open and I just didn’t want to go down that path. 

Eventually the light bulb turned on and I realised that Ubuntu simply must be caching it itself and I needed to identify the service and restart it. 

Scrolling right down the Stack post I found my answer. Although there’s several answers with differing methods of restarting the service, I went for the systemctl approach. 

sudo systemctl restart systemd-resolved.service

a ping and a dig confirms we’re good to go:

dave@redbox1804:~$ ping subdomain1.example.url
PING subdomain1.example.url (192.168.2.111) 56(84) bytes of data.
64 bytes from ubology.dav3 (192.168.2.111): icmp_seq=1 ttl=64 time=0.874 ms
64 bytes from ubology.dav3 (192.168.2.111): icmp_seq=2 ttl=64 time=0.295 ms
64 bytes from ubology.dav3 (192.168.2.111): icmp_seq=3 ttl=64 time=0.268 ms
^C
--- subdomain1.example.url ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2006ms
rtt min/avg/max/mdev = 0.268/0.479/0.874/0.279 ms
dave@redbox1804:~$ dig subdomain1.example.url

; <<>> DiG 9.11.3-1ubuntu1.1-Ubuntu <<>> subdomain1.example.url
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21571
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;subdomain1.example.url.		IN	A

;; ANSWER SECTION:
subdomain1.example.url.	3589	IN	A	192.168.2.111

;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Mon Aug 27 00:09:44 AEST 2018
;; MSG SIZE  rcvd: 61

dave@redbox1804:~$ 

Some other untested suggestions in the Stack post revolve around restarting the network manager, killing the process, flushing the cache using a command line switch (I probably should have tried that one, just out of curiosity), and restarting the service via init.d, and others. Posted for posterity and “next time” 

sudo service dns-clean
sudo service network-manager restart
sudo /etc/init.d/nscd restart
sudo kill -HUP $(pgrep dnsmasq)
sudo pkill -HUP $(pgrep dnsmasq)
sudo systemd-resolve --flush-caches
sudo systemctl restart systemd-resolved.service

Problem solved!

Now, for the next one:

I need to get a reverse proxy up and running. I did give it a go with squid, but I was obviously not getting something right. Further readings suggest HAProxy might be the way to go. That will probably be my next pfsense adventure.

Will this haircut still be in my head on Monday?

Will this haircut still be in my head on Monday?

Jason really likes his haircut. I think he’s afraid it’ll grow out before he gets to school. 

I convince him that it’ll still be there on Monday. 

Then Harry asks, “what about Tuesday?”

Yes, it’ll be there Tuesday buddy. 

We then went through the rest of the week. 

It might be a bit short for mum’s liking, but he loves it. That’s all that matters. 

Each morning…

Each morning when I open my eyes I say to myself: I, not events, have the power to make me happy or unhappy today. I can choose which it shall be. Yesterday is dead, tomorrow hasn’t arrived yet. I have just one day, today, and I’m going to be happy in it

Groucho Marx

Lawlessness on the roads

how to navigate an intersection

I have long come to realise that Australia is a nanny state of rules and regulations. There’s a rule for just about everything, and I’m starting to realise this is beginning to hamper us when in reality these sort of things should be making our lives better. Safer.

After having spent a lot of time in Thailand and using Thai roads and the lawlessness that apparently goes with driving in Thailand, you can come to both appreciate the rules we have on one hand, but despise them at the same time.

There are times where (especially in the middle of the night) where you can seemingly spend an eternity at a set of traffic lights for no apparent reason. I have a set of traffic lights at the end of my street, and I have to admit, they are pretty speedy on letting traffic out of our street. You only have to stop there for a moment and the lights begin to change. This appears to be the case even if it has only just turned red. The wait is never very long.

Yet there are traffic lights where they seemingly take forever. What’s with that?

Why are there stop signs for turning left on roads where you can see the approaching traffic for half a kilometre?

Why are there red arrows against turning right when there’s no approaching traffic at all? This one can be especially infuriating.

Paris Traffic

I remember seeing a photograph of a classic Parisian roundabout when I was at school, and I remember thinking “How do they do it? It’s chaos! How does anyone not get killed?”

As a kid I couldn’t see it. As a young adult I couldn’t see it either. Our lives are built around a network of safety measures and molly coddling. Rules and regulations designed to give us a “right of way” and a feeling of security as well as entitlement.

It’s this entitlement that breeds complacency on the roads. A righteousness that forgoes common sense. Road rage built around “he just cut me off,” or “I have the right of way!”

But what if we took that entitlement away? What would happen then?

People speed through intersections because they feel “safe” in the knowledge that other traffic should be giving way to them. But what if other traffic didn’t have to give way?

What if we took away the very things that apparently made our roads safer?

One of my favourite presenters, “99% invisible” teams up with Vox and illustrate just how it’s done elsewhere and removes some of the “molly coddling.”