jlj

The musings of an aspiring carver of space

I've been furloughed

It's official now, so I'll simply say that I've been furloughed. I'm certainly not alone, even in my company, and I'm still better off than many. It's worrying, but that isn't why I've been away.


I completed my Architect practical lab yesterday. (Or around 3am today, more precisely. I needed 18 of the 24 hours, in the end.) That's my shorthand for what Splunk calls their Splunk Enterprise Deployment Practical Lab Public Class, by the way. Quite a mouthful. It'll be a few days before I get my results. There isn't much feedback, apparently. And, they are completely upfront about it being a subjective evaluation, just as it would be with a customer, waiting to be paid.

I'm disappointed, regardless of the outcome. The whole process has highlighted some glaring gaps in my knowledge, particularly around apps and technology add-ons. I was deep on trying to extract fields from syslog before a colleague pointed out that putting the *nix TA on my search head would do the job for me. (I'd only installed it on the universal forwarder that was gathering the data.) And my regex is pitiful: I tried to use the Add Data wizard on a custom XML data source, and was so thrown by the stream of gobbledygook it spat out that I lost hours and never properly parsed the stuff, in the end. I tried to justify that point, amongst others, in my report to the 'customer'; we'll soon see how it all went down.

It was a real blow to my ego. Especially now, seeing my manager finish the same lab in seven hours today. I'm so impressed by her intelligence, work ethic and thought process, but, try as I might, I still use moments like this to knock myself down. Ah, well, I'm a work in progress, as we all are.

End of Day 29

(I know I've missed a few actual days, but I've decided to ignore the gap.)

jlj@Fosstodon #100DaysToOffload

I'm writing this as part of the 100 Days To Offload project; join us at: https://100daystooffload.com/

Community, in the 'multiverse'

Not across universes, literally; I can't think of a good word for an association that transcends the fediverse, and to IRL beyond. I don't know why I keep coming back to this idea. It'll probably be the first topic in my zettelkasten, when I finally decide on a medium for it. If I could (non-disruptively) give my 20-year-old self anything, it'd be a zettelkasten; an actual piece of furniture, I guess, as Windows 95 was the height of innovation back then. But, man, I know this post would be an order of magnitude better for it.

Clay Shirky talks about how you can have a small community — on the Internet; I'll explicitly reference IRL when I mean otherwise — and, particularly with a dedicated core of a few friends committed to your vision, the centre will likely hold. (I recently rejoined IRC, and TooCool@EFNet pointed me to a piece that also explored this idea.) But, with success, comes danger: there is a size, beyond which, your vision will be threatened. This came back to me while reading about Cooper coming on as a community manager @Fosstodon, so I want to say it's around 10000 members (but don't quote me on that). And this is where I reference the cliche of an ellipsis, followed by the Shangri-La of 'profit'. Seriously, though, there are probably tens of thousands of examples for every birdsite. (Not that I'd hold that up as my ideal, mind.)

I joined Lemmy(Net) yesterday, curious about the ReverseEagle project that bubbled up in /r/Linux. I liked the message, and the tone. I'm a sucker for humility, and the opening apology regarding a perceived lack of professionalism really struck a chord. I mean, they sound very measured to me, and they're talking about simply offering genuine alternatives. That seems far more professional than, say, the early days of Slashdot, where Bill Gates as Borg was considering the starting point for any conversation outside of FOSS. (I can't resist bragging about my five-digit user id: 68393; yeah, it was mid-morning before I realised CmdrTaco had opened registration.) And then you have other models, like Tildes' sponsorship (lite) one.

I want to know my IRL neighbours (I think). But I'm not good in meat space; particularly after some of the difficulties I've had in recent years. Years ago I tried Streetlife, but that registration process alone was far too intrusive, especially for the limited benefit of what became a local Gumtree knock-off, from what I could see, before closing. Freegle was OK, pre-COVID, but, again, fairly intrusive, and a mixed bag of, well, exactly what you'd probably get IRL if a bunch of people came together for the primary purpose of barter, in effect. I had some positive experiences with Meetup, but the clue is in the name, for me: too heavy on IRL; I want a community that is rich here, and naturally extends beyond that over time.

I'm going to end it there. More to come, though — fuelled by my zettelkasten, I hope!

End of Day 28

— jlj #100DaysToOffload

I'm writing this as part of the 100 Days To Offload project; join us at: https://100daystooffload.com/

The elephant in the room

It's been a tough day. I'm still not at liberty to explain why. Mind you, I don't imagine it would take much of a leap to guess the reasons. Still, I'll leave it there, as an apology for what will probably be a lacklustre post.


More practice today: the Splunk Enterprise Deployment Practical Lab test is this Monday. So there goes my weekend. I'm up to almost 90% of my monthly EC2 usage on the AWS free tier, so I switched to GCP today. I have to say, I like the interface more. It's still early days, obviously, but it's a better experience, so far. For example, they explicitly state that you will not be billed for your free kit; period. It will just stop working at the end of the trial period, presumably, if you haven't made a payment. I like that, as I've had a low level of worry rumbling along this week about waking up to a bill from Amazon. (Although, probably a very small one, in their defence.)

A word of warning if you're thinking about using Keybase's private storage heavily: don't overextend yourself. It's obvious, with hindsight, that performing lots of writes and deletes on a (remote) encrypted filesystem is a bad idea, but it took being in a real mess to open my eyes. Actually, it was my sloppiness that included the path for what would be the synced copy of those files (in my home directory) in the actual copy I was making to Keybase's storage that kicked off the facing mirrors, reflections ad infinitum, hard disk drive thrashing itself to death. Obvious, as I said, but this cautionary tale is the single benefit I can wrestle out of that agonising spell yesterday.

End of Day 27

— jlj #100DaysToOffload

I'm writing this as part of the 100 Days To Offload project; join us at: https://100daystooffload.com/

How cheap IS disk space?

What's on my mind? Keybase. I saw that they offered file storage as part of my general tour of their services upon first signing up a few weeks ago. Today, I noted that — thanks to FUSE, coincidentally on my radar for the second time this week — the desktop client provides simple access to this storage. And then I thought something went wrong, because Nemo told me I had approximately 248GB of free space on that filesystem.

The documentation confirms it: every single user gets 250GB of free storage space. I do not understand how this is good business, but, then again, I've never had a head for business. (Still, I can't help but wonder whether Zoom will revisit this policy.) Regardless, it's very generous. (Thank you, Keybase!) More than ten times what Google offers, while foregoing the profits Google makes by selling our data to boot.

So I've been playing around with rsync and cron today, adjusting my backup solution to take advantage of this bounty.

Oh, and I created a team — beardlovers, for fans of YouTuber Wheezy Waiter — so I could play around with that feature as well. Pop in, and I'll tell you all about the wonderful show that is The Good Stuff.


I had some news at work today. I'm not at liberty to share it right now, but let's just say I'm a bit preoccupied at the moment. I'm also worried about my upcoming 24-hour game show of a test that is my Splunk Architect practical lab; I keep running into indexer cluster issues, and so haven't had much time to practice field extractions, creating sourcetypes, or dashboarding. Oh, well. A fun weekend activity, I guess?

End of Day 26

— jlj #100DaysToOffload

I'm writing this as part of the 100 Days To Offload project; join us at: https://100daystooffload.com/

Great expectations, and coping

I was hoping to share a screenshot of Conky today, with the weather conditions beautifully displayed. Instead, I'll have to settle for the results on the command line, conditionally formatted:

weather script output

I failed to account for the necessary transition to pixels when working with Conky. I think I'll need to manipulate the returned JSON directly — as opposed to calling a script to parse it — or reconsider how I want the results formatted. A decision for another day.


I'm a lot better at dealing with disappointment these days; and on a much bigger scale than some failed hacking, let me quickly add! Not long after I first moved to the UK — more than a decade ago now — I blew a PSU.

And completely lost it.

It's embarrassing to remember, frankly, and really scared my ex at the time. Most of that embarrassment is due to the extent of my overreaction, but a small part of it is because, while I did read the instructions on the PSU, I failed to take in that its wide tolerance of voltage was predicated on throwing a (very prominent, in their defence) switch on the back of it. Unbelievably, it was the only casualty of that mistake: the motherboard, and all the attached components, served me well for another eight years. (In fact, the hard disk drive is still working well in my current tower.)

Fast forward ten years, and that reaction seems completely alien to me. Part of that is having young children: things get smashed, lost, they explode; life loses much of its predictability. Part of it is one of the many happy consequences of years of psychotherapy. Part of it is simply getting older.

It feels good. Particularly in the light of the criticism I've been levelling at myself recently: what could, ungenerously, be deemed a regression in my anxiety levels as lockdown has continued.

End of Day 25 — a quarter of the way!

— jlj #100DaysToOffload

I'm writing this as part of the 100 Days To Offload project; join us at: https://100daystooffload.com/

Filesystems and grey-matter systems

What. A. Day.

My head is spinning. I've learned so much, and yet I have so many questions, as I close out my work day. I can't remember a more recursive day in recent memory: where you park your reading on a topic to delve into another subtopic, return, continue reading on that topic, then delve into another subtopic, and a deeper subtopic from there, etc. There were some functions I didn't even call — to stretch a metaphor to breaking; I just made notes in the list of questions that always sits on my desktop and ploughed on.

FUSE was one of those. I don't know how this passed me by; what an elegant idea. I was trying to get an SSH private key from my desktop to my Mac, so I could continue working on my AWS indexer cluster from there, but ext4 is a mystery to Catalina. ext4fuse to the rescue, along with some, well, magic, from my point of view to make sure my user was in the proper group for mounting filesystems. I need to get more comfortable with macOS; it feels so alien, compared with Ubuntu and Mint.

The good news is, it all worked, and I've made notes about the research I still need to do, in slower time, before I'll be able to properly explain the process to anyone. (My ultimate goal, these days, as I want to be a mentor in our company, somewhere well down the road.)


Also, quickly, I hit Yes on a support page — you know, that, typically annoying, “Did you find this useful?” bit at the bottom — for the first time in ages today: it was a short, clear, spectacularly useful page, hosted by Amazon, on how to grow EBS volumes and partitions, and then resize the associated filesystems. Within ten minutes, my indexer cluster stopped complaining about the walls closing it. Result!


Finally, HN pointed me to a wonderful, beautiful piece by Eugene Yan on zettelkasten and Roam. I can't remember... Gosh, is it a bad sign that a post of this length is causing deja vu, about forgetting things?! I was so excited by this piece. Just, beside myself, really. I'm so unhappy with my knowledge management system right now; nothing's joined up. It's like an archaeological dig, going back through old notebooks, text files on old backups. Making connections is serendipity, at its most frustrating.

Until now.

Or, it would be, if Roam wasn't closed to new users. (D'oh!) The bright side is that the application form for their wait list was a lot of fun. I'm not joking. I really enjoyed it. It didn't make up for not having access to the software, but it certainly took away some of the sting.

End of Day 24

— jlj #100DaysToOffload

I'm writing this as part of the 100 Days To Offload project; join us at: https://100daystooffload.com/

Full on

Boy, I'm beat. I finished Phase 1 of the (practice) Splunk Architect lab tasks today, and got well stuck in with editing config files as part of Phase 2. Happily, I learned — just moments ago — that the AWS free tier includes 750 hours of t2.micro instances per month, so I'm leaving all my hard work up and running overnight; then it'll be ready for me to pick it up right away tomorrow morning. (Even with eight concurrent instances, I'll be good for a few days.)


Late last night, I decided that I'd park the Python code I was working on to parse Met Office JSON. I will come back to it — when I finally decide on a course — but, for now, a script calling jq will work fine. (I only want a few elements from the current, and following, day to display in Conky.) I was pretty chuffed with my progress last night, although it meant I was late getting to bed.

It was tough seeing my partner leave this morning, double pram full of the most precious cargo in my world. My boy is back now, though — I can hear him squealing downstairs — and seems very happy.


This is a bit of a non-sequitur: I was reading the latest Brain Food (No. 371) from Farnam Street and an excerpt from a Bloomberg article on Musk caught my eye. It was so simple: he acknowledges that he shoots from the hip on birdsite; that's why he likes it. No press releases or faff — just his mind, to the public. Which, he also acknowledges, means that lots of his tweets are “dumb.” And, that if that's the price of doing business in this way, he's happy to pay it.

I don't think that having that policy, as such an influential person, occurred to me. My sleepy brain can't fathom the pitfalls of such a policy at the moment, but I can certainly see its benefits. My tendency is to think that good communication is based on expectation management: knowing what your intended audience expects, and them knowing what to expect from you. Putting a message like this out — an implicit caveat to every tweet — certainly sets clear expectations.

End of Day 23

— jlj #100DaysToOffload

I'm writing this as part of the 100 Days To Offload project; join us at: https://100daystooffload.com/

Bunker day

Sundays are hard. I have enough data now to say it's a pattern. And I'm now taking it seriously. I've just ducked out of the family zoom. I'm not going fedi, or on birdsite. (My BBC apps went with bookface.) I'm adding virtual lockdown to my continued physical one.

I am OK, so far; better than, maybe, as I handled my three-year-old's nap-time tantrum with aplomb. I think overconfidence has been part of the problem; assuming good days mean that will just continue. But, no, vigilance is required.


Saw on Reddit that someone was asking about FOSS keyboards for Android. I'd seen that Microsoft bought SwiftKey, which had started the idea of a change buzzing about in the back of my mind; the suggestion of OpenBoard was the tipping point.

I'm happy with it so far. Nice and simple. I do miss gesture typing: I started using Swype nine years ago, I guess. (And gestures on my Palm V long before that!) But the developer has said that it's part of his plan, once he has the time. I have to say, though, I've disabled enough of Android and/or the defaults that I'm reaching another tipping point: time for a change of operating system? I hadn't even heard of Pinephone before jumping on Fosstodon, so that's probably in the mix too.

I think I'm fine for the moment, but the seed has certainly been planted well.

End of Day 22

— jlj #100DaysToOffload

I'm writing this as part of the 100 Days To Offload project; join us at: https://100daystooffload.com/

A nice day

I had a friend. His mum passed away, and, going through her old photos, so many of them were captioned simply, “A nice day,” on the back. So he created a blog that showcased a different one each day, partly for his extended family, and the conversations that would come out of identifying any subjects, and partly as a tribute to the gratitude that was so much a part of her philosophy. And, I guess, partly to help conjure some of the latter in his own life.

He's still alive and well (last I heard), but he isn't my friend now. We didn't even talk about it. I assumed that my actions over here were relayed to him in some form, and he — completely justifiably — decided I wasn't someone he cared to have in his life. He stopped responding to my emails anyway. It's still pretty raw for me, eight years later. A fifteen-year friendship, doing the maths.


I was thinking the subject line fits today as well. (That's what got me thinking about all that again.) I got this well-loved, circus-style pup tent on Freegle last year, and — pretty ingeniously, I thought — I decided to deploy it in the new paddling pool today to keep the sun off the kids while they splashed. It was a big hit all 'round.

I spent a bit of time learning git as well. Renamed my GitHub account, proved it with Keybase, uploaded an SSH key, and then cloned the examples from that Terraform course I took a while back. I don't know whether cloning that makes sense now, to be honest, but I wanted to give credit for the bulk of this code I'm using to stand up my lab environment. Anyway, for now I've committed my code as an enhancement to Ned's, with an associated project covering where I want to take it. I'm jlj77 there, if you're curious.

End of Day 21

— jlj #100DaysToOffload

I'm writing this as part of the 100 Days To Offload project; join us at: https://100daystooffload.com/

We had an indexer cluster, ladies and gentlemen!

This entry will be Splunk related. Heavily Splunk related. But I'll open with how I was asked to present my Terraform code for setting up a practice lab in AWS at our company's weekly tech talk. I was pretty proud of the slide deck I put together. When my turn came, I shared my desktop and started running through it. Halfway through, someone piped up, “Are we supposed to be seeing something? It sounds like you're expecting us to see something.” Turns out, they could see my gallery view of the meeting, 'cause my slide deck was in another workspace. And then, when I tried to share Chrome specifically instead — where I was presenting from — our meeting application complained that it needed more permissions (that wouldn't be issued until I left the meeting as part of a restart). I got through it, in the end, but I felt a bit silly.

I ran that code today, thinking I'd get much further along in my practising. And, initially, things looked good. But once I'd finished the initial indexer cluster configuration, I wasn't clear on what the contents should be of that initial cluster bundle to be sent to the peers. The Clustering Administration class had quickly gone from everything's working to let's bust an indexer and see how failover works. I did eventually find a relevant page, deep in Splunk Docs, only to find out that, while the master node was happy to validate said bundle, it was not about to push anything out to the peers.

Early on I'd noticed that the peers were in automatic detention, but, in my mind, that went in the category of, oh, they don't have any data to index yet. My colleague had spent a long time troubleshooting a problem that essentially boiled down to, oh, you won't see that until you start forwarding data to the cluster. I decided all sorts of anomalies were acceptable while my cluster was in a simple racing block position, as it were.

Silly me. Silly, silly me. Thankfully, one of our longest serving splunkers took pity on my whining in the team chat at 10pm on her Friday night. Turns out, Splunk likes to have plenty of breathing room by default. Five gigabytes of it, in fact. Without that, things... stop. EBS volumes default to 8GB. The typical Splunk Enterprise install comes in at about 3.5GB. Notice a problem?

Once I'd gone 'round my peers, knocking that default well on the head — minFreeSpace under diskUsage in server.conf, for your notes — making sure that while I did it in each peer's slave-apps, I also did it in the master's master-apps bundle directory that would soon be clobbering the former changes (I hoped), everything was tickety-boo.

And, once my runbook was up-to-date, I typed terraform destroy again. D-:

End of Day 20

— jlj #100DaysToOffload

I'm writing this as part of the 100 Days To Offload project; join us at: https://100daystooffload.com/