Putting the Meaning in Platform Engineering

I have been working for a few years in technology, mostly doing platform engineering. I’ve done work for small companies but more often for larger companies. Software engineering is my vocation. When I was little I wanted to be an inventor, and programming is my outlet to make useful gizmos like I always wanted to. So in general, its a pretty good thing to be doing. With that said, I want to talk about one thing that bothers me:

On a day-to-day basis, few tasks relate to the impact of our work on people’s lives.

This is not to say that there is no impact of the tasks we do, simply that it is not often the focus of our efforts. We augment our software and remove technical deficiencies, but no-one asks us to question if the system will be useful.

The problem is not specific to platform or software engineering. In large engineering projects, many people are needed and the individual’s work may not be useful in isolation. The traditional view would be something like this:

Engineers do practical, quantifiable things to shape the world, and subjective things like feelings and emotions are nebulous and unquantifiable. Furthermore, there’s a division of labour, and the strengths of engineers lie in abstract reasoning and attention-to-detail, not in emotional intelligence.

From what I see in software engineering, this view is incomplete. There is too much churn in technologies and too many ‘best-practices’ rabbit holes to get lost in. The purpose of the work is everyone’s responsibility. We can all gain if each person can engage more with the outcomes of their work.

Managing personal finances like a developer with SQL

There are quite a few services around that purport to help you manage your personal finances. They require access to your bank credentials and scrape information. Then, transactions can get automatically categorised and visualised so you can see trends in your spending.

There are two problems with this:

• Exposing bank information to third parties is an uncomfortable experience at best and giving access to them tends to be in violation of the bank’s ToS (in the UK at least)
• In practical terms, many services are unreliable because of their unsanctioned status
• In many cases categorisation is not particularly good or suited to personal needs (for sufficient flexibility it would be good to have a Query Language)

Securing communications in Kubernetes

As more and more organisations are bringing Kubernetes clusters to production, we are seeing a lot more concerns about availability and security surfacing. And while there are quite a number of tutorials and scripts for bringing up out there for getting up and running (see Lorenzo’s very thorough and understandable guide), it remains relatively difficult to bring up a maintainable setup that also follows best practices. In this post we will look at an areas where best practices are quite well understood but documentation is unfortunately still a bit lacking. That is, how privacy and authentication can be provided for communication between cluster components.

Summer Exhibitionism

It’s lots of pictures of buildings, grouped by their function. They are things like water-towers, gas holders and sawmills. The buildings are quite old and seem to belong to those parts of our towns that are unloved and decaying. Simultaneously we see the care and attention that went into their designs. Despite that care, these are not glamorous or celebrated buildings. They are just doing the job they need to do.

Monorepos just dont work everywhere

Dependency management in Go has always worked quite differently from in other languages. Although Go baked in a standardised dependency management process, it seems to have been based on the authors’ experience working in Google. Unfortunately, things work differently in Google than in other places, and so the default approach is found lacking by most developers. This has led to a reliance on ad hoc and third-party approaches, and as yet the community has not arrived at a de facto standard. This leaves the new Golang developer with some learning to do in order to choose the best tool for the job.

Kung-Fu Panda 3, Identity, and the Not-Self

On my long flight to Indonesia I was watching the film Kung Fu Panda 3, which did a great job of keeping me entertained for a couple of hours. Like the previous films, it is a lighthearted take on kung-fu that nevertheless (generally) manages to seriously engage with the buddhist/taoist/confucian themes that are important in that culture. The format is that the eponymous panda has to master an esoteric kung-fu technique in order to defeat an evil demon and save everyone he cares about. His mastery is driven by a personal revelation which enables him to “level up” just in time to beat the demon.

In this third movie, the skill he has to master is the control of Chi - that mystical life-force energy. In itself, the concept of chi is an interesting thing. But what was especially interesting was that developing this skill was tied into his journey to identify as a Panda (remember he was brought up by an adopted parent, a Goose). So, as well as a battle of good and evil, the movie is also depicting a spiritual journey. In this journey, our panda seeks to answer the question, “Who Am I?”.

This seems like an important question - its something we’ve surely all asked ourselves. At least, I have asked myself this before many times. Usually when things aren’t going the way I want somehow. I think, “Who am I?”, “What is unique about me?”, “What do I bring to bear that no-one else does?”. There is some feeling that answering this question is fundamental to finding meaning in our lives. But this is one place where the film seems to stray from its Buddhist underpinnings.

Going Deeper in Docker

There’s been a couple of awesome resources come out in Docker over the last few weeks. First was the “Hitler uses Docker” video, which gives a sample of the common complaints that people have about Docker distilled down into a couple of minutes. And some of the complaints are totally valid! Watching this video really made me want to understand better the status of the many claims being made. How often do problems come from abuse of the technology, how often from missing features, and how often from fundamental limitations? To do this we need to look underneath the orchestration tools and the packaging format and look at the technologies that are being built upon.

Exhibition Review: Double Je (Le Palais de Tokyo)

On my visit to Paris, I luckily had a free day before running the marathon on the Sunday. My brother recommended to me that the Palais de Tokyo was one of the best places in Paris to see a nice exhibition or two. And it did not disappoint - there were a few good exhibitions on when I visited but the best of all was Double Je. The concept of this exhibition was pretty interesting - to tell the story of the mystery behind the apparent murder of an artist through a series of installed spaces containing the work of artists and artisans.

The problem I always have with objets d’arte in an exhibition is that they are out of context, the isolated pieces don’t seem to make a clear statement or impact in the same way that individual works of fine art do. This is not the case here. These constructed spaces - the apartment of the artist, his workshop, his garage, etc. - consist of a melange of objects, both functional and decorative, finished and unfinished. The effect en ensemble is to show the fascinations, the obsessiveness and the creativity of this individual expressed though these spaces. Some links are drawn between the free creative exploration of the artist and the anti-social behaviour of the criminal.

As the exhibition goes on, I increasingly feel like I’m falling down the rabbit hole. The links between artist and criminal are strengthened and we feel as though we are arriving at a conclusion. What seems at first as prolific expression of creativity is subverted into something more and more dark and ominous.

Maybe it was my bad French understanding, but I don’t think we were ever told in plain terms what had happened, but it did all the same feel that there was a payoff to the journey. This is good. Usually in a work of art we don’t want to be told what the meaning is, just to let our response arise naturally to what is presented. And what is presented here is both visceral and liberating.

TL/DR: Kubecon

It was a great couple of days at Kubecon - so much going on, so many great speakers and so much to process! Surely I missed quite a lot, given that there was four tracks, but this is my summary and synthesis of some of the major (and minor) points that I learned.

New Year's Resolutions 2016

Instead of new year’s resolutions in 2015, I looked at different ways to do behavioural changes. The idea was that, when making resolutions we are only thinking about the destination and not about the journey [1]. We look more at the goals and less at how we get there. It seemed interesting to look at things very much the opposite way - to think only about habits, how I had been spending time, and how I could improve that. Going into 2016, it’s a good time to review that approach and to look forward.

Layers and Creativity

In the construction of something, there are usually some distinct levels, or aspects, to them. For example, to make a painting, there is first the level of perspective and construction, whereby we provide the outline of geometrical forms occurring in a fairly abstract (but nontheless almost always Euclidian) space. In other words, the first step is quite precisely to project the 3D forms onto a 2D surface - first with the most important forms and then moving onto the details.

What I've been reading: Charismatic Edition

Notes from The Charisma Myth by Olivia Fox Cabane:

• When you are in discomfort, make changes to your environment to alleviate it. If you cannot, consider rescheduling an important conversation. Other people can sense your unease but do not necessarily know the reason for it. Naturally they may consider whether it relates to themselves or to a quirk of your personality.
• Before going into a performance, just imagining a loved one embracing you can give you a shot of oxytocin and calm your nervers.
• It’s important to practice emitting warmth - one way you can do this is to take up the Buddhist practice of Metta meditation
• Charisma is the simultaneous projection of both power and warmth whilst being fully present

What I've been reading: Zen Edition

Moon in a Dewdrop - Zen Master Dogen

*Master, what should we do if hot or cold comes?
Why don’t you go to where there is no hot or cold?

• Dongshan*

In the buddhist tradition, hot and cold refers to birth and death. Therefore, this could be read as a master prompting his student to try to escape the cycle of samsara (suffering).

I feel this is also meaningful from a secular, phenomonolgical perespective. Hot or cold seems like anger or fear, like confrontation or avoidance, like desire or aversion.

The War Of Art

In 2021, the city-state of Bienalia was established as a result of political fractures in the Venetian republic. The political radicals, led by the Biennale organizing committee, staged a coup at Venezia city hall on the 14 March 2021. Flanked by militiamen, the Venetian counsillors were forced to hand over power to the Biennale organizing commitee. Once in power, the commitee immediately announced a state of emergency. This effectively meant an indefinite suspension of the normal democratic process and the institution of a single party state.

Infrastructure as Code is not Configuration Management

For the past few months, I have been concerned with deploying complex applications from the ground up on top of my company’s IaaS platform, Interoute VDC. Wanting not to re-invent the wheel , we have spent quite some time looking at tools that could do the job. In this search I somewhere came across the sentiment that forms the title of this piece, and for some time, I was quite confused about this [1]. And it seems like lots of others are confused too. There are a plethora of plugins adding cloud provisioning capabilities to traditional Configuration Management tools such as Puppet, Chef, or Ansible. But, we eventually settled on a different solution for our provisioning: a new framework called Terraform. This is designed with IaaS specific problems in mind, which leads to some big wins, as we will see. There are three main points of differentiation:

The Blue Pill or the Red Pill

Many people like to travel, myself included, and if travel has to be just one thing, it is adventure. Most people would agree that travel is a type of adventure. But, how is it adventurous? We all have googles, tripadvisors and lonely planets falling out of our eyeballs. We know in advance exactly what we want to do and when we want to do it. Is that really adventure? On the other hand, if we take a trip that is unstructured and unplanned, we tend not to stumble across those amazing things we were hoping to find (although perhaps sometimes we do). So, where is the adventure to be found?

New Years' Substitutions

As I have been learning a lot about psychology this year, I thought to make some changes to my New Years' Resolutions. Specifically, for the bad habits I want to cut down on, I will now rely on substitution.

Substitution, as a strategy for habit change, is based on a psychological theory of behaviours. The first thing this says that the behaviour can be broken down into a trigger, a routine and a reward:

• The trigger includes both the context and the specific prompt for the behaviour
• The routine describes the sequence of physical actions and the though processes being carried out in the behaviour
• The reward specifies which results of the behaviour stimulate any of the various reward centres of the brain [1]

On Hedonism

"The bigger the bill, the harder you ball
Well I'm throwing mine, cause my money long
The quicker you here, the faster you go
That's why where I come from the only thing we know is

Work hard, play hard"

-- Wiz Khalifa ("Work Hard, Play Hard") [1]

What does Hedonism mean to you? Perhaps the first things that come to mind will be sex, drugs, and rock n' roll. Or indeed, money, hoes and dro. This is surely a hedonistic way of living life. But is this, in fact, everything that hedonism can mean, and is hedonism an immature and irresponsible way to live life - suited best to rockstars and other self-destructive types?

Repeat Sales means House Prices

There is a problem with the way houses are traded. Since houses are so heterogeneous, it is hard to estimate what they are worth according to any consistent methodology - and it is hard to place them into a trend. If I look at a stock, for example, it is frequently traded so I know that the price I would buy at reflects the market value. The history of its price growth is available and hence we have some clues as to whether this value seems to reflect an underconfident or alternatively an overinflated market (one overinflated might currently growing at 20% per year, for example). However, this is currently not the case for those looking to buy a house - the asking price is the single price point available, and whether it is market value can normally be only determined by the level of interest, by ad hoc comparisons, or by an expensive valuation process. We need a method to be able to value houses on a local basis, whilst taking into account their heterogeneous nature.

Everyone Is An Artist

“Everyone is an artist”. This is a highly interesting quote. Because, on the face of it, it is clearly untrue. Not everyone is a painter, not everyone is a sculptor, nor is everyone a draughtsman. In fact, most people are none of these things. But, although these are extremely important as mediums, these are not pre-requisites for art. Art, in its purest form, is about the expression of the intangible aspects of the world we live in. This is what is really interesting - what is seen by one person cannot ordinarily be seen by anyone else. You may capture an image, but it is rather more difficult to capture a subjective experience.

Why We Spend Expenses Wrongly

My company, like many others, provides a fixed daily amount to recompense travellers for their daily expenses. On most occasions this ends up being a little more than your expenses naturally sum to. Now, in theory, this means that once you know the duration of that trip (and assuming you trust payroll) you are sure to receive a fixed amount of money. However, people do not quite react accordingly. In particular, there is a tendency to fixate on the daily amount to guide decision-making. When I have been with my colleagues to dinner, we tend to spend more when expenses are being paid than when they are not. People want to spend the allowance - and when they spend the allowance, they stop spending.

According to rational economic theory, this makes no sense at all. The daily allowance has no relevance to your spending - you should spend as you normally would for dinner. Simply, you just buy according to your preferences. However, prospect theory comes in to play to explain this behaviour, as follows.

A Quick Review Of HTTP Essentials

Request and Response

The paradigm conceives of two actors: a client and a server. The client sends a request to the server, of a particular type, and with certain parameters. In reply the server returns a response message. The response message includes a status code in the header - giving information about the requested information, and will also include a body which is the content at the requested URI. Lets look at the process in more detail.

A Friendly FAQ For Unit Testing

The Basics

Unit testing, in its modern form, is the process of writing automated test scripts that run against your units of code. This is usually tied together with OO programming practices, but also can be applied elsewhere. The benefits are as follows:

• Lowers chance of regression in the codebase. This in turn encourages (enables!) refactoring
• Shortens the feedback loop; problems are found earlier
• Is a precise documentation of the system, and the intent of pieces of code
• Encourages good design; it makes you think about the coupling and interfaces of the design

The form of the test is usually quite simple - you will have:

1. A method to set up resources than can be used by all tests
2. A method to initialize before every test common dependencies/common state
3. The test method itself - which includes:
   • Initialization of state and dependencies that are specific to that test method
   • Call to the method under test
   • Assertions of the expected state that is produced by the test method
4. Test-specific cleanup
5. Cleanup of shared resources after all tests have run

Tests will pass when the assertions all return true, and no exceptions are encountered during the test execution. Otherwise, the test will fail.

Becoming a Certified Java Person

Last weekend, I became an Oracle Certified Java Professional. It took some efforts to prepare for this exam, but in the end I was able to pass with flying colours. So, I thought this would be a good time to share some tips on preparing for the exam - and what you might hope to get from that work beyond a certificate.

First off, I have to say, this is not an easy exam. I do not work with Java every day, but I have been using it sporadically and in my own projects for more than two years. Still, it took me a month of hard (albeit part-time) study including a week of heavy cramming to be really well-versed on all the material. Part of the reason for this is that, as a user of the language, you are not necessarily aware of all the different possibilities; once you find some patterns that work you can just re-use them without knowing about all the other options that the language provides for you.

Seven-A-Day

According to a new study that has been doing the rounds in the news, the traditional 5-a-day doesn’t really cut it and we should be eating 7 portions of fruit or veg every day. What’s more, the majority of those portions should be vegetables rather than fruit. For a non-vegetable eating person like me, this certainly seems like a lot. But, the health and lifestyle benefits are scientifically proven, and we should all listen more to solid science. Some of the benefits of eating more fruit and vegetables are:

• Lowering the risk over obesity/type 2 diabetes as vegetables are filling and not high in calories
• Lower the risk of cardiovascular diseases and some cancer
• Keep you healthy and energised with vital vitamins and nutrients you need

I certainly want to get on board with each of these benefits. But, I have a “pragmatic” approach to food - if it is quick, and tasty, that’s what I tend to go for. And, I know many people are in the same boat with me. So, we will have to devise some kinds of strategies to get us eating more healthily - and being more healthy.

Electric Dreams or Electric Relaxation?

I’ve been thinking a little recently about the electric car market. I tend get very excited about the prospect of electric cars taking over our streets (despite the fact that I very rarely need to drive, at the moment). I like the technology, and I like what they stand for. Eco-friendly. Intense acceleration and low centres of mass. This means guilt-free driving pleasure. But, sometimes it seems unclear whether what technical underpinnings lie under this sea change towards electric propulsion (if indeed it is so). And it has seemed obvious until now that the Tesla cars are the stars of the show in regards to electric transportation. These are by far and away the most exciting cars; they get far more hype and they get far more praise than any other electric car on the market. But why? What is really the state of the art and where does it reside? And most importantly, how long will it be until I own my own electric sportscar?

This is a story that can be illustrated with two simple graphics.

My Little Theory About Expansion

Sometimes nature is cold, and the chest heaves with shock as we haul ourselves off to work. She is anti-social. But, she is pretty sometimes.

Even when she is destructive she is surprising. She threatens to break down our schemes and our ideals of law and order. Things may become less useful therein, but do they also become less attractive? And do we sometimes gain when things fall apart?

Using Vim: Just the Essentials

Vim is a text editor with a lot of history and a lot of advocates. It is also universally available on Unix-based systems. And, for this reason, some things become much easier when you know how to use it. It is more confusing initially than other editors - after not using it for a while and coming back to it I have found myself perplexed as how to actually edit the text! But, there are only a small number of things you need to remember to use it effectively.

Android: Editing Lists of Values

I have been working with lists in the Android app I’m working on, and I found that they have a bit of a learning curve. Its always the case in programming - there are some things, that seem like they should be easy, which can often be quite complicated. And, lists are one of those things in Android: not that intuitive; you have to learn to do things the “Android way”. However, after coming to understand the tools that the Android framework gives us, we can quite quickly construct a flexible and robust solution.

Learning about Design with C-K Theory

C-K (“Concept-Knowledge”) theory is a popular and successful model to describe and give insight to the process of design. At its heart, it posits a distinction between the space of knowledge and the space of concepts, and it builds upon this to describe the process of design as a sequence of operations that transform knowledge into concepts, and vice versa.

We may seek to understand it in order to achieve a more structured methodical approach to design, and also to see how the creative design process is unified with the discipline of problem-solving, which might perhaps be considered more scientific in nature.

ScientTUBEism

On different days, the seemingly fixed world that surrounds you becomes surprisingly divergent .

Sakura Falls

One year already since the last cherry blossoms fell.

Sleepy

Sleepy eyes open,
Humid air presses gently.
Memories are lost.

The Mechanics of Productivity

If there’s one thing that I was used to at university, as a diligent physics student, it was long study-sessions. No matter how early I tried to start, there was never quite enough time before exams to learn everything as well as I had been used to. I have a perfectionist streak, so this left a feeling of having left things unfinished with me. And I spent some effort in trying to think of how I can avoid this. After having tried out a number of different productivity systems, not entirely successfully, I decided that I needed to learn much more about the underlying psychology behind concentration and studying. And, it turns out that there are a number of fairly simple psychological principles that can help you in forming effective working habits.

Willpower is a limited resource

There is a psychological phenomenon called ‘ego depletion’, in which subjects who had needed to exert willpower in a task performed more poorly in a subsequent cognitively demanding test [1]. Since maintaining concentration on a task usually takes some willpower[2], this suggests that maximising the amount of time you spend in work is not always the best way to get the most done. Interestingly, exerting willpower or engaging in heavy cognition lowers blood glucose level, and it has been found that ingestion of glucose helps to counteract the effect of ego depletion.That is, a low level of glucose is to be avoided. It is for this reason that I always used to take energy drinks with me to a study session, and, when I was looking at other students around the library, I saw that I was not the only one to have picked up this habit.

When I saw this book I laughed. But when I started to read...

“Made to stick” is a book about sticky ideas. It seems most relevant to marketers. But, the message that it conveys would be useful to anyone. Its all about story-telling, and who doesnt need to tell stories?
The authors advice: in order to communicate a message for maximum recall, we should not rely on facts or figures, but should instead tell stories. Even if not telling a story, we should include some story-like aspects, which are identified as the following:

Simplicity - finding the core of the message. And making it compact.

• “Its the economy, stupid” = What you remember from Bill Clintons presidential campaign. Even if you didnt see the campaign.

Unexpectedness

• Get attention by breaking a pattern. Were good at recognising patterns, and dealing with them without giving them much attention. Break this system!
• Create mystery; highlight a knowledge gap.

Concreteness - Minimise abstraction in factor of examples

• Dont fall prey to the curse of knowledge. A successful story will naturally be an over-simplification. But people can only learn things one at a time. The abstractions that an expert can deduce from experience will not be meaningful to someone without those experiences.
• The Velcro theory of memory - the more hooks, the better
• Sour grapes - the best explanation of cognitive dissonance. From Aesops ancient fable, The Fox and the Grapes

Credibility

• Showing convincing evidence. If you believe that h.pylori bacteria causes ulcers, then drink a solution full of them and see what the result is! This may not be sensible, but the result is convincing. One way or the other.
• Having external credibility. As a doctor, your medical advice would be credible. But, the smoker with lung cancer also gains credibility through their experience of making the wrong decisions. Or how about the Financial Pro Who Lost his House, he has both types of external credibility!
• Minor, quirky details make a story more believable, as well as more memorable.
• To show competence - use examples that pass the Sinatra Test: If you can make it there, you can make it anywhere
• Make your claim easy for people to verify. The logic doesnt have to be perfect. “Wheres the beef?”,leads people to judge a burger by the amount of beef in it.

Emotionality

• If I look at the mass, I will never act. If I look at the one, I will act. We are easily overwhelmed; keep things simple, and on a scale we can hope to tackle. Ask for donations to a single child, or village, in a starving nation, not to solve the entire problem.
• Appeal to self-interest. But, the noble forms of self-interest are the most powerful, self-actualization and so forth. We all have needs described by Maslows hierarchy, but in fact we tackle them all simultaneously.

• “They laughed when I sat down to play the piano. But when I began to play” = one of the most successful marketing slogans of all time. Its aspirational.

• Appeal to Identity. How to stop macho Texans from littering? The message:Dont mess with Texas! , delivered by Texan sport stars.

Why should we want to use stories? Well, firstly, for inspiration: if we see that Jared (of Subway fame) was able to lose so much weight, we feel our problems should be easy to solve. And, to give guidance as how to act; particular cautionary tales will stick in our minds so we dont make the same mistakes.

It is a good book. I dont have too many quibbles. The main criticism would be that none of the advice is that surprising. But, it does do a good job of identifying important issues and showing how often we fail to address them. And, the book mostly tries to follow its own advice, which is reassuring. This book is a useful pointer in the right direction. It makes sense that pairing stories with ideas will make for much more effective communication. Indeed, it seems to unify a lot of ideas about good exposition that I had started to identify through experience.