Keeping things manageable
2024-01-03
I'd like to share some of my notes about how I keep things manageable in my consulting engagements. These engagements have gone on long enough to reveal which tactics are valid and which ones are unhelpful or even dangerous.
- I host my services and batch jobs on Ubuntu virtual machines. Some of the FOSS services I use are packaged as Docker images, so I tend to use Docker on my virtual machines, too. Ubuntu and Docker are both popular, so if something goes wrong, I like my odds of being able to find help.
- I write important things down. My technical and business notes live in plaintext (.txt or .md) format in a folder I sync to Dropbox. Having portable notes has been a lifesaver for me and my bad memory.
- I use technologies with a fast feedback loop. Python is my daily driver because I primarily work in data engineering, but I have used Go and Elixir for my side projects. I find all three of these languages to be engaging due to how easy it is to test code I write. This helps me stay connected to the outcomes I want to achieve.
- I scrutinize dependencies. Every dependency I add to a project becomes another point of failure as time passes and versions drift. Some of them are helpful for projects to come online quickly, but a surprising number are not and should thus be removed or avoided. If I do add a dependency, I do my best to ensure that I can reproduce the build.
One thing I never compromise on now is security. I have several layers of access control to my virtual machines in place that sometimes make it difficult for me to reach my own servers if I am away from my primary workstation. I also do my best to avoid hosting any sensitive data such as PII on internet-exposed services at all so that even if there is a breach, nothing of value is compromised. These measures make it slightly more difficult to do my job, but they are well worth it.
Please note that I normally develop and maintain my clients' systems alone and that I know how to operate each tool I mentioned here. If you are working in a team or if you don't have an equivalent breadth of skills yet, these notes may not necessarily apply to you.