ryanvgates

[ryanvgates]

AWS MSK Create & List Topics

Problem I needed to created topics in Amazon Web Services(AWS) Managed Streaming for Apache Kafka(MSK) and I wanted to list out the topics after they were created to verify. Solution This solution is written in python using the confluent-kafka package. It connects to the Kafka cluster and adds the new topics. Then it prints out all of the topics for verification This file contains…

View On WordPress

#amazon web services #apache kafka #aws #confluent-kafka #create #kafka #kafka topic #list #managed streaming for apache kafka #msk #python #topic

[ryanvgates]

Generate all PlantUML diagrams

Problem You have many C4 PlantUML diagrams in nested folders that you would like to render without having to run the PlantUML cli on each one. Solution Ensure that your prerequisites are met Install PlantUML on Mac This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden…

View On WordPress

#c4 #c4 plantuml #diagram #diagrams #exec #find #find exec #plantuml #plantuml diagrams #puml #uml

[ryanvgates]

Install PlantUML on Mac

Problem You would like to create PlantUML diagrams on your Mac but don’t have the prerequisites or cli installed. Solution Steps Install java with brew install java Install PlantUML with brew install plantuml Output for Java Your output will look like the below if it is a new installation This file contains bidirectional Unicode text that may be interpreted or compiled differently than…

View On WordPress

#brew #cli #diagram #homebrew #install #java #mac #osx #plantuml #plantuml diagrams #uml #uml modeling #uml modeling with plantuml

[ryanvgates]

AWS Missing Required Attribute

Problem I was trying to create an Elastic Container Service(ECS) service on an ECS cluster that used EC2 instances in AWS. However the tasks would fail to deploy with the following cryptic error. service my-service was unable to place a task because no container instance met all of its requirements. The closest matching container-instance 1234a5b6cd7890e2f34gh5678i8jk is missing an attribute…

View On WordPress

[ryanvgates]

Bash query json

Problem You want to know all of the applications that your team supports across environments. You are working with an api and it gives you a distinct list of all the apps across all environments. This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode…

View On WordPress

#api #array #arrays #bash #jq #read #read array #zsh

[ryanvgates]

Vault CLI Create Policy

Problem You need to grant permissions to secret paths in a uniform way for multiple users and/or roles. You may even need to explain the access to someone working in security or compliance. This becomes much more difficult if you are granting access separately to each user and/or role. Solution Fortunately HashiCorp Vault provides a policy. With a policy, you can define a set of paths and…

View On WordPress

#access #cli #hashicorp #hashicorp vault #hcl #policy #vault #vault cli #vault policy #vault policy write

[ryanvgates]

Terraform staying current

Problem Depending on your CI/CD story for terraform and the sensitivity of your environments, you may need to apply terraform changes manually. The challenge that can happen is that changes in terraform may not be applied. This creates a challenge when you do try to apply a change but see historic changes as well that were never applied. Solution The first step is to understand what changes…

View On WordPress

#apply #arrays #bash #grep #plan #stacks #terraform #terraform apply #terraspace #terraspace list #terraspace plan

[ryanvgates]

Tmux tickets

Problems Like many engineers, I have a support shift. When doing so, much of the work is based around tickets. At the same time, it’s very useful to understand what changes are made for each ticket. This helps for future reference and knowledge sharing as well as understanding changes made if anything goes sideways. The question is how to do this without making it add to the burden of…

View On WordPress

[ryanvgates]

Pipenv Null Markers

Problem While trying to use pipenv to manage python dependencies, I noticed that I got the following error when trying to convert the Pipfile.lock to a requirements.txt file. This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about…

View On WordPress

#pip #pipenv #pipfile.lock #pipfile2req #python #virtual environment

[ryanvgates]

Terraform Linting in CI

Problem Hashicorp Terraform provides a cli with the command terraform fmt which will format terraform files according to the canonical style. This is a great tool. The challenge is that you don’t want to give yourself or team members one more thing they need to remember to do. It would be ideal to integrate it into the Continuous Integration(CI) system so it would always be run but no one would…

View On WordPress

#ci #continuous integration #fmt #terraform #terraform fmt

[ryanvgates]

MongoDB Update Certificate

Problem You try to start your mongo db service and get an error like the following This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters Show hidden characters my-user@my-server:~$ sudo systemctl status…

View On WordPress

#certificate #certificate expired #encryption #expired #mongo #mongod #mongodb #mongos #pem #ssl #systemctl #systemctl restart #tls

[ryanvgates]

Chef Mount Network Drive

Problem You need to mount a network drive onto your Linux server and you have a previously established pipeline using Chef and Ruby. There needs to be a way to do this in a straight forward, repeatable and way that is easy to debug. Solution This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an…

View On WordPress

#action #chef #chef mount #chef recipe #cifs #linux server #mount #mount network drive #mount point #network drive #retries #retry_delay #ruby #verbose

[ryanvgates]

HA Proxy Multiple Sockets

HA Proxy Multiple Sockets

Problem You are leveraging HA Proxy and need to monitor it closely to ensure good performance for your site or service. In most cases, this will mean monitoring. In my case, I leveraged telegraf and sensu for monitoring the socket. However I ran into permission problems trying to have both sensu and telegraf use the same user/group to read the stats. I got the below error. This file contains…

View On WordPress

#haproxy #haproxy cfg #haproxy configuration #haproxy stats #http #sensu #sensu check #socket #stat #stats enable #telegraf

[ryanvgates]

FireEye Agent Unattended Install

FireEye Agent Unattended Install

Problem Need to install the FireEye Agent version 34.28.1 on Windows in a simple and unattended manner. Solution Assuming your msi and configuration file are both in C:\temp then you can run msiexec C:\temp\xagtSetup_34.28.1_universal.msi /qn /quiet /norestart INSTALLSERVICE=1 CONFJSONDIR=C:\temp References StackOverflow: Silent installation of a MSI packageSilent Install BuilderFireEye…

View On WordPress

#fireeye #install #unattended install #Windows #windows unattended install

[ryanvgates]

Chef Foodcritic exit code 3

Chef Foodcritic exit code 3

Problem I was running bundle exec foodcritic -f correctness and it was returning an exit code of 3. This causes builds to fail, like mine in TeamCity. FC007: Ensure recipe dependencies are reflected in cookbook metadata: cookbooks/my_cookbook/recipes/backup.rb:1 I had the following line referencing a recipe in the same cookbook include_recipe 'firewall' Solution The solution was to fully…

View On WordPress

#chef #chef cookbook #chef foodcritic #chef recipe #cookbook #foodcritic #include #recipe

[ryanvgates]

Removing Double Quotes from Token

Removing Double Quotes from Token

Problem You’re authenticating with an api and receiving a token that you need to use for subsequent calls. However the issue is that there are double quotes around response of your token. Without removing the double quotes, the future calls will fail due to an invalid token. This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To…

View On WordPress

#authentication #authorization #bearer #bearer token #curl #double quotes #quotation marks #token #tr

[ryanvgates]

Update Sensu Config

Problem You need to update sensu configuration but would like to avoid redeploying servers. Ideally it needs to be something that can be done simply across multiple servers ideally with one line. The sensu service will need to be restarted as well to apply the new configuration. Solution The below commands will update the configuration and restart the service. This file contains bidirectional…

View On WordPress

#backup configuration #configuration #jq #json #sensu #sensu configuration #systemctl #systemctl restart #systemctl status #tee