P45Bot - Solving a problem that has already been solved

Design decisions are a double edged sword. In one instance they give clear guidance as to why something is like it is. On the other they serve as a shield as to why something has been implemented in an a less than ideal manner :) Invariably as engineers we bump up against both sides of this coin. One particular example of this was recently operating with terraform. The design decision was "identity is the perimeter". Fair enough but this led to UPNs of users being used in terraform code and as the inevitable churn occurred then pipelines would break. The problem already has a solution and that is to front calls to groups and populate groups with users. Alas this was an abstraction too far for the powers that be but the upshot of this is it gives us an excuse to play around with Go and over engineer a solution to the problem :)

The Plan

So, we know what we want to solve and that is finding UPNs in terraform HCL files that are no longer valid in our domain. To achieve this we are going to do the following:

• Write a cli tool - this means we can use cobra-cli
• Allow for multiple configurations to be used - means we get to use viper
• Find UPNs in terraform files
• Look these up and check for validity (we are using Azure AD as our source of truth for this).

The hope is that by avoiding hardcoding the configuration we can make this cli tool modular and plumb in different use cases or actions. Today we will just report on what we find but maybe tomorrow I want to search YAML manifests for strings and then shout to MS Teams or raise an automated PR with a fix. If we configure this correctly we should be able to achieve that.

The Design

By using a combination of cobra and viper we can very quickly get a useable framework of a cli up. We can quickly define the top level commands, flags and config structures we want. Granted these will do nothing initially except fmt.Println but a lot of the complexity will be removed. Then we can start looking at how we write the individual pieces of the puzzle and add the in the actual meat.

Top Level Commands

• validate - will validate the config be that input or core config as well as ensure that required environment variables are set.
• set - will set the various bits of core configuration via flags and then written to config file using viper.
• scan - will actually do the work of scanning the terraform files and checking the UPNs in AAD. It will take an input manifest/config file.

Flags

• config - each command should take a config path so that multiple core configs can be passed in. This should allow for different domains/tenants to be used or different alert shouts to be used.
• manifest - these will be individual config files that detail what to search for, in what location and what type of filetype is being scanned.
• dryrun - always useful to have something that details what will be carried out by the program before it actually happens. Will most likely be more useful when we get to automating the fixing of the UPNs but have included it here.
• verbose - make the output verbose and give us all the detail we are after when debugging or just looking to try and understand what is happening behind the scenes. Each command may well end up with its own collection of flags (for example I think the scan should take a -Directory flag and then probably a -Recurse option). Also SPOLIER ALERT I haven't actually written or tested this. I therefore reserve the right to abandon the plan and freestyle my way to acceptability :)

The Coding Bit

Lets get things setup. You will of course need go installed, an editor, git and cobra-cli (eases the setup).

Cobra can be installed easily enough using the following:

go install github.com/spf13/cobra-cli@latest

Now we can get things rolling:

mkdir p45bot
cd p45bot
go mod init p45bot

Then you can stand up the app with the following command:

cobra-cli init p45bot --viper

The flag --viper tells cobra that we want to use the viper tool for configuration. This makes importing config from file, variables etc so much easier.

We should now have something like this:

Then we can add in the commands we discussed above with the following:

cobra-cli add scan
cobra-cli add validate
cobra-cli add set

Which should give us:

We can test that it has all worked by running it and can even pass in the commands:

go run main.go

A longer description that spans multiple lines and likely contains
examples and usage of using your application. For example:

Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.

Usage:
  p45bot [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  scan        A brief description of your command
  set         A brief description of your command
  validate    A brief description of your command

Flags:
      --config string   config file (default is $HOME/.p45bot.yaml)
  -d, --dryrun          dryrun only
  -h, --help            help for p45bot
  -t, --toggle          Help message for toggle
  -v, --verbose         verbose output

Use "p45bot [command] --help" for more information about a command.

Or we could run:

go run main.go scan

scan called

Adding flags in we need to actually open up our editor. We want to add in some global flags for verbosity and dryrun behaviour so we open up cmd/root.go and add in the following lines:

var Verbose bool
var DryRun bool

This defines the variables that go will use and we then need to tell cobra to use them in the commands:

  func init() {
    cobra.OnInitialize(initConfig)
    // Here you will define your flags and configuration settings.
    // Cobra supports persistent flags, which, if defined here,
    // will be global for your application.
    rootCmd.PersistentFlags().StringVar(&cfgFile, "config", "", "config file (default is $HOME/.p45bot.yaml)")
    rootCmd.PersistentFlags().BoolVarP(&Verbose, "verbose", "v", false, "verbose output")
    rootCmd.PersistentFlags().BoolVarP(&DryRun, "dryrun", "d", false, "dryrun only")
    // Cobra also supports local flags, which will only run
    // when this action is called directly.
    rootCmd.Flags().BoolP("toggle", "t", false, "Help message for toggle")
}

For flags we need to provide the type of flag, the type of the data, variable to use, longname, shortname, default value and a description. The Verbose flag therefore has the following information:

• PersistentFlags() - This allows the flag to be set at the root and be available to all sub-commands. So we only have to configure it once to have it available throughout the app.
• BoolVarP - This defines a bool flag.
• &Verbose - This argument points to the variable we defined earlier to store the value in
• "verbose" - is the longform name of the flag
• "v"- is the shortform name of the flag
• false- is the default value for the flag
• "verbose output" - provides a usage description visible in the help of the command when we run with --help.

We can add in the other flags I spoke about earlier in a similar manner.

Next Steps

We now have a cli app and it has taken very little effort. It also does not actually do anything of merit except print a bunch of stuff to stdout. I plan to split the lifecycle of this cli app out over a number of blogs as I write it. So the next step I think would be to spend some time completing the help information of the cli. Cobra has given us a great framework and if we are decent engineers then we should be ensuring that the next poor sap that inherits our tooling can get up to speed as quickly as possible. Part of that comes with naming things logically within the code but a decent help guide goes a long way. After that we can start adding in actual code to go off and start doing things. Hopefully you will come back for the next update!