Cloud automation setup missing Gen3

Maybe we should have a category for this, I'd hate to add yet another topic on this...but here goes.

  1. I believe this assumption is correct, but I assume the adminVM being created as instructed here: https://github.com/uc-cdis/cloud-automation/blob/master/doc/csoc-free-commons-steps.md is an EC2 instance (assuming an AWS cloud?)

  2. I did the following:

sudo bash cloud-automation/gen3/bin/kube-setup-workvm.sh (didn't seem to work for some reason even though echo $GEN3_NOPROXY was set to no)

sudo su -
GEN3_NOPROXY bash cloud-automation/gen3/bin/kube-setup-workvm.sh (seems to work. Then copied the relevant portions to my user's $HOME/.bashrc)

gen3 gives me command not found under my user.

sudo su -
gen3 as root gives me command not found.

Anything I've missed? the setup completed successfully as far as I can tell.

Site Note: I can get the gen3 binary from compose-services but I assume that's not the correct way of doing this?

Hi @csgeek!

The command bash cloud-automation/gen3/bin/kube-setup-workvm.sh works properly if user has sudo access. You don't need to add sudo or switch to the root user to run the command , just ensure your user is in sudo group. If the user has no sudo access, the command will fail. It looks like your user has sudo access, so I would suggest to run

bash cloud-automation/gen3/bin/kube-setup-workvm.sh

without sudo and under your user (not root).

@Viktorija or anyone on here that's knowledgeable.

I'm essentially trying to setup and evaluate the gen3 cloud automation to see if this will work for our use cases and don't have all the pieces in place yet.

For example according to the guide I should point it to a valid dictionary schema:

dictionary_url="https://s3.amazonaws.com/dictionary-artifacts/YOUR/DICTIONARY/schema.json"

is that an example schema I can use?

Hi @csgeek!

You could use this schema for example: https://s3.amazonaws.com/dictionary-artifacts/datadictionary/develop/schema.json

It is used in Compose Services as a sample schema.

If you are interested in creating your own or modifying dictionary, you can find these resources useful: Creating New Dictionary, Data Types and Templates.

Thank you. I added the dictionary and moving further down on the instruction list.

I create a certificate and update the config as best as I can with all the values requested. Now, when I move on the next step I get an error:

gen3 tfplan
Running terraform plan   --var-file ./config.tfvars out plan.terraform /home/ubuntu/cloud- 
automation/tf_files/aws/commons/
gen3_aws_run terraform plan --var-file ./config.tfvars -out plan.terraform /home/ubuntu/cloud- 
automation/tf_files/aws/commons/
Usage: terraform plan [options] [DIR-OR-PLAN]

The rest is truncated but it's the terraform help file it seems ending with:

 non zero exit code from terraform plan: 1

Any thoughts on what I should do next or how to debug the next step?

Hi @csgeek!

Yes, probably there is an error. Maybe there is additional information in the output, could you please post complete output of the command?

below is the complete output.

gen3 tfplan
Running terraform plan   --var-file ./config.tfvars out plan.terraform /home/ubuntu/cloud-automation/tf_files/aws/commons/
gen3_aws_run terraform plan --var-file ./config.tfvars -out plan.terraform /home/ubuntu/cloud-automation/tf_files/aws/commons/
Usage: terraform plan [options] [DIR-OR-PLAN]

  Generates an execution plan for Terraform.

  This execution plan can be reviewed prior to running apply to get a
  sense for what Terraform will do. Optionally, the plan can be saved to
  a Terraform plan file, and apply can take this plan file to execute
  this plan exactly.

  If a saved plan is passed as an argument, this command will output
  the saved plan contents. It will not modify the given plan.

Options:

  -destroy            If set, a plan will be generated to destroy all resources
                      managed by the given configuration and state.

  -detailed-exitcode  Return detailed exit codes when the command exits. This
                      will change the meaning of exit codes to:
                      0 - Succeeded, diff is empty (no changes)
                      1 - Errored
                      2 - Succeeded, there is a diff

  -input=true         Ask for input for variables if not directly set.

  -lock=true          Lock the state file when locking is supported.

  -lock-timeout=0s    Duration to retry a state lock.

  -module-depth=n     Specifies the depth of modules to show in the output.
                      This does not affect the plan itself, only the output
                      shown. By default, this is -1, which will expand all.

  -no-color           If specified, output won't contain any color.

  -out=path           Write a plan file to the given path. This can be used as
                      input to the "apply" command.

  -parallelism=n      Limit the number of concurrent operations. Defaults to 10.

  -refresh=true       Update state prior to checking for differences.

  -state=statefile    Path to a Terraform state file to use to look
                      up Terraform-managed resources. By default it will
                      use the state "terraform.tfstate" if it exists.

  -target=resource    Resource to target. Operation will be limited to this
                      resource and its dependencies. This flag can be used
                      multiple times.

  -var 'foo=bar'      Set a variable in the Terraform configuration. This
                      flag can be set multiple times.

  -var-file=foo       Set variables in the Terraform configuration from
                      a file. If "terraform.tfvars" or any ".auto.tfvars"
                      files are present, they will be automatically loaded.
non zero exit code from terraform plan: 1

Thanks! I'm just checking... The right order before running tfplan is:

  • run gen3 workon <aws profile> <commons-name>
  • run gen3 cd
  • and edit config.tfvars

Have you run all these commands? If you run gen3 cd and then gen3 tfplan - do you see the same result?

Correct. I always run the following when i start working on this:

gen3 workon cdistest commons-test
gen3 cd

Then edit any updates to config.tfvars and trying to apply the changes where it fails:

gen3 tfplan

Hi @csgeek!

Thank you for confirming your steps. There could be syntax error in the configuration.

Sometimes the causes of the error are slack_webhook and secondary_slack_webhook parameters leaved with default values FILL THIS IN FOR CLOUDWATCH ALARMS. Default value is missing quotes, that could be the reason for an error. If you have these fields in your configuration and don't want to use them for now, just put empty string "" for values or remove them.

Let me know if that is the cause of the error. If not - I'll be glad to help further.

I'm assuming this is in the config.tvars file? I've looked for that pattern in the current directory globally as well config.tvars and can't find anything relevant.

grep -iR "slack" *
grep -iR "CLOUDWATCH" *

Nothing found.

Yes, config.tvars file, sorry if it was not clear.

Thanks for checking. I would like to show your configuration to our developers to help hunting possible issue, but it contains private information and I don't want you to expose on the public forum. Is it OK for you to send me first 25 lines of your config.tvars (it should not contain too sensitive information) via private message? Or maybe you would like to join our slack channel, then you can send me direct message in the slack.