Issues when installing on AWS - Cloud automation

Dear all

I am newbie trying to install Gen3 on AWS.
following the guide in https://github.com/uc-cdis/cloud-automation/blob/master/doc/csoc-free-commons-steps.md I feel like I am missing some information. For example nowhere I read I could not use Amazon Linux and Debian is needed.

Now I am stuck with the third step of the installation:

gen3 workon default commons-test

I get the following error:

An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:sts::xxxxxxx:assumed-role/Ec2-gen3/i-0c1xxxxxxxxxcfc is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::COMMONS-ACCOUNT-ID-HERE:role/csoc_adminvm

WhileI can understand the specific AWS Role, I am missing to understand the other part: COMMONS-ACCOUNT-ID-HERE:role/csoc_adminvm

  • When / where that role "csoc adminvm has been created?
  • where to populate COMMONS-ACCOUNT-ID-HERE account

Any pointer of all I am missing to run smoothly the installation is appreciated

Thanks
-Dan

Hi @DanResearch! Welcome to the Gen3 forum :slightly_smiling_face:

In part 1 step 5 there is description of {HOME}/.aws/config file where you add a profile for your commons. I believe the default profile contains the following lines:

[default]
output = json
region = us-east-1
# Comment these out if not running on adminvm
role_arn = arn:aws:iam::COMMONS-ACCOUNT-ID-HERE:role/csoc_adminvm
credential_source = Ec2InstanceMetadata

You may want to comment out the role_arn = ... line if you are not running on adminvm.

Thanks for your reply Viktorija.

Unfortunately, that line is not in my profile..

I believe my question might be more generic: I am very proficient with AWS, however I find the Gen3 installation guide is really above me... What do I need to read before understanding fully the steps in the installation guideline? It seems to me that the guidelines take for granted some Gen3 knowledge that user like me donĀ“t have.. like in your reply ... you are assuming I know the difference between adminvm or not.
What do you recommend to read?

Hi @DanResearch,

I agree with you, Cloud Automation might require some effort to install and I'm sorry you are running into issues and questions.

Here is our generic documentation that might help you to understand Gen3 Cloud Automation structure better, and particularly this part is about Independent Setup and CSOC in general, and this part is about CSOC-attached setup and adminvm role. Initially, Cloud Automation was developed for internal use within our center and it was bound to our internal CSOC. Later we added the possibility to use Cloud Automation for third parties and removed dependencies to our CSOC and created this documentation about SCOC free commons steps :slight_smile:

Another resource for documentation might be /docs folder. You can use the gen3 help command ( gen3 help gitops or gen3 gitops help will open /docs/gitops.md) command or just grep command.

About role_arn line I'm confused because {HOME}/.aws/config is the only place I might think of. My understanding is that this folder is created on Part 1, step 3 with bash cloud-automation/gen3/bin/kube-setup-workvm.sh in, and according to the script the role_arn line should be placed in the [default] profile.
Can you check and confirm if it is in the [default] profile or not? That line is placed in the [default] profile, not the profile you added in the same file. The {HOME}/.aws/config file should contain two profiles: [default] and your commons.

Thanks for the reply Viktorija.

Now I understand a bit more of the overall architecture. By the way, the link to the CSOC architecture is not working (https://github.com/uc-cdis/cdis-wiki/blob/master/ops/CSOC_Documentation.md).

Please allow me to ask, as I am progressing, about section " Second part start gen3" on the guide at cloud automation in step 3:

It talks about the following variable "peering_vpc_id" and "peering-cidr". In an enviroment where there will not be an adminvm, as it installed outside the CSOC environment.

What to do with them? the "gen3 tfapply" fails to that VPC: " **module.cdis_vpc.data.aws_route_tables.control_routing_table: data.aws_route_tables.control_routing_table: no matching route tables found for vpc with id vpc-e2b51d99**".
(and no VPC was given, so not sure where it got the VPC id was obtained from)

Hi @DanResearch!

Maybe this explanation would help. Here is an example of the configuration:

vpc_cidr_block="192.168.144.0/20"
csoc_managed=false
peering_cidr="172.25.64.0/24"
peering_vpc_id="vpc-aaaaaaaaaaaaa"

The vpc_cidr_block variable is where your Gen3 resources will be, so choose whatever you want or feel comfortable with. The peering_cidr, though, must be the CIDR in which the adminVM (or the instance you are running gen3 commands) belongs, and the same for vpc_id. I believe the value you are seeing is the default value that comes from this file.csoc_managed is a boolean type variable, so it should be without quotes of any kind csoc_managed=false.

More information about VPC and CIDR is available on AWS documentation

Maybe you want to join our slack channel, where Gen3 enthusiasts share their experiences?