Issue with Compose Services

Zander_Giuffrida · January 23, 2020, 7:15pm

Hey @Viktorija could you send me an invite to the slack channel? Thanks!

Viktorija · January 23, 2020, 8:17pm

We sent you an invite @Zander_Giuffrida

Viktorija · January 29, 2020, 4:11pm

Hi @Zander_Giuffrida!

Our developers suggest to run this command before starting Compose Services:

sudo sysctl -w vm.max_map_count=262144

This would help to start esproxy-service. Could you try this and let me know the result?

Zander_Giuffrida · January 29, 2020, 5:34pm

Hey @Viktorija,

Yes that seems to be working. I'll let you know if I run into any other issues.

Thanks!

Zander

Zander_Giuffrida · February 4, 2020, 6:50pm

Hello @Viktorija,

I've been trying to get data upload to work on my local gen3 installation, and ran into some issues similar to what's described in this post:

I added the self-signed certificate to trusted certificates on my OS (Ubuntu). After a few tries I was able to configure a profile using the following command:
gen3-client configure --profile=zander --cred=~/compose-services/credentials.json --apiendpoint=https://localhost

When I run
gen3-client auth --profile=zander
it returns:
Screenshot from 2020-02-04 13-42-17

However, when I run
gen3-client upload --profile=zander --upload-path=~/Documents/text.txt
I get the following error:

Could you help me resolve this issue?

Thanks!

Zander

Viktorija · February 4, 2020, 7:41pm

File submission is not fully working in Compose Services, but it should not give you Internal Server Error. Could you share your logs?

Also I wonder if you are able to create program/project in your commons like described here

Zander_Giuffrida · February 19, 2020, 9:45pm

Hello @Viktorija. Sorry for the delay in my response!

It seems that I'm unable to get the portal to show up on localhost again. Here's a link to the zip I generated from the dump.sh script.

Viktorija · February 19, 2020, 11:03pm

Hi @Zander_Giuffrida!

Looks like fence is not happy trying to reach AWS buckets and getting errors. I will consult with developers on this and let you know. Do you experience the same with docker-compose down and then docker-compose up -d ?

Viktorija · February 20, 2020, 6:56pm

Hi @Zander_Giuffrida!

Could you please try this:

Shutdown compose services with docker-compose down
Comment out these lines in the ngninx.conf:

#        location /guppy/ {
#                proxy_pass http://guppy-service/;
#        }

Modify the AWS credentials block and add aws_access_key_id: '' and aws_secret_access_key: '' to credentials in the fence-config.yaml

AWS_CREDENTIALS:
  'CRED1':
    aws_access_key_id: ''
    aws_secret_access_key: ''
  'CRED2':
    aws_access_key_id: ''
    aws_secret_access_key: ''
S3_BUCKETS:
  bucket1:
    cred: 'CRED1'
  bucket2:
    cred: 'CRED2'
  bucket3:
    cred: '*'
  bucket4:
    cred: 'CRED1'
    role-arn: 'arn:aws:iam::role1'

Start docker compose with docker-compose up -d

Please let me know the result

Zander_Giuffrida · February 20, 2020, 8:21pm

Hey @Viktorija

I'm able to get the login screen to come up, but when I try to login it throws an Internal Server Error.

Here's a link to the logs

Viktorija · February 20, 2020, 9:01pm

Hi @Zander_Giuffrida,

From your new logs I see Fence complains about AWS buckets and fence-config does not contain aws_access_key_id and aws_secret_access_key fields in the AWS_CREDENTIALS block. Could you please modify fence-config.yaml as described in Step 3 in the post above and see if it helps?

Zander_Giuffrida · February 20, 2020, 10:09pm

Hey @Viktorija

Those fields were already present in fence-config.yaml. Here's a screenshot of it.

Viktorija · February 20, 2020, 10:18pm

It's interesting that fence didn't pick changes up. Did you change fence-config.yaml in the templates folder or in Secrets folder? The change should go to Secrets/fence-config.yaml

Zander_Giuffrida · February 20, 2020, 10:19pm

I changed Secrets/fence-config.yaml

Viktorija · February 20, 2020, 10:26pm

Do I understand correctly, if you re-run docker compose with modified fence-config.yaml on your new logs you don't see your changes to fence-config.yaml? I will check with developers what's happening.

Viktorija · February 21, 2020, 3:27pm

Oh, you are right! dump.sh script removes creds information for privacy reasons:

github.com

uc-cdis/compose-services/blob/f1d2be4c3c893bfecfc19ee54bb30d8ea2c9a689/dump.sh#L45


	mkdir -p $dirname/config/
fi


if $get_config; then
	echo "Copying config files"
	cp docker-compose.yml $dirname/config/
	cp Secrets/user.yaml $dirname/config/
	cp Secrets/*config.* $dirname/config/
	cp Secrets/*settings.* $dirname/config/


	# remove lines containing creds
	if [[ "$OSTYPE" == "linux-gnu" ]]; then
		sed -i "/key/Id" $dirname/config/*
		sed -i "/secret/Id" $dirname/config/*
		sed -i "/password/Id" $dirname/config/*
	elif [[ "$OSTYPE" == "darwin"* ]]; then # MacOS
		sed -i "" "/[Kk][Ee][Yy]/d" $dirname/config/*
		sed -i "" "/[Ss][Ee][Cc][Rr][Ee][Tt]/d" $dirname/config/*
		sed -i "" "/[Pp][Aa][Ss][Ss][Ww][Oo][Rr][Dd]/d" $dirname/config/*
	else
		echo "WARNING: did not remove lines with creds (unknown OS $OSTYPE)"

Zander_Giuffrida · February 21, 2020, 5:29pm

I've pulled the latest images but still get 'internal service error' when I try to login. Do you think it would be worthwhile to try a clean install?

Viktorija · February 21, 2020, 6:12pm

I would try
I asked our developers to look at your logs again, it might take some time and I'll let you know their opinion on what could break the work.

Zander_Giuffrida · February 24, 2020, 3:45pm

Unfortunately the clean install didn't work. Here are the latest logs.

Viktorija · February 24, 2020, 5:03pm

Thank you for new logs. Let me check with developers and Ill get back to you.