Issue with Compose Services

Do I understand correctly, if you re-run docker compose with modified fence-config.yaml on your new logs you don't see your changes to fence-config.yaml? I will check with developers what's happening.

Oh, you are right! script removes creds information for privacy reasons:

I've pulled the latest images but still get 'internal service error' when I try to login. Do you think it would be worthwhile to try a clean install?

I would try :slight_smile:
I asked our developers to look at your logs again, it might take some time and I'll let you know their opinion on what could break the work.

Unfortunately the clean install didn't work. Here are the latest logs.

Thank you for new logs. Let me check with developers and Ill get back to you.

Hi @Zander_Giuffrida,

I just want to share that we were able to reproduce this behavior. We are looking how to fix it. I'll keep you updated.


Hi @Viktorija - I'm running into similar issues with compose services.

I've got the portal up and running, but I'm still getting a 500 error on the Fence service when I run the smoke script -- I get an Internal Service Error when I try to log-in.

Here's my dump link


Hi @steviejacs! Welcome to the forum! :slight_smile:

It looks the problem is on our side, we are working to fix it. I'll update you when have more information.

@Viktorija super! in the meantime, would you mind sending me an invite to the slack channel?

thanks for the support!

@steviejacs, the invite was sent your way :slight_smile:

Hey @Viktorija,

Here is a link to my logs with the latest docker images for compose services.

I will be testing out cloud-automation on AWS soon. I saw that you previously mentioned this site for help with making a budget. Are there any resources you'd recommend I look at for making a budget of cloud provider costs with gen3?

Also do you know if users have had issues with cloud-automation similar to what I'm experiencing with compose-services?



Hi @Zander_Giuffrida and @steviejacs!

I'm sorry it took so long but we have a fix:

  • shutdown compose services with docker-compose down
  • update your repository with git pull
  • update images with docker-compose pull
  • change fence-config.yaml in your Secrets folder as shown here leaving an empty list for AWS_CREDENTIALS and S3_BUCKETS (we updated template for this file and in the new setup it will automatically create a fixed version in Secrets, but in your current setup you already have a copy of the old fence configuration under your Secrets folder so need to update it manually this time).
  • start Compose Services with docker-compose up

This should fix a fence error.

Hi @Zander_Giuffrida!

I don't think I have other resources for making a budget at this point. I'll ask other people maybe they will suggest something and I'll share with you.

For Cloud Automation it is hard to predict, it could go easy or not, but my recommendation would be to raise any questions you have on the gen3_community slack channel - there is a lot of expertise from our developers and also from independent groups who already have this experience and you can get a good advice from them.

Hi @Viktorija - I'm still running into the same issue :thinking:

did this fix for others?

Here's a link to the dump

Thanks again for the help!

I will take a look tomorrow and update you. Did you pull new images? Thanks for the logs!

Actually it is another error from fence:

OSError: missing public key file; expected file to exist: /fence/keys/2020-03-12T20:43:34Z/jwt_public_key.pem
[36mfence-service        |[0m unable to load app 0 (mountpoint='') (callable not found or import error)

Do you have jwt_public_key.pem file in your Secrets/fenceJwtKeys/2020-03-12T20:43:34Z folder? It should be created automatically in this step by script

Odd - I ran the script (when i first set it up) but it didn't create the jwt_public_key.pem file...

just ran it again and still same issue with the new secrets folder.

Hm, I can not reproduce this behavior on my setup. Also I downloaded fresh compose services and still can not reproduce it. Do you have openssl installed as mentioned here in dependencies? I'm asking because script uses openssl to generate keys, please see here.