Hello,
I have been trying to set up Arvados on the AWS cloud. For now I want a minimal installation, where I have one EC2 instance for the Core (API + Controller), Keep(Storage etc.) the Workbench and the Websocket, and one other instance for the SSO server. The only reason I have the SSO on a different machine is that it says in the docks that only ruby v2.3 is supported, and not the 2.5 I am using in the first one. The problem I have is the following:
I can see the Workbench, and when I click on the Login button it takes me to the SSO server, where I input my credentials but then I get a We're sorry, but something went wrong. error. If I click Login again, it tells me that I have logged in, and that I can go to Arvados (providing a link). When I click on that link, it takes me to localhost:3000. Anyway, I don’t think the last part is directly correlated to my problem (localhost:3000 looks like it was hardcoded?).
Another detail is this: When I try to create a Trusted Client Flag, like it says in the Workbench installation docs, I run include CurrentApiClient, but instead of => true, I get => Object. This could just be a change in the API, but it felt worth mentioning. In the same console, ApiClient.all returns:
irb(main):003:0> ApiClient.all
=> #<ActiveRecord::Relation [#<ApiClient id: 1, uuid: "abcde-ozdt8-fz4w0ycr4rjh84l", owner_uuid: "abcde-tpzed-000000000000000", modified_by_client_uuid: nil, modified_by_user_uuid: "abcde-tpzed-000000000000000", modified_at: "2020-06-25 15:33:55", name: nil, url_prefix: "https://workbench.ClusterID.example.com/", created_at: "2020-06-25 15:33:55", updated_at: "2020-06-25 15:33:55", is_trusted: true>, #<ApiClient id: 2, uuid: "abcde-ozdt8-tj6fasaz2ta9j96", owner_uuid: "abcde-tpzed-000000000000000", modified_by_client_uuid: nil, modified_by_user_uuid: "abcde-tpzed-000000000000000", modified_at: "2020-06-26 17:26:49", name: nil, url_prefix: "https://localhost/", created_at: "2020-06-26 17:26:49", updated_at: "2020-06-26 17:26:49", is_trusted: true>, #<ApiClient id: 3, uuid: "abcde-ozdt8-csdzwgi3y6hrt9z", owner_uuid: "abcde-tpzed-000000000000000", modified_by_client_uuid: nil, modified_by_user_uuid: "abcde-tpzed-000000000000000", modified_at: "2020-06-26 22:52:17", name: nil, url_prefix: "https://13.53.137.27", created_at: "2020-06-26 22:52:17", updated_at: "2020-06-26 22:52:17", is_trusted: true>, #<ApiClient id: 4, uuid: "abcde-ozdt8-8r13xigjuvidqre", owner_uuid: "abcde-tpzed-000000000000000", modified_by_client_uuid: nil, modified_by_user_uuid: "abcde-tpzed-000000000000000", modified_at: "2020-06-27 11:32:14", name: nil, url_prefix: "https://13.53.137.27:8443", created_at: "2020-06-27 11:32:14", updated_at: "2020-06-27 11:32:14", is_trusted: true>]>
The relevant parts of the configuration are the following (feel free to ask me for more):
My config.yml file :
Clusters:
abcde:
SystemRootToken: "..."
ManagementToken: "..."
API:
RailsSessionSecretToken: "..."
Collections:
BlobSigningKey: "SYFZLtXLUuKp8TtBowlKBfiMXeAUtEEFHp7L7OOY5brSlDvVOe"
PostgreSQL:
Connection:
host: localhost
user: arvados
password: ...
dbname: arvados_production
Services:
Controller:
ExternalURL: "https://13.53.137.27"
InternalURLs:
"http://localhost:8003": {}
RailsAPI:
# Does not have an ExternalURL
InternalURLs:
"http://localhost:8004": {}
Keepstore:
# No ExternalURL because they are only accessed by the internal subnet.
InternalURLs:
"http://localhost:25107": {} # keepstore server on the same machine
Keepproxy:
ExternalURL: "https://13.53.137.27:8005"
InternalURLs:
"http://localhost:25108": {}
WebDAVDownload:
ExternalURL: "https://13.53.137.27:8006" # some free port
WebDAV:
ExternalURL: "https://13.53.137.27:8007" # some other free port
InternalURLs:
"http://localhost:9002": {}
Keepbalance:
InternalURLs:
"http://localhost:9005/": {}
SSO:
ExternalURL: "https://13.48.47.196"
Workbench1:
ExternalURL: "https://13.53.137.27:8443"
Websocket:
InternalURLs:
"http://localhost:8008/": {}
ExternalURL: wss://13.53.137.27:8445/websocket
Users:
AnonymousUserToken: "3i2xoi6d364mb0w9i872ae3jzzs51qpdghjv0ra8gdnlxy9apq"
AutoAdminFirstUser: true
NewUsersAreActive: true
AutoSetupNewUsers: true
Workbench:
SecretKeyBase: ...
Login:
ProviderAppID: "arvados-server"
ProviderAppSecret: ...
Volumes:
abcde-nyw5e-000000000000000:
AccessViaHosts:
"http://localhost:25107": {}
Driver: Directory
DriverParameters:
# The directory that will be used as the backing store.
Root: /home/ubuntu/storage
# How much replication is performed by the underlying
# filesystem. (for example, a network filesystem may provide
# its own replication). This is used to inform replication
# decisions at the Keep layer.
Replication: 1
# If true, do not accept write or trash operations, only
# reads.
ReadOnly: false
# Storage classes to associate with this volume.
StorageClasses: null
Relevant snippets from the corresponding nginx config files:
arvados-api-and-controller.conf :
proxy_http_version 1.1;
geo $external_client {
default 1;
127.0.0.0/24 0;
10.20.30.0/24 0;
13.48.47.196/32 0; # make the other instance be considered internal
# 1.2.3.4/32 0;
}
# This is the port where nginx expects to contact arvados-controller.
upstream controller {
server localhost:8003 fail_timeout=10s;
}
server {
listen *:443 ssl;
#server_name xxxxx.example.com;
ssl on;
ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt;
ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key;
# Refer to the comment about this setting in the passenger (arvados
# api server) section of your Nginx configuration.
client_max_body_size 128m;
location / {
... haven't changed anything
}
}
server {
listen localhost:8004;
server_name localhost-api;
root /var/www/arvados-api/current/public;
index index.html index.htm index.php;
passenger_enabled on;
client_max_body_size 128m;
}
arvados-workbench.conf :
server {
listen 80;
#listen 13.53.137.27:8005;
#listen 8005;
#return 301 https://workbench.ClusterID.example.com$request_uri;
#return 301 https://$host$request_uri;
return 301 https://13.53.137.27:8443$request_uri;
}
server {
listen *:8443 ssl;
#listen 13.53.137.27:443 ssl;
ssl on; # possible it's not needed
ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt;
ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key;
root /var/www/arvados-workbench/current/public;
index index.html;
passenger_enabled on;
passenger_friendly_error_pages on;
client_max_body_size 128m;
}
arvados-sso.conf (on the 2nd instance) :
server {
listen 443 ssl;
ssl on;
ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt;
ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key;
root /var/www/arvados-sso/current/public;
index index.html;
passenger_enabled on;
}
application.yml on the instance hosting the SSO:
production:
uuid_prefix: abcde
secret_token: ...
allow_account_registration: true
require_email_confirmation: false
In all the logs I’ve searched, the most helpful snippet I found was this, in /var/www/arvados-workbench/current/log/production.log :
{"method":"GET","path":"/users/welcome","format":"html","controller":"UsersController","action":"welcome","status":200,"duration":4.51,"view":3.52,"request_id":"req-67gfszbcc9j7jkfz6klm","params":{"return_to":"/"},"@timestamp":"2020-06-26T22:53:00.363Z","@version":"1","message":"[200] GET /users/welcome (UsersController#welcome)"}
#<ActionController::RoutingError: Path not found>
#<ActionView::MissingTemplate: Missing template links/404, application/404 with {:locale=>[:en], :formats=>["text"], :variants=>[], :handlers=>[:raw, :erb, :html, :builder, :ruby, :coffee]}. Searched in:
* "/var/www/arvados-workbench/current/themes/default/views"
* "/var/www/arvados-workbench/current/app/views"
Any help would be appreciated at this point. Thanks in advance and sorry for the long post!