Hello Team,
We have successfully upgraded our arvados server to 2.7.0. Everything seems to work and small
jobs are being completed, however, if we submit a job which require a bit more resources (whole genome
analysis) the some of the intermediate jobs are getting cancelled because of timeouts. Also, sometimes
when we browse the workbench the requests are getting stuck and the only way to bring it back is to
restart the controller. Here are some of the errors in our logs:
Error retrieving services list: Get “https://cborg.cbrc.kaust.edu.sa/arvados/v1/keep_services/accessible”: context deadline exceeded (Client.Timeout exceeded while awaiting headers) (retrying in 3s)
cborg-dz642-rl2ukbq8e3npyxq: error fetching container record: Get “https://cborg.cbrc.kaust.edu.sa/arvados/v1/containers/cborg-dz642-rl2ukbq8e3npyxq”: context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Any suggestions?