Check the 'Source Control Management' section, and validate the LAST COMMIT is correct1. Viewing the status of the Application:
| Code Block |
|---|
nex-cli apps:info my-app |
If the status is 'ACTIVE'the App Details section "STATUS" is "active", and the Source Control Management section "LAST COMMIT" is correct, if so, your deployment should be ok & running.
If the status is '"restarting' " and it has beens some time, then it may be having troubles starting and is worth investigating the cubes.
2. Viewing the status of the cubes, and checking the logs
List the cubes for your app:
| Code Block |
|---|
nex-cli cubes --app my-app --all |
Most likely the cube that has crashed status will be "performing:starting".. SSH into the rack via the cubes using the crashed cube's IP address listed from the command above with:
| Code Block |
|---|
nex-cli racks:ssh 10.0.1.234 |
List the docker containers to check the statusViewing the logs for the cube:
| Code Block |
|---|
sudo su docker pslogs -a |
If a container "exited" (should say in the status), there is something wrong. View the logs:
| Code Block |
|---|
docker logs CONTAINER-NAME |
Hopefully you have identified your issue :)
Resetting the Nex! rack lock
...
CUBE-INSTANCE-ID |
3. Logs don't look like your new code has been applied?
Whilst ssh'ed in the rack, list the docker containers to check the status:
| Code Block |
|---|
sudo su
docker ps -a |
If the status of the container with the same NAME as the crashed cube instance ID has exited hours or days ago, and you recently deployed, then the new code hasn't been applied properly.
4. Deleting the Nex! Locks & Restarting the Cubes
The lock files are used to control the state of actions currently undertaken on containers. If a containers crashes during provisioning the lock will stay there and the container will be forever considered as “starting” or “provisioning”.
SSH into the rack, and then delete the Nex! Locks:
| Code Block |
|---|
rm -fr /var/lock/nex/* |
Scale applications down as many times as there are Cube
| Code Block |
|---|
nex-cli apps:down my-app |
After the apps:info status is "stopped" for all Cube Instances, scale apps up as many times are there are Cube Instances:
| Code Block |
|---|
nex-cli apps:up my-app |
When all Cubes apps:info status is "running", see if it's now working.