Failed to start Dynatrace Server

[Vuceti]

hello,
I have a question about how I can correct this server.sh service, after I have updated my operating system from red hat 7.9 to 8.6, after the restart it is not loading the server.sh.
I get these details.

Redirecting to /bin/systemctl status dynatrace-server.service
● dynatrace-server.service - Dynatrace Server
Loaded: loaded (/etc/systemd/system/dynatrace-server.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Fri 2023-09-15 13:44:51 -05; 7min ago
Process: 171705 ExecStart=/opt/dynatrace-managed/server/services/server.sh start (code=exited, status=2)
Tasks: 2 (limit: 32768)
Memory: 341.7M
CGroup: /system.slice/dynatrace-server.service
└─7664 /opt/dynatrace-managed/server/dynatraceserver -config /opt/dynatrace-managed/server/conf/dynatraceserver.ini -bg

Sep 15 13:44:51 dynaservamt3.vuce.gob.pe systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Sep 15 13:44:51 dynaservamt3.vuce.gob.pe systemd[1]: Starting Dynatrace Server...
Sep 15 13:44:51 dynaservamt3.vuce.gob.pe server.sh[171705]: Starting Server ...
Sep 15 13:44:51 dynaservamt3.vuce.gob.pe server.sh[171705]: Launcher PID: 7664, main PID: , NOT listening on ports: 8021
Sep 15 13:44:51 dynaservamt3.vuce.gob.pe server.sh[171705]: Starting Server ... the launcher is running, it should restart main process soon
Sep 15 13:44:51 dynaservamt3.vuce.gob.pe systemd[1]: dynatrace-server.service: Control process exited, code=exited status=2
Sep 15 13:44:51 dynaservamt3.vuce.gob.pe systemd[1]: dynatrace-server.service: Failed with result 'exit-code'.
Sep 15 13:44:51 dynaservamt3.vuce.gob.pe systemd[1]: Failed to start Dynatrace Server.
Redirecting to /bin/systemctl status dynatrace-security-gateway.service
● dynatrace-security-gateway.service - Dynatrace Active Gate
Loaded: loaded (/etc/systemd/system/dynatrace-security-gateway.service; enabled; vendor preset: disabled)
Active: inactive (dead) since Fri 2023-09-15 13:04:03 -05; 48min ago
Process: 160363 ExecStop=/opt/dynatrace-managed/security-Gateway/services/security-gateway.sh stop (code=exited, status=0/SUCCESS)
Process: 32886 ExecStart=/opt/dynatrace-managed/security-Gateway/services/security-gateway.sh start (code=exited, status=0/SUCCESS)
Main PID: 34668 (code=exited, status=143)

[natanael_mendes]

The error message you're seeing in the systemctl status output indicates that the Dynatrace Server service failed to start. Here are some steps you can follow to diagnose and potentially correct the issue:

1. Check the logs: Look into the logs generated by Dynatrace Server for more detailed error messages. You can find these logs in the Dynatrace Server's log directory, which is typically located at `/var/log/dynatrace/`. Look for any specific error messages or stack traces that might help you pinpoint the issue.

2. Review the systemd service unit file: Make sure that the systemd service unit file `/etc/systemd/system/dynatrace-server.service` is correctly configured for your Dynatrace Server installation. Check that the `ExecStart` line points to the correct `server.sh` script and that all required environment variables or configurations are set. You might need to update this file if paths or configurations have changed after your OS upgrade.

3. Permissions and Ownership: Ensure that the `server.sh` script and all related files and directories have the correct permissions and ownership. The user account running the service should have appropriate access to all necessary files.

4. Check for missing dependencies: After upgrading your operating system, some dependencies or libraries required by Dynatrace Server may have been updated or removed. Check if there are any new dependencies needed by Dynatrace Server, and make sure they are installed on your system.

5. Systemd Service Reload: After making any changes to the systemd service unit file or other configurations, reload the systemd manager with the following command to ensure it recognizes the changes:

```bash
sudo systemctl daemon-reload
```

6. Try starting the service manually: You can attempt to start the Dynatrace Server manually to see if it provides more detailed error messages. Use the following command:

```bash
sudo /opt/dynatrace-managed/server/services/server.sh start
```

This might give you more immediate feedback on what's causing the issue.

7. Check for resource constraints: Ensure that there are no resource constraints like low memory or disk space that might be preventing Dynatrace Server from starting properly.

8. Check for configuration changes: Review any configuration files that Dynatrace Server relies on (e.g., `dynatraceserver.ini`) to ensure they are still valid and compatible with the upgraded OS.

9. Check for SELinux or AppArmor: If you're using SELinux or AppArmor, ensure that the policies are correctly configured to allow Dynatrace Server to run.

10. Contact Dynatrace Support: If you continue to face issues, consider reaching out to Dynatrace support for assistance. They may have specific troubleshooting steps or patches for compatibility with Red Hat 8.6.

Remember to make backups of any configuration files or settings before making changes, and proceed with caution when troubleshooting system services.

[Taylor-Sanchez · Accepted solution]

Hi @Vuceti ,

I think this question could best be answered by the Dynatrace Support team by adding some additional files from the logs that are collected by Dyntrace to troubleshoot the issue. 

Link: https://support.dynatrace.com/

In addition, this would allow for you to give context to Support Engineers by providing the Log Output. The most relevant logs in the Dynatrace files would be: 

• Server Launcher Logs
• Server.0.0.log
• launc-logging.log

You can also download the SupportArchives as a Zip file and append it as a whole. 

[pradeepboda]

Could you please let me know how did you fix the issue.

I am also facing same error.

● dynatrace-server.service - Dynatrace Server
Loaded: loaded (/etc/systemd/system/dynatrace-server.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2023-10-10 14:38:21 EDT; 18s ago
Process: 32199 ExecStart=/dynatrace/server/services/server.sh start (code=exited, status=2)
Tasks: 2 (limit: 32768)
CGroup: /system.slice/dynatrace-server.service
└─11664 /dynatrace/server/dynatraceserver -config /dynatrace/server/conf/dynatraceserver.ini -bg