topic Re: JDBC driver settings when building a custom extension in Extensions

JDBC driver settings when building a custom extension

Adrian_I — Thu, 27 Mar 2025 08:17:30 GMT

Hello,

We have recently built a custom extension to fetch data from a SQL data source.

Our underlying data source is an Exasol cluster and in order to establish the connection we have enabled the JDBC driver of the vendor according to the official instructions documented here.

Initially everything went according to plan, the data was fetched every minute so we started expanding the number of jdbcRemote endpoints to have complete coverage according to our needs.

After a few days of observing we noticed that from time to time some endpoints are failing randomly with the following error message:

[out]org.apache.tomcat.jdbc.pool.PoolExhaustedException: [pool-4-thread-1] Timeout: Pool empty. Unable to fetch a connection in 30 seconds, none available[size:1; busy:1; idle:0; lastwait:30000].

This was not a limitation on the data source side as the connection pool there is way larger and we were also not even seeing an attempt to connect. We quickly concluded that the default connection pool size of our JDBC driver is actually 64, so what mechanism in the extension decides to overwrite the default connection pool size and "shrink" it to 1 (as seen in the message above)?

Furthermore, this message was randomly affecting endpoints that were grouped under the same monitoring configuration and "evicting" the endpoint to its own monitoring configuration remediated the issue.

We need more details around how the Extension is building a JDBC connector as right now there is nothing in the extension log that hints to how to connector is initialised.

Thanks,
Adrian

Re: JDBC driver settings when building a custom extension

Sustenance — Wed, 16 Apr 2025 03:22:49 GMT

Hey Adrian

I recently faced the same issue while creating an SQL oracle custom extension. Definitely agreed, we need more visibility over how the extensions are configured and way to configure the same.

Regards,

Akash

Re: JDBC driver settings when building a custom extension

Adrian_I — Tue, 13 May 2025 11:42:33 GMT

Small update on this request.

I've eventually opened a support ticket where I got recommended to explore introducing custom timeouts: https://docs.dynatrace.com/docs/ingest-from/extensions20/data-sources/sql/sql-reference#timeouts

According to the document above doing so should achieve parallel queries, haven't tried it out yet, but it looks promising. I am leaving this update here for reference if someone else faces the same challenges.

Later edit:

Introducing custom timeouts will result in parallel queries. It's not the most flexible option, but at least it can be used as a workaround for the initial problem.