Data Engine is running but during various tasks will suddenly logs me out, refreshing the tab does not enable me to log back in, I have to close the tab and select Data Engine from the 9 dot menu.
Data engine scheduled reports have failed.
Some DataHub outages were causing the issue temporarily and rectified by our hosting team. For example, there were some DataHub issues where It has been identified that there is connectivity issues with the RabbitMQ server, which drives the Datahub messaging system. Reports of some applications receiving errors from datahub. After investigation it appears there is a connectivity issue with queues not all in sync. Our product team have temporarily paused the message flow and restored the RabbitMQ.
In relation to the performance issues, we could see that you have a big number of tasks (from which, ones are very large) and schedules with a small timeframe between. They are using a shared infrastructure, so queued tasks are expected somehow as there are busy times during the day when we have a large amount of traffic going through Data Engine. We only have a limited number of threads available that can process tasks. All customers could be affected in some way during this early morning period.
We recommend you organise the schedule in order to not have many tasks running at the same time or with small timeframe between them.
We cannot guarantee that schedules and tasks will execute and complete on the time they were scheduled for. The infrastructure is shared and during peak times (usually early morning) there are a lot of tasks which need executing and tasks are queued.
Data Engine is a data warehouse solution to enable business intelligence reporting, by pulling data from data sources, transforming the data and presenting data to Access Analytics. It's primary use case isn't to support real time reporting. Schedules can be configured to run every 30min, and the challenge is around the busy periods.
With the above in mind, we are exploring ways to improve the experience for whoever requires time critical high frequency schedules. This exploration, is both a technical one and a commercial one. It's only an exploration at this point in time, so I can't offer any guarantees, but please be assured we are aware of the challenges and are looking into resolving.