Forums
Dec 12Too Much Sleep
by in Best of the Sidebar on 2013-01-22I've frequently posted about my attempts to speed up our system being thwarted by sleepy management decisions about application performance. Our application is essentially: query data, crunch data and save results. Each of those tasks took approximately 1/3 of the run time. A large part of my job is to make the application run more quickly. Every time I fixed something, something else would break, not because I coded something incorrectly, but because of the fragility of our application, other associated applications and the database. For instance...
Sleep for the database
We run four instances of our application. We also perform about 30 load-data queries. Most of them query tables with 3+ billion rows. If all four instances of the app do queries at the same time and each query is run in its own thread, that's 120 parallel queries just from our application server instances. The database simply could not keep up. The original developer created five threads for thirty parallel queries. Then, to add insult to injury, he added sleeps after each query (in each thread) to give the database a chance to catch up. Part of the problem turned out to be that the DBAs only allocated 512M of temp space for all users. They refused to fix it.