How often have you seen IT professionals point their finger to another technical area, when a problem arises? To me it happens far too often. How do you think it will look to your peers, or even better, your management, if you were to point the finger at other groups and it turns out to be your problem? To me it wouldn’t look good!
Recently at one of my assignments, we had a performance issue on Oracle database servers that have SAN connected devices. At certain times during the day, the commit wait time would increase drastically and cause application connections to time out. The problem most closely resembled a SAN or storage-related issue. The application teams concluded storage as the culprit, as well.
On the infrastructure side of the house, we realized the problem could have been with Oracle, the operating system, device drivers, firmware, or as the application team suspected, the SAN/Storage. Each team did their job opening support tickets with the vendors (EMC, RedHat, HP and Oracle), to which each vendor provided recommendations for resolving the issue, in addition to validating that their supported areas or responsibility appeared healthy and functioning properly. Oracle suggested applying a patch and verifying storage health with the SAN/Storage team. RedHat suggested changing the I/O scheduler and verifying storage health with the SAN/Storage team. Dell EMC also suggested adjusting the I/O scheduler and to setting the HBA and Disk Devices max queue depth on the affected host system.
In an attempt to address the performance issue, we applied the Oracle patches, adjusted the Linux I/O scheduler and set max queue depth. While those changes were being implemented, we also worked closely with EMC to collect system logs for further analysis. In the end, the root cause turned out to be a recently identified bug/issue with the EMC VPLEX, and we ended-up applying new code to resolve the issue.
This is a perfect example of how each of the technical teams worked together, did their due diligence to rule-out their specific supported technology as the potential cause of the problem, instead of just taking the easy route, by assigning blame to other technical teams involved.
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
Cookie | Duration | Description |
---|---|---|
cookielawinfo-checbox-analytics | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics". |
cookielawinfo-checbox-functional | 11 months | The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional". |
cookielawinfo-checbox-others | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other. |
cookielawinfo-checkbox-necessary | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary". |
cookielawinfo-checkbox-performance | 11 months | This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance". |
viewed_cookie_policy | 11 months | The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data. |
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.