When IT audits go very wrong: the story of Boeing and SoX
You can count on Francine's site to yield some interesting stories about audit work, and the latest post concerning the audit woes at Boeing is without a doubt the wildest epic of chaos and confusion wrought by a poor SOX job I've really ever seen.
The amount of money blown testing controls that had no chance of passing brings to mind a recent post by GeekLawyer bashing his profession, criticizing poor lawyers for spoiling their profession's reputation.
A lot of the highlighted points in Francine's article either look outlandishly grim or unintelligible, depending on your familiarity with the world of audit.
Deloitte performed the external audit, while Jefferson Wells and PricewaterhouseCoopers did internal audit work for Boeing.
Among other reasons to have them present, internal audits can test how your companies controls are working before the external auditors arrive. Since Boeing was paying a lot of money for PwC and JW to send people to test everything - the smart thing to do would be to leverage their work.
For example, if the internal audits finds that you can't rely on the controls, Deloitte shouldn't test them - they will already know the controls are 'broken'. According to the article, though, "Deloitte decided it would do its own tests to come to its own conclusion about control effectiveness". I don't know if the reporter Francine quotes has all the facts - there's a lot of he-said/she-said finger-pointing evident in the article - but assuming that the controls really weren't working, Deloitte just wasted Boeing's money by going to town with their own tests.
IT auditors have the luxury, you see, of being able to say "we can't test your system, we already know you're going to fail", and stopping there. The financial auditors still have to go in and see if everything's in order, which is a costly headache for everyone involved, but at least the IT auditors don't contribute to the bill.
The worst thing you can do, in this context, is to do full-on testing, and generate a huge bill, only to realize that the controls don't work and the financial audit team is going to have to do their intense testing.
If things are as bad as the article makes them sound, I wouldn't be surprised to see Boeing looking for a new auditor in the near future.
There are a few sensationalistic points in the article worth taking a closer look at. First, the article mentions allegations of internal audit fraud at Boeing. This would imply someone lied about whether or not they did their job. If that's the case, someone probably got a severe reprimand, or was even fired. The article doesn't go into detail, though, so just mentioning the topic is idle speculation which is rather weak journalism.
The same comment applies to the allegations of "infighting" between the two internal audit teams from JW and PwC. I suppose to depends on how much you count on the Seattle Post-Intelligencer to give you a true account of the "thousands" of internal e-mails they read in preparing their article.
Hopefully you noticed something strange there. The P-I got thousand of internal Boeing e-mails.
Talk about an information leak. How did that happen? It sounds, no doubt, like disgruntled employees were willing to talk and share a lot of information. I wonder how many people are glad Boeing 'just' makes planes, and doesn't actually safeguard personal information of the flying public?
P-I allows me to make a rare mini-case study, through use of an angry employee is quoted in the article - and his e-mail to the people auditing him is shared as a PDF, where he laments the trouble they're giving him. It turns out that a program the man was involved in setting up was being questioned by the auditors. A common step when auditing changes to a company's software is to find out whether someone approved the installation of the new program. There's other steps too, but this "authorization" is one of them - and Michael Du Pas became justifiably upset because he explains that he got evidence to prove someone provided the needed authorization.
Calling SOX "a terrorist attack on America" - great line, by the way - things get murky, fast.
His first claim is that it's "obvious" the program is approved. His second argument is that he had an e-mail to back up his claim.
I'm curious to know whether his claim that it approval is "obvious" because he had evidence, or because it's wishful thinking. I don't know, but it would matter in making a call on what happened there. In the same way, the e-mail that is cited in the PDF file is another thing an auditor would have to take a look at. Was it composed before or after the fact? The implication is the that the comment specifying when the approval was given was dated at some point in time before the program was actually installed, but that's yet another mystery.
This episode, the conclusion to which is unclear, handily illustrates the observation that "the level of emotion seemed unusual" at Boeing. And it's a shame, really. The article makes it sound like the hardy folks used to seeing auditors dropping by - the finance department denizens - were compliant with SoX demands, since they're used to being audited. The IT workers, on the other hand, are depicted as free spirits who get crushed by rules, or openly rebel against them.
That certainly happens, but when it does it's a sign that someone really dropped the ball.
People hate having their work audited in general. And depending on your attitude, it can be really grating to see some young punk subjecting you to the annual ritual of producing evidence that you dotted your i's and crossed your t's.
What makes the Boeing scenario interesting is that I've found it often easier to work with IT departments than the finance groups. While in all well managed, professional firms, you'll find friendly amiable people, there are bad eggs here and there. And some companies just have nasty work cultures.
In the spirit of over-broad generalizations, I've found that the finance department people often - though not always - are the group that's more resistant to auditors because they're used to dealing with fresh faced kids that do the low level field work on many audit jobs. The finance folks may have been doing their jobs longer than some auditors have been alive - not a far-fetched scenario when an A/P clerk is pushing 50 while the auditor is a young 20-something.
IT people, on the other hand, are usually more welcoming to intelligent auditors who treat them with the respect they feel they deserve. The counterpoint is that even if you give the curmudgeons in the finance area respect, they'll just use that as a tool to make your life tougher.
I don't mean to say Boeing's IT department has an unusually high ratio of curmudgeons to friendly people, nor do I imply that their auditors are incompetent bumbling fools who alienate their client.
I don't have enough information to make that claim. But that is the message delivered through anecdotal evidence from the P-I.
It must have been an uncomfortable day for the people in the Seattle accounting firms' offices today when this article came out. Deloitte declined to comment on the P-I story, which isn't terribly surprising given the negative spin: the P-I points out that "When it comes to telling shareholders all that it should, Deloitte
does not have a spotless record, according to government records." The article points out some findings from the Public Company Accounting Oversight Board, which criticize Deloitte for "dozens of decisions that made audit results appear rosier." This includes a finding in 2006 where the PCAOB wrote up "one Deloitte audit for certifying information
technology controls that the firm had not sufficiently tested."
The PCAOB is the American version of Canada's CPAB, a group that polices the police. They review some sample of all major audit firms' work, and it's a point of pride for any competent auditor to get a 'clean' opinion on their work.
Having said that, the P-I's move is a bit of a low blow. It'll be interesting to see if this just blows over or if a storm cloud is going to settle over this topic.