Long Hours in the Business World Our two previous posts on the effects of long hours on errors and reduced productivity focused on two professions, doctors and lawyers. We picked those two rather than further explore the issue in the military because long hours for doctors and lawyers are purposely selected by managers whereas soldiers work long during wartime or training for wartime. This choice to impose long hours implies acceptance of the costs and benefits that result; the professions also have higher duties to the public, not just their patients and clients, that are codified in various ethical codes as well as actual statutes.

The original article sparking the discussion, however, came not from the professions but from the business world, and so we thought it would be helpful to discuss these issues in light of two additional fields where long hours are already on the table: consultants and business in general. We think that exploring the reasons why long hours in these fields might not be so readily discovered as a source of problems will help design appropriate experiments for firms to undertake as they seek continuous improvement.

Let' start with consultants. We have had substantial experience with the consulting world, and long hours are almost a badge of honor in the field. Travel is an additional duty for consultants, and an extra 10 hours a week of travel is not unusual, even for those who work all week at a single client location. If long hours reduces productivity and increases the likelihood of errors, why aren't these problems apparent or acknowledged by consultants?

First, like lawyers, consultants typically bill by time (even if their firms do not, consultants are tracked by hours per project). This hourly approach creates, if not a perverse incentive, the complication that it is hard to distinguish between lots of optimal work and lots of inefficient work. This lack of clarity affects the consultant, the firm, and the client, all without any bad faith. Indeed, benchmarking one consultant against another to compare the time taken for various tasks would likely show the same sorts of performance losses and hence be taken for normal performance. In other words, two folks performing at 80% would each give the appearance of 100% if matched against one another; of course, that's the wrong question.

Second, unlike doctors, whose "mistakes" often have immediate effects, any errors caused by consultants may never be discovered or be revealed to have any effect. (Of course, the counter-argument is that if a mistake doesn't have any ill effect, it's not a mistake.) But the lack of revelation is different from the actual mistake. For example, imagine a spreadsheet that contains errors in formulas used to support part of a decision analysis. If the right decision doesn't get made, in part because of the spreadsheet calculations, it may be invisible to the participants, but the end result will not be. Companies regularly miss earnings projections (at least they would if they weren't so heavily managed) and often fail to earn their cost of capital. Those are certainly "mistakes," broadly construed, and virtually impossible to connect to a specific action.

It's clear that this second point is the one most likely at work in the corporate world. Once we moved most work from factory or manufacturing work that is relatively easy to measure to much fuzzier knowledge work, we exposed ourselves to productivity problems and cures of all kinds, all equally undefinable and unreliable. In the same way that it is difficult to determine the effects of fatigue on productivity, it is difficult to sort out what benefits in performance may be expected from other changes. This disconnection is one reason that usability experts are still focused on getting businesses to implement changes that are easy to measure, such as intranet structure for common activities.

If it's hard to observe and measure real-world effects of chronic fatigue and long hours, where can we get evidence about the likely effects that is convincing enough to allow leaders to implement changes, or at least tests, in their organizations? Well, some of that research already exists and was referenced in the original article. The Belenky article describes the pattern of failure from sleep deprivation. Performance slowly degrades until a critical failure is reached because the time available to make a decision or analysis arrives while the decision-making process is not complete. "Thus, a gradual decline in performance during simulations or laboratory studies maps into a long period of apparently adequate, if not good, performance in actual operations, and then, suddenly, failure. " This paradigm is supported by our personal experience in Ranger School (described in the article as 3.6 hours of sleep -- we wish we saw that much every day!), in military training, in Ivy League graduate school, in careers in professional services organizations, and in our current roles leading our own portfolio of businesses.

Is knowledge work like that described by Col. Belenky? We think so. The dichotomy between the sustained ability to complete physical tasks and the degraded ability to maintain situational (or strategic) awareness, the context for those physical tasks, describes very well the difficulties facing people in the corporate world. It remains possible to read and type, to even modestly edit and review presentations and spreadsheets and documents, but the strategic viewpoint, the stream of constant background analysis that is the hallmark of good decision-making, is lost. This continuing failure to appreciate the big picture, if it affects all those folks involved in strategic decision making as a result of long hours at work over time, could explain the almost random performance of corporations and the failures we have seen to make even the most basic decisions right on a consistent basis, namely ensuring that the firm earns its cost of capital.

This degradation results in a constant watering-down of analysis since the simple tasks are done and the obvious connections made. But the competition can be assumed to make the same simple connections as well. Working smarter, not harder, has been a theme for the last 15 years, since automation and knowledge management become more accessible through the ready availability of information technology resources to almost all workers. While that may be true, What we're learning, however, is that working harder is almost certainly not working smarter.

Labels: leadership, performance

Tuesday, January 08, 2008 :: posted by Rick Colosimo @ 10:11 AM

Big law firm hours explained - competing interests This is the second in our series of three posts relating to the idea that long hours, substantially over 40 hours per week, lead to reduced cognitive function and poor performance, most notably lower productivity per unit of time.

Our first post reframed and rehashed these arguments in the context of the medical profession, notably the long hours of medical residents, which some medical groups now suggest should be limited to an average of 80 hours per week.

We wrote how even if you exempted malpractice in the legal, actionable sense from the discussion, professional ethics should lead doctors to reduce hours for residents. (Question: are all hospital administrators practicing physicians? Certainly shareholders are not.) The next profession we are going to analyze is the legal profession. Not only do we have direct experience here, we have also been in the position of law firm client (too many times, most would say). The situation for lawyers is akin to that of doctors: malpractice is an actionable claim for injured clients; professional ethics create obligations governing one's practice.

If you accept the general premise that lengthy hours have a dual effect of both reducing productivity and increasing the likelihood of mistakes, then lawyers, and by extension their law firms, are probably committing professional errors (meaning a breach of the general code of ethics requiring competence), if not actual malpractice, by increasing billable hours per associate. Various times throughout the last ten years (and obviously much before that as well), first-year lawyers have been the recipients of dramatic increases in salary. Of course, there's no such thing as a free lunch, and the tradeoff for associates has typically been increased billable hour requirements. Firm have typically chosen to increase the hours of associates and partners rather than maintain the hour requirements and increase the numbers of lawyers. This increase in hours is driven by the increase in salaries, which in turn drives tuition hikes.

So, where are the studies that show the increase in mistakes for lawyers, like the ones for doctors? They don't exist. Why is that? Several reasons:

1. Big law firms typically bill clients by the hour for legal services. If productivity declines and it takes longer to resolve matters, the first and most obvious result in increased billable hours. So, rather than poor medical outcomes for patients, which are noticeable to those outside the hospital, we have larger bills, which are certainly noticeable to the clients but not distinguishable from "business as usual."
2. Errors that would lead to malpractice seldom lead to actual malpractice claims. Not only are errors often caught by other lawyers involved in a matter, but the errors themselves are harder to identify as errors, since they seldom have immediate effects and the true outcome may not be known for months in a transactional matter or even years in a litigation. Many errors are harmless because the circumstances that would trigger them fail to arise; a choice of law clause only becomes relevant when litigation is contemplated. Even when matters do turn for the worse, the idea that a negative outcome could be traced to a single error, which could then be traced to acute or chronic sleep deprivation, is harder to imagine.
   
3. It's possible that even when operating at a sub-par level, the types of lawyers who work at big firms may still perform at the standard of care, which is more generally based on lawyers overall. This argument has a lot of assumptions and perceptions built into it, which should be made explicit:
1. Associates who go to big firms are commonly perceived (at least among those seeking to go to big firms and among the big firms hiring them) to be of higher quality than those who aim differently following law school. They are perceived to be generally more capable legally.
   
2. Associates at big firms are perceived to be harder workers, or at least to have worked harder, in terms of more hours, during law school. Big firms prize law review membership and high grades, assuming that the former's hourly commitment combined with the latter's intellectual performance will translate well to the firm environment.
3. One of the key skills for lawyers is attention to detail, and that is another one of the skills for which good grades and law review time are supposed to be proxies.
4. Another difference between medical practice and legal practice, related to point #1 is that because of the fee-sharing restrictions in legal ethics codes, lawyers still run their own practices. Not always from a day-to-day perspective, where smarter firms are hiring non-lawyers to manage the business while leadership is to be provided by senior lawyers. The medical field has not had such a restriction in a very long time, exacerbated perhaps by early concerns about the effects of medical malpractice exposure to individual doctors. That led to the ability to restrict professional liability and to growth to spread risk. Many of those larger entities became public entities with shareholders and other investors. While it is obviously common for hospitals to be run by doctors, it is not required. The key argument for why law firms should not be owned by non-lawyers is that the profession does not want lawyers to be subject to people who did not have the same ethical obligations as the lawyers. Where this tension currently exists, however, is in corporate legal departments, and the complications are clearly there.
   

So these assumptions might explain why the rate of error would be lower, but does it really make any sense that lawyers would actually err less than doctors? Doctors constitute a much smaller universe than lawyers, and few lawyers who would claim that medicine is an "easier" field for those with less powerful intellect. If doctors can make apparently clearly measurable errors, wouldn't it make sense that lawyers make at least the same number of errors? If anything, the type of errors committed and the circumstances have a greater impact on what outcomes are measured and the perception of those errors.

Do firms have an ethical obligation to reduce attorney billable hours to reduce risk of poor performance? Do they have an ethical obligation to reduce billable hours to reduce low productivity, which results in larger bills for the same work? What about clients? Certainly the primary clients of large law firms are large corporations, who might be more knowledgeable than individuals about these effects. Also, given the outside legal budgets of large corporations, they probably experience low-probability events on a regular basis and so can expect that there is some harm to them from these issues. Does anyone know of large corporations that have firm restrictions on the number of hours that may be billed to them by a single lawyer, or by such a lawyer in a week, month, or year?

We are attempting to identify some future research opportunities in this field. Does anyone have connections inside a legal malpractice insurer? We would be interested in tracking claim experience by firm size and hour requirements. Similarly, we are preparing a proposal for corporate counsel at large commercial firms to evaluate lawyer performance in light of these issues.

Business people inside corporations and acting as consultants are our next area of inquiry in this sleep deprivation/overwork trifecta.

Labels: leadership, performance

Sunday, January 06, 2008 :: posted by Rick Colosimo @ 11:02 AM

Fatigue - Doctors Would Know Better With More Rest

The IGDA is the International Game Developers Association, which includes all those people who create videogames, from programmers to testers to artists and many, many more. The linked article is a quasi-response to another post in which someone said to be the spouse of an Electronic Arts employee said that programmers were working mandatory 87.5 hour workweeks. The author of the article writes, not strictly about videogames, but about workers in general. He refers to several items in the history of the 40-hour workweek, tracing the development and changes in hours over time. The takeaway: the perception that hours can be costlessly increased for salaried knowledge workers is false, and in a big way. Productivity declines rapidly, such that the work done in 8 60-hour weeks actually approximates that completed in 8 40-hour weeks. That's a pretty amazing drop-off, and it's one that deserves further study beyond what's cited in the article.

A profession is defined by three things: specialized knowledge, a public trust, and an ethical code enforced by self-government. In this first of three posts related to the fatigue brought on by long hours and little sleep, we explore the article as it relates to the medical profession. Doctors have been debating the issue of 24-hour shifts in earnest over the past several years. Doctors are almost certainly committing professional errors, by which we mean breaches of ethics codes governing the profession, if not actual malpractice, by rigidly sticking to the 24 hours on/off rules and long hours for interns & residents. While that field is slowly asking the right questions, they don't seem terribly concerned about moving on, even for the sake of the patients. Nurses work similar odd schedules, typically 3x 12 hour shifts and then overtime after that. Both argue that continuity of care is a moderating factor, and so that should indeed be taken into consideration, but the number of accidents is certainly a knowable number, even if the alternative isn't. But given the number of institutions in the country, and the world, there should be some comparative data.

This article http://www.ncbi.nlm.nih.gov/pubmed/15509817 notes that residents made substantially more errors when on a 24-hour shift schedule. We think this is reason enough for the public to demand changes, but the real concern is why aren't doctors and hospitals taking this more seriously. Since this article was published, there have been articles in the medical journals and related forums outlining the risks, and the AMA has taken some steps to recognize the problems, although they are too slight to be described as mitigating the potential harm to patients (the proposed maximum is 80 hours/week. From a professional ethics perspective, particularly one that starts with thou shalt do no harm, we don't understand why doctors don't feel compelled to make the necessary changes. The arguments for the long shift are weak when measured against preventable harm: continuity of care is a common excuse, and "toughening" up residents is another.

Continuity of care makes more sense with nurses, who often work three 12-hour shifts per night, and can be assigned in ratios as low as 1:1. Doctors, on the other hand, are typically running from patient to patient on rounds in-between emergencies and urgent consults. Similarly, nurses typically have more varied work duties, allowing them to somewhat tailor their current task to their alertness level whereas residents can be called on to make the most difficult decisions of their shift at anytime, whether hour 1 or hour 21.

Toughening up residents is a faux argument. The grueling schedule is entirely man-made. This is not a situation at all like military special operations training courses, such as the US Army Ranger School, where sleep deprivation is a specific part of the program to increase stress and help these future leaders learn how they and their soldiers react under some of the conditions they will face on the battlefield. But that's training, and no military leader would deliberately handicap troops in battle by depriving them of sleep in the absence of a mission-critical requirement. The more you sweat in training, the less you bleed in combat.

(We contemplated asking family members who are now at the chief resident level, but we decided it was a little too combative to call them unprofessional to their faces.)

Friday, January 04, 2008 :: posted by Rick Colosimo @ 11:07 PM

Eisenhower Quote on Planning "In preparing for battle I have always found that plans are useless, but planning is indispensable."


We use this phrase a lot in two contexts: first, we freely admit, and accept, that our plans may not work out as expected, which is a risk. We then discuss how we manage that risk by a thorough planning process that takes into account the existence of scenarios as opposed to a singular rosy vision of the world. In other words, good plans identify probabilities of various events and then seek to plot a course that is responsive to the array of possible futures. In other fields with different problems, this approach is sometimes described as game theory.

The second context in which we find ourselves repeating this maxim is when we review business "plans" of startups seeking our help either through our Coordinated Market Entry service offering or through our affiliated investing entity, Namahagi Ventures. A startup seeking millions in investment needs a real plan, if only to provide evidence to investors that when the plan turns out to be wrong, the founders will have already thought through the issues and tradeoffs and will be immediately prepared to undertake a new course of action.

Scenario planning is one project we often undertake where this quote could be shortened to "planning is indispensable." It is the embodiment of planning in an environment of uncertainty, in which no one know what is actually going to happen. Please note that there is a difference between business plans and scenario planning. Good business planners plan for the expected and are prepared for the unexpected. Scenario planning takes that process further into the future by preparing for likely changes based on expectations about macroeconomic trends and similar big-picture events that should trigger a change in, or at least a review of, strategy.

Labels: quotes

Thursday, January 03, 2008 :: posted by Rick Colosimo @ 4:02 PM

GiveWell fiasco -- what's different? Well, the time to start an organization related to "charity ratings" in any way: not right now. GiveWell, mentioned in an earlier post, has admitted to astroturfing charges. (Astroturfing is when you pretend that there's a grassroots supporters of your business or cause. Basically, one of the founders asked questions under a fake name and then answered them under a different name.) What a mess, and what a shame.

Perhaps this episode highlights the key distinction between GiveWell's approach: rather than think that we know better than the nonprofits themselves how to solve the problems they're working at every day, we believe that unlocking and sharing their knowledge, and revealing that to donors in a coherent manner, will reveal the better ways to solve the problems. What does that actually mean? It means that while we think we know better than most how to present and compare information, we're not likely to try to substitute our judgment for someone else's -- which makes the GiveWell activity much less likely for us just in terms of underlying incentive. In other words, when you think you're smarter than other people, you start wanting to make decisions for them. Our approach is founded in the most historical American notions of freedom: the freedom for everyone to make up her own mind. Also, we have specifically decided to preserve the strategic element in our comparisons by only discussing & comparing organizations in each strategy within a sector.

By presenting the information we receive from organizations in a standard format, we hope to work from the other side of the river from WhatWorks.org, building a bridge that should eventually join the two projects. While they are creating standards and forms, an idea we support, we recognize that harnessing the wisdom of the organizations by the organizations has immediate potential for impact. In other words, isn't a nonprofit more likely to adopt a strategy knowing it has worked elsewhere than through a standard or form that describes or implements that same strategy? It shouldn't be that way, but it is.

Operational excellence is evolutionary -- it is refined by recurring methodical review of what went well, what went wrong, and what should be changed as a result. Our goal is to capture information organizations in different stages of those reviews, collate it into a cohent body of work, and share it openly, allowing it to be review and commented on by the community. Why should there be a community approach rather than a top-downish (and we don't mean that in a perjorative way) approach like Whatworks.org? The right answer is that the system needs both. We've always assumed that after the collection of "practices" in a sector, we would actually draw out the "best practices" identified by the community, create a workable plan primarily by ourselves, and then expose that plan to the community for further comment.

There is a lot of discussion about evaluations and their cost. That problem is one triggered by the top-down approach. We're saying something similar and something different. The first point is that we absolutely agree that evaluations have to take place or we all risk wasting an awful lot of time. The second is that it makes sense, from an efficiency perspective, to see what organizations are doing themselves to evaluate their efficiency and effectiveness. That much the GiveWell folks have right: the proxies often touted are primarily useful for identifying waste, not for identifying true program impact. If that weren't true, then why would the Robin Hood Foundation expend the time and money to create an evaluation system? Wouldn't they just use the financial data umbrella sites? The fact that those smart people with money to spend, just like similar programs at other large foundations, want to spend it wisely. Some of them, more than one, have decided to evaluate impact in some way. We want to support that by making it easier, rather than harder, for nonprofits to engage donors in those evaluation discussion.

Labels: nonprofit

:: posted by Rick Colosimo @ 1:13 PM

Measuring Nonprofit Performance - other approaches In refining our vision for the Wolfhound Fund, we've come across different ideas that are relevant to our plans but don't capture the entire idea. In this post, we'll discuss some other approaches, but note that we will not go over what we've already discussed, namely Robin Hood.

The NYT recently published an article on GiveWell, a new organization that seeks to collect information about measurable results obtained by charities and uses that information to rank charities it has evaluated. They are incredibly open in their approach and vow to make the information, the evaluations, and their methodology available to the public. We think this is absolutely the right approach. The most disturbing thing they reveal is that many of the larger foundations they contacted would not disclose their rationales, decision-making process, or grant recipient information to the GiveWell parent foundation. We are shocked; we have been hoping to, in the course of developing our metrics in an open platform that we would be supported by foundations who have presumably spent a lot of time evaluating various programs. We have not yet contacted these organizations to "seed" the discussions with existing metrics, but it's certainly on the list.

The WSJ recently published an article on due diligence on donations. They note the typical first step of evaluating along standard financial measures, using the excellent resources that make those results, and the Form 990s from which they are derived, available. Notably, they identify GiveWell as one of the new organizations helping evaluate charities in a different way. The article mentions several other groups that take a similar approach to evaluating measured results. One theme is that many organizations do not attempt to research and measure their results; this statement does not surprise us given our experience with organizations.

First, for-profit businesses seldom actually evaluate whether they are creating value, and the myriad financial publications seem almost unwilling to identify those companies that are regularly destroying value -- not just those that are actually losing money or not producing cash, but those that are underperforming by not earning the cost of capital. This notion led us to believe philosophically that many nonprofits were not instituting similar rigor in their operations.

Second, our personal experience in forming nonprofits and advising founders indicates that often, the same fervent belief and desire to help that sparks the impulse to start an organization and bear those early burdens also negates the natural skeptic within those same founders; they start organizations because they believe. This mindset isn't to be ridiculed; it's to be understood. All organizations, whether for-profit, nonprofit, family, or recreational, start with some leap of faith, however small or large. Our response and reaction to this circumstance is to rely on a liberal arts education and quote Milton: "I can not praise a fugitive or cloistered virtue;" our second response: "The unaimed arrow never misses." More melodramatic is Socrates: The unexamined life is not worth living.

WSJ also gives a sidebar mention and link to The Center for What Works. They provide a variety of benchmarking materials to nonprofits, and even provide basic education about benchmarking and how the process works. They give a great introduction for their audience, who typically doesn't come from a big-firm corporate culture where benchmarking, the development of Key Performance Indicators, and a dashboard/balanced scorecard/3x5x15 report to track it all are more common. While WhatWorks has benchmarking information, it does not

Their initial work on developing a standard taxonomy (PDF) for various types of outcomes is exactly the type of thing we want to create. Building on this standardization effort by adding domain/sector-specific metrics is one way of visualizing the Wolfhound Fund's approach. In addition, they've taken this general taxonomy and actually applied it to 14 specific program areas, revealed at this page. If anything, these magnificent efforts clearly show the way and exemplify the type of standardized tools we believe should be produced for, and more importantly by, nonprofits and the extended community of stakeholders, including donors and program recipients.

What separates the Wolfhound Fund from these organizations who have already started collecting research and making evaluations? Three things: first, we come from a strong background combining military leadership with traditional financial analysis, which has made us experts in the tools, language, and very idea of converting strategy into action through leadership, training, execution, and management; second, we believe that the organizations that are measuring their results are the best place to start the discussion -- we hope to build on the thousands of internal discussions in dozens of organizations who have already addressed these questions rather than seek to substitute our judgment for theirs; third, we aim to collect this information from a broader cross-section of the charitable universe so that donors interested in a sector, such as homelessness, can evaluate charities with different strategies that are not directly comparable. That second order decision, of choosing a strategy to combat a problem, is not the primary inflection point the Wolfhound Fund seeks to influence but one that we hope to illuminate just the same.

Fundamentally, we believe that an open-source, continuous improvement model is the best way to create self-correcting, self-improving, and self-proving tools for creating positive change. This is a small part of the overall problem, and we in no way compare ourselves, sitting alongside board members and donors, as remotely as committed as those who actually run homeless shelters, who show up to hearings for protective orders, who perform surgeries in developing countries, who counsel and guide troubled youth. Our goal is to help the outsiders find those people who do these jobs better than everyone else and who have built programs to put themselves out of business.

Labels: nonprofit, wolfhound fund

Tuesday, January 01, 2008 :: posted by Rick Colosimo @ 1:26 PM