Iatrogenic Legal Assistance?

Harvard Professors Jim Greiner and Cassandra Pattanayak have posted a remarkable randomized experiment (“What Difference Representation?“) with evidence showing that offers for free legal representation from the?Harvard Legal Aid Bureau (HLAB) ended up hurting unemployment claimants.

HLAB is a “student-run, faculty-overseen” legal service clinic at Harvard Law School.? It is “the oldest student legal services organization in the country.”? In the experiment, unemployment benefit claimants (who were pursuing “first-level” appeals) were randomized into one of two groups: a treatment group which was offered free HLAB representation, and a control group which was not offered representation.? Prior to randomization all claimants agreed to participate in a randomized study.? (“If the randomization was not to offer, the student-attorney so informed the claimant by telephone and provided her with names and telephone numbers of other legal services provides in the area who might take her case.”)

The claimants who were offered representation were no more (or less) likely to win their administrative appeal – but “the offer caused a delay in the proceeding.”? The claimants offered representation had to wait on average 42 percent longer (53.1 vs. 37.3 days) before they received a decision of an Administrative Law Judge.

The results are particularly striking because not everyone who was offered representation was represented, and because those who were not offered HLAB representation were sometimes represented by alternative organizations.

The study highlights, again, the simple power of randomized control studies.? There is a persuasive transparency to randomized control trials.? The randomization doesn’t tell us why the offers caused a delay, but we should be fairly confident that those who were lucky enough not to be offered free legal assistance by HLAB had a better shot of cashing unemployment checks sooner.? This initial study’s main limitation is that its sample size is only 207.? Still, that is sufficient to raise the serious concern that HLAB’s offers of representation are hurting its potential clients. In medicine, iatrogenic effects are adverse side effects caused by medical treatment – this study points to a legal analogue in which well-intentioned legal assistance ends up resulting in adverse “side effects” for the clients.

This study raises deep ethical questions both for HLAB and other legal service providers.? Does HLAB have a duty to stop offering representation or to change its modus operandi?? Does it at least have an ethical duty to disclose the results of the study to prospective clients?? Can other student legal service organizations ethically ignore the results of the study?

Will other organizations submit themselves to the institutional risk that their services will be found lacking?? When in doubt, bet on narrow “head in the sand” self-interest.? The authors report that another Boston-based provider of similar services “did not limit its opposition to a refusal to participate on its own part. Instead, when it discovered that HLAB was conducting a randomized evaluation, it halted its previous?practice of suggesting that clients it could not itself represent call HLAB.?And as of the time of this writing, this provider is currently using its power over the intake system of a third organization to prevent this third group from conducting its own randomized evaluation.”

A Last Minute Charitable Gift Suggestion

The HLAB story also motivated me to redirect some of my year-end charitable giving.? In the past, I’ve given to causes (such as?A Better Chance) which made me feel good but which turned out to have an?abysmal record or at least no reputable evidence of success.? But this year, I’ve given money to two charities, MIT’s?Poverty Action Lab (PAL) and?Innovations for Poverty Research (IPA), which are dedicated to using randomized control studies to find out which public policy interventions work to alleviate poverty.? (Disclosure:? I have number-crunching friends at both charities.)? Would microcredit organizations do better using statistical credit scores instead of traditional subjective committee decision-making?? Does providing free chlorine dispensers at water sources reduce child diarrhea?? Scholars associated with these charities ran randomized experiments (described?here and?here) to find out.? What I love about these charities is that they add to our knowledge – even when they establish that a particular intervention doesn’t work.?? As a reader of this blog, if you’re inclined to support data-driven decisionmaking, you could do a lot worse than contributing to these non-profits.


Drill-Baby-Drill Drill Team

If a benefits claim process involves more LAWYERS than SOCIAL WORKERS then it is doomed.

If an engineering problem involves more lawyers than engineers then it is doomed.

IF a medical procedure involves more lawyers than doctors then it is doomed.

And if a law process involves more lawyers than lay citizens, it is doomed. Has anyone noticed that the majority of our legislatures are lawyers, and deadlock is the norm?

Yep, America has more lawyers per capita than any other country. Armies of Lawyers. Progress to put up a simple super efficient green windmill is delayed in courts for months then years.

MikeM

"If the randomization was not to offer, the student-attorney so informed the claimant by telephone and provided her with names and telephone numbers of other legal services provides in the area who might take her case."

There it is RIGHT THERE. Perhaps rather than the offer slowing down the process, the opposite happened. The alternative was to offer names and numbers of competent legal service providers, which sped up the process.

To continue the medical analogy, that's like giving your control group a known effective drug and then claiming the drug you're studying retards recovery compared to people who don't receive it.

Eric M. Jones

Even though it fits with my prejudice, I allow for the possibility that the study has huge gaping holes.

AaronS

While I would be interested in how well other providers performed (i.e., is this about ALL involved lawyers...or just the Harvard students?), there might be a couple of things at play here....

First, it might be that there is a natural defensiveness in the bureaucracy against those who use a lawyer. Maybe it is perceived that such a person has means that make them ineligible for unemployment. Or perhaps some folks just like to give lawyers a hard time because it is just the right thing to do.

Second, it might be a case of knowing too much. For instance, if you are a river guide on a wild river, you have a pretty good idea of all that can go wrong if you go over that ledge...so maybe you avoid it. The newbie, however, oblivious to the risk, goes over and does well. Likewise, maybe lawyers, knowing all the particulars, just get too many forms and so forth in motion, while the untrained person just takes everything at face value, perhaps making it easier on the bureaucrats.

Lastly, maybe these students, be they ever so bright, are being so careful not to miss anything--or not to blow a good grade or opportunity to make a good impression--that they are going overboard, making life more difficult than it needs to be.

Read more...

Jim Greiner

Hi, Ian, thanks so much for the shout-out re the article.

One thing your post highlights is the bravery of the Harvard Legal Aid Bureau's students in undertaking the randomized evaluation. They were under no obligation to do so (they initially turned us down before reconsidering), and as you highlight, not all legal services providers have embraced the idea of gold-standard study.

My impression is that randomized trials are the exception, rather than the norm, in almost all fields except medicine. Given their power to provide important and useful information, and given resistance from some quarters, how can we encourage their widespread acceptance? I think by doing as you suggest in your post: by supporting (financially and otherwise) entities that agree to undergo rigorous evaluation. With that in mind, the Harvard Legal Aid Bureau and the charities you highlight deserve congratulations and support.

Read more...

Surprise

HLA at least means well and were willing to have this printed.

An organization to avoid giving a nickel to, working for, or (Lord help you!) being a client of is the Jewish Vocational Service. (I worked there as a contractor.) While my experience working there was mostly painless, that of clients is another matter. If you like being a sausage then by all means make yourself a client of JVS. If you dare fancy yourself something more than that, avoid them at all costs!

Rod Joyce

As one whose career has involved both pleading and judging court cases my suspicion would be that this was an entirely understandable case of too much academic knowledge and too little real life, making the right judgment calls, experience of the realities of litigation including what actually and economically wins cases.

Browning

Some "legal" issues are really just fact issues, of no great complexity. Lawyers can't help much. Persons who own little and have no children can represent themselves in obtaining a no-fault divorce and lawyers don't change the outcome - they all get divorced. Here, the applicants either meet the criteria for benefits, or they don't.

G Wolf

I don't think you're using the word "iatrogenic" correctly. It usually means an effect that is brought about by something a physician says or does to a patient, rather than through the treatment.

Pat S

This study is, unfortunately, worthless because of the very high incidence of people in the control group (49%) actually getting legal aid help from other sources. That, as has been mentioned above, is like a drug study advising their control group where they can buy the tested drug on their own and having 50% of people take them up on it.

This completely destroys randomization, since not only is half the control group getting the experimental treatment, but the remaining half who did not are no longer a random group, having self-selected (or been selected by being turned down by other groups) to not receive the treatment.

In addition, the article cites the fact that the control group had significantly higher success in their appeals than is typical. This raises the question of whether the "half hour intake screening" also offered advice helpful in making a successful appeal (ie -- "Your case is really very simple. You just need to get hold of the following documentation and then the judge will allow your claim. You don't really need a lawyer."

What the study does prove is that the problem of crossover is very important in studies involving human subjects who have the right to exercise choices involving their own situation -- one of the major things that haunts many medical, sociological, and psychological studies.

Read more...

BeckleyLawyer

As an attorney with 35+ years experience doing unemployment cases in legal services, all for employees, I think that success depends upon the advocate's worldview. That is, to win for employees, you have to understand and sympathize with the power imbalance in the workplace. You also have to have a good idea of what must be established, even when the facts seem to be against you. Experienced advocates know these issues; and self-represented claimants instinctively know the workplace. I may be wrong about this; but I would bet that Harvard Law students have little real work experience and are naturally management oriented. Over those 35+ years I have been very unselective, taking over 95% of the cases that walked in the door, rejecting only exceptionally clear losers and a few when excessive time demands made representation impossible. Of those cases, over 90% of my clients ultimately won benefits (although some had to go through the appeals process as far as the state Supreme Court).

Read more...

Richard B.

It's highly cynical to look at the success or failure of an administrative appeal without looking at the merits of the case. Why does everyone seem to assume that an appeal may be won or lost based solely on the lawyers?

Ty

The really fascinating product of the study was the probability numbers the researchers were able to put to certain legal outcomes.

I'm hoping the "Moneyball" revolution will come to the legal industry soon. But, as the study showed, it can be difficult and vexing to categorize and put firm numbers to legal dispute outcomes.

JoelP

"To continue the medical analogy, that's like giving your control group a known effective drug and then claiming the drug you're studying retards recovery compared to people who don't receive it."

Yeah, that's exactly how we do it in medicine. You compare the drug in question to the best known alternative. It's unethical to give them nothing unless you believe nothing is an appropriate treatment. You ask if the Harvard Legal Aid Bureau is effective? Well, you compare it to giving a list of lawyers they can call and letting them make their own decision. That's the exact right control group.

If you wanted to do a different study and say "here's a self-help guide" vs "here's a list of lawyers" that would be another interesting study to look at the effectiveness of the self-help guide in question. If you found one that was helpful (or even found that a blank paper was as helpful as the list of lawyers) then you could use that as a control. Today you can't.

Read more...

JoelP

Richard: That's what randomized control trials are for. If you randomize people to one group (aid) or another (a list of names that they may or may not choose to call), you get to assume that the people in both groups have similar proportions of meritorious cases. Obviously you need to avoid systematic problems (like nonrandom assignments of people) or low n - but avoid those and you never need to look at the merits of the cases.

bill fischel

Ian, thanks for using the term "disclosure" instead of "full disclosure." Reporters and opinionators who say "full disclosure" are either sloppy or trying to lull their readers into acceptance of everything else they say.

Victoria S

While this article does bring up some good questions, I wish you had emphasized more that the study only focused on unemployment claims. There have been similar (as in truly randomized) studies that have found that counsel can have a significant positive impact on the outcome of a claim. What this study tells us, more than anything, is that we need more studies.

There's an interview with the study's author that does a much better job of discussing the significance of the study's findings and the difficulty of doing a randomized study of legal services. It can be found at accesstojustice.net blog (the January 14th post.)

In the interview, Jim Grenier observed that IF we can truly say that free representation is not helpful to unemployment claimants, this may be due to the accessibility of the unemployment claims system, which, in turn, may be due to the efforts of legal aid service providers to make the system pro se friendly. As a volunteer legal aid attorney, I found this comment heartening.

Read more...